Debian GSoC Kotlin projecthttps://java-team.pages.debian.net/gsoc-kotlin-blog/2020-06-01T15:38:00+05:00Kotlin Update2020-06-01T15:38:00+05:002020-06-01T15:38:00+05:00Samyak Jaintag:java-team.pages.debian.net,2020-06-01:/gsoc-kotlin-blog/2020/06/01/kotlin-update/<h3>A Quick Recap from last year:</h3>
<p><strong>Kotlin</strong> is being packaged under the Google Summer of Code within the Debian organization itself. The major reason behind bringing Kotlin in Debian is to update all the Android packages which are now heavily dependent upon the Kotlin libraries.</p>
<p>The major work to bring …</p><h3>A Quick Recap from last year:</h3>
<p><strong>Kotlin</strong> is being packaged under the Google Summer of Code within the Debian organization itself. The major reason behind bringing Kotlin in Debian is to update all the Android packages which are now heavily dependent upon the Kotlin libraries.</p>
<p>The major work to bring Kotlin into Debian is done for the version 1.3.30, by <em>Saif Abdul Cassim</em> (goes by <em>m36</em> on IRC) as a part of his GSoC'2019.
All his contributions to the team can be found in <a href="https://java-team.pages.debian.net/gsoc-kotlin-blog/">his blog posts</a>.</p>
<p>So, for now, we have a bootstrap package and a Kotlin package for the version with 1.3.30.
There were still changes needed as we lacked some of the dependencies for Kotlin, and the source package lacked copyright information and didn’t comply with Debian standards.</p>
<h3>What's the present year brought for Kotlin?</h3>
<p>To be specific the following were mainly left dependencies for Kotlin:</p>
<ul>
<li>JLine3</li>
<li>intellij-community-idea</li>
<li>kotlin-bootstrap</li>
</ul>
<p>And, we lack documentation for the newbies in order to get them started :(</p>
<p>Most importantly the crucial part was and still is, to figure out how to upload the package?</p>
<p>For GSoC'<em>20</em>, three students are selected as a part of project <strong>Android SDK tools in Debian</strong>.</p>
<h4>What's the work done/left?</h4>
<p><strong>Work Done</strong></p>
<ul>
<li>
<p>A couple of dependencies were completed and reside in NEW Queue, those include <code>Jline3</code> (done by <code>@samyak-jn</code>, myself), and <code>intellij-community-idea</code> (finished by <code>@The_LoudSpeaker</code>, Raman Sarda).</p>
</li>
<li>
<p>The <code>kotlin</code> package residing in m36’s repository had a couple of issues that were needed to be fixed to meet Debian standards, but Kotlin was building fine locally with the mentioned dependencies. :D</p>
</li>
<li>
<p>I (Samyak Jain) took the work for converting all the commits to the <code>patches</code> as all the changes were made directly to the source, and henceforth fixed <code>rules</code> and <code>control</code> files to meet Debian Standards. Debian is very particular about its license policies. The copyright was a pending task that was completed for Good.
The newer package exists at <a href="https://salsa.debian.org/samyak-jn/kotlin">Samyak's repo</a>.</p>
</li>
<li>
<p>I set up an initial wiki page for Kotlin as well, so everyone can follow. Thanks, Hans (<code>@_hc</code>) for the help with that. The wiki page for Kotlin exists <a href="https://wiki.debian.org/Kotlin">here</a>.</p>
</li>
</ul>
<p><strong>What's Blocking?</strong></p>
<ul>
<li>The most uncertain thing is to decide, how Kotlin will be <strong>uploaded</strong> to the Debian Archive?</li>
</ul>
<p><strong>What is the problem being faced?</strong></p>
<p>The Kotlin-Bootstrap package consists of JAR files for various dependencies of kotlin such as Gradle, kotlin compiler, and kotlinx. The package is added to the build-depends of the main package so that the JAR files can be provided. Since the kotlin-bootstrap consists of binaries (JAR files), it is not feasible to upload the package as free software.</p>
<p>The other workaround was the Gradle 6.4 version, which consists of Kotlin files and generates a suitable JAR. But since the package needed Kotlin language itself, it was never updated, as it created a cyclic dependency.</p>
<p>Final workaround came, which proposed <em>Kotlin to build from itself</em>, that was a pretty impressive suggestion. But, we still have to look if the solution is feasible? Because, as far as I last checked and conversed with ebourg on the mailing list <a href="https://lists.debian.org/debian-java/2020/05/msg00048.html">here</a>, Emmanuel Bbourg mentioned very clearly that the <em>rebuilt package is our interest</em>. So, this is under WIP.</p>
<p>But, I fail to acknowledge the fact if we can drop the kotlin-bootstrap package totally, Kotlin will not be able to be built because each and every JAR file present in the bootstrap is needed.</p>
<p>That pretty much is the ongoing work and the update on the kotlin package. We intend to bring Kotlin to the Debian Archive as soon as possible :)</p>
<h4>Have any queries or suggestions for Kotlin?</h4>
<p>Please feel to drop a message at <code>#debian-mobile</code> channel on OFTC.</p>Beginning of the end2019-09-08T17:09:26+04:002019-09-08T17:09:26+04:00Saif Abdul Cassimtag:java-team.pages.debian.net,2019-09-08:/gsoc-kotlin-blog/2019/09/08/begining-of-the-end/<h2>Work done</h2>
<p>Hey all, since the last page of this post we have come so far into packaging Kotlin 1.3.30.
I am glad to announce that Kotlin 1.3.30's dependencies are completely packaged and only refining work
on intellij-community-java (which is the source package of the IntelliJ …</p><h2>Work done</h2>
<p>Hey all, since the last page of this post we have come so far into packaging Kotlin 1.3.30.
I am glad to announce that Kotlin 1.3.30's dependencies are completely packaged and only refining work
on intellij-community-java (which is the source package of the IntelliJ related JARs that Kotlin depended on) and Kotlin remain.</p>
<p>I have roughly packaged Kotlin, the debian folder is pretty much done, and have pushed it <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30">here</a>.
Also the bootstrap package can be found <a href="https://salsa.debian.org/m36-guest/kotlin-bootstrap">here</a>.</p>
<p>The links to all the dependencies of Kotlin 1.3.30 can be found in my previous blog pages, but I’ll list them here for convinience of the reader.</p>
<ol>
<li>→ java-compatibility-1.0.1 → https://github.com/JetBrains/intellij-deps-java-compatibility (DONE: <a href="https://salsa.debian.org/m36-guest/intellij-java-compatibility">here</a>)</li>
<li>→ jps-model → https://github.com/JetBrains/intellij-community/tree/master/jps (DONE: <a href="https://salsa.debian.org/java-team/intellij-community-idea/tree/wip/m36/master">here</a>)</li>
<li>→ intellij-core → https://github.com/JetBrains/intellij-community/tree/183.5153 (DONE: <a href="https://salsa.debian.org/java-team/intellij-community-idea/tree/wip/m36/master">here</a>)</li>
<li>→ streamex-0.6.7 → https://github.com/amaembo/streamex/tree/streamex-0.6.7 (DONE: <a href="https://salsa.debian.org/m36-guest/streamex">here</a>)</li>
<li>→ guava-25.1 → https://github.com/google/guava/tree/v25.1 (DONE: Used guava-19 from libguava-java)</li>
<li>→ lz4-java → https://github.com/lz4/lz4-java/blob/1.3.0/build.xml (DONE: <a href="https://salsa.debian.org/m36-guest/lz4-java">here</a>)</li>
<li>→ libjna-java & libjna-platform-java recompiled in jdk 8. → https://salsa.debian.org/java-team/libjna-java (DONE: <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/commit/4c3fcc86535bad68df6c48b01fd8ef52c5a8662e">commit</a>)</li>
<li>→ liboro-java recompiled in jdk8 → https://salsa.debian.org/java-team/liboro-java (DONE: <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/commit/f2d3f051f5bb57052fff54a03419daa929b1884d">commit</a>)</li>
<li>→ picocontainer-1.3 refining → https://salsa.debian.org/java-team/libpicocontainer-1-java (DONE: <a href="https://salsa.debian.org/java-team/libpicocontainer-1-java">here</a>)</li>
<li>→ platform-api → https://github.com/JetBrains/intellij-community/tree/183.5153/platform (DONE: <a href="https://salsa.debian.org/java-team/intellij-community-idea/tree/wip/m36/master">here</a>)</li>
<li>→ util → https://github.com/JetBrains/intellij-community/tree/183.5153/platform (DONE: <a href="https://salsa.debian.org/java-team/intellij-community-idea/tree/wip/m36/master">here</a>)</li>
<li>→ platform-impl → https://github.com/JetBrains/intellij-community/tree/183.5153/platform (DONE: <a href="https://salsa.debian.org/java-team/intellij-community-idea/tree/wip/m36/master">here</a>)</li>
<li>→ extensions → https://github.com/JetBrains/intellij-community/tree/183.5153/platform (DONE: <a href="https://salsa.debian.org/java-team/intellij-community-idea/tree/wip/m36/master">here</a>)</li>
<li>→ jengeleman:shadow:4.0.3 -→ https://github.com/johnrengelman/shadow (<a href="https://salsa.debian.org/m36-guest/jengelman-shadow">DONE</a>)</li>
<li>→ trove4j 1.x → https://github.com/JetBrains/intellij-deps-trove4j (<a href="https://salsa.debian.org/java-team/libtrove-intellij-java">DONE</a>)</li>
<li>→ proguard:6.0.3 in jdk8 (DONE: released as libproguard-java 6.0.3-2)</li>
<li>→ io.javaslang:2.0.6 → https://github.com/vavr-io/vavr/tree/javaslang-v2.0.6 (<a href="https://salsa.debian.org/m36-guest/javaslang">DONE</a>)</li>
<li>→ jline 3.0.3 → https://github.com/jline/jline3/tree/jline-3.3.1 (<a href="https://salsa.debian.org/m36-guest/jline3">DONE</a>)</li>
<li>→ protobuf-2.6.1 in jdk8 (<a href="https://salsa.debian.org/java-team/protobuf-2">DONE</a>)</li>
<li>→ com.jcabi:jcabi-aether:1.0 → the file that requires this is commented out; can be seen <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/commit/cb298ba550ca9f727ff66e4ffca0cb73e9ee03f1">here</a> and <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/commit/1503110b35bd701388a30cd7358bca780b2ab2ce">here</a></li>
<li>→ org.sonatype.aether:aether-api:1.13.1 → the file that requires this is commented out; can be seen <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/commit/cb298ba550ca9f727ff66e4ffca0cb73e9ee03f1">here</a> and <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/commit/1503110b35bd701388a30cd7358bca780b2ab2ce">here</a></li>
</ol>
<h2>Important Notes</h2>
<p>It should be noted that at this point in time, 8th September 2019, the kotlin package only aims to package the jars generated by the ":dist" task of the kotlin build scripts. This task builds the kotlin
home. So thats all we have, we don't have the kotlin-gradle-plugins or kotlinx or anything that isn't part of the kotlin home.</p>
<p>It can be noted that the kotlin boostrap package has kotlin-gradle-plugin, kotlinx and kotlin-dsl jars. The eventual plan is to build kotlin-gradle-plugins and kotlinx from kotlin source itself
and to build kotlindsl from <a href="https://salsa.debian.org/java-team/gradle">gradle source</a> using kotlin as a dependency for gradle. After we do that we can get rid of the kotlin bootstrap package.</p>
<p>It should also be noted that this kotlin package as of 8th September 2019 may not be perfect and might contain a ton of bugs, this is because of 2 reasons; partly because I have ignored some code
that depended on jcabi-aether(mentioned above with link to commit) and mostly because the platform-api.jar and patform-impl.jar from intellij-community-idea are not the same as their upstream counterpart
but minimum files that are required to make kotlin compile without errors; I did this because they needed packaging new dependencies and at this time it didn't look like it was worth it.</p>
<h2>Work left to be done</h2>
<p>Now I believe most of the building blocks of packaging kotlin are done and whats left is to remove this pesky bootstrap. I believe this can be counted as the completion of my GSoC (officially ended in August 26).
The tasks left are as follows:</p>
<h3>Major Tasks</h3>
<ol>
<li>Make kotlin build just using openjdk-11-jdk; now it builds iwth openjdk-8-jdk and openjdk-11-jdk.</li>
<li>Build kotlin-gradle-plugins.</li>
<li>Build kotlinx.</li>
<li>Build kotlindsl from <a href="https://salsa.debian.org/java-team/gradle">gradle</a>.</li>
<li>Do 2, 3 and 4 and make Kotlin build without bootstrap.</li>
</ol>
<h3>Things that will help the kotlin effort</h3>
<ol>
<li>refine <a href="https://salsa.debian.org/java-team/intellij-community-idea/tree/wip/m36/master">intellij-community-idea</a> and do its copyrights file proper.</li>
<li>import <a href="https://github.com/JetBrains/kotlin/tree/1.3.30">kotlin 1.3.30</a> into a new debian-java-maintainers repository.</li>
<li>move kotlin changes(now maintained as git commits) to quilt patches. link to kotlin → <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30">here</a>.</li>
<li>do kotlin's copyrights file.</li>
<li>refine kotlin.</li>
</ol>
<h2><a href="#authorsnotes">Authors Notes.</a></h2>
<p>Hey guys its been a wonderful ride so far. I hope to keep doing this and maintain kotlin in debian. I am only a final year student and my career fare starts this october 17nth 2019 so I have to prepare for
coding interviews and start searching jobs. So until late November 2019 I'll only be taking on the smaller tasks and be doing them. Please note that I won't be doing it as fast as I used to up until now
since I am going to be a little busy during this period. I hope I can land a job that lets me keep doing this :)</p>
<p>I would love to take this section to thank _hc, ebourg, andrewsh and seamlik for helping and mentoring me trough all this.</p>
<p>So if any of you want to help please kindly take on any of these tasks.</p>
<p>NOTE — ping me if you want to build kotlin in your system and are stuck!</p>
<p>You can find me as <code>m36</code> or <code>m36[m]</code> on <code>#debian-mobile</code> and <code>#debian-java</code> on OFTC.</p>
<p>I’ll try to maintain this blog and post the major updates.</p>Packaging Dependencies Part 2; and plan on how to2019-08-04T17:22:26+05:302019-08-04T17:22:26+05:30Saif Abdul Cassimtag:java-team.pages.debian.net,2019-08-04:/gsoc-kotlin-blog/2019/08/04/packaging-dependencies-part-2-and-plan-on-how-to/<h3>Mapping and packaging dependencies part 1</h3>
<p>Hey all, I had my exams during weeks 8 and 9, so I couldn't update my blog nor get
much accomplished; but last week was completely free so I managed to finish packaging
all the dependencies from packaging dependencies part 1. Since some of …</p><h3>Mapping and packaging dependencies part 1</h3>
<p>Hey all, I had my exams during weeks 8 and 9, so I couldn't update my blog nor get
much accomplished; but last week was completely free so I managed to finish packaging
all the dependencies from packaging dependencies part 1. Since some of you may not
remember how I planned to tackle pacakging dependencies I'll mention it here one more
time.</p>
<p>I split this task into two sub tasks that can be done independently. The 2 subtasks are as follows:</p>
<ul>
<li>part 1: make the entire project build successfully without :buildSrc:prepare-deps:intellij-sdk:build<ul>
<li>part 1.1: package these dependencies</li>
</ul>
</li>
<li>part 2: package the dependencies in :buildSrc:prepare-deps:intellij-sdk:build; i.e. try to recreate whatever is in it.</li>
</ul>
<p>This is taken from my last blog which was specifically on packaging dependencies in part 1.
Now I am happy to tell all of you that packaging dependencies for part 1 is now complete
and all the needed packages are either in the new queue or already in sid archive as of
04 August 2019. I would like to thank <code>ebourg</code>, <code>seamlik</code> and <code>andrewsh</code> for helping me with this.</br></p>
<h3>How to build kotlin 1.3.30 after dependency packaging part 1 and design choices</h3>
<p>Before I go into how to build the project as it is now I'll briefly talk of some of the
choices I made while packaging dependencies in part 1 and general things you should know.</p>
<p>Two dependencies in part 1 were <code>Jcabi-aether</code> and <code>sonatype-aether</code>, both of these
are incompatible with maven-3 and these were only used in one single file in the entire dist
task graph. Considering the time it would take to migrate these dependencies to maven-3 I chose to
patch out the one file that needed both of these and that change is denoted by this <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/commit/cb298ba550ca9f727ff66e4ffca0cb73e9ee03f1">commit</a>.
Also it must be noted that so far we are only trying to build the <code>dist</code> task which only and only builds
the basic Kotlin compiler; it doesn't build the maven artifacts with poms nor does it build the kotlin-gradle-plugin.
Those things are built and installed in the local maven repository (.m2 file in surce project when you invoke debuild)
using the <code>install</code> task which I am planning to do once we finish successfully building the <code>dist</code> task. Invoking the
<code>install</code> task in our master as of Aug 04, 2019 will build and install all available maven artifacts into the local maven
repo but this again will not have kotlin-gradle-plugin or such since I have removed those subprojects as they aren't needed
by the <code>dist</code> task. Keeping them would mean that I have to convert and patch them to groovy if they are written in <code>.kts</code>
since they are evaluated during the initialization phase.</p>
<p>Now we are ready to build the project. I have written a simple makefile which copies all the needed bootstrap jars and prebuilts to
their proper places. All you need to build the project is:</p>
<div class="highlight"><pre><span></span>1.git clone https://salsa.debian.org/m36-guest/kotlin-1.3.30.git
2.cd kotlin-1.3.30
3.git checkout buildv1
4.debian/pseudoBootstrap bootstrap
5.debuild -b -rfakeroot -us -uc
Note that we need only do steps 1 though 4 the very first time you are building this project. everytime after that just invoke step 5
</pre></div>
<h3>Packaging dependencies part 2</h3>
<p>Now packaging dependencies part 2 involves <code>package the dependencies in :buildSrc:prepare-deps:intellij-sdk:build</code>. This is
the folder that is taking up the most space in <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30-temp-requirements.git">Kotlin-1.3.30-temp-requirements</a>.
The sole purpose of this task is reduce the jars in this folder and substitute them with JAR from the Debian environment. I have managed to
map out the needed jars from these for the <code>dist</code> task graph and they are</p>
<div class="highlight"><pre><span></span>saif@Hope:/srv/chroot/KotlinCh/home/kotlin/kotlin-1.3.30-debian-maintained/buildSrc/prepare-deps/intellij-sdk/repo/kotlin.build.custom.deps/183.5153.4$ ls -R
.:
intellij-core intellij-core.ivy.xml intellijUltimate intellijUltimate.ivy.xml jps-standalone jps-standalone.ivy.xml
./intellij-core:
asm-all-7.0.jar intellij-core.jar java-compatibility-1.0.1.jar
./intellijUltimate:
lib
./intellijUltimate/lib:
asm-all-7.0.jar guava-25.1-jre.jar jna.jar log4j.jar openapi.jar picocontainer-1.2.jar platform-impl.jar trove4j.jar
extensions.jar jdom.jar jna-platform.jar lz4-1.3.0.jar oro-2.0.8.jar platform-api.jar streamex-0.6.7.jar util.jar
./jps-standalone:
jps-model.jar
</pre></div>
<p>This folder is treated as an ant repository and the code to that is <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/blob/master/buildSrc/src/main/kotlin/intellijDependencies.kt#L30">here</a>. Build.gradle files use this via methods like <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/blob/master/compiler/ir/ir.tree/build.gradle#L8">this</a> which tells the project to take only
the needed jars from the collection. I am planning on replacing this with just plain old maven repository resolution using format like <code>compile(groupID:artifactId:version)</code> but we will
need the jars to be in our system anyways, atleast now we know that this particular file structure can be avoided.</p>
<p>Please note that these jars listed above by me are only needed for the <code>dist</code> task and the ones needed for other subprojects in the original <code>install</code> task can still be found <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30-temp-requirements/tree/master/build/repo/kotlin.build.custom.deps/183.5153.4">here</a>.</p>
<p>The <a href="#dependencies-to-package">following</a> are the dependencies need for part 2. <code>*</code> denotes what I am not sure of. Contact me before you attempt to pacakge any of the intellij dependencies as we only need parts from those and I have a script to tell what we need.</p>
<ol>
<li>→ java-compatibility-1.0.1 → https://github.com/JetBrains/intellij-deps-java-compatibility (DONE: <a href="https://salsa.debian.org/m36-guest/intellij-java-compatibility">here</a>)</li>
<li>→ jps-model → https://github.com/JetBrains/intellij-community/tree/master/jps (DONE: <a href="https://salsa.debian.org/java-team/intellij-community-idea/tree/wip/m36/master">here</a>)</li>
<li>→ intellij-core → https://github.com/JetBrains/intellij-community/tree/183.5153 (DONE: <a href="https://salsa.debian.org/java-team/intellij-community-idea/tree/wip/m36/master">here</a>)</li>
<li>→ streamex-0.6.7 → https://github.com/amaembo/streamex/tree/streamex-0.6.7 (DONE: <a href="https://salsa.debian.org/m36-guest/streamex">here</a>)</li>
<li>→ guava-25.1 → https://github.com/google/guava/tree/v25.1 (DONE: Used guava-19 from libguava-java)</li>
<li>→ lz4-java → https://github.com/lz4/lz4-java/blob/1.3.0/build.xml (DONE:<a href="https://salsa.debian.org/m36-guest/lz4-java">here</a>)</li>
<li>→ libjna-java & libjna-platform-java recompiled in jdk 8. → https://salsa.debian.org/java-team/libjna-java (DONE: <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/commit/4c3fcc86535bad68df6c48b01fd8ef52c5a8662e">commit</a>)</li>
<li>→ liboro-java recompiled in jdk8 → https://salsa.debian.org/java-team/liboro-java (DONE: <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/commit/f2d3f051f5bb57052fff54a03419daa929b1884d">commit</a>)</li>
<li>→ picocontainer-1.3 refining → https://salsa.debian.org/java-team/libpicocontainer-1-java (DONE: <a href="https://salsa.debian.org/java-team/libpicocontainer-1-java">here</a>)</li>
<li>→ platform-api → https://github.com/JetBrains/intellij-community/tree/183.5153/platform (DONE: <a href="https://salsa.debian.org/java-team/intellij-community-idea/tree/wip/m36/master">here</a>)</li>
<li>→ util → https://github.com/JetBrains/intellij-community/tree/183.5153/platform (DONE: <a href="https://salsa.debian.org/java-team/intellij-community-idea/tree/wip/m36/master">here</a>)</li>
<li>→ platform-impl → https://github.com/JetBrains/intellij-community/tree/183.5153/platform (DONE: <a href="https://salsa.debian.org/java-team/intellij-community-idea/tree/wip/m36/master">here</a>)</li>
<li>→ extensions → https://github.com/JetBrains/intellij-community/tree/183.5153/platform (DONE: <a href="https://salsa.debian.org/java-team/intellij-community-idea/tree/wip/m36/master">here</a>)</li>
</ol>
<p>So if any of you want to help please kindly take on any of these and package them.</p>
<p>NOTE — ping me if you want to build Kotlin in your system and are stuck!</p>
<p><a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30">Here</a> is a link to the work I have done so
far. You can find me as <code>m36</code> or <code>m36[m]</code> on <code>#debian-mobile</code> and <code>#debian-java</code> on OFTC.</p>
<p>I’ll try to maintain this blog and post the major updates weekly.</p>Packaging Dependencies Part 1; and plan on how to2019-07-07T17:21:23+05:302019-07-07T17:21:23+05:30Saif Abdul Cassimtag:java-team.pages.debian.net,2019-07-07:/gsoc-kotlin-blog/2019/07/07/packaging-dependencies-part-1-and-plan-on-how-to/<h3>Finished downgrading the project to be buildable by gradle 4.4.1</h3>
<p>I have finished downgrading the project to be buildable using gradle 4.4.1. The project
still needed a part of gradle 4.8 that I have successfully patched into sid gradle.
<a href="https://salsa.debian.org/java-team/gradle/commit/f6623d76ade6f515423669f55d0b3508f9e99c6b">here</a> is the link to the …</p><h3>Finished downgrading the project to be buildable by gradle 4.4.1</h3>
<p>I have finished downgrading the project to be buildable using gradle 4.4.1. The project
still needed a part of gradle 4.8 that I have successfully patched into sid gradle.
<a href="https://salsa.debian.org/java-team/gradle/commit/f6623d76ade6f515423669f55d0b3508f9e99c6b">here</a> is the link to the changes that I have made.</p>
<p>Now we are officially done with making the project build with our gradle; so we can now go on ahead
and finally start mapping out and packaging the dependencies.</p>
<h3>Packaging dependencies for Kotlin-1.3.30</h3>
<p>I split this task into two sub tasks that can be done independently. The 2 subtasks are as follows:</p>
<ul>
<li>part 1: make the entire project build successfully without :buildSrc:prepare-deps:intellij-sdk:build<ul>
<li>part1.1:package these dependencies</li>
</ul>
</li>
<li>part 2: package the dependencies in :buildSrc:prepare-deps:intellij-sdk:build; i.e. try to recreate whatever is in it.</li>
</ul>
<p>The task has been split into this exact model because <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30-temp-requirements">this</a> folder has a variety of JARs that the project uses and we’ll have to
minimize it and package the needed jars from it. Also the project uses other plugins and JARs other than this one main folder which we can simultaneously
map and package out.</p>
<p>I now have successfully mapped out the dependencies need by part 1: all that remains is to package em.
I have copied the dependencies from the original cache(one created when I build the project using <code>./gradlew -Pteamcity=true dist</code>) to /usr/share/maven-repo
so some of these dependencies still need to have their dependencies clearly defined as in which of their dependencies we can omit and which we need. I have marked such
dependencies with a <code>*</code>. So here are the dependencies:</p>
<ul>
<li>jengeleman:shadow:4.0.3 → https://github.com/johnrengelman/shadow (DONE: https://salsa.debian.org/m36-guest/jengelman-shadow)</li>
<li>trove4j 1.x → https://github.com/JetBrains/intellij-deps-trove4j (DONE: https://salsa.debian.org/java-team/libtrove-intellij-java)</li>
<li>proguard:6.0.3 in jdk8 (DONE: released as libproguard-java 6.0.3-2)</li>
<li>io.javaslang:2.0.6 → https://github.com/vavr-io/vavr/tree/javaslang-v2.0.6 (DONE:https://salsa.debian.org/m36-guest/javaslang)</li>
<li>jline 3.0.3 → https://github.com/jline/jline3/tree/jline-3.3.1 (DONE: https://salsa.debian.org/m36-guest/jline3)</li>
<li>protobuf-2.6.1 in jdk8 (DONE: https://salsa.debian.org/java-team/protobuf-2)</li>
<li>com.jcabi:jcabi-aether:1.0 → the file that requires this is commented out; can be seen <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/commit/cb298ba550ca9f727ff66e4ffca0cb73e9ee03f1">here</a> and <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/commit/1503110b35bd701388a30cd7358bca780b2ab2ce">here</a></li>
<li>org.sonatype.aether:aether-api:1.13.1 → the file that requires this is commented out; can be seen <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/commit/cb298ba550ca9f727ff66e4ffca0cb73e9ee03f1">here</a> and <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/commit/1503110b35bd701388a30cd7358bca780b2ab2ce">here</a></li>
</ul>
<p>NOTE: Please note that I might have missed out some; I’ll add them to the list once I get them mapped out proper!</p>
<p>So if any of you kind souls wanna help me out please kindly take on any of these and package them.</p>
<p>NOTE — ping me if you want to build kotlin in your system and are stuck!</p>
<p><a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30">Here</a> is a link to the work I have done so
far. You can find me as <code>m36</code> or <code>m36[m]</code> on <code>#debian-mobile</code> and <code>#debian-java</code> in OFTC.</p>
<p>I’ll try to maintain this blog and post the major updates weekly.</p>Finished converting all the buildfiles to groovy and downgraded to Gradle 4.4.1; week 3+ update2019-06-19T23:19:12+05:302019-06-19T23:19:12+05:30Saif Abdul Cassimtag:java-team.pages.debian.net,2019-06-19:/gsoc-kotlin-blog/2019/06/19/finished-converting-all-the-buildfiles-to-groovy-and-downgraded-to-gradle-441-week-3-update/<h3>Converting build files to groovy</h3>
<p>During the third week I mainly spent my time converting all the build files in the "dist" task graph to groovy from kotlin-dsl.</p>
<p>I finished converting all the build files from kotlin-dsl to groovy. I then proceeded to build the entire project with only the …</p><h3>Converting build files to groovy</h3>
<p>During the third week I mainly spent my time converting all the build files in the "dist" task graph to groovy from kotlin-dsl.</p>
<p>I finished converting all the build files from kotlin-dsl to groovy. I then proceeded to build the entire project with only the subprojects required for the dist task so that we can avoid
converting all the unneeded subproject build files to groovy. Ran tests on the binary obtained from the newly converted project and compared it to the test result on an original unconverted
project. Since the new project only contains the needed subprojects this new project is unable to run all the needed tests. So in order to overcome this we copy the binaries built by our new project
and run the tests using the original unaltered projects. The compiler test task we need is "compilerTest"; this is the only applicable test for out build binary from the "dist" task. I have run "distTest"
for the unaltered project and uploaded it <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/blob/master/UnalteredDistTest.txt">here</a>; "distTest" task encompasses compilerTest task within it. <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/blob/master/GroovyTranslationCompilerTest.txt">Here</a>
is the log of the "compilerTest" run on the generated binaries.</p>
<h3>Downgrading the Project to Gradle 4.4.1</h3>
<p>I also have downgraded the project to Gradle 4.4.1 decrementaly from 5.1 to 4.7 to 4.4.1. Since Gradle 4.4.1 isn't by default compatible with openjdk-11 (default Java version in Debian Sid) I used an external
JDK-9 to build the project with an external Gradle 4.4.1 and confirmed it has indeed been downgraded. Next step would be beginning to make this project buildable with Debian Sid Gradle.</p>
<h3>Closing notes and things to note</h3>
<p>I figure that to those unfamiliar with Debian packaging it would be useful to know that we are going to upload a non-free pre-built package containing only the necessary binaries for bootstrapping. Then
using this we can build this project in a more Debian way. Also I am posting the environment variables followed by build command to build this project as of this blog.</p>
<div class="highlight"><pre><span></span>export JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64"
export JDK_16="/usr/lib/jvm/java-8-openjdk-amd64"
export JDK_17="/usr/lib/jvm/java-8-openjdk-amd64"
export JDK_18="/usr/lib/jvm/java-8-openjdk-amd64"
export JDK_9="/home/Java/jdk-9.0.4_linux-x64_bin/jdk-9.0.4"
./gradlew -Pteamcity=true dist
</pre></div>
<p><a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30">Here</a> is a link to the work I have done so
far. You can find me as <code>m36</code> or <code>m36[m]</code> on <code>#debian-mobile</code> and <code>#debian-java</code> on OFTC.</p>
<p>I’ll try to maintain this blog and post the major updates weekly.</p>Converting build files to Groovy; week 2 update2019-06-06T17:21:23+05:302019-06-06T17:21:23+05:30Saif Abdul Cassimtag:java-team.pages.debian.net,2019-06-06:/gsoc-kotlin-blog/2019/06/06/converting-build-files-to-groovy-week-2-update/<p>I spent the first two weeks on updating build files of Kotlin to groovy so that we can reduce the dependency on kotlin-dsl while packaging Kotlin.</p>
<p><code>task("dist")</code> is the task that we call in order to build Kotlin 1.3.30 so it would be
enough to translate and …</p><p>I spent the first two weeks on updating build files of Kotlin to groovy so that we can reduce the dependency on kotlin-dsl while packaging Kotlin.</p>
<p><code>task("dist")</code> is the task that we call in order to build Kotlin 1.3.30 so it would be
enough to translate and deal with only those subprojects which are involed in
the "dist" task graph. I wrote this code <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30/commit/c6d4445a7ef816acde62394f9626266dd63d52c1">here</a> find out exactly which tasks from
which subprojects are being called; the build files of these subprojects are also
shown.</p>
<p>There was a total of about 83 subprojects involved in the build process, out of
these about 67 had build files written in kotlin-dsl. During this week I
translated about 40 of those build files, so only 27 more remain. I have also
translated the root projects build file. My work could be seen <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30">here</a>.</p>
<p>The build logic for the build files are written in Kotlin and placed in the
<code>$rootDir/buildSrc</code> directory. Kotlin supports writing custom extension functions
like fun <code>String.customfunction()</code> but groovy doesnt support this behaviour, so I
had to introduce intermediate functions for these functions in the build logic
so that these function can be invoked from within the groovy buildfiles.</p>
<p>I am also ignoring all publishing code logic also the test logics that look like
it may take a lot of time to translate. However I translate test logics as long
as they don’t take up much time. Once I finish with translating business mostly
by next week I will go on ahead with trying to build the project with only the
relevant subprojects as child projects.</p>
<p>Also one thing I noticed is that I forgot to mention the source of intellij-core
and the version we need to package Kotlin. <a href="https://github.com/JetBrains/intellij-community/tree/183.5153">Here</a> is the source of intellij-core
183.5153 which is the version we need. <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30">Here</a> is a link to the work I have done so
far. You can find me as <code>m36</code> or <code>m36[m]</code> on <code>#debian-mobile</code> and <code>#debian-java</code> on OFTC.</p>
<p>I’ll try to maintain this blog and post the major updates weekly.</p>Welcome to the Debian Kotlin GSoC blog2019-05-28T12:55:30+05:302019-05-28T12:55:30+05:30Saif Abdul Cassimtag:java-team.pages.debian.net,2019-05-28:/gsoc-kotlin-blog/2019/05/28/welcome-to-the-debian-kotlin-gsoc-blog/<p>I’ll be using this blog to track progress on packaging Kotlin and report on what I am doing during the GSoC period. So let me go on a head and start with the current progress in packaging Kotlin.</p>
<h2>Packaging Kotlin</h2>
<h3>Kotlin 1.1.1</h3>
<p>During GSoC 2018 I tried …</p><p>I’ll be using this blog to track progress on packaging Kotlin and report on what I am doing during the GSoC period. So let me go on a head and start with the current progress in packaging Kotlin.</p>
<h2>Packaging Kotlin</h2>
<h3>Kotlin 1.1.1</h3>
<p>During GSoC 2018 I tried to package kotlin-1.1.1 because it was built with ant and looked easy, but as it turned out it had its shortcomings. For starters, since it was very old, it missed a lot of the dependencies that were supposed to be up and hosted by Jetbrains. I had to patch up these dependencies with the closest matching versions and got things building. Recently it ran into a massive hiccup and the problem just couldn't be solved as I couldn't figure which dependency was causing it. I opened an <a href="https://youtrack.jetbrains.com/issue/KT-31041">issue</a> with Jetbrains and the conclusion I arrived at was that Kotlin 1.1.1 is too old and not supported anymore.</p>
<p>So following the advice of the Kotlin devs I have proceeded to take on the task of packaging Kotlin 1.3.30 (a couple months old version) into Debian.</p>
<h3>Kotlin 1.3.30</h3>
<p>This is my Google Summer of Code 2019 project.</p>
<p>This is the <a href="https://github.com/JetBrains/kotlin/tree/1.3.30">source</a> for Kotlin 1.3.30. Unlike Kotlin 1.1.1, Kotlin 1.3.30 had everything it needed to build successfully already hosted. So I went on ahead and split the work into 3 subtasks as follows:</p>
<ol>
<li>Convert all gradle build scripts from kts to groovy so that we dont have
to depend on kotlin-gradle-dsl.</li>
<li>Downgrade the build from gradle 5.1 to gradle 4.4.1 which is the version
available in Debian Sid.</li>
<li>Package the dependencies of Kotlin 1.3.30 so that it builds fine using
packages from the Debian Sid software environment alone.</li>
</ol>
<p>A major blocker for this would in 3, when we are packaging the dependencies of
Kotlin 1.3.30. We would have to package IDEA and this will be a huge task.
Also yesterday I noted that Jetbrains has stopped hosting Kotlin <a href="https://plugins.gradle.org/m2/org/jetbrains/kotlin/kotlin-gradle-plugin/">1.3.30-eap-28</a>
gradle plugin which is the bootstrap for Kotlin 1.3.30. I have gotten things to
build with 1.3.30 gradle plugin and have opened an <a href="https://youtrack.jetbrains.com/issue/KT-31642">issue</a> with Jetbrains asking
them to prolong hosting the dependecies for Kotlin 1.3.30.</p>
<p>You can see my work on kotlin 1.3.30 <a href="https://salsa.debian.org/m36-guest/kotlin-1.3.30">here</a> and I will be posting here in this
blog once every week with the major updates. You can find me as <code>m36[m]</code> or <code>m36</code>
in <code>#debian-java</code> or <code>#debian-mobile</code> channels hosted at OFTC.</p>