Mapping and packaging dependencies part 1
Hey all, I had my exams during weeks 8 and 9, so I couldn't update my blog nor get much accomplished; but last week was completely free so I managed to finish packaging all the dependencies from packaging dependencies part 1. Since some of you may not remember how I planned to tackle pacakging dependencies I'll mention it here one more time.
I split this task into two sub tasks that can be done independently. The 2 subtasks are as follows:
- part 1: make the entire project build successfully without :buildSrc:prepare-deps:intellij-sdk:build
- part 1.1: package these dependencies
- part 2: package the dependencies in :buildSrc:prepare-deps:intellij-sdk:build; i.e. try to recreate whatever is in it.
This is taken from my last blog which was specifically on packaging dependencies in part 1.
Now I am happy to tell all of you that packaging dependencies for part 1 is now complete
and all the needed packages are either in the new queue or already in sid archive as of
04 August 2019. I would like to thank
andrewsh for helping me with this.
How to build kotlin 1.3.30 after dependency packaging part 1 and design choices
Before I go into how to build the project as it is now I'll briefly talk of some of the choices I made while packaging dependencies in part 1 and general things you should know.
Two dependencies in part 1 were
sonatype-aether, both of these
are incompatible with maven-3 and these were only used in one single file in the entire dist
task graph. Considering the time it would take to migrate these dependencies to maven-3 I chose to
patch out the one file that needed both of these and that change is denoted by this commit.
Also it must be noted that so far we are only trying to build the
dist task which only and only builds
the basic Kotlin compiler; it doesn't build the maven artifacts with poms nor does it build the kotlin-gradle-plugin.
Those things are built and installed in the local maven repository (.m2 file in surce project when you invoke debuild)
install task which I am planning to do once we finish successfully building the
dist task. Invoking the
install task in our master as of Aug 04, 2019 will build and install all available maven artifacts into the local maven
repo but this again will not have kotlin-gradle-plugin or such since I have removed those subprojects as they aren't needed
dist task. Keeping them would mean that I have to convert and patch them to groovy if they are written in
since they are evaluated during the initialization phase.
Now we are ready to build the project. I have written a simple makefile which copies all the needed bootstrap jars and prebuilts to their proper places. All you need to build the project is:
1.git clone https://salsa.debian.org/m36-guest/kotlin-1.3.30.git 2.cd kotlin-1.3.30 3.git checkout buildv1 4.debian/pseudoBootstrap bootstrap 5.debuild -b -rfakeroot -us -uc Note that we need only do steps 1 though 4 the very first time you are building this project. everytime after that just invoke step 5
Packaging dependencies part 2
Now packaging dependencies part 2 involves
package the dependencies in :buildSrc:prepare-deps:intellij-sdk:build. This is
the folder that is taking up the most space in Kotlin-1.3.30-temp-requirements.
The sole purpose of this task is reduce the jars in this folder and substitute them with JAR from the Debian environment. I have managed to
map out the needed jars from these for the
dist task graph and they are
saif@Hope:/srv/chroot/KotlinCh/home/kotlin/kotlin-1.3.30-debian-maintained/buildSrc/prepare-deps/intellij-sdk/repo/kotlin.build.custom.deps/183.5153.4$ ls -R .: intellij-core intellij-core.ivy.xml intellijUltimate intellijUltimate.ivy.xml jps-standalone jps-standalone.ivy.xml ./intellij-core: asm-all-7.0.jar intellij-core.jar java-compatibility-1.0.1.jar ./intellijUltimate: lib ./intellijUltimate/lib: asm-all-7.0.jar guava-25.1-jre.jar jna.jar log4j.jar openapi.jar picocontainer-1.2.jar platform-impl.jar trove4j.jar extensions.jar jdom.jar jna-platform.jar lz4-1.3.0.jar oro-2.0.8.jar platform-api.jar streamex-0.6.7.jar util.jar ./jps-standalone: jps-model.jar
This folder is treated as an ant repository and the code to that is here. Build.gradle files use this via methods like this which tells the project to take only
the needed jars from the collection. I am planning on replacing this with just plain old maven repository resolution using format like
compile(groupID:artifactId:version) but we will
need the jars to be in our system anyways, atleast now we know that this particular file structure can be avoided.
Please note that these jars listed above by me are only needed for the
dist task and the ones needed for other subprojects in the original
install task can still be found here.
The following are the dependencies need for part 2.
* denotes what I am not sure of. Contact me before you attempt to pacakge any of the intellij dependencies as we only need parts from those and I have a script to tell what we need.
- → java-compatibility-1.0.1 → https://github.com/JetBrains/intellij-deps-java-compatibility (DONE: here)
- → jps-model → https://github.com/JetBrains/intellij-community/tree/master/jps (DONE: here)
- → intellij-core → https://github.com/JetBrains/intellij-community/tree/183.5153 (DONE: here)
- → streamex-0.6.7 → https://github.com/amaembo/streamex/tree/streamex-0.6.7 (DONE: here)
- → guava-25.1 → https://github.com/google/guava/tree/v25.1 (DONE: Used guava-19 from libguava-java)
- → lz4-java → https://github.com/lz4/lz4-java/blob/1.3.0/build.xml (DONE:here)
- → libjna-java & libjna-platform-java recompiled in jdk 8. → https://salsa.debian.org/java-team/libjna-java (DONE: commit)
- → liboro-java recompiled in jdk8 → https://salsa.debian.org/java-team/liboro-java (DONE: commit)
- → picocontainer-1.3 refining → https://salsa.debian.org/java-team/libpicocontainer-1-java (DONE: here)
- → platform-api → https://github.com/JetBrains/intellij-community/tree/183.5153/platform (DONE: here)
- → util → https://github.com/JetBrains/intellij-community/tree/183.5153/platform (DONE: here)
- → platform-impl → https://github.com/JetBrains/intellij-community/tree/183.5153/platform (DONE: here)
- → extensions → https://github.com/JetBrains/intellij-community/tree/183.5153/platform (DONE: here)
So if any of you want to help please kindly take on any of these and package them.
NOTE — ping me if you want to build Kotlin in your system and are stuck!
Here is a link to the work I have done so
far. You can find me as
#debian-java on OFTC.
I’ll try to maintain this blog and post the major updates weekly.