Dependency management is a complex thing. A typical gradle project has several dependencies, each of which in turn has multiple other dependencies and so on. This continues and what forms as a result is called a dependency graph. While this is great, unused dependencies is a big problem. For instance, say your project depends on dependency A which then depends on B, C and D. Your project also depends directly with another dependency E which in turn depends on F and G. The dependency graph would look something like this:
Your Project -> A -> (B, C, D)
Your Project -> E -> (F, G)
Now, your project probably only uses code in A which only then directly depends on some code in B. Also, maybe you also have some code that depends on E but that code in E doesn’t depend directly on neither F, nor G. This means that C, D, F and G are your unused dependencies.
If you knew this from the start, you could just exclude those libraries. However, tracking this manually is incredibly difficult as the more complex the libraries are, the more complex the web of inter-dependencies becomes. I’ve been looking for an automated solution to this for a while and recently, I found that Netflix has a plugin that could help. This is called the gradle-lint-plugin.
Its part of Netflix’s OSS and is available on github to download. Its artifacts are available in JCenter maven repository.
To test it out, I quickly got a gradle project up and running by following the spring boot tutorial. To make things a slightly more interesting, I added the org.apache:httpcomponents:httpclient library as a dependency in my build.gradle file:
|
1 |
compile group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.2' |
Also, for reference, here’s my gradle version:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
C:\Users\manthanhd\Documents\projects\gradle-lint-test>gradle -v ------------------------------------------------------------ Gradle 2.12 ------------------------------------------------------------ Build time: 2016-03-14 08:32:03 UTC Build number: none Revision: b29fbb64ad6b068cb3f05f7e40dc670472129bc0 Groovy: 2.4.4 Ant: Apache Ant(TM) version 1.9.3 compiled on December 23 2013 JVM: 1.7.0_10 (Oracle Corporation 23.6-b04) OS: Windows 8 6.2 amd64 |
I ran the gradle clean build command to assess the initial size of the build. As expected, the war file was massive at 14.3 megabytes. For a small project that has a simple healthcheck endpoint, this is huge. So, to streamline this, I followed the gradle-lint-plugin guide to use and apply the plugin. I added some rules of my own too. Here’s what my build.gradle file looks like:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
buildscript { repositories { mavenCentral() jcenter() } dependencies { classpath("org.springframework.boot:spring-boot-gradle-plugin:1.3.5.RELEASE") } } plugins { id 'nebula.lint' version '0.30.12' } allprojects { apply plugin: 'nebula.lint' gradleLint.rules = ['all-dependency', 'unused-dependency', 'unused-exclude-by-dep'] // add as many rules here as you'd like } apply plugin: 'java' apply plugin: 'eclipse' apply plugin: 'idea' apply plugin: 'spring-boot' jar { baseName = 'gs-spring-boot' version = '0.1.0' } repositories { mavenCentral() jcenter() } sourceCompatibility = 1.7 targetCompatibility = 1.7 dependencies { compile group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.2' // tag::jetty[] compile("org.springframework.boot:spring-boot-starter-web") { exclude module: "spring-boot-starter-tomcat" } compile("org.springframework.boot:spring-boot-starter-jetty") // end::jetty[] // tag::actuator[] compile("org.springframework.boot:spring-boot-starter-actuator") // end::actuator[] testCompile("junit:junit") } task wrapper(type: Wrapper) { gradleVersion = '2.1' } |
Once that was out of the way, I simply ran gradle clean build command.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
C:\Users\manthanhd\Documents\projects\gradle-lint-test>gradle clean build :clean :compileJava :processResources UP-TO-DATE :classes :findMainClass :jar :bootRepackage :assemble :compileTestJava UP-TO-DATE :processTestResources UP-TO-DATE :testClasses UP-TO-DATE :test UP-TO-DATE :check UP-TO-DATE :build :lintGradle This project contains lint violations. A complete listing of the violations follows. Because none were serious, the build's overall status was unaffected. warning unused-dependency one or more classes in org.springframework:spring-web:4.2.6.RELEASE are required by your code directly warning unused-dependency one or more classes in org.springframework.boot:spring-boot:1.3.5.RELEASE are required by your code directly warning unused-dependency one or more classes in org.springframework.boot:spring-boot-autoconfigure:1.3.5.RELEASE are required by your code directly warning unused-dependency one or more classes in org.springframework:spring-context:4.2.6.RELEASE are required by your code directly warning unused-dependency one or more classes in org.springframework:spring-web:4.2.6.RELEASE are required by your code directly warning unused-dependency one or more classes in org.springframework.boot:spring-boot:1.3.5.RELEASE are required by your code directly warning unused-dependency one or more classes in org.springframework.boot:spring-boot-autoconfigure:1.3.5.RELEASE are required by your code directly warning unused-dependency one or more classes in org.springframework:spring-context:4.2.6.RELEASE are required by your code directly warning unused-dependency this dependency is unused and can be removed build.gradle:39 compile group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.2' warning unused-dependency this dependency is unused and can be removed build.gradle:39 compile group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.2' ? build.gradle: 10 problems (0 errors, 10 warnings) To apply fixes automatically, run fixGradleLint, review, and commit the changes. BUILD SUCCESSFUL Total time: 15.742 secs |
Excellent! It found the unused dependencies. I was quite pleased with this result so I ran gradle fixGradleLint command. According to the wiki, this command should fix the issues that it found by making changes to the build.gradle file. Unfortunately, this didn’t work as it resulted in an error.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
C:\Users\manthanhd\Documents\projects\gradle-lint-test>gradle fixGradleLint :fixGradleLint FAILED FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':fixGradleLint'. > com.netflix.nebula.lint.jgit.api.errors.PatchApplyException: Cannot apply: HunkHeader[36,7->36,14] * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. BUILD FAILED Total time: 10.808 secs |
I’ve already raised a github issue to draw their attention to this problem. For information on progress, feel free to watch or add to the issue.
So, even if the automated fix didn’t work, I wasn’t done yet. I looked around in the build directory looking for something that I can use to apply for the fix myself. After all, since gradle lintGradle and gradle fixGradleLint are two separate commands, independent of each other, it must store some form of state somewhere. Fortunately I found it. It stores its state in a file called lint.patch directly under the build directory at the root of your project.
Hurrah! I quickly opened it. It was a normal git patch file. While looking at it, I was slightly confused with the additions that it was making. Here’s what mine looked like:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
diff --git a/build.gradle b/build.gradle --- a/build.gradle +++ b/build.gradle @@ -36,7 +36,14 @@ targetCompatibility = 1.7 dependencies { + compile 'org.springframework:spring-web:4.2.6.RELEASE' + compile 'org.springframework.boot:spring-boot:1.3.5.RELEASE' + compile 'org.springframework.boot:spring-boot-autoconfigure:1.3.5.RELEASE' + compile 'org.springframework:spring-context:4.2.6.RELEASE' + compile 'org.springframework:spring-web:4.2.6.RELEASE' + compile 'org.springframework.boot:spring-boot:1.3.5.RELEASE' + compile 'org.springframework.boot:spring-boot-autoconfigure:1.3.5.RELEASE' + compile 'org.springframework:spring-context:4.2.6.RELEASE' - compile group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.2' // tag::jetty[] compile("org.springframework.boot:spring-boot-starter-web") { exclude module: "spring-boot-starter-tomcat" |
Looking at that file, I could immediately tell that it was adding some compile time dependencies and was removing one httpcomponents dependency. While I agree with the removal of the httpcomponents dependency, I was slightly confused why it wasn’t replacing the existing spring-boot dependencies with the ones it was adding. It should’ve replaced them because having both provides no benefit. What its trying to add is more specific than the dependencies that are already there! To prove myself right, I removed all the existing compile dependencies (expect testCompile ones) and added the ones that it was going to add from the patch file. Here’s what the dependency section of my build.gradle looked like:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
dependencies { compile 'org.springframework:spring-web:4.2.6.RELEASE' compile 'org.springframework.boot:spring-boot:1.3.5.RELEASE' compile 'org.springframework.boot:spring-boot-autoconfigure:1.3.5.RELEASE' compile 'org.springframework:spring-context:4.2.6.RELEASE' compile 'org.springframework:spring-web:4.2.6.RELEASE' compile 'org.springframework.boot:spring-boot:1.3.5.RELEASE' compile 'org.springframework.boot:spring-boot-autoconfigure:1.3.5.RELEASE' compile 'org.springframework:spring-context:4.2.6.RELEASE' /*compile group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.2' // tag::jetty[] compile("org.springframework.boot:spring-boot-starter-web") { exclude module: "spring-boot-starter-tomcat" } compile("org.springframework.boot:spring-boot-starter-jetty") // end::jetty[] // tag::actuator[] compile("org.springframework.boot:spring-boot-starter-actuator")*/ // end::actuator[] testCompile("junit:junit") } |
I quickly ran gradle clean build and sure enough, this time the build size had shrunk to almost a third of its original size – 5.5 megabytes! Also, the lintGradle task at the end of the build didn’t report any errors.
I can certainly see the value of this plugin in a continuous integration system as unused libraries cause host of memory related issues like lack of permgen space in java web containers. Also, its better to let developers focus on core development and let the periphery issues such as these be handled by an automated plugin like this one.
I hope the developers on the github project fix the issues related to the fixGradleLint task soon.
Check out the source for this test on github.