Gradle vs Maven: Performance Comparison
- Business Cases for Faster Builds
- Gradle is up to 100 times faster than Maven
- Performance Advantages over Maven that make this possible
- Taking Performance Further
- Your Turn
Business Cases for Faster Builds
Gradle aims to help organizations ship better software, faster. Faster builds is one of the most direct ways of achieving this; an engineer has more opportunity to deliver software when not waiting for a software build. CI instances that spend less time rebuilding unnecessarily require fewer CPU resources and alert engineers earlier.
In 2017, we worked with a 600-engineer team. On average, the engineers each ran about 42 builds per week. The team estimated the cost per-minute per engineer to be US$1.42 and they work at least 44 weeks annually.
This means every minute they can make the developer build faster, they save $1,600,000 in waiting time for their engineers.
600 engineers * $1.42/minutes * 42 builds/week * 44 work weeks/year = $1,600,000/year
What if the build were 90% faster instead?
Performance improvements have additional strong economic impact. For example CI instances that spend less time rebuilding unnecessarily also require fewer CPU resources, alert engineers earlier, and build a smaller change set which makes debugging CI failures easier.
What about you?
We have found time and again that organizations that invest into build performance generally recoup their investment quickly, ship faster, and have happier and more productive teams.
A faster build have less obvious benefits as well. In a primer on Speed, Performance, and Human Perception, Ilya Grigorik posits that mental context switches likely occur after just a few seconds — engineers using extremely fast build tooling are more likely to remain in a state of “flow” to overall be much more productive.
Gradle is up to 100 times faster than Maven
We have run performance measurements on a variety of projects in common scenarios to approximate the amortization of the cost of migrating from Maven to Gradle.
Results below were generated on a Dell XPS 15, i7-4712HQ CPU @ 2.30GHz, 16 GB of RAM, SSD, Linux Mint 18.1, using mean of 10 runs.
Learn about the test projects and how you can reproduce the performance results yourself using the instructions below.
Scenario: Java Library
Gradle is 1.7x faster for running tests, and up to 72x faster building when the build cache is used!
Scenario: small multi-project build
Here are the results for common tasks for a 10 module multi-project build, most similar to a collection of microservices. Each subproject has 50 source files and 50 test source files.
Gradle is 2-3x faster for clean builds, about 10x faster for incremental changes, and up to 40x faster when Gradle task outputs are cached.
This GIF shows the
clean build scenario side-by-side so you can get a feel for the difference (without build cache).
Scenario: medium multi-project build
Here are the results for common tasks for a 100 module multi-project build living in a single repository. Each subproject has 100 source files and 100 test files.
Gradle is 4-5x faster for clean builds, about 40x faster for incremental changes, and up to 100x faster when Gradle task outputs are cached.
Scenario: large multi-project build
Here are the results for common tasks for a 500 module multi-project build living in a single repository. Each subproject has 100 source files and 100 test files.
Gradle is 3-10x faster for clean builds, about 100x faster for incremental changes, and up to 50x faster when Gradle task outputs are cached.
Scenario: large monolithic application
While it is rare to have all the code in one project, it is very common to have multi-module builds where the majority of the code lives in one or a few modules which are much larger than the rest. This scenario is an approximation of such projects — a single project with 50000 source files and 50000 test files.
Gradle is 2-3x faster for clean builds, about 10x faster for incremental changes, and up to 17x faster when Gradle task outputs are cached.
Summary of performance results
- Across all results, Gradle is at least 2 times faster in every single scenario.
- Gradle is between 10 and 100 times faster than Maven when building incremental changes; benefits increase with number of subprojects.
- Gradle builds are 17 to 100 times faster than Maven builds when task outputs can be resolved Gradle’s build cache.
Performance Advantages over Maven that make this possible
By choosing Gradle, you’ll benefit from huge performance improvements compared to Apache Maven: Gradle implements a wide range of strategies to make your builds faster:
- the Gradle Daemon is a long-lived process that keeps build information “hot” in memory
- incremental task inputs and outputs for various types of tasks makes it unnecessary to run
- incremental compilation analyzes the dependencies between your sources and classes and recompiles only those which are affected by changes
- the build cache fetches results from a cache when switching branches or running a clean build and the same output has already been produced somewhere else in the organization.
- Gradle’s smart classpath analyzer avoids unnecessary compilation when the binary interface of a library hasn’t changed
- better modelling of dependencies using the Java Library plugin reduces the size of the compile classpath which has a large, positive impact on performance
All those features combined together make a big difference, the performance results above show.
Taking Performance Further
You have likely noticed that the most extreme performance gains from the measurements above were those that leveraged Gradle’s build cache.
Gradle Enterprise comes with a remote build cache out-of-the-box with convenient management tools, not to mention tools to beautifully visualize and compare builds and see all builds across your organization. Get a faster, smoother migration so you can focus on shipping software.
Reproducing Results and Methodology
For our measurements, we have used the following test projects, corresponding to four scenarios commonly encountered by real developers.
- 10-module project, each module having 50 source and 50 test files
- 100-module project, each module having 100 source and 100 test files
- 500-module project, each module having 100 source and 100 test files
- Large monolithic app, with 50000 source and 50000 test files in a single project.
You can generate these projects yourself by following the instructions posted on GitHub.
Why is the time for clean builds so important?
Gradle, in contrast to Maven, can execute fast, reliable, incremental builds. Nonetheless the time it takes to do a clean build is a relevant metric. It reflects important real world scenarios:
- A CI build that always uses a fresh slave with no local state preserved.
- A culture where CI builds are always done as clean builds.
- A developer builds from a fresh checkout
- A developer builds where a lot of code changed. This is not strictly a clean build. But as almost everything needs to be rebuilt, the clean build time reflects this scenario best.
For Gradle, we’re measuring
clean assemble, which cleans all outputs, then compiles classes, process resources and builds the jars. It does not execute tests. For Maven, we’re measuring
clean package, which does the same, but we need to explicitly exclude test execution, using
About Incremental Builds
A typical developer workflow is to edit one or a few source files and rebuild. Maven’s incremental compile is broken (see this bug, this other one or this bug) and you are forced to always do a clean build. In contrast, Gradle builds reliably and much faster (compared to a clean build) when only a few source files have changed.
In the ABI-compatible change scenario, we apply a change to the implementation of a method, which is a change that does not change the Application Binary Interface (ABI) of a component.
In the ABI-incompatible change scenario, we apply a change to the public signature of a public method. You can learn more about this in our blog post about compile avoidance and incremental compilation.
About Gradle’s Build Cache
The Gradle build cache reuses the outputs of Gradle tasks locally and shares task outputs between machines. In many cases, this will accelerate the average build time. Learn more in the Introducing Gradle Build Cache blog post.
The build cache is also extremely helpful when switching between branches as the build outputs from the previous builds are preserved and don’t have to be recreated. The performance savings are comparable to the cached builds above, where Gradle is 17 to 100x faster than Maven for the test projects.
Extreme performance gains are now achievable with the Gradle build tool.
To make it efficient for you to achieve and maintain those numbers, we have created the Gradle Enterprise platform that comes with:
- A highly scalable remote build cache with convenient management tools
- Build scans, that allow your organization to gather deep insights into builds run locally by developers or on CI servers. Leverage this data to reduce outages and continuously improve the speed and reliability of your builds. Furthermore, they transform debugging complex build issues from very difficult to very efficient.
Performance is only one important quality of a build system among other ones, like dependency management and working effectively with multiple repositories. Check out the feature comparison matrix between Gradle and Maven.
Traction and adoption are also important criteria when deciding on a build system. After all, migrating a build is a significant decision and a long-term decision. Gradle was recently ranked in the Top 20 of all open source projects.