Chapter 44. Dependency Management

The version of the jar must be easy to recognize. Sometimes the version is in the Manifest file of the jar, often not. And even if, it is rather painful to always look into the Manifest file to learn about the version. Therefore we think that you should only use jars which have their version as part of their file name. If you are using transitive dependency management you are forced to do this in any case.

Why do we think this is important? Without a dependency management as described above, your are likely to burn your fingers sooner or later. If it is unclear which version of a jar your are using, this can introduce subtle bugs which are very hard to find. For example there might be a project which uses Hibernate 3.0.4. There are some problems with Hibernate so a developer installs version 3.0.5 of Hibernate on her machine. This did not solve the problem but she forgot to roll back Hibernate to 3.0.4. Weeks later there is an exception on the integration machine which can't be reproduced on the developer machine. Without a version in the jar name this problem might take a long time to debug. Version in the jar names increases the expressiveness of your project and makes it easier to maintain.

When we talk about transitive dependency management, we mean any technique that enables to distinguish between what are the first level dependencies and what are the transitive ones. We will about different techniques for this later on.

Why is transitive dependency management so important? If you don't know which dependencies are first level dependencies and which ones are transitive you will soon lose control over your build. Even a non enterprise project Gradle has already 100+ jars. An enterprise project using Spring, Hibernate, etc. easily ends up with many more jars. There is no way to memorize where all these jars come from. If you want to get rid of a first level dependency you can't be sure which other jars you should remove. Because a dependency of a first level dependency might also be a first level dependency itself. Or it might be a transitive dependency of another of your first level dependencies. Many first level dependencies are runtime dependencies and the transitive dependencies are of course all runtime dependencies. So the compiler won't help you much here. The end of the story is, as we have seen very often, no one dares to remove any jar any longer. The project classpath is a complete mess and if a classpath problem arises, hell on earth invites you for a ride. In one of our former projects, we found some ldap related jar in the classpath, whose sheer presence, as we found out after much research, accelerated LDAP access. So removing this jar would not have led to any errors at compile or runtime.

Gradle offers you different ways to express what are first level and what are transitive dependencies. Gradle allows you for example to store your jars in CVS or SVN without XML descriptor files and still use transitive dependency management. Also, not all techniques for transitive dependency management deal with the problem described above equally well.

Conflicting versions of the same jar should be detected and either resolved or cause an exception. If you don't use transitive dependency management, version conflicts are undetected and the mostly accidental fragile order of the classpath will determine, what version of a dependency will win. For example adding a dependency with a particular version to a subproject might change that order and then will led to all kind of surprising side effects. You might also want to learn where conflicting versions are used as you might want to consolidate on a particular version of an dependency across your organization. With a good conflict reporting that information can be used to communicate with the teams to solve this.

It is common that different dependencies rely on different versions of another dependency which leads to a version conflictm as The JVM unfortunately does not offer yet any easy way, to have different versions of the same jar in the classpath (see Section 44.2.5, “Dependency management and Java”).

Gradle offers following conflict resolution strategies:

Gradle provides means to resolve version conflicts:

To deal with problems due to version conflicts, reports with dependency graphs are also very helpful. Such reports are another feature of dependency management.

Sometimes, you always want to use the latest version of a particular dependency, or the latest in a range of versions. You can easily do this using a dynamic version. A dynamic version can be either a version range (eg. 2.+) or it can be a placeholder for the latest version available (eg. latest.integration).

Alternatively, sometimes the module you request can change over time, even for the same version. An example of this type of changing module is a maven SNAPSHOT module, which always points at the latest artifacts published.

The main difference between a dynamic version and a changing module is that when you resolve a dynamic version, you'll get the real, static version as the module name. When you resolve a changing module, the artifacts are named using the version you requested, but the underlying artifacts may change over time.

By default, Gradle caches dynamic versions and changing modules for 24 hours. You can override the default cache modes using command line options. You can change the cache expiry times in your build using the resolution strategy (see Section 44.8.3, “Fine-tuned control over dependency caching”).

Traditionally, Java has offered no support at all for dealing with libraries and versions. There are no standard ways to say that foo-1.0.jar depends on a bar-2.0.jar. This has led to proprietary solutions. The most popular ones are Maven and Ivy. Maven is a complete build system whereas Ivy focuses solely on dependency management.

Both approaches rely on descriptor XML files, which contains information about the dependencies of a particular jar. Both also use repositories where the actual jars are placed together with their descriptor files. And both offer resolution for conflicting jar versions in one form or the other. Yet we think the differences of both approaches are significant in terms of flexibility and maintainability. Originally Gradle did use Ivy under the hood for its dependency management. This has been replaced with a native Gradle dependency resolution engine. This resolution engine supports both pom and ivy descriptor files.

External module dependencies are the most common dependencies. They refer to a module in an external repository.

dependencies {
    runtime group: 'org.springframework', name: 'spring-core', version: '2.5'
    runtime 'org.springframework:spring-core:2.5', 'org.springframework:spring-aop:2.5'
    runtime(
        [group: 'org.springframework', name: 'spring-core', version: '2.5'],
        [group: 'org.springframework', name: 'spring-aop', version: '2.5']
    )
    runtime('org.hibernate:hibernate:3.0.5') {
        transitive = true
    }
    runtime group: 'org.hibernate', name: 'hibernate', version: '3.0.5', transitive: true
    runtime(group: 'org.hibernate', name: 'hibernate', version: '3.0.5') {
        transitive = true
    }
}

Please see the DependencyHandler for more examples and complete reference. Please read on to get thorough understanding of the Gradle's dependency management.

Gradle provides different notations for module dependencies. There is a string notation and a map notation. A module dependency has an API which allows for further configuration. Have a look at ExternalModuleDependency to learn all about the API. This API provides properties and configuration methods. Via the string notation you can define a subset the properties. With the map notation you can define all properties. To have access to the complete API, either with the map or with the string notation, you can assign a single dependency to a configuration together with a closure.

If you declare a module dependency, Gradle looks for a corresponding module descriptor file (pom.xml or ivy.xml) in the repositories. If such a module descriptor file exists, it is parsed and the artifacts of this module (e.g. hibernate-3.0.5.jar) as well as its dependencies (e.g. cglib) are downloaded. If no such module descriptor file exists, Gradle looks for a file called hibernate-3.0.5.jar to retrieve. In Maven a module can only have one and only one artifact. In Gradle and Ivy a module can have multiple artifacts. Each artifact can have a different set of dependencies.

dependencies {
    runtime "org.groovy:groovy:1.8.8@jar"
    runtime group: 'org.groovy', name: 'groovy', version: '1.8.8', ext: 'jar'
}

compile "org.gradle.test.classifiers:service:1.0:jdk15@jar"
    otherConf group: 'org.gradle.test.classifiers', name: 'service', version: '1.0', classifier: 'jdk14'

To use the external dependencies of a configuration:

task listJars << {
    configurations.compile.each { File file -> println file.name }
}

> gradle -q listJars
hibernate-core-3.6.7.Final.jar
antlr-2.7.6.jar
commons-collections-3.1.jar
dom4j-1.6.1.jar
slf4j-api-1.6.1.jar
hibernate-commons-annotations-3.2.0.Final.jar
hibernate-jpa-2.0-api-1.0.1.Final.jar
jta-1.1.jar

Client module dependencies enable you to declare transitive dependencies directly in your build script. They are a replacement for a module descriptor XML file in an external repository.

dependencies {
    runtime module("org.codehaus.groovy:groovy-all:1.8.8") {
        dependency("commons-cli:commons-cli:1.0") {
            transitive = false
        }
        module(group: 'org.apache.ant', name: 'ant', version: '1.8.4') {
            dependencies "org.apache.ant:ant-launcher:1.8.4@jar", "org.apache.ant:ant-junit:1.8.4"
        }
    }
}

This declares a dependency of your project on Groovy. Groovy itself has dependencies. But Gradle does not look for an XML descriptor to figure them out but gets the information from the build file. The dependencies of a client module can be normal module dependencies or artifact dependencies or another client module. Have also a look at the API documentation: ClientModule

In the current release client modules have one limitation. Let's say your project is a library and you want this library to be uploaded to your company's Maven or Ivy repository. Gradle uploads the jars of your project to the company repository together with the XML descriptor file of the dependencies. If you use client modules the dependency declaration in the XML descriptor file is not correct. We will improve this in a future release of Gradle.

Gradle distinguishes between external dependencies and dependencies on projects which are part of the same multi-project build. For the latter you can declare Project Dependencies.

dependencies {
    compile project(':shared')
}

For more information see the API documentation for ProjectDependency

Multi-project builds are discussed in Chapter 50, Multi-project Builds.

File dependencies allow you to directly add a set of files to a configuration, without first adding them to a repository. This can be useful if you cannot, or do not want to, place certain files in a repository. Or if you do not want to use any repositories at all for storing your dependencies.

To add some files as a dependency for a configuration, you simply pass a file collection as a dependency:

dependencies {
    runtime files('libs/a.jar', 'libs/b.jar')
    runtime fileTree(dir: 'libs', include: '*.jar')
}

File dependencies are not included in the published dependency descriptor for your project. However, file dependencies are included in transitive project dependencies within the same build. This means they cannot be used outside the current build, but they can be used with the same build.

You can declare which tasks produce the files for a file dependency. You might do this when, for example, the files are generated by the build.

dependencies {
    compile files("$buildDir/classes") {
        builtBy 'compile'
    }
}

task compile << {
    println 'compiling classes'
}

task list(dependsOn: configurations.compile) << {
    println "classpath = ${configurations.compile.collect {File file -> file.name}}"
}

> gradle -q list
compiling classes
classpath = [classes]

You can declare a dependency on the API of the current version of Gradle by using the DependencyHandler.gradleApi() method. This is useful when you are developing custom Gradle tasks or plugins.

dependencies {
    compile gradleApi()
}

You can declare a dependency on the Groovy that is distributed with Gradle by using the DependencyHandler.localGroovy() method. This is useful when you are developing custom Gradle tasks or plugins in Groovy.

dependencies {
    groovy localGroovy()
}

You can exclude a transitive dependency either by configuration or by dependency:

configurations {
    compile.exclude module: 'commons'
    all*.exclude group: 'org.gradle.test.excludes', module: 'reports'
}

dependencies {
    compile("org.gradle.test.excludes:api:1.0") {
        exclude module: 'shared'
    }
}

If you define an exclude for a particular configuration, the excluded transitive dependency will be filtered for all dependencies when resolving this configuration or any inheriting configuration. If you want to exclude a transitive dependency from all your configurations you can use the Groovy spread-dot operator to express this in a concise way, as shown in the example. When defining an exclude, you can specify either only the organization or only the module name or both. Have also a look at the API documentation of Dependency and Configuration.

Not every transitive dependency can be excluded - some transitive dependencies might be essential for correct runtime behavior of the application. Generally, one can exclude transitive dependencies that are either not required by runtime or that are guaranteed to be available on the target environment/platform.

Should you exclude per-dependency or per-configuration? It turns out that in majority of cases you want to use the per-configuration exclusion. Here are the some exemplary reasons why one might want to exclude a transitive dependency. Bear in mind that for some of those use cases there are better solutions than exclusions!

Basically, in most of the cases excluding the transitive dependency should be done per configuration. This way the dependency declaration is more explicit. It is also more accurate because a per-dependency exclude rule does not guarantee the given transitive dependency does not show up in the configuration. For example, some other dependency, which does not have any exclude rules, might pull in that unwanted transitive dependency.

Other examples of the dependency exclusions can be found in the reference for ModuleDependency or DependencyHandler.

All attributes for a dependency are optional, except the name. It depends on the repository type, which information is need for actually finding the dependencies in the repository. See Section 44.6, “Repositories”. If you work for example with Maven repositories, you need to define the group, name and version. If you work with filesystem repositories you might only need the name or the name and the version.

dependencies {
    runtime ":junit:4.10", ":testng"
    runtime name: 'testng' 
}

You can also assign collections or arrays of dependency notations to a configuration:

List groovy = ["org.codehaus.groovy:groovy-all:1.8.8@jar",
               "commons-cli:commons-cli:1.0@jar",
               "org.apache.ant:ant:1.8.4@jar"]
List hibernate = ['org.hibernate:hibernate:3.0.5@jar', 'somegroup:someorg:1.0@jar']
dependencies {
    runtime groovy, hibernate
}

In Gradle a dependency can have different configurations (as your project can have different configurations). If you don't specify anything explicitly, Gradle uses the default configuration of the dependency. For dependencies from a Maven repository, the default configuration is the only available one anyway. If you work with Ivy repositories and want to declare a non-default configuration for your dependency you have to use the map notation and declare:

dependencies {
    runtime group: 'org.somegroup', name: 'somedependency', version: '1.0', configuration: 'someConfiguration'
}

To do the same for project dependencies you need to declare:

dependencies {
    compile project(path: ':api', configuration: 'spi')
}

You can generate dependency reports from the command line (see Section 11.6.3, “Listing project dependencies”). With the help of the Project report plugin (see Chapter 39, The Project Report Plugin) such a report can be created by your build.

Since Gradle 1.2 there is also a new programmatic API to access the resolved dependency information. The dependency reports (see the previous paragraph) are using this API behind the hood. The API lets you to walk the resolved dependency graph and provides information about the dependencies. With the coming releases the API will grow to provide more information about the resolution result. For more information about the API please refer to the javadocs on ResolvableDependencies.getResolutionResult(). Potential usages of the ResolutionResult API:

To add the central Maven 2 repository (http://repo1.maven.org/maven2) simply add this to your build script:

repositories {
    mavenCentral()
}

Now Gradle will look for your dependencies in this repository.

To use the local Maven cache as a repository you can do:

repositories {
    mavenLocal()
}

Gradle uses the same logic as maven to identify the location of your local maven cache. If a local repository location is defined in a settings.xml, this location will be used. The settings.xml in USER_HOME/.m2 takes precedence over the settings.xml in M2_HOME/conf. If no settings.xml is available, Gradle uses the default location USER_HOME/.m2/repository.

For adding a custom Maven repository you can do:

repositories {
    maven {
        url "http://repo.mycompany.com/maven2"
    }
}

Sometimes a repository will have the POMs published to one location, and the JARs and other artifacts published at another location. To define such a repository, you can do:

repositories {
    maven {
        // Look for POMs and artifacts, such as JARs, here
        url "http://repo2.mycompany.com/maven2"
        // Look for artifacts here if not found at the above location
        artifactUrls "http://repo.mycompany.com/jars"
        artifactUrls "http://repo.mycompany.com/jars2"
    }
}

Gradle will look at the first URL for the POM and the JAR. If the JAR can't be found there, the artifact URLs are used to look for JARs.

repositories {
    maven {
        credentials {
            username 'user'
            password 'password'
        }
        url "http://repo.mycompany.com/maven2"
    }
}

If you want to use a (flat) filesystem directory as a repository, simply type:

repositories {
    flatDir {
        dirs 'lib'
    }
    flatDir {
        dirs 'lib1', 'lib2'
    }
}

This adds repositories which look into one or more directories for finding dependencies. If you only work with flat directory resolvers you don't need to set all attributes of a dependency. See Section 44.4.8, “Optional attributes”

To use an Ivy repository with a standard layout:

repositories {
    ivy {
        url "http://repo.mycompany.com/repo"
        layout "maven"
    }
}

See IvyArtifactRepository for details.

repositories {
    ivy {
        url "http://repo.mycompany.com/repo"
        layout 'pattern', {
            artifact "[module]/[revision]/[artifact].[ext]"
            ivy "[module]/[revision]/ivy.xml"
        }
    }
}

repositories {
    ivy {
        artifactPattern "http://repo.mycompany.com/3rd-party-artifacts/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"
        artifactPattern "http://repo.mycompany.com/company-artifacts/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"
        ivyPattern "http://repo.mycompany.com/ivy-files/[organisation]/[module]/[revision]/ivy.xml"
    }
}

repositories {
    ivy {
        credentials {
            username 'user'
            password 'password'
        }
        artifactPattern "http://repo.mycompany.com/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"
    }
}

To access a repository:

println repositories.localRepository.name
    println repositories['localRepository'].name

To configure a repository:

repositories {
    flatDir {
        name 'localRepository'
    }
}
repositories {
    localRepository {
        dirs 'lib'
    }
}
repositories.localRepository {
    dirs 'lib'
}

Gradle, thanks to Ivy under its hood, is extremely flexible regarding repositories:

Let's say, you declare a dependency on the junit:junit:3.8.2 library. Now how does Gradle find it in the repositories? Somehow the dependency information has to be mapped to a path. In contrast to Maven, where this path is fixed, with Gradle you can define a pattern that defines what the path will look like. Here are some examples: ^[14]

// Maven2 layout (if a repository is marked as Maven2 compatible, the organization (group) is split into subfolders according to the dots.)
someroot/[organisation]/[module]/[revision]/[module]-[revision].[ext]

// Typical layout for an ivy repository (the organization is not split into subfolder)
someroot/[organisation]/[module]/[revision]/[type]s/[artifact].[ext]

// Simple layout (the organization is not used, no nested folders.)
someroot/[artifact]-[revision].[ext]

To add any kind of repository (you can pretty easy write your own ones) you can do:

repositories {
    ivy {
        ivyPattern "$projectDir/repo/[organisation]/[module]-ivy-[revision].xml"
        artifactPattern "$projectDir/repo/[organisation]/[module]-[revision](-[classifier]).[ext]"
    }
}

An overview of which Resolvers are offered by Ivy and thus also by Gradle can be found here. With Gradle you just don't configure them via XML but directly via their API.

You can fine-tune certain aspects of caching using the ResolutionStrategy for a configuration.

By default, Gradle caches dynamic versions for 24 hours. To change how long Gradle will cache the resolved version for a dynamic version, use:

configurations.all {
    resolutionStrategy.cacheDynamicVersionsFor 10, 'minutes'
}

By default, Gradle caches changing modules for 24 hours. To change how long Gradle will cache the meta-data and artifacts for a changing module, use:

configurations.all {
    resolutionStrategy.cacheChangingModulesFor 30, 'days'
}

For more details, take a look at the API documentation for ResolutionStrategy.

There is another way to deal with transitive dependencies without XML descriptor files. You can do this with Gradle, but we don't recommend it. We mention it for the sake of completeness and comparison with other build tools.

The trick is to use only artifact dependencies and group them in lists. That way you have somehow expressed, what are your first level dependencies and what are transitive dependencies (see Section 44.4.8, “Optional attributes”). But the draw-back is, that for the Gradle dependency management all dependencies are considered first level dependencies. The dependency reports don't show your real dependency graph and the compile task uses all dependencies, not just the first level dependencies. All in all, your build is less maintainable and reliable than it could be when using client modules. And you don't gain anything.