Chapter 28. Dependency Management

Why do we think this is necessary? Without a dependency management as described above, your are likely to burn your fingers sooner or later. If it is unclear which version of a jar your are using, this can introduce subtle bugs which are very hard to find. For example there might be a project which uses Hibernate 3.0.4. There are some problems with Hibernate so a developer installs version 3.0.5 of Hibernate on her machine. This did not solve the problem but she forgot to roll back Hibernate to 3.0.4. Weeks later there is an exception on the integration machine which can't be reproduced on the developer machine. Without a version in the jar name this problem might take a long time to debug. Version in the jar names increases the expressiveness of your project and makes it easier to maintain.

Why is transitive dependency management so important? If you don't know which dependencies are first level dependencies and which ones are transitive you will soon lose control over your build. Even Gradle has already 20+ jars. An enterprise project using Spring, Hibernate, etc. easily ends up with 100+ jars. There is no way to memorize where all these jars come from. If you want to get rid of a first level dependency you can't be sure which other jars you should remove. Because a dependency of a first level dependency might also be a first level dependency itself. Or it might be a transitive dependency of another of your first level dependencies. Many first level dependencies are runtime dependencies and the transitive dependencies are of course all runtime dependencies. So the compiler won't help you much here. The end of the story is, as we have seen very often, no one dares to remove any jar any longer. The project classpath is a complete mess and if a classpath problem arises, hell on earth invites you for a ride. In one of my former projects, I found some ldap related jar in the classpath, whose sheer presence, as I found out after much research, accelerated LDAP access. So removing this jar would not have led to any errors at compile or runtime.

Gradle offers you different ways to express what are first level and what are transitive dependencies. Gradle allows you for example to store your jars in CVS or SVN without XML descriptor files and still use transitive dependency management. Gradle also validates your dependency hierarchy against the reality of your code by using only the first level dependencies for compiling.

In your dependency description you tell Gradle which version of a dependency is needed by another dependency. This frequently leads to conflicts. Different dependencies rely on different versions of another dependency. The JVM unfortunately does not offer yet any easy way, to have different versions of the same jar in the classpath (see Section 28.2.4, “Dependency management and Java”). What Gradle offers you is a resolution strategy, by default the newest version is used. To deal with problems due to version conflicts, reports with dependency graphs are also very helpful. Such reports are another feature of dependency management.

Traditionally, Java has offered no support at all for dealing with libraries and versions. There are no standard ways to say that foo-1.0.jar depends on a bar-2.0.jar. This has led to proprietary solutions. The most popular ones are Maven and Ivy. Maven is a complete build system whereas Ivy focuses solely on dependency management.

Both approaches rely on descriptor XML files, which contains information about the dependencies of a particular jar. Both also use repositories where the actual jars are placed together with their descriptor files. And both offer resolution for conflicting jar versions in one form or the other. Yet we think the differences of both approaches are significant in terms of flexibility and maintainability. Beside this, Ivy fully supports the Maven dependency handling. So with Ivy you have access to both worlds. We like Ivy very much. Gradle uses it under the hood for its dependency management. Ivy is most often used via Ant and XML descriptors. But it also has an API. We integrate deeply with Ivy via its API. This enables us to build new concepts on top of Ivy which Ivy does not offer itself.

Right now there is a lot of movement in the field of dependency handling. There is OSGi and there is JSR-294. ^[19] OSGi is available already, JSR-294 is supposed to be shipped with Java 7. These technologies deal, amongst many other things, also with a painful problem which is neither solved by Maven nor by Ivy. This is enabling different versions of the same jar to be used at runtime.

Dependencies are grouped in configurations. Configurations have a name, a number of other properties, and they can extend each other. For examples see: Section 8.1, “Artifact configurations”. If you use the Java plugin, Gradle adds a number of pre-defined configurations to your build. The plugin also assigns configurations to tasks. See Section 18.4, “Dependency management” for details. Of course you can add your add custom configurations on top of that. There are many use cases for custom configurations. This is very handy for example for adding dependencies not needed for building or testing your software (e.g. additional JDBC drivers to be shipped with your distribution). The configurations are managed by a configurations object. The closure you pass to the configurations object is applied against its API. To learn more about this API have a look at ConfigurationHandler .

Module dependencies are the most common dependencies. They correspond to a dependency in an external repository.

dependencies {
    runtime group: 'org.springframework', name: 'spring-core', version: '2.5'
    runtime 'org.springframework:spring-core:2.5', 'org.springframework:spring-aop:2.5'
    runtime(
        [group: 'org.springframework', name: 'spring-core', version: '2.5'],
        [group: 'org.springframework', name: 'spring-aop', version: '2.5']
    )
    runtime('org.hibernate:hibernate:3.0.5') {
        transitive = true
    }
    runtime group: 'org.hibernate', name: 'hibernate', version: '3.0.5', transitive: true
    runtime(group: 'org.hibernate', name: 'hibernate', version: '3.0.5') {
        transitive = true
    }
}

Gradle provides different notations for module dependencies. There is a string notation and a map notation. A module dependency has an API which allows for further configuration. Have a look at ExternalModuleDependency to learn all about the API. This API provides properties and configuration methods. Via the string notation you can define a subset the properties. With the map notation you can define all properties. To have access to the complete API, either with the map or with the string notation, you can assign a single dependency to a configuration together with a closure.

If you declare a module dependency, Gradle looks for a corresponding module descriptor file (pom.xml or ivy.xml) in the repositories. If such a module descriptor file exists, it is parsed and the artifacts of this module (e.g. hibernate-3.0.5.jar) as well as its dependencies (e.g. cglib) are downloaded. If no such module descriptor file exists, Gradle looks for a file called hibernate-3.0.5.jar to retrieve. In Maven a module can only have one and only one artifact. In Gradle and Ivy a module can have multiple artifacts. Each artifact can have a different set of dependencies.

dependencies {
    runtime "org.groovy:groovy:1.5.6@jar"
    runtime group: 'org.groovy', name: 'groovy', version: '1.5.6', ext: 'jar'
}

compile "org.gradle.test.classifiers:service:1.0:jdk15@jar"
    otherConf group: 'org.gradle.test.classifiers', name: 'service', version: '1.0', classifier: 'jdk14'

Client module dependencies enable you to declare transitive dependencies directly in your build script. They are a replacement for a module descriptor XML file in an external repository.

dependencies {
    runtime module("org.codehaus.groovy:groovy-all:1.5.6") {
        dependency("commons-cli:commons-cli:1.0") {
            transitive = false
        }
        module(group: 'org.apache.ant', name: 'ant', version: '1.7.0') {
            dependencies "org.apache.ant:ant-launcher:1.7.0@jar", "org.apache.ant:ant-junit:1.7.0"
        }
    }
}

This declares a dependency of your project on Groovy. Groovy itself has dependencies. But Gradle does not look for an XML descriptor to figure them out but gets the information from the build file. The dependencies of a client module can be normal module dependencies or artifact dependencies or another client module. Have also a look at the javadoc: ClientModule

In the current release client modules have one limitation. Let's say your project is a library and you want this library to be uploaded to your company's Maven or Ivy repository. Gradle uploads the jars of your project to the company repository together with the XML descriptor file of the dependencies. If you use client modules the dependency declaration in the XML descriptor file is not correct. We will improve this in a future release of Gradle.

Gradle distinguishes between external dependencies and dependencies on projects which are part of the same multi-project build. For the latter you can declare Project Dependencies.

dependencies {
    compile project(':shared')
}

For more information see the javadoc for ProjectDependency

Multi-project builds are discussed in Chapter 31, Multi-project Builds.

Transitive compile dependencies are disabled by default. They can be enabled using:

configurations.compile.transitive = true

File dependencies allow you to directly add a set of files to a configuration, without first adding them to a repository. This can be useful if you cannot, or do not want to, place certain files in a repository. Or if you do not want to use any repositories at all for storing your dependencies.

To add some files as a dependency for a configuration, you simply pass a file collection as a dependency:

dependencies {
    runtime files('libs/a.jar', 'libs/b.jar')
    runtime fileTree(dir: 'libs', includes: ['*.jar'])
}

File dependencies are not included in the published dependency descriptor for your project. However, file dependencies are included in transitive project dependencies within the same build. This means they cannot be used outside the current build, but they can be used with the same build.

You can declare which tasks produce the files for a file dependency. You might do this when, for example, the files are generated by the build.

dependencies {
    compile files("$buildDir/classes") {
        builtBy 'compile'
    }
}

task compile << {
    println 'compiling classes'
}

task list(dependsOn: configurations.compile) << {
    println "classpath = ${configurations.compile.collect {File file -> file.name}}"
}

> gradle -q list
compiling classes
classpath = [classes]

You can exclude a transitive dependency either by configuration or by dependency:

configurations {
    compile.exclude module: 'commons'
    all*.exclude group: 'org.gradle.test.excludes', module: 'reports'
}

dependencies {
    compile("org.gradle.test.excludes:api:1.0") {
        exclude module: 'shared'
    }
}

If you define an exclude for a particular configuration, the excluded transitive dependency will be filtered for all dependencies when resolving this configuration or any inheriting configuration. If you want to exclude a transitive dependency from all your configurations you can use the Groovy spread-dot operator to express this in a concise way, as shown in the example. When defining an exclude, you can specify either only the organization or only the module name or both. Have also a look at the javadoc of Dependency and Configuration .

All attributes for a dependency are optional, except the name. It depends on the repository type, which information is need for actually finding the dependencies in the repository. See Section 28.5, “Repositories”. If you work for example with Maven repositories, you need to define the group, name and version. If you work with filesystem repositories you might only need the name or the name and the version.

dependencies {
    runtime ":junit:4.4", ":testng"
    runtime name: 'testng' 
}

You can also assign collections or arrays of dependency notations to a configuration:

List groovy = ["org.codehaus.groovy:groovy-all:1.5.4@jar",
               "commons-cli:commons-cli:1.0@jar",
               "org.apache.ant:ant:1.7.0@jar"]
List hibernate = ['org.hibernate:hibernate:3.0.5@jar', 'somegroup:someorg:1.0@jar']
dependencies {
    runtime groovy, hibernate
}

In Gradle a dependency can have different configurations (as your project can have different configurations). If you don't specify anything explicitly, Gradle uses the default configuration of the dependency. For dependencies from a Maven repository, the default configuration is the only available one anyway. If you work with Ivy repositories and want to declare a non-default configuration for your dependency you have to use the map notation and declare:

dependencies {
    runtime group: 'org.somegroup', name: 'somedependency', version: '1.0', configuration: 'someConfiguration'
}

To do the same for project dependencies you need to declare:

dependencies {
    compile project(path: ':api', configuration: 'spi')
}

You can generate dependency reports from the command line (see Section 9.5, “Obtaining information about your build”). With the help of the Project report plugin (see Chapter 27, The Project Report Plugin) such a report can be created by your build.

The Gradle repository management, based on Apache Ivy, gives you have a lot of freedom regarding the repository layout and the retrieval policies. Additionally Gradle provides a couple of convenience method to add preconfigured repositories.

To add the central Maven2 repository (http://repo1.maven.org/maven2) simply type:

repositories {
    mavenCentral()
}

Now Gradle looks for your dependencies in this repository.

Quite often certain jars are not in the official Maven repository for licensing reasons (e.g. JTA), but its poms are.

repositories {
    mavenCentral name: 'single-jar-repo', urls: "http://repo.mycompany.com/jars"
    mavenCentral name: 'multi-jar-repos', urls: ["http://repo.mycompany.com/jars1", "http://repo.mycompany.com/jars2"]
}

Gradle looks first in the central Maven repository for the pom and the jar. If the jar can't be found there, its looks for it in the other repositories.

For adding a custom Maven repository you can say:

repositories {
    mavenRepo urls: "http://repo.mycompany.com/maven2"
}

To declare additional repositories to look for jars (like above in the example for the central Maven repository), you can say:

repositories {
    mavenRepo urls: ["http://repo2.mycompany.com/maven2", "http://repo.mycompany.com/jars"]
}

The first URL is used to look for poms and jars. The subsequent URLs are used to look for jars.

org.apache.ivy.util.url.CredentialsStore.INSTANCE.addCredentials("REALM", "HOST", "USER", "PASSWORD");

If you want to use a (flat) filesytem directory as a repository, simply type:

repositories {
    flatDir name: 'localRepository', dirs: 'lib'
    flatDir dirs: ['lib1', 'lib2']
}

This adds repositories which look into one or more directories for finding dependencies. If you only work with flat directory resolvers you don't need to set all attributes of a dependency. See Section 28.3.7, “Optional attributes”

The methods above for creating preconfigured repositories share some common behavior. For all of them, defining a name for the repository is optional. If no name is defined a default name is calculated, depending on the type of the repository. You might want to assign a name, if you want to access the declared repository. For example if you want to use it also for uploading your project artifacts. An explicit name might also be helpful when studying the debug output.

The values passed as arguments to the repository methods can be of any type, not just String. The value that is actually used, is the toString result of the argument object.

When Gradle downloads dependencies from remote repositories it stores them in a local cache located at USER_HOME/.gradle/cache. When Gradle downloads dependencies from one of its predefined local resolvers (e.g. Flat Directory resolver), the cache is not used as an intermediate storage for dependency artifacts. The cache is always used for caching module descriptors.

Gradle, thanks to Ivy under its hood, is extremely flexible regarding repositories:

Let's say, you declare a dependency on the junit:junit:3.8.2 library. Now how does Gradle find it in the repositories? Somehow the dependency information has to be mapped to a path. In contrast to Maven, where this path is fixed, with Gradle you can define a pattern that defines what the path will look like. Here are some examples: ^[22]

// Maven2 layout (if a repository is marked as Maven2 compatible, the organization (group) is split into subfolders according to the dots.)
someroot/[organisation]/[module]/[revision]/[module]-[revision].[ext]

// Typical layout for an ivy repository (the organization is not split into subfolder)
someroot/[organisation]/[module]/[revision]/[type]s/[artifact].[ext]

// Simple layout (the organization is not used, no nested folders.)
someroot/[artifact]-[revision].[ext]

To add any kind of repository (you can pretty easy write your own ones) you can do:

repositories {
    add(new org.apache.ivy.plugins.resolver.FileSystemResolver()) {
        name = 'repo'
        addIvyPattern "$projectDir/repo/[organisation]/[module]-ivy-[revision].xml"
        addArtifactPattern "$projectDir/repo/[organisation]/[module]-[revision](-[classifier]).[ext]"
        descriptor = 'optional'
        checkmodified = true
    }
}

An overview of which Resolvers are offered by Ivy and thus also by Gradle can be found here. With Gradle you just don't configure them via XML but directly via their API.

There is another way to deal with transitive dependencies without XML descriptor files. You can do this with Gradle, but we don't recommend it. We mention it for the sake of completeness and comparison with other build tools.

The trick is to use only artifact dependencies and group them in lists. That way you have somehow expressed, what are your first level dependencies and what are transitive dependencies (see Section 28.3.7, “Optional attributes”). But the draw-back is, that for the Gradle dependency management all dependencies are considered first level dependencies. The dependency reports don't show your real dependency graph and the compile task uses all dependencies, not just the first level dependencies. All in all, your build is less maintainable and reliable than it could be when using client modules. And you don't gain anything.