Chapter 43. Dependency Management

43.1. Introduction

This chapter gives an overview of issues related with dependency management and presents how Gradle can be used to overcome them.

Gradle offers a very good support for dependency management. If you are familiar with Maven or Ivy approach you will be delighted to learn that:

  • All the concepts that you already know and like are still there and are fully supported by Gradle. The current dependency management solutions all require to work with XML descriptor files and are usually based on remote repositories for downloading the dependencies. Gradle fully supports this approach.

  • Gradle works perfectly with your existent dependency management infrastructure, be it Maven or Ivy. All the repositories you have set up with your custom POM or ivy files can be used as they are. No changes necessary.

  • Additionally, Gradle offers a simpler approach, which might be better suited for some projects.

43.2. Dependency management overview

We think dependency management is very important for almost any project. Yet the kind of dependency management you need depends on the complexity and the environment of your project. Is your project a distribution or a library? Is it part of an enterprise environment, where it is integrated into other projects builds or not? But all types of projects share the following requirements:

  • The version of the jar must be easy to recognize. Sometimes the version is in the Manifest file of the jar, often not. And even if, it is rather painful to always look into the Manifest file to learn about the version. Therefore we think that you should only use jars which have their version as part of their file name.

  • It hopes to be clear what are the first level dependencies and what are the transitive ones. There are different ways to achieve this. We will look at this later.

  • Conflicting versions of the same jar should be detected and either resolved or cause an exception.

43.2.1. Versioning the jar name

Why do we think this is necessary? Without a dependency management as described above, your are likely to burn your fingers sooner or later. If it is unclear which version of a jar your are using, this can introduce subtle bugs which are very hard to find. For example there might be a project which uses Hibernate 3.0.4. There are some problems with Hibernate so a developer installs version 3.0.5 of Hibernate on her machine. This did not solve the problem but she forgot to roll back Hibernate to 3.0.4. Weeks later there is an exception on the integration machine which can't be reproduced on the developer machine. Without a version in the jar name this problem might take a long time to debug. Version in the jar names increases the expressiveness of your project and makes it easier to maintain.

43.2.2. Transitive dependency management

Why is transitive dependency management so important? If you don't know which dependencies are first level dependencies and which ones are transitive you will soon lose control over your build. Even Gradle has already 20+ jars. An enterprise project using Spring, Hibernate, etc. easily ends up with 100+ jars. There is no way to memorize where all these jars come from. If you want to get rid of a first level dependency you can't be sure which other jars you should remove. Because a dependency of a first level dependency might also be a first level dependency itself. Or it might be a transitive dependency of another of your first level dependencies. Many first level dependencies are runtime dependencies and the transitive dependencies are of course all runtime dependencies. So the compiler won't help you much here. The end of the story is, as we have seen very often, no one dares to remove any jar any longer. The project classpath is a complete mess and if a classpath problem arises, hell on earth invites you for a ride. In one of my former projects, I found some ldap related jar in the classpath, whose sheer presence, as I found out after much research, accelerated LDAP access. So removing this jar would not have led to any errors at compile or runtime.

Gradle offers you different ways to express what are first level and what are transitive dependencies. Gradle allows you for example to store your jars in CVS or SVN without XML descriptor files and still use transitive dependency management. Gradle also validates your dependency hierarchy against the reality of your code by using only the first level dependencies for compiling.

43.2.3. Version conflicts

In your dependency description you tell Gradle which version of a dependency is needed by another dependency. This frequently leads to conflicts. Different dependencies rely on different versions of another dependency. The JVM unfortunately does not offer yet any easy way, to have different versions of the same jar in the classpath (see Section 43.2.5, “Dependency management and Java”).

Gradle offers following conflict resolution strategies:

  • Newest - used by default by Gradle - the newest version of the dependency is used. This strategy has been in Gradle since early days.
  • Fail - fail eagerly on version conflict. Useful if you need extra control and manage the conflicts manually. Introduced in 1.0-milestone-6. See ResolutionStrategy for reference on managing the conflict resolution strategies.

Gradle provides means to resolve version conflicts:

  • Configuring a first level dependency as forced. The feature has been in Gradle since early days. This approach is useful if the dependency incurring conflict is already a first level dependency. See examples in DependencyHandler
  • Configuring any dependency (transitive or not) as forced. The feature was introduced in 1.0-milestone-7. This approach is useful if the dependency incurring conflict is a transitive dependency. It also can be used to force versions of first level dependencies. See examples in ResolutionStrategy

To deal with problems due to version conflicts, reports with dependency graphs are also very helpful. Such reports are another feature of dependency management.

43.2.4. Dynamic Versions and Changing Modules

Sometimes, you always want to use the latest version of a particular dependency, or the latest in a range of versions. You can easily do this using a dynamic version. A dynamic version can be either a version range (eg. 2.+) or it can be a placeholder for the latest version available (eg. latest.integration).

Alternatively, sometimes the module you request can change over time, even for the same version. An example of this type of changing module is a maven SNAPSHOT module, which always points at the latest artifacts published.

The main difference between a dynamic version and a changing module is that when you resolve a dynamic version, you'll get the real, static version as the module name. When you resolve a changing module, the artifacts are named using the version you requested, but the underlying artifacts may change over time.

By default, Gradle caches dynamic versions and changing modules for 24 hours. You can override the default cache modes using command line options. You can change the cache expiry times in your build using the resolution strategy (see Section 43.8.3, “Fine-tuned control over dependency caching”).

43.2.5. Dependency management and Java

Traditionally, Java has offered no support at all for dealing with libraries and versions. There are no standard ways to say that foo-1.0.jar depends on a bar-2.0.jar. This has led to proprietary solutions. The most popular ones are Maven and Ivy. Maven is a complete build system whereas Ivy focuses solely on dependency management.

Both approaches rely on descriptor XML files, which contains information about the dependencies of a particular jar. Both also use repositories where the actual jars are placed together with their descriptor files. And both offer resolution for conflicting jar versions in one form or the other. Yet we think the differences of both approaches are significant in terms of flexibility and maintainability. Beside this, Ivy fully supports the Maven dependency handling. So with Ivy you have access to both worlds. We like Ivy very much. Gradle uses it under the hood for its dependency management. Ivy is most often used via Ant and XML descriptors. But it also has an API. We integrate deeply with Ivy via its API. This enables us to build new concepts on top of Ivy which Ivy does not offer itself.

Right now there is a lot of movement in the field of dependency handling. There is OSGi and there is JSR-294, Improved Modularity Support in the JavaTM Programming Language. OSGi is available already, JSR-294 is supposed to be shipped with Java 7. These technologies deal, amongst many other things, also with a painful problem which is neither solved by Maven nor by Ivy. This is enabling different versions of the same jar to be used at runtime.

43.3. Dependency configurations

In Gradle dependencies are grouped into configurations. Configurations have a name, a number of other properties, and they can extend each other. Many Gradle plugin add pre-defined configurations to your project. The Java plugin, for example, adds some configurations to represent the various classpaths it needs. see Section 23.5, “Dependency management” for details. Of course you can add your add custom configurations on top of that. There are many use cases for custom configurations. This is very handy for example for adding dependencies not needed for building or testing your software (e.g. additional JDBC drivers to be shipped with your distribution).

A project's configurations are managed by a configurations object. The closure you pass to the configurations object is applied against its API. To learn more about this API have a look at ConfigurationContainer.

To define a configuration:

Example 43.1. Definition of a configuration

build.gradle

configurations {
    compile
}

To access a configuration:

Example 43.2. Accessing a configuration

build.gradle

println configurations.compile.name
println configurations['compile'].name

To configure a configuration:

Example 43.3. Configuration of a configuration

build.gradle

configurations {
    compile {
        description = 'compile classpath'
        transitive = true
    }
    runtime {
        extendsFrom compile
    }
}
configurations.compile {
    description = 'compile classpath'
}

43.4. How to declare your dependencies

There are several different types of dependencies that you can declare:

Table 43.1. Dependency types

Type Description
External module dependency A dependency on an external module in some repository.
Project dependency A dependency on another project in the same build.
File dependency A dependency on a set of files on the local filesystem.
Client module dependency A dependency on an external module, where the artifacts are located in some repository but the module meta-data is specified by the local build. You use this kind of dependency when you want to override the meta-data for the module.
Gradle API dependency A dependency on the API of the current Gradle version. You use this kind of dependency when you are developing custom Gradle plugins and task types.
Local Groovy dependency A dependency on the Groovy version used by the current Gradle version. You use this kind of dependency when you are developing custom Gradle plugins and task types.

43.4.1. External module dependencies

External module dependencies are the most common dependencies. They refer to a module in an external repository.

Example 43.4. Module dependencies

build.gradle

dependencies {
    runtime group: 'org.springframework', name: 'spring-core', version: '2.5'
    runtime 'org.springframework:spring-core:2.5', 'org.springframework:spring-aop:2.5'
    runtime(
        [group: 'org.springframework', name: 'spring-core', version: '2.5'],
        [group: 'org.springframework', name: 'spring-aop', version: '2.5']
    )
    runtime('org.hibernate:hibernate:3.0.5') {
        transitive = true
    }
    runtime group: 'org.hibernate', name: 'hibernate', version: '3.0.5', transitive: true
    runtime(group: 'org.hibernate', name: 'hibernate', version: '3.0.5') {
        transitive = true
    }
}

Please see the DependencyHandler for more examples and complete reference. Please read on to get thorough understanding of the Gradle's dependency management.

Gradle provides different notations for module dependencies. There is a string notation and a map notation. A module dependency has an API which allows for further configuration. Have a look at ExternalModuleDependency to learn all about the API. This API provides properties and configuration methods. Via the string notation you can define a subset the properties. With the map notation you can define all properties. To have access to the complete API, either with the map or with the string notation, you can assign a single dependency to a configuration together with a closure.

If you declare a module dependency, Gradle looks for a corresponding module descriptor file (pom.xml or ivy.xml) in the repositories. If such a module descriptor file exists, it is parsed and the artifacts of this module (e.g. hibernate-3.0.5.jar) as well as its dependencies (e.g. cglib) are downloaded. If no such module descriptor file exists, Gradle looks for a file called hibernate-3.0.5.jar to retrieve. In Maven a module can only have one and only one artifact. In Gradle and Ivy a module can have multiple artifacts. Each artifact can have a different set of dependencies.

43.4.1.1. Depending on modules with multiple artifacts

As mentioned earlier, a maven module has only one artifact. So, when your project depends on a maven module it's obvious what artifact is the actual dependency. With Gradle or Ivy the case is different. Ivy model of dependencies (ivy.xml) can declare multiple artifacts. For more information, see Ivy reference for ivy.xml. In Gradle, when you declare a dependency on an ivy module you actually declare dependency on the 'default' configuration of that module. So the actual list of artifacts (typically jars) your project depends on, are all artifacts that are attached to the default configuration of that module. This is very important in following exemplary use cases:
  • The default configuration of some module contains some artifacts you don't want on the classpath. You might need to configure a dependency on specific artifact(s) of given module, rather than pulling all artifacts of the default dependency
  • The artifact you need on the classpath has been published in a different configuration than the default one. This means this artifact will not be pulled in by Gradle. Unless you explicitly declare what configuration of the module you depend on.
There are other situations where it is necessary to fine-tune the dependency declaration. Please see the DependencyHandler for examples and complete reference on declaring dependencies.

43.4.1.2. Artifact only notation

As said above, if no module descriptor file can be found, Gradle by default downloads a jar with the name of the module. But sometimes, even if the repository contains module descriptors, you want to download only the artifact jar, without the dependencies. [12] And sometimes you want to download a zip from a repository, that does not have module descriptors. Gradle provides an artifact only notation for those use cases - simply prefix the extension that you want to be downloaded with '@' sign:

Example 43.5. Artifact only notation

build.gradle

dependencies {
    runtime "org.groovy:groovy:1.5.6@jar"
    runtime group: 'org.groovy', name: 'groovy', version: '1.5.6', ext: 'jar'
}


An artifact only notation creates a module dependency which downloads only the artifact file with the specified extension. Existing module descriptors are ignored.

43.4.1.3. Classifiers

The Maven dependency management has the notion of classifiers. [13] Gradle supports this. To retrieve classified dependencies from a maven repository you can write:

Example 43.6. Dependency with classifier

build.gradle

compile "org.gradle.test.classifiers:service:1.0:jdk15@jar"
    otherConf group: 'org.gradle.test.classifiers', name: 'service', version: '1.0', classifier: 'jdk14'

As you can see in the example, classifiers can be used together with setting an explicit extension (artifact only notation).

To use the external dependencies of a configuration:

Example 43.7. Usage of external dependency of a configuration

build.gradle

task listJars << {
    configurations.compile.each { File file -> println file.name }
}

Output of gradle -q listJars

> gradle -q listJars
hibernate-core-3.6.7.Final.jar
antlr-2.7.6.jar
commons-collections-3.1.jar
dom4j-1.6.1.jar
slf4j-api-1.6.1.jar
hibernate-commons-annotations-3.2.0.Final.jar
hibernate-jpa-2.0-api-1.0.1.Final.jar
jta-1.1.jar

43.4.2. Client module dependencies

Client module dependencies enable you to declare transitive dependencies directly in your build script. They are a replacement for a module descriptor XML file in an external repository.

Example 43.8. Client module dependencies - transitive dependencies

build.gradle

dependencies {
    runtime module("org.codehaus.groovy:groovy-all:1.8.4") {
        dependency("commons-cli:commons-cli:1.0") {
            transitive = false
        }
        module(group: 'org.apache.ant', name: 'ant', version: '1.8.2') {
            dependencies "org.apache.ant:ant-launcher:1.8.2@jar", "org.apache.ant:ant-junit:1.8.2"
        }
    }
}

This declares a dependency of your project on Groovy. Groovy itself has dependencies. But Gradle does not look for an XML descriptor to figure them out but gets the information from the build file. The dependencies of a client module can be normal module dependencies or artifact dependencies or another client module. Have also a look at the API documentation: ClientModule

In the current release client modules have one limitation. Let's say your project is a library and you want this library to be uploaded to your company's Maven or Ivy repository. Gradle uploads the jars of your project to the company repository together with the XML descriptor file of the dependencies. If you use client modules the dependency declaration in the XML descriptor file is not correct. We will improve this in a future release of Gradle.

43.4.3. Project dependencies

Gradle distinguishes between external dependencies and dependencies on projects which are part of the same multi-project build. For the latter you can declare Project Dependencies.

Example 43.9. Project dependencies

build.gradle

dependencies {
    compile project(':shared')
}

For more information see the API documentation for ProjectDependency

Multi-project builds are discussed in Chapter 49, Multi-project Builds.

43.4.4. File dependencies

File dependencies allow you to directly add a set of files to a configuration, without first adding them to a repository. This can be useful if you cannot, or do not want to, place certain files in a repository. Or if you do not want to use any repositories at all for storing your dependencies.

To add some files as a dependency for a configuration, you simply pass a file collection as a dependency:

Example 43.10. File dependencies

build.gradle

dependencies {
    runtime files('libs/a.jar', 'libs/b.jar')
    runtime fileTree(dir: 'libs', include: '*.jar')
}

File dependencies are not included in the published dependency descriptor for your project. However, file dependencies are included in transitive project dependencies within the same build. This means they cannot be used outside the current build, but they can be used with the same build.

You can declare which tasks produce the files for a file dependency. You might do this when, for example, the files are generated by the build.

Example 43.11. Generated file dependencies

build.gradle

dependencies {
    compile files("$buildDir/classes") {
        builtBy 'compile'
    }
}

task compile << {
    println 'compiling classes'
}

task list(dependsOn: configurations.compile) << {
    println "classpath = ${configurations.compile.collect {File file -> file.name}}"
}

Output of gradle -q list

> gradle -q list
compiling classes
classpath = [classes]

43.4.5. Gradle API Dependency

You can declare a dependency on the API of the current version of Gradle by using the DependencyHandler.gradleApi() method. This is useful when you are developing custom Gradle tasks or plugins.

Example 43.12. Gradle API dependencies

build.gradle

dependencies {
    compile gradleApi()
}

43.4.6. Local Groovy Dependency

You can declare a dependency on the Groovy that is distributed with Gradle by using the DependencyHandler.localGroovy() method. This is useful when you are developing custom Gradle tasks or plugins in Groovy.

Example 43.13. Gradle's Groovy dependencies

build.gradle

dependencies {
    groovy localGroovy()
}

43.4.7. Excluding transitive dependencies

You can exclude a transitive dependency either by configuration or by dependency:

Example 43.14. Excluding transitive dependencies

build.gradle

configurations {
    compile.exclude module: 'commons'
    all*.exclude group: 'org.gradle.test.excludes', module: 'reports'
}

dependencies {
    compile("org.gradle.test.excludes:api:1.0") {
        exclude module: 'shared'
    }
}

If you define an exclude for a particular configuration, the excluded transitive dependency will be filtered for all dependencies when resolving this configuration or any inheriting configuration. If you want to exclude a transitive dependency from all your configurations you can use the Groovy spread-dot operator to express this in a concise way, as shown in the example. When defining an exclude, you can specify either only the organization or only the module name or both. Have also a look at the API documentation of Dependency and Configuration.

Not every transitive dependency can be excluded - some transitive dependencies might be essential for correct runtime behavior of the application. Generally, one can exclude transitive dependencies that are either not required by runtime or that are guaranteed to be available on the target environment/platform.

Should you exclude per-dependency or per-configuration? It turns out that in majority of cases you want to use the per-configuration exclusion. Here are the some exemplary reasons why one might want to exclude a transitive dependency. Bear in mind that for some of those use cases there are better solutions than exclusions!

  • The dependency is undesired due to licensing reasons.
  • The dependency is not available in any of remote repositories.
  • The dependency is not needed for runtime.
  • The dependency has a version that conflicts with a desired version. For that use case please refer to Section 43.2.3, “Version conflicts” and the documentation on ResolutionStrategy for a potentially better solution to the problem.

Basically, in most of the cases excluding the transitive dependency should be done per configuration. This way the dependency declaration is more explicit. It is also more accurate because a per-dependency exclude rule does not guarantee the given transitive dependency does not show up in the configuration. For example, some other dependency, which does not have any exclude rules, might pull in that unwanted transitive dependency.

Other examples of the dependency exclusions can be found in the reference for ModuleDependency or DependencyHandler.

43.4.8. Optional attributes

All attributes for a dependency are optional, except the name. It depends on the repository type, which information is need for actually finding the dependencies in the repository. See Section 43.6, “Repositories”. If you work for example with Maven repositories, you need to define the group, name and version. If you work with filesystem repositories you might only need the name or the name and the version.

Example 43.15. Optional attributes of dependencies

build.gradle

dependencies {
    runtime ":junit:4.8.2", ":testng"
    runtime name: 'testng' 
}

You can also assign collections or arrays of dependency notations to a configuration:

Example 43.16. Collections and arrays of dependencies

build.gradle

List groovy = ["org.codehaus.groovy:groovy-all:1.8.4@jar",
               "commons-cli:commons-cli:1.0@jar",
               "org.apache.ant:ant:1.8.2@jar"]
List hibernate = ['org.hibernate:hibernate:3.0.5@jar', 'somegroup:someorg:1.0@jar']
dependencies {
    runtime groovy, hibernate
}

43.4.9. Dependency configurations

In Gradle a dependency can have different configurations (as your project can have different configurations). If you don't specify anything explicitly, Gradle uses the default configuration of the dependency. For dependencies from a Maven repository, the default configuration is the only available one anyway. If you work with Ivy repositories and want to declare a non-default configuration for your dependency you have to use the map notation and declare:

Example 43.17. Dependency configurations

build.gradle

dependencies {
    runtime group: 'org.somegroup', name: 'somedependency', version: '1.0', configuration: 'someConfiguration'
}

To do the same for project dependencies you need to declare:

Example 43.18. Dependency configurations for project

build.gradle

dependencies {
    compile project(path: ':api', configuration: 'spi')
}

43.4.10. Dependency reports

You can generate dependency reports from the command line (see Section 11.5.3, “Listing project dependencies”). With the help of the Project report plugin (see Chapter 39, The Project Report Plugin) such a report can be created by your build.

43.5. Working with dependencies

For the examples below we have the following dependencies setup:

Example 43.19. Configuration.copy

build.gradle

configurations {
    sealife
    alllife
}

dependencies {
    sealife "sea.mammals:orca:1.0", "sea.fish:shark:1.0", "sea.fish:tuna:1.0"
    alllife configurations.sealife
    alllife "air.birds:albatros:1.0"
}

The dependencies have the following transitive dependencies:

shark-1.0 -> seal-2.0, tuna-1.0

orca-1.0 -> seal-1.0

tuna-1.0 -> herring-1.0

You can use the configuration to access the declared dependencies or a subset of those:

Example 43.20. Accessing declared dependencies

build.gradle

task dependencies << {
    configurations.alllife.dependencies.each { dep -> println dep.name }
    println()
    configurations.alllife.allDependencies.each { dep -> println dep.name }
    println()
    configurations.alllife.allDependencies.findAll { dep -> dep.name != 'orca' }.each { dep -> println dep.name }
}

Output of gradle -q dependencies

> gradle -q dependencies
albatros

albatros
orca
shark
tuna

albatros
shark
tuna

dependencies returns only the dependencies belonging explicitly to the configuration. allDependencies includes the dependencies from extended configurations.

To get the library files of the configuration dependencies you can do:

Example 43.21. Configuration.files

build.gradle

task allFiles << {
    configurations.sealife.files.each { file ->
        println file.name
    }
}

Output of gradle -q allFiles

> gradle -q allFiles
orca-1.0.jar
shark-1.0.jar
tuna-1.0.jar
seal-2.0.jar
herring-1.0.jar

Sometimes you want the library files of a subset of the configuration dependencies (e.g. of a single dependency).

Example 43.22. Configuration.files with spec

build.gradle

task files << {
    configurations.sealife.files { dep -> dep.name == 'orca' }.each { file ->
        println file.name
    }
}

Output of gradle -q files

> gradle -q files
orca-1.0.jar
seal-2.0.jar

The Configuration.files method always retrieves all artifacts of the whole configuration. It then filters the retrieved files by specified dependencies. As you can see in the example, transitive dependencies are included.

You can also copy a configuration. You can optionally specify that only a subset of dependencies from the original configuration should be copied. The copying methods come in two flavors. The copy method copies only the dependencies belonging explicitly to the configuration. The copyRecursive method copies all the dependencies, including the dependencies from extended configurations.

Example 43.23. Configuration.copy

build.gradle

task copy << {
    configurations.alllife.copyRecursive { dep -> dep.name != 'orca' }.allDependencies.each { dep ->
        println dep.name
    }
    println()
    configurations.alllife.copy().allDependencies.each { dep ->
        println dep.name
    }
}

Output of gradle -q copy

> gradle -q copy
albatros
shark
tuna

albatros

It is important to note that the returned files of the copied configuration are often but not always the same than the returned files of the dependency subset of the original configuration. In case of version conflicts between dependencies of the subset and dependencies not belonging to the subset the resolve result might be different.

Example 43.24. Configuration.copy vs. Configuration.files

build.gradle

task copyVsFiles << {
    configurations.sealife.copyRecursive { dep -> dep.name == 'orca' }.each { file ->
        println file.name
    }
    println()
    configurations.sealife.files { dep -> dep.name == 'orca' }.each { file ->
        println file.name
    }
}

Output of gradle -q copyVsFiles

> gradle -q copyVsFiles
orca-1.0.jar
seal-1.0.jar

orca-1.0.jar
seal-2.0.jar

In the example above, orca has a dependency on seal-1.0 whereas shark has a dependency on seal-2.0. The original configuration has therefore a version conflict which is resolved to the newer seal-2.0 version. The files method therefore returns seal-2.0 as a transitive dependency of orca. The copied configuration only has orca as a dependency and therefore there is no version conflict and seal-1.0 is returned as a transitive dependency.

Once a configuration is resolved it is immutable. Changing its state or the state of one of its dependencies will cause an exception. You can always copy a resolved configuration. The copied configuration is in the unresolved state and can be freshly resolved.

To learn more about the API of the configuration class see the API documentation: Configuration.

43.6. Repositories

Gradle repository management, based on Apache Ivy, gives you a lot of freedom regarding repository layout and retrieval policies. Additionally Gradle provides various convenience method to add pre-configured repositories.

You may configure any number of repositories, each of which is treated independently by Gradle. If Gradle finds a module descriptor in a particular repository, it will attempt to download all of the artifacts for that module from the same repository. Although module meta-data and module artifacts must be located in the same repository, it is possible to compose a single repository of multiple URLs, giving multiple locations to search for meta-data files and jar files.

There are several different types of repositories you can declare:

Table 43.2. Repository types

Type Description
Maven central repository A pre-configured repository that looks for dependencies in Maven Central.
Maven local repository A pre-configured repository that looks for dependencies in the local Maven repository.
Maven repository A Maven repository. Can be located on the local filesystem or at some remote location.
Ivy repository An Ivy repository. Can be located on the local filesystem or at some remote location.
Flat directory repository A simple repository on the local filesystem. Does not support any meta-data formats.

43.6.1. Maven central repository

To add the central Maven 2 repository (http://repo1.maven.org/maven2) simply add this to your build script:

Example 43.25. Adding central Maven repository

build.gradle

repositories {
    mavenCentral()
}

Now Gradle will look for your dependencies in this repository.

43.6.2. Local Maven repository

To use the local Maven cache as a repository you can do:

Example 43.26. Adding the local Maven cache as a repository

build.gradle

repositories {
    mavenLocal()
}

43.6.3. Maven repositories

For adding a custom Maven repository you can do:

Example 43.27. Adding custom Maven repository

build.gradle

repositories {
    maven {
        url "http://repo.mycompany.com/maven2"
    }
}

Sometimes a repository will have the POMs published to one location, and the JARs and other artifacts published at another location. To define such a repository, you can do:

Example 43.28. Adding additional Maven repositories for JAR files

build.gradle

repositories {
    maven {
        // Look for POMs and artifacts, such as JARs, here
        url "http://repo2.mycompany.com/maven2"
        // Look for artifacts here if not found at the above location
        artifactUrls "http://repo.mycompany.com/jars"
        artifactUrls "http://repo.mycompany.com/jars2"
    }
}

Gradle will look at the first URL for the POM and the JAR. If the JAR can't be found there, the artifact URLs are used to look for JARs.

43.6.3.1. Accessing password protected Maven repositories

To access a Maven repository which uses basic authentication, you specify the username and password to use when you define the repository:

Example 43.29. Accessing password protected Maven repository

build.gradle

repositories {
    maven {
        credentials {
            username 'user'
            password 'password'
        }
        url "http://repo.mycompany.com/maven2"
    }
}

It is advisable to keep your username and password in gradle.properties rather than directly in the build file.

43.6.4. Flat directory repository

If you want to use a (flat) filesystem directory as a repository, simply type:

Example 43.30. Flat repository resolver

build.gradle

repositories {
    flatDir {
        dirs 'lib'
    }
    flatDir {
        dirs 'lib1', 'lib2'
    }
}

This adds repositories which look into one or more directories for finding dependencies. If you only work with flat directory resolvers you don't need to set all attributes of a dependency. See Section 43.4.8, “Optional attributes”

43.6.5. Ivy repositories

To use an Ivy repository with a standard layout:

Example 43.31. Ivy repository

build.gradle

repositories {
    ivy {
        url "http://repo.mycompany.com/repo"
        layout "maven"
    }
}

See IvyArtifactRepository for details.

43.6.5.1. Defining custom patterns for an Ivy repository

To define an Ivy repository with a non-standard layout, you can define a pattern layout for the repository:

Example 43.32. Ivy repository with pattern layout

build.gradle

repositories {
    ivy {
        url "http://repo.mycompany.com/repo"
        layout 'pattern', {
            artifact "[module]/[revision]/[artifact].[ext]"
            ivy "[module]/[revision]/ivy.xml"
        }
    }
}

43.6.5.2. Defining different artifact and ivy file locations for an Ivy repository

To define an Ivy repository which fetches ivy files and artifacts from different locations, you can explicitly define complete URL patterns for artifacts and ivy files:

Example 43.33. Ivy repository with custom patterns

build.gradle

repositories {
    ivy {
        artifactPattern "http://repo.mycompany.com/3rd-party-artifacts/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"
        artifactPattern "http://repo.mycompany.com/company-artifacts/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"
        ivyPattern "http://repo.mycompany.com/ivy-files/[organisation]/[module]/[revision]/ivy.xml"
    }
}

43.6.5.3. Accessing password protected Ivy repositories

To access an Ivy repository which uses basic authentication, you specify the username and password to use when you define the repository:

Example 43.34. Ivy repository

build.gradle

repositories {
    ivy {
        credentials {
            username 'user'
            password 'password'
        }
        artifactPattern "http://repo.mycompany.com/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"
    }
}

43.6.6. Working with repositories

To access a repository:

Example 43.35. Accessing a repository

build.gradle

println repositories.localRepository.name
    println repositories['localRepository'].name

To configure a repository:

Example 43.36. Configuration of a repository

build.gradle

repositories {
    flatDir {
        name 'localRepository'
    }
}
repositories {
    localRepository {
        dirs 'lib'
    }
}
repositories.localRepository {
    dirs 'lib'
}

43.6.7. More about Ivy resolvers

Gradle, thanks to Ivy under its hood, is extremely flexible regarding repositories:

  • There are many options for the protocol to communicate with the repository (e.g. filesystem, http, ssh, ...)

  • Each repository can have its own layout.

Let's say, you declare a dependency on the junit:junit:3.8.2 library. Now how does Gradle find it in the repositories? Somehow the dependency information has to be mapped to a path. In contrast to Maven, where this path is fixed, with Gradle you can define a pattern that defines what the path will look like. Here are some examples: [14]

// Maven2 layout (if a repository is marked as Maven2 compatible, the organization (group) is split into subfolders according to the dots.)
someroot/[organisation]/[module]/[revision]/[module]-[revision].[ext]

// Typical layout for an ivy repository (the organization is not split into subfolder)
someroot/[organisation]/[module]/[revision]/[type]s/[artifact].[ext]

// Simple layout (the organization is not used, no nested folders.)
someroot/[artifact]-[revision].[ext]

To add any kind of repository (you can pretty easy write your own ones) you can do:

Example 43.37. Definition of a custom repository

build.gradle

repositories {
    ivy {
        ivyPattern "$projectDir/repo/[organisation]/[module]-ivy-[revision].xml"
        artifactPattern "$projectDir/repo/[organisation]/[module]-[revision](-[classifier]).[ext]"
    }
}

An overview of which Resolvers are offered by Ivy and thus also by Gradle can be found here. With Gradle you just don't configure them via XML but directly via their API.

43.7. How dependency resolution works

Gradle takes your dependency declarations and repository definitions and attempts to download all of your dependencies by a process called dependency resolution. Below is a brief outline of how this process works.

  • Given a required dependency, Gradle first attempts to resolve the module for that dependency. Each repository is inspected in order, searching first for a module descriptor file (pom or ivy file) that indicates the presence of that module. If no module descriptor is found, Gradle will search for the presence of the primary module artifact file indicating that the module exists in the repository.

    • If the dependency is declared as a dynamic version (like 1.+), Gradle will resolve this to the newest available static version (like 1.2) in the repository. For maven repositories, this is done using the maven-metadata.xml file, while for ivy repositories this is done by directory listing.

    • If the module descriptor is a pom file that has a parent pom declared, Gradle will recursively attempt to resolve each of the parent modules for the pom.

  • Once each repository has been inspected for the module, Gradle will choose the 'best' one to use. This is done using the following criteria:

    • For a dynamic version, a 'higher' static version is preferred over a 'lower' version.
    • Modules declared by a module descriptor file (ivy or pom file) are preferred over modules that have an artifact file only.
    • Modules from earlier repositories are preferred over modules in later repositories.

    When the dependency is declared by a static version and a module descriptor file is found in a repository, there is no need to continue searching later repositories and the remainder of the process is short-circuited.

  • All of the artifacts for the module are then requested from the same repository that was chosen in the process above.

43.8. The dependency cache

Gradle contains a highly sophisticated dependency caching mechanism, which seeks to minimise the number of remote requests made in dependency resolution, while striving to guarantee that the results of dependency resolution are correct and reproducible.

The Gradle dependency cache consists of 2 key types of storage:

  • A file-based store of downloaded artifacts, including binaries like jars as well as raw downloaded meta-data like pom files and ivy files. The storage path for a downloaded artifact includes the SHA1 checksum, meaning that 2 artifacts with the same name but different content can easily be cached.

  • A binary store of resolved module meta-data, including the results of resolving dynamic versions, module descriptors, and artifacts.

Separating the storage of downloaded artifacts from the cache metadata permits us to do some very powerful things with our cache that would be difficult with a transparent, file-only cache layout.

The Gradle cache does not allow the local cache to hide problems and creating mysterious and difficult to debug behavior that has been a challenge with many build tools. This new behavior is implemented in a bandwidth and storage efficient way. In doing so, Gradle enables reliable and reproducible enterprise builds.

43.8.1. Key features of the Gradle dependency cache

43.8.1.1. Separate metadata cache

Gradle keeps a record of various aspects of dependency resolution in binary format in the metadata cache. The information stored in the metadata cache includes:

  • The result of resolving a dynamic version (eg 1.+) to a concrete version (eg 1.2).
  • The resolved module metadata for a particular module, including module artifacts and module dependencies.
  • The resolved artifact metadata for a particular artifact, including a pointer to the downloaded artifact file.
  • The absence of a particular module or artifact in a particular repository, eliminating repeated attempts to access a resource that does not exist.

Every entry in the metadata cache includes a record of the repository that provided the information as well as a timestamp that can be used for cache expiry.

43.8.1.2. Repository caches are independent

As described above, for each repository there is a separate metadata cache. A repository is identified by its URL, type and layout. If a module or artifact has not been previously resolved from this repository, Gradle will attempt to resolve the module against the repository. This will always involve a remote lookup on the repository, however in many cases no download will be required (see Section 43.8.1.3, “Artifact reuse”, below).

Dependency resolution will fail if the required artifacts are not available in any repository specified by the build, regardless whether the local cache has retrieved this artifact from a different repository. Repository independence allows builds to be isolated from each other in an advanced way that no build tool has done before. This is a key feature to create builds that are reliable and reproducible in any environment.

43.8.1.3. Artifact reuse

Before downloading an artifact, Gradle tries to determine the checksum of the required artifact by downloading the sha file associated with that artifact. If the checksum can be retrieved, an artifact is not downloaded if an artifact already exists with the same id and checksum. If the checksum cannot be retrieved from the remote server, the artifact will be downloaded (and ignored if it matches an existing artifact).

As well as considering artifacts downloaded from a different repository, Gradle will also attempt to reuse artifacts found in the local Maven Repository. If a candidate artifact has been downloaded by Maven, Gradle will use this artifact if it can be verified to match the checksum declared by the remote server.

43.8.1.4. Checksum based storage

It is possible for different repositories to provide a different binary artifact in response to the same artifact identifier. This is often the case with Maven SNAPSHOT artifacts, but can also be true for any artifact which is republished without changing it's identifier. By caching artifacts based on their SHA1 checksum, Gradle is able to maintain multiple versions of the same artifact. This means that when resolving against one repository Gradle will never overwrite the cached artifact file from a different repository. This is done without requiring a separate artifact file store per repository.

43.8.1.5. Cache Locking

The Gradle dependency cache uses file-based locking to ensure that it can safely be used by multiple Gradle processes concurrently. The lock is held whenever the binary meta-data store is being read or written, but is released for slow operations such as downloading remote artifacts.

43.8.2. Command line options to override caching

43.8.2.1. Offline

The --offline command line switch tells Gradle to always use dependency modules from the cache, regardless if they are due to be checked again. When running with offline, Gradle will never attempt to access the network to perform dependency resolution. If required modules are not present in the dependency cache, build execution will fail.

43.8.2.2. Refresh

At times, the Gradle Dependency Cache can be out of sync with the actual state of the configured repositories. Perhaps a repository was initially misconfigured, or perhaps a "non-changing" module was published incorrectly. To refresh all dependencies in the dependency cache, use the --refresh-dependencies option on the command line.

The --refresh-dependencies option tells Gradle to ignore all cached entries for resolved modules and artifacts. A fresh resolve will be performed against all configured repositories, with dynamic versions recalculated, modules refreshed, and artifacts downloaded. However, where possible Gradle will attempt to if the previously downloaded artifacts are valid before downloading again. This is done by comparing published SHA1 values in the repository with the SHA1 values for existing downloaded artifacts.

43.8.3. Fine-tuned control over dependency caching

You can fine-tune certain aspects of caching using the ResolutionStrategy for a configuration.

By default, Gradle caches dynamic versions for 24 hours. To change how long Gradle will cache the resolved version for a dynamic version, use:

Example 43.38. Dynamic version cache control

build.gradle

configurations.all {
    resolutionStrategy.cacheDynamicVersionsFor 10, 'minutes'
}

By default, Gradle caches changing modules for 24 hours. To change how long Gradle will cache the meta-data and artifacts for a changing module, use:

Example 43.39. Changing module cache control

build.gradle

configurations.all {
    resolutionStrategy.cacheChangingModulesFor 30, 'days'
}

For more details, take a look at the API documentation for ResolutionStrategy.

43.9. Strategies for transitive dependency management

Many projects rely on the Maven Central repository. This is not without problems.

  • The Maven Central repository can be down or has a very long response time.

  • The pom.xml's of many projects have wrong information (as one example, the POM of commons-httpclient-3.0 declares JUnit as a runtime dependency).

  • For many projects there is not one right set of dependencies (as more or less imposed by the pom format).

If your project relies on the Maven Central repository you are likely to need an additional custom repository, because:

  • You might need dependencies that are not uploaded to Maven Central yet.

  • You want to deal properly with wrong metadata in a Maven Central pom.xml.

  • You don't want to expose people who want to build your project, to the downtimes or sometimes very long response times of Maven Central.

It is not a big deal to set-up a custom repository. [15] But it can be tedious, to keep it up to date. For a new version, you have always to create the new XML descriptor and the directories. And your custom repository is another infrastructure element which might have downtimes and needs to be updated. To enable historical builds, you need to keep all the past libraries and you need a backup. It is another layer of indirection. Another source of information you have to lookup. All this is not really a big deal but in its sum it has an impact. Repository Manager like Artifactory or Nexus make this easier. But for example open source projects don't usually have a host for those products.

This is a reason why some projects prefer to store their libraries in their version control system. This approach is fully supported by Gradle. The libraries can be stored in a flat directory without any XML module descriptor files. Yet Gradle offers complete transitive dependency management. You can use either client module dependencies to express the dependency relations, or artifact dependencies in case a first level dependency has no transitive dependencies. People can check out such a project from svn and have everything necessary to build it.

If you are working with a distributed version control system like Git you probably don't want to use the version control system to store libraries as people check out the whole history. But even here the flexibility of Gradle can make your life easier. For example you can use a shared flat directory without XML descriptors and yet you can have full transitive dependency management as described above.

You could also have a mixed strategy. If your main concern is bad metadata in the pom.xml and maintaining custom XML descriptors, Client Modules offer an alternative. But you can of course still use Maven2 repo and your custom repository as a repository for jars only and still enjoy transitive dependency management. Or you can only provide client modules for POMs with bad metadata. For the jars and the correct POMs you still use the remote repository.

43.9.1. Implicit transitive dependencies

There is another way to deal with transitive dependencies without XML descriptor files. You can do this with Gradle, but we don't recommend it. We mention it for the sake of completeness and comparison with other build tools.

The trick is to use only artifact dependencies and group them in lists. That way you have somehow expressed, what are your first level dependencies and what are transitive dependencies (see Section 43.4.8, “Optional attributes”). But the draw-back is, that for the Gradle dependency management all dependencies are considered first level dependencies. The dependency reports don't show your real dependency graph and the compile task uses all dependencies, not just the first level dependencies. All in all, your build is less maintainable and reliable than it could be when using client modules. And you don't gain anything.



[12] Gradle supports partial multiproject builds (seeChapter 49, Multi-project Builds).

[14] At http://ant.apache.org/ivy/history/latest-milestone/concept.html you can learn more about ivy patterns.

[15] If you want to shield your project from the downtimes of Maven Central things get more complicated. You probably want to set-up a repository proxy for this. In an enterprise environment this is rather common. For an open source project it looks like overkill.