The current dependency management solutions all require to work with XML descriptor files and are usually based on remote repositories for downloading the dependencies. Gradle fully supports this approach. Gradle works perfectly with your existent dependency management infrastructure, be it Maven or Ivy. All the repositories you have set up with your custom pom or ivy files can be used as they are. No changes necessary. But Gradle offers also a simpler approach which might be better suited for many projects.
We think dependency management is very important for almost any project. Yet the kind of dependency management you need depends on the complexity and the environment of your project. Is your project a distribution or a library? Is it part of an enterprise environment, where it is integrated into other projects builds or not? But all types of projects share the following requirements:
Why do we think this is necessary? Without a dependency management as described above, your are likely to burn your fingers sooner or later. If it is unclear which version of a jar your are using, this can introduce subtle bugs which are very hard to find. For example there might be a project which uses Hibernate 3.0.4. There are some problems with Hibernate so a developer installs version 3.0.5 of Hibernate on her machine. This did not solve the problem but she forgot to roll back Hibernate to 3.0.4. Weeks later there is an exception on the integration machine which can’t be reproduced on the developer machine. Without a version in the jar name this problem might take a long time to debug. Version in the jar names increases the expressiveness of your project and making it easier to maintain.
Why is transitive dependency management so important? If you don’t know which dependencies are first level dependencies and which ones are transitive you are loosing very soon control over your build. Even Gradle has already 20+ jars. An enterprise project using Spring, Hibernate, etc. easily ends up with 100+ jars. There is no way to memorize where all this jars belong to. If you want to get rid of a first level dependency you can’t be sure which other jars you should remove. Because a dependency of your first level dependency might also be a first level dependency itself. Or it might be a transitive dependency of another of your first level dependency. Many first level dependencies are runtime dependencies and the transitive dependencies are of course all runtime dependencies. So the compiler won’t help you much here. The end of the story is, as we have seen very often, nobody dares to remove any jar any longer. The project classpath is a complete mess and if a classpath problems arises, hell on earth invites you for a ride. In one of my former projects, I’ve found some ldap related jar in the classpath, which sheer presence, as I’ve found out after much research, accelerated LDAP access. So removing this jar would not have led to any errors at compile or runtime.
Gradle offers you different ways to express what are first level and what are transitive dependencies. Gradle allows you for example to store your jars in CVS or SVN without XML descriptor files and still use transitive dependency management. Gradle also validates your dependency hierarchy against the reality of your code by using only the first level dependencies for compiling.
In your dependency description you tell Gradle which version of a dependency is needed by another dependency. This leads frequently to conflicts. Different dependencies rely on different versions of another dependency. The JVM unfortunately does not offer yet any easy way, to have different versions of the same jar in the classpath (See 12.1.4). What Gradle offers you is a resolution strategy, by default the newest version is used. To deal with problems due to version conflicts, reports with dependency graphs are also very helpful. Such reports are another feature of dependency management.
Traditionally, Java has offered no support at all for dealing with libraries and versions. There are no standard ways to say that foo-1.0.jar depends on a bar-2.0.jar. This has led to proprietary solutions. The most popular ones are Maven and Ivy. Maven is a complete build system whereas Ivy focuses solely on dependency management.
Both approaches rely on descriptor xml files, which contains information about the dependencies of a particular jar. Both also use repositories where the actual jars are placed together with there descriptor files. And both offer resolution for conflicting jar versions in one form or the other. Yet we think the differences of both approaches are gigantic in terms of flexibility and maintainability. Beside this, Ivy fully supports the Maven dependency handling. So with Ivy you have access to both worlds. We like Ivy very much. Gradle uses it under the hood for its dependency management. Ivy is most often used via Ant and XML descriptors. But it also has a very good API. We integrate deeply with Ivy via its API. This enables us to build new concepts on top of Ivy which Ivy does not offer itself.
Right now there is a lot of movement in the field of dependency handling. There is OSGi and there is JSR-277. OSGi is available already, JSR-277 is supposed to be shipped with Java 7. These technologies deal also with a painful problem which is neither solved by Maven nor by Ivy. This is enabling different versions of the same jar to be used at runtime.
People who know Ivy have come across most the concepts we are going to introduce now. But Gradle does not use any XML for declaring the dependencies (e.g. no ivy.xml file). It has its own notations which is part of the Gradle build file.
Dependencies are grouped in configurations. Configurations have a name and they can extend each other. If you use the Java or Groovy plugin, Gradle adds a number of related configurations to your build. The plugin also associates configurations with tasks. See section 12.3 for details. Of course you can add your own configurations on top of that. This is very handy for example for adding dependencies not needed for building or running your software (e.g. additional JDBC drivers). You might want to add such dependencies to a distribution of your software.
A dependency can belong to more than one configuration although that might be rather the exception. What is a dependency after all and how do you add it to a configuration?
To add an artifact dependency to let’s say the compile configuration you simply type:
An artifact dependency is just a file, usually a jar file. The notation for such a dependency follows the pattern: [group]:[artifact]:[version]@[extension] or [group]:[artifact]:[version]:[classifier]@[extension].1 Gradle needs a repository somewhere where this artifact can be found, but there is no xml descriptor required in the repository. If there is one, it is ignored. In the simplest case such a repository is a folder on your local machine.
Here is an example for a module dependency:
The notation here is the same as for artifact dependencies except that you don’t specify a file extension. If you declare a module dependency, Gradle looks in the repositories if there is either a Maven pom file or an ivy.xml file for this module. If there is no such XML file, Gradle either tries to download a jar with the same signature as the module dependency or throws an exception (or both, if this jar can’t be found). It is configurable whether a XML descriptor for a module dependency is required. This is a per-repository configuration.
If the module descriptor (e.g. pom.xml) declares dependencies you don’t want you have, you can exclude them. Gradle offers the following notation:
Client module dependencies enable you to declare transitive dependencies directly in your build script. Client module dependencies can be nested.
This declares a dependency of your project on Groovy. Groovy itself has dependencies. But Gradle does not look for an xml descriptor to figure them out but gets the information from the build file. The dependencies of a client module can be normal module dependencies or artifact dependencies or another client module.
In the current release Client Modules have one limitation. For example if your project is a library and you want this library to be uploaded to your company’s Maven or Ivy repository. Gradle uploads the jars of your project to the company repository together with the XML descriptor file of the dependencies. If you use Client Modules the dependency declaration in the XML descriptor file is not correct. We try to fix this for the next release of Gradle.
Gradle distinguishes between external dependencies and dependencies on projects which are part of the same multi-project build. For the latter you can declare Project Dependencies.
Multi-project builds are discussed in chapter 14.
Gradle offers its own notation to describe dependencies. Under the hood we transform them into ivy dependency objects. Our notation provides only a subset of what is possible with Ivy. If you want or need to use options of Ivy not offered by our notation you can pass ivy dependency objects directly to Gradle.
In the examples above we have already configured the dependency manager. How does this work? The project object offers a property called dependencies which points to an instance of org.gradle.api.DependencyManager. If you call dependencies with a closure, all statements within the closure are delegated first to the dependency manager. This is the same mechanism which is used for configuring tasks.
The Java Plugin preconfigures the dependency management. It adds a set of configurations and links the tasks like compile and test to this configuration. Table 12.1 shows the details. The Java plugin defines also where to find the jars and zips to be uploaded. Finally it does a couple of settings for multi-project builds. If you have for example declared a project dependency for the compile configuration you don’t want this project to be build if you execute a clean.
|
Many projects rely on the Maven2 repository. This is not without problems.
If your project relies on the IBibilio repository you are likely to need an additional custom repository, because:
It is not a big deal to set-up a custom repository.2 But it is tedious, to keep it up to date. For a new version, you have always to create the new XML descriptor and the directories. And your custom repository is another infrastructure element which might have downtimes and needs to be updated. To enable historical builds, you need to keep all the past libraries. It is another layer of indirection, which contains information relevant for your build and you have to navigate to the lengthy path with your browser to get the information. All this is not really a big deal but in its sum it has a relevant impact on your agility. Why should you cope with this? You pay something, so you should also get something, right? So we end up with the question: Does it make sense to store the dependency information (and the dependencies) in remote repositories? Our answer is: It depends.
In a larger enterprise environment you are likely to have a number of independent builds, possibly many different, large multi-project builds. Here a repository infrastructure makes a lot of sense. You have already infrastructure that guarantees a high uptime. Multiple projects can share there information about certain common dependencies they have. There is a central place to look for dependency information. And the repositories are used to integrate the internal projects with each other. One project publishes itself to the company repository, with all its dependency information stored in an ivy.xml or pom.xml file.3 Together with a continous integration server this offers a strong integration for projects with separate builds. As you have seen, Gradle fully supports this approach.
But what about other environments. What about a typical open source projects like Gradle or environments which consists of a single multi-project build . They don’t integrate with other projects in the same environment or need to share dependency information in this environment. In such a (very common) case we think dependency management based on remote repositories is an unnecessary indirection which makes things more complicated than necessary. Gradle itself (which is build by Gradle) stores its dependencies in svn. There is a lib folder which contains all jars needed to build Gradle. Yet we have a complete management of our dependencies. For Gradle the lib folder is a local repository containing all the jars (in a flat structure). There are no xml descriptors. We use either client module dependencies to express our dependency relations, or artifact dependencies in case a first level dependency has no transitive dependencies. People can check out Gradle from svn and have everything necessary to build it. The same applies to our source distribution.
You might argue that such an approach works only for projects like Gradle, which is not used as a library but is simply a distribution. What about open source library projects (e.g. commons-httpclient) that need to upload themselves to the Maven2 repo together with a pom.xml file. As soon as the limitation mentioned in subsection 12.2.4 has been overcome, your project can upload to such a repository without relying on XML descriptor files themselves.
You could also have a mixed strategy. If your main concern is bad metadata in the and maintaining custom XML descriptors, Client Modules offer an alternative. But you can of course still use Maven2 repo and your custom repository as a repository for jars only and still enjoy transitive dependency management.
There is another way to deal with transitive dependencies without xml descriptor files. You can do this with Gradle, but we don’t recommend it. We mention it for the sake of completeness and comparison with other build tools.
The trick is to use only artifact dependencies and group them in lists. That way you have somehow expressed, what are your first level dependencies and what are transitive dependencies.
But for the Gradle dependency management all dependencies are considered first level dependencies. The dependency reports don’t show your real dependency graph and the compile task uses all dependencies, not just the first level dependencies. All in all, your build is less maintainable and reliable than it could be when using client modules. And you don’t gain anything.
Gradle (Ivy) distinguishes between resolvers and repositories. A repository is the physical source for your dependencies. A resolver is responsible for retrieving a dependency (usually from a repository but not necessarily). As a Gradle or Ivy user, resolvers are the objects your are working with. You can also aggregate multiple resolvers in a certain order (not just linked lists). It is one of the very strengths of Ivy, how versatile you can aggregate resolvers and that you can write easily your own. Gradle for example makes use of this to implement client modules. If you add a resolver, you always have to give it a name. This name is used to internally address a resolver but is also used for the dependency logging. The Java plugin does not add any default resolver to your projects dependency manager. Gradle offers a couple of convenience methods to add resolvers. You can also always add an instance of an Ivy resolver object directly.
To add a resolver for the Maven2 repository (http://repo1.maven.org/maven2) simply type:
The added resolver has the name MavenRepo and is added to the classpathResolvers (see 12.5.5).
If you want to add a custom repository with a Maven layout you can type:
Quite often certain jars are not in the official Maven repository for licensing reasons (e.g. JTA). But the poms are there and you want to use them. Gradle (and Ivy) use the same resolver to look for the pom and the corresponding artifact. It is not possible to define a chain of resolvers, where one resolver is used for retrieving the pom and another resolver is used for retrieving the artifact. For the dependencies of the artifact the chain of resolvers can be used. To solve this problem
Now Gradle looks only in the official Maven repository for the poms, but looks in both the official Maven repository and the two repositories specified above for the artifacts belonging to the pom. Let’s look at an example to understand this better.
We have 4 files in different repositories. For this example let’s assume that repotest-1.0.pom is in the official Maven repository and that the other files are in http://repo.gradle.org.
Let’s define our resolvers like this:
Gradle finds repotest-1.0.pom in the official Maven repo. After this it tries to find the corresponding artifact repotest-1.0.jar. It looks only in the Maven repo for this file as it expects to find pom and artifact by the same resolver. The build fails.
Let’s try to improve this:
Gradle finds repotest-1.0.pom in the Maven repo. After this it tries to find the corresponding artifact repotest-1.0.jar and finds it in http://repo.gradle.org. But what about testdep-1.0.pom? As Gradle looks for poms only in the Maven repo it can’t find this pom file.4
The final correct version to solve our use case is:
If you want to use a (flat) filesytem directory as a repository, simply type:
In the example we assign multiple directories to a flat directory resolver5 . The first argument is the name of the resolver. The group attributes in the dependency declaration is empty (but there has to be a colon at the beginning). Specifying the extension (e.g. jar) is optional and defaults to jar. You could add a group attribute if you are in a chain of resolvers. This resolver simply ignores it. The repositories could look like this:
The addFlatDirResolver methods adds a resolver to the classpathResolvers (see 12.5.5)
You can add easily file system repositories with a different layout (see 12.5.6)
When Gradle downloads dependencies from remote repositories it stores them in a local cache located at USER_HOME/.gradle/cache. When Gradle downloads dependencies from one of its predefined local resolvers (e.g. Flat Directory resolver), the cache is not used as an intermediate storage for dependencies.
There are three hooks for adding resolvers. Via the resolvers assigned to the classpathResolvers property of the dependency manager, the compile, testCompile and test tasks retrieve there dependencies. The uploadLibs and uploadDists tasks have both an uploadResolvers property. You can assign one resolver to multiple hooks.
All of those hooks are instances of org.gradle.api.dependencies.ResolverContainer. This class provided methods for adding new resolvers at a particular position in the resolve chain.6 Ivy offers more complex structures for resolvers than simple chains. The resolver container does not offer particular support for this. But by manipulating its properties directly you could set up such other structures.
Gradle, thanks to Ivy under its hood, is extremely flexible regarding repositories:
If you declare a dependency like junit:junit:3.8.2 how does Gradle finds it in the repositories. Somehow the dependency information has to be mapped to a path. In contrast to Maven, where this path is fixed, with Gradle you can define a pattern how this path should look like. Here are some examples:7
To add any kind of repository (you can pretty easy write your own ones) you can do:
WebdavResolver is a class that implements an Ivy interface and is part of the Gradle distribution. An overview of which Resolvers are offered by Ivy and thus also by Gradle can be found here. With Gradle you just don’t configure them via xml but directly via there API.