Ubuntu Wiki

Summary

A lot of Java applications use Maven (and associated maven plugins) as their build system. In order to package this software as appropriate Debian packages, a lot of changes need to be applied to the usual Maven process. This document outlines the known issues and proposed solutions.

Release Note

It is now possible to package Maven-based Java applications like Geronimo or Glassfish in main or universe, using their native build system.

Rationale

Most recent complex Java applications leverage ASF's Maven system and artifact repository to ease the dependency resolving and building of their software. Unfortunately the usual Maven process downloads/updates needed artifacts (JARs), Maven plugins and build/runtime dependency information (.pom files) from the Maven repositories into a local repository cache and works from there. This is incompatible with the way our buildd system works, and with the idea of repeatable builds in general. Given the power of the Maven plugins-based build system, working around Maven to build those complex packages without it is almost impossible.

Use Cases

• Alan in an Ubuntu developer. He wants to package Geronimo in universe, but Geronimo uses Maven. He uses our modified Maven, the packaged Java libraries and Maven plugins to write a fully-offline and repeatable package, compatible with our buildd system.
• Betty is a Java developer. She wants to use Maven to build her own Java software. Fortunately, the new Maven doesn't prevent her from using Maven the usual way, setting up a local repository that downloads and updates files from the official Maven repositories.

Assumptions

Software using Maven to build is usually very large and requires lots of dependencies. In order to be useful, the modified Maven will require lots of external packaging work to have all the required Java libraries available. This might just be the visible tip of the iceberg.

Design

Identified incompatibilities with buildd

• Online dependency download/update: Maven downloads needed build/runtime dependency information (.pom files) and resulting artifacts (Java libraries and Maven plugins) from Maven repositories on the Internet. For each item already present in the local repository, it checks if the version is up to date. We need to provide all those dependencies as part of the source or as packaged build dependencies, and disable Maven freshness check.
• Use of a specific repository: Maven doesn't use the libraries installed in /usr/share/java, it only uses its own local repository. Since the required Java libraries build dependencies will be pulled as packages, Maven needs to make use of those.
• Maven relies on specific versions of libraries: in the .pom files, each artifact can depend on a very specific version of a given JAR, which is usually different from the one in the Ubuntu repositories. A dependency mapping system needs to be supported in order to match the very specific required version with the installed one while keeping Maven happy.

Additional issues

• .pom files shipping: these files need to be available for maven to resolve dependencies and build order. We have to either ship them with each Java library we package, or as a big blob of .pom files together with the source, or have a way of reconstructing them from existing Debian Depends information before calling maven.
• Maven plugins: we'll need to package the most common Maven plugins, and make Maven support the use of these system-installed plugins.

JPackage implementation (used in Fedora for example)

JPackage patched maven to solve these issues. They provide a mvn-jpp wrapper that enables those patches:

• Really offline run
• JPP repository support: makes maven use system-installed JARs
• depmap.xml: maps whatever is in the POM file to the location of the library in the filesystem. Each package adds its own dependencies in a central file that is used by mvn-jpp, so it rarely needs a specific dependency map spec.

Then they ship .pom files in various ways : in a big maven2-common-poms package for the libraries that don't ship them, and together with Java libraries in their package. The package-provided ones are preferred in case of conflict.

See the [http://cvs.fedoraproject.org/viewcvs/rpms/maven2/F-9/ patches] and the README here: [http://fedoraproject.org/wiki/Java/JPPMavenReadme].

Each Java package must provides:

• jars in /usr/share/java/...
• .pom files corresponding to each jar as /usr/share/maven2/poms/JPP.path.to-jarname.pom
• depmap file fragments in /etc/maven/fragments/ mapping .pom artifacts to the real jars
• a postinst regenerating /etc/maven/maven2-depmap.xml from all depmap fragments in /etc/maven/fragments/

The maven2 package provides:

• An /usr/share/maven2/repository/JPP -> /usr/share/java symlink

• An /etc/maven/fragments/maven2 depmap fragment and the postinst hook to regenerate /etc/maven/maven2-depmap.xml

The maven2-common-poms package provides:

• Various .pom files in /usr/share/maven2/default_poms/
• A versionless depmap for all those poms /etc/maven/maven2-versionless-depmap.xml

• A /usr/share/java/maven2/default_poms -> /usr/share/maven2/default_poms symlink

The patches we would need to port are: maven2-jpprepolayout.patch, maven2-JPackageRepositoryLayout.java and maven2-MavenJPackageDepmap.java

These patches apply almost cleanly to the version in Intrepid. A proof-of-concept package integrating them is available in [https://launchpad.net/~tcarrez/+archive this PPA].

Here are the extra options defined in the patchset:

• maven2.offline.mode - This property tells maven to use the pre-configured jpackage repositories.
• maven2.ignore.versions - This property tells maven to ignore versions when looking at the dependency map.
• maven2.usejppjars - With this property supplied, maven will try to use jpackage jars whenever possible. It's main intention is to allow users to use /usr/share/java files when available, and fetch jars from the net if not. It may be combined with the above two.
• maven2.jpp.depmap.file - This option tells maven where to find custom dependency mappings

To enable a buildd-compatible mode, you should run with the -Dmaven2.offline.mode -Dmaven2.ignore.versions -Dmaven2.usejppjars options.

Implementation

The proposed implementation is to follow an approach close to the JPackage one, with a few adaptations.

Known problems to solve with the JPackage implementation

• Fedora shipped a maven2-common-poms package as a workaround for existing libraries that don't ship the required .pom and depmap fragments. It's probably cleaner to fix all affected packages in the release cycle where this would be implemented.
• Fedora uses a single maven-depmap.xml file regenerated at postinstall from fragments. It might be cleaner to patch the patchset so that it supports directly reading into a depmap.d directory.
• Depmap fragments (and the resulting autogenerated maven-depmap.xml) are located in /etc/maven. Since those aren't configuration files but rather information needed to integrate the library with maven, it should probably be located somewhere in /usr/share to conform to FHS and Debian policy.

A packaging-compliant Maven

• Modify the JPP patchset to fix the known problems above

mvn-jpp-compliant Java libraries

mvn-jpp-compliant Maven plugins

Test/Demo Plan

In order to test the new maven features, we need a fake "hello world" maven project to package and probably a few fake Java libraries dependencies packaged with the required .pom files and depmap fragments. Testing it on large packages will not be possible before most of the Java libraries are modified or packaged.

Outstanding Issues

• .pom files and depmap fragments duplicate information found in debian/control and others. It's probably too complicated to patch the JPP patchset to look into /var/lib/pkg for dependency info. What could be done in the future is writing some dh_javapom helper that would help generating the pom and depmap fragment for a Java library package.

Discussion

CategorySpec