Maven 3 : avoiding duplication of information, make stuff easy to maintain

Leave a comment

This post in the 3rd one of a series on maven3. You can find the information related to the demo project used in example here.

One of the painful thing with all build management system, is the maintenance.

Setting it up at the beginning is ok, but maintaining it over time take time. Ensuring that all tools run on all modules, with same version, that the experiment you made on one module, or on a subset of module do not brake the whole things ….

So, in this post will see some tips to avoid information duplication in order to ease maintenance.

1: use pomParent mechanism to share most of the configuration

The pom parent mechanism is an “inclusion like” mechanism. It’s well defined here.
In the demo project, you have it located in the PomParent directory, and it’s used in all the others pom files.
In the following pomParent I am :

  • defining common properties (java version, encoding, plugin version) use in all pom files
  • defining plugin management element for all plugin used in one of the pom, in order to have plugin version centralized in one location
  • defining developer, license, organisation in one location

PomParent content :

   <project>
<modelVersion>4.0.0</modelVersion>

<!– The Basics –>
<groupId>com.persistentsas.mavenTraining.PomParent</groupId>
<artifactId>parent</artifactId>
<version>1.9-SNAPSHOT</version>

<packaging>pom</packaging>
<properties>
<!– build mechanism property –>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<!– Plugin having a custom setting –>
<maven.pmd.plugin>2.7.1</maven.pmd.plugin>
<pmd.java.version>1.7</pmd.java.version>
….
</properties>
<dependencyManagement>
…..
</dependencyManagement>
<modules></modules>
<!– Build Settings –>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<configuration>
<encoding>${maven.resources.plugin.encoding}</encoding>
</configuration>
</plugin>
……
</plugins>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-release-plugin</artifactId>
<version>${maven.release.plugin}</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-site-plugin</artifactId>
<version>${maven.site.plugin}</version>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>findbugs-maven-plugin</artifactId>
<version>${maven.findbugs.plugin}</version>
</plugin>
…………….
</plugins>
</pluginManagement>
</build>
<reporting></reporting>

<!– More Project Information –>
<name>parent</name>
<description></description>
<url></url>
<inceptionYear></inceptionYear>
<licenses>
<license>
<name>Apache License, Version 2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0.txt</url&gt;
<distribution>repo</distribution>
<comments>A business-friendly OSS license</comments>
</license>
</licenses>
<organization>
<name>PersistentSAS</name>
<url>http://www.persistentsas.fr</url&gt;
</organization>
<developers>
<developer>
<id>LaurentT</id>
<name>Laurent Tardif</name>
<email>Laurent.Tardif at persistentsas.com</email>
<url>http://www.persistentsas.fr</url&gt;
<organization>PersistentSAS</organization>
<organizationUrl>http://www.persistentsas.fr</organizationUrl&gt;
<roles>
<role>architect</role>
<role>developer</role>
</roles>
<timezone>+1</timezone>
<properties>
<picUrl>http://www.persistentsas.fr</picUrl&gt;
</properties>
</developer>
</developers>
<contributors></contributors>

<!– Environment Settings –>
<issueManagement></issueManagement>
<ciManagement></ciManagement>
<mailingLists></mailingLists>
<scm></scm>
<prerequisites></prerequisites>
<repositories></repositories>
<pluginRepositories></pluginRepositories>
<!– not inherited –>
<distributionManagement>
<site>
<id>${project.artifactId}-site</id>
<url>file:///c:/temp/site/${project.artifactId}/${project.artifactId}.${project.packaging}</url>
</site>
<repository>
<id>GrenobleWrite</id>
<name>artifatoryWrite</name>
<url>http://MyBinaryRepo:8081/artifactory/libs-release-local/</url&gt;
</repository>
<snapshotRepository>
<id>GrenobleWrite</id>
<url>http://MyBinaryRepo:8081/artifactory/libs-snapshot-local/</url&gt;
</snapshotRepository>
</distributionManagement>
<profiles></profiles>
</project>

Inclusion in other pom :

<parent>
        <groupId>com.persistentsas.mavenTraining.PomParent</groupId>
        <artifactId>parent</artifactId>
        <version>1.9-SNAPSHOT</version>
<!– hint path, not mandatory –>
        <relativePath>.\PomParent\parent.pom</relativePath>
    </parent>

2: use properties to centralize in a common place important informations.

This is the first advise I’ll give. Use properties to put in a common place, informations your need to share.

you can define as many properties you want, and reuse them in any part of the pom file. If you define them in a PomParent, you can reuse them in all pom files. That’s help having easy to maintain project.

Among the practices i liked (but not every one agreed :-), I’m agree also to say that in the example, it’s a bit too much :-), but it’s an exercise … ), is to duplicate variable definition, in order to know where it’s used. That make also my life easier when the value need to be changed locally.

<properties>
<!– build mechanism property –>

        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>${project.build.sourceEncoding}</project.reporting.outputEncoding>
        <maven.resources.plugin.encoding>${project.build.sourceEncoding}</maven.resources.plugin.encoding>
        <maven.compiler.plugin.encoding>${project.build.sourceEncoding}</maven.compiler.plugin.encoding>

        <maven.deploy.plugin>2.8.2</maven.deploy.plugin>
        <maven.checkstyle.plugin>2.7</maven.checkstyle.plugin>
….
</properties>

 3) define common build step in the pom Parent

In the pom parent you can define common build section in order to centralized that.

Now Imaging you have 4 modules, let’s say two standard jar libraries, and two webservices with code generation, and so on….

The build steps are not common to the 4 modules, but in this case you can make 3 pomParent :

1) first one common to every one (with dependencies version, common variables, … )

2) Second one, including first one, and defining build step for jar

3) 3rd one , including first one, and defining , source code generation, and so on for webservices.

Advertisements

Maven 3: some useful tips

Leave a comment

Maven is a build tool, but not only. It’s able to manage to whole lifecycle of your library/application.

I’ll use the project describe in a previous post and the source is available on github.

 

 1 : Work online / offline

One of the common usage of maven, is to work most of the time connected. Before going deeper, let explain quickly how maven work. when you run the following command to display the project hierarchy :

mvn dependency:tree

It will check on you build repository for new snapshots, or dependencies update to display it. There’s lot of command that will lead to an update of either your project dependencies, or the plug-in used by maven to build your project.

In order to control when you check update or not, you have the –offline to disable all remote access, and the -U as example, to force verification of updates. The maven –help command list all the options possible at Maven level, it’s independent of the content of your pom, or of any setting you defined.

mvn –offline dependency:tree

 

2: Use several thread to work

By default, maven will use One thread to work. Since maven3, you can use several thread to do the job.

You have 2 options to define the number of threads to use

  • The first one is by explicitly specifying the number of threads to use (2 in the example)

mvn -T2 compile

  • The second one is by specifying the number of thread relatively to the number of core you have on the PC (2 thread per core in the example)

mvn -T2C compile

The second option is very interesting on a heterogeneous build farm or in script, where you can adapt you number of thread regarding the number of cores available at run time.

 

3: Build partially your project

You can also optimise your build time , when running  :

mvn compile

Maven will build the list of project/module to build, and will determine a build order.  For each module, it will check if building it make sense (if no sources, tests or ressources change, there’s no point to rebuild the project).

Even if the verification is efficient, doing it on large project make no sense when, as a developper, you know where you change stuff, and want to do only a partial build.

Let’s see some running examples :

mvn compile

will generate a full build

[INFO] Scanning for projects…
[INFO] ————————————————————————
[INFO] Reactor Build Order:
[INFO]
[INFO] module1
[INFO] module2
[INFO] project1
[INFO] P2module1
[INFO] project2
[INFO] Root
[INFO]
[INFO] ————————————————————————
[INFO] Building module1 1.9-SNAPSHOT
[INFO] ————————————————————————
[INFO] Compiling 1 source file to C:\Dev\GITHUB\mavenTraining\Project1\module1\target\classes
[INFO]
[INFO] ————————————————————————
[INFO] Building module2 1.9-SNAPSHOT
[INFO] ————————————————————————
[INFO] — maven-compiler-plugin:2.5.1:compile (default-compile) @ module2 —
[INFO] Nothing to compile – all classes are up to date
[INFO]
[INFO] ————————————————————————
[INFO] Building project1 1.9-SNAPSHOT
[INFO] ————————————————————————
[INFO]
[INFO] ————————————————————————
[INFO] Building P2module1 1.9-SNAPSHOT
[INFO] ————————————————————————
[INFO]
[INFO] — maven-compiler-plugin:2.5.1:compile (default-compile) @ module1 —
[INFO] Nothing to compile – all classes are up to date
[INFO]
[INFO] ————————————————————————
[INFO] Building project2 1.9-SNAPSHOT
[INFO] ————————————————————————
[INFO] ————————————————————————
[INFO] Building Root 1.9-SNAPSHOT
[INFO] ————————————————————————
[INFO] ————————————————————————
[INFO] Reactor Summary:
[INFO]
[INFO] module1 ……………………………………. SUCCESS [  1.020 s]
[INFO] module2 ……………………………………. SUCCESS [  0.017 s]
[INFO] project1 …………………………………… SUCCESS [  0.001 s]
[INFO] P2module1 ………………………………….. SUCCESS [  0.013 s]
[INFO] project2 …………………………………… SUCCESS [  0.001 s]
[INFO] Root ………………………………………. SUCCESS [  0.001 s]
[INFO] ————————————————————————
[INFO] BUILD SUCCESS
[INFO] ————————————————————————
[INFO] Total time: 1.240 s
[INFO] Finished at: 2014-10-07T10:00:38+01:00
[INFO] Final Memory: 11M/214M
[INFO] ————————————————————————

With such small project, optimizing build time in a intellectual game, but can save lot of time on large project 🙂 So, let’s optimize it.

You can launch the build from a module level (ex: module2). We are going to use the maven “-rf”  parameter to do so (resume from).

[INFO] Scanning for projects…
[INFO] ————————————————————————
[INFO] Reactor Build Order:
[INFO]
[INFO] module2
[INFO] project1
[INFO] P2module1
[INFO] project2
[INFO] Root
[INFO]
[INFO] Using the builder org.apache.maven.lifecycle.internal.builder.singlethreaded.
count of 1
[INFO]
[INFO] ————————————————————————
[INFO] Building module2 1.9-SNAPSHOT
[INFO] ————————————————————————
[INFO]
[INFO] — maven-compiler-plugin:2.5.1:compile (default-compile) @ module2 —
[INFO] Nothing to compile – all classes are up to date
[INFO]
[INFO] ————————————————————————
[INFO] Building project1 1.9-SNAPSHOT
[INFO] ————————————————————————
[INFO]
[INFO] ————————————————————————
[INFO] Building P2module1 1.9-SNAPSHOT
[INFO] ————————————————————————
[INFO][INFO]
[INFO] — maven-compiler-plugin:2.5.1:compile (default-compile) @ module1 —
[INFO] Nothing to compile – all classes are up to date
[INFO]
[INFO] ————————————————————————
[INFO] Building project2 1.9-SNAPSHOT
[INFO] ————————————————————————
[INFO]
[INFO] ————————————————————————
[INFO] Building Root 1.9-SNAPSHOT
[INFO] ————————————————————————
[INFO] ————————————————————————
[INFO] Reactor Summary:
[INFO]
[INFO] module2 ……………………………………. SUCCESS [  0.689 s]
[INFO] project1 …………………………………… SUCCESS [  0.002 s]
[INFO] P2module1 ………………………………….. SUCCESS [  0.016 s]
[INFO] project2 …………………………………… SUCCESS [  0.002 s]
[INFO] Root ………………………………………. SUCCESS [  0.003 s]
[INFO] ————————————————————————
[INFO] BUILD SUCCESS
[INFO] ————————————————————————
[INFO] Total time: 0.900 s
[INFO] Finished at: 2014-10-07T10:04:05+01:00
[INFO] Final Memory: 7M/214M
[INFO] ————————————————————————

 

or even directly from a project :

[INFO] Scanning for projects…
[INFO] ————————————————————————
[INFO] Reactor Build Order:
[INFO]
[INFO] project2
[INFO] Root
[INFO]
[INFO] ————————————————————————
[INFO] Building project2 1.9-SNAPSHOT
[INFO] ————————————————————————
[INFO]
[INFO] ————————————————————————
[INFO] Building Root 1.9-SNAPSHOT
[INFO] ————————————————————————
[INFO] ————————————————————————
[INFO] Reactor Summary:
[INFO]
[INFO] project2 …………………………………… SUCCESS [  0.003 s]
[INFO] Root ………………………………………. SUCCESS [  0.068 s]
[INFO] ————————————————————————
[INFO] BUILD SUCCESS
[INFO] ————————————————————————
[INFO] Total time: 0.252 s
[INFO] Finished at: 2014-10-07T10:04:53+01:00
[INFO] Final Memory: 6M/214M
[INFO] ————————————————————————

 

Not so bad, we reduce build time by 80% 🙂

As you see, maven will scan for project,  construct the ordered list of projects, and will start build at the module or project.

You have several option to control more precisely the resume, let see some :

Options:
-am,–also-make                         If project list is specified, also build projects required by the  list
-amd,–also-make-dependents            If project list is specified, also build projects that depend on  projects on the list
-B,–batch-mode                         Run in non-interactive (batch)  mode
-fae,–fail-at-end                         Only fail the build afterwards;  allow all non-impacted builds to  continue
-ff,–fail-fast                                   Stop at first failure in reactorized builds
-N,–non-recursive                     Do not recurse into sub-projects
-pl,–projects <arg>                   Comma-delimited list of specified  reactor projects to build instead  of all projects.
A project can be  specified by [groupId]:artifactId  or by its relative path.
-rf,–resume-from <arg>          Resume reactor from specified project

Maven 3 standard layout

2 Comments

I have help a team recently to move to maven 3, and ask for “what’s the standard way to organise a project ?”.

Using Maven, you are encourage to follow convention to organise your library sources, but working at a project level you have more freedom.

When you use Maven since years, yo know there’s some good / bad practices that will make your project easy to maintain or not. Recently, some plug-in maintainer have advise a structure, and I’m pretty align with it.

So let’s describe it, and see how to benefit from it. For those interested, you can clone the repository on github, to have the files locally on your PC ( https://github.com/LaurentTardif/mavenTraining )

The project is made of :

  • 3 java libraries (2 in project 1 : module1 and module2, and 1 in project2)
  • one build-tools project
  • one pom parent project

We’ll saw later the benefit of this structure.

+—build-tools
|   \—src
|       \—main
|           \—resources
|               \—build
+—PomParent
+—Project1
|   +—module1
|   |   \—src
|   |       +—main
|   |       |   \—java
|   |       |       \—org
|   |       |           \—persistentsas
|   |       |               \—mavenTraining
|   |       \—test
|   |           \—java
|   |               \—org
|   |                   \—persistentsas
|   |                       \—mavenTraining
|   \—module2
|       \—src
|           +—main
|           |   \—java
|           |       \—org
|           |           \—persistentsas
|           |               \—mavenTraining
|           \—test
|               \—java
|                   \—org
|                       \—persistentsas
|                           \—mavenTraining
+—Project2
|   \—module1
|       \—src
|           +—main
|           |   \—java
|           |       \—org
|           |           \—persistentsas
|           |               \—mavenTraining
|           \—test
|               \—java
|                   \—org
|                       \—persistentsas
|                           \—mavenTraining
\—src
\—site
\—markdown
\—training

Now, let add some files inside this structure ;

  • Some java files in sources and tests. The current setup is minimalist, and contains one file per project.
  • A pom files per library, and some others to make life easier
  • Some files in the build-tools repository, these files will be use to share some common configuration or settings among  libraries.
  • A package-info.java to demonstrate package documentation in Java
  • a Site repository with some markdown file (.md), to demonstrate how to provide documentation up to date with your project.

C:.
|   .gitignore
|   LICENSE
|   pom.xml
|   README.md
|
+—build-tools
|   |   pom.xml
|   |
|   \—src
|       \—main
|           \—resources
|               \—build
|                       checkstyle.xml
|                       lib-filter.xml
|                       LICENSE.txt
|
+—PomParent
|       parent.pom
|
+—Project1
|   |   pom.xml
|   |
|   +—module1
|   |   |   pom.xml
|   |   |
|   |   \—src
|   |       +—main
|   |       |   \—java
|   |       |       \—org
|   |       |           \—persistentsas
|   |       |               \—mavenTraining
|   |       |                       HelloWorld.java
|   |       |                       package-info.java
|   |       |
|   |       \—test
|   |           \—java
|   |               \—org
|   |                   \—persistentsas
|   |                       \—mavenTraining
|   |                               HelloWorldTest.java
|   |
|   \—module2
|       |   pom.xml
|       |
|       \—src
|           +—main
|           |   \—java
|           |       \—org
|           |           \—persistentsas
|           |               \—mavenTraining
|           |                       HelloWorld2.java
|           |
|           \—test
|               \—java
|                   \—org
|                       \—persistentsas
|                           \—mavenTraining
|                                   HelloWorld2Test.java
|
+—Project2
|   |   pom.xml
|   |
|   \—module1
|       |   pom.xml
|       |
|       \—src
|           +—main
|           |   \—java
|           |       \—org
|           |           \—persistentsas
|           |               \—mavenTraining
|           |                       HelloWorld3.java
|           |
|           \—test
|               \—java
|                   \—org
|                       \—persistentsas
|                           \—mavenTraining
|                                   HelloWorld3Test.java
|
\—src
\—site
|   site.xml
|
\—markdown
|   index.md
|
\—training
Training.md

Now, let describe what are the benefit of such layout.

  1. You have all items related to a project, inside a project (pom, source code, test code, resources)
  2. You are able to share common build related configuration file using the build-tools repository
  3. you have several pom to make a hierarchical build, let’s call them aggregator pom for the moment
  4. you have “master” pom at the root of the project, that allow you to easily build, tests all your projects, but all to run command on a dedicated module.
  5. you do not duplicated informations inside build (versionning, behavior, …) using pomParent mechanism.

We’ll see in the next posts, how these functionalities works.