Maven 102: Advanced Basics

Alternate title:  What you need to do to be a good Maven citizen

Java module layout

Here is the default layout for a java/jar maven module.  You can reconfigure source and output directories, but you may not want to, as I’ll discuss in a second.

source/input directories:

src/main/java
src/main/resources
src/test/java
src/test/resources

output directories:

target
target/classes
target/test-classes

Build Flow

A general flow for Maven in such a project would be to:

1. compile src/main/java to target/classes
2. copy src/main/resources to target/classes
3. compile src/test/java to target/test-classes
4. copy src/test/resources to target/test-classes
5. run unit tests, output to target/surefire-reports
6. build jar, source jar, and test jars, place in target

Again, you can change much of this, but with Maven, the more you fight with it, the more you will have to fight with it.  It kind of snowballs and will cause annoyances down the line, during releases and documentation and site generation and such.

Repositories

It is not terrible to set up a repository, but the holy grail (for the users, anyway) is to get your libraries uploaded to Maven Central — this will make the process quite seamless for adding a new compiler to a project because access to Maven Central is basically required for Maven operation.

Custom Language Compilation

Another important task for easy maven integration is compilation.  This will generally require a special maven plugin that can essentially execute  the custom language compiler and conform to something similar to the input and output directories above.  This is not too tricky, especially if there is a java entry point to do the work.

Modularity

Related to this is a topic of modularity:  how do you break up a project?  How do you resolve circular references across multiple langugaes? For example, you will generally get yourself into trouble if you have:

src/main/java
src/main/netrexx
src/main/javafx

all in the same module, and are trying to compile them all at once.  Unless you have a single compiler that can process all your source at once, this won’t work.  You will be far happier to make multiple modules, use maven to describe the dependencies, and do one language at a time.

What Maven does right

Maven teaches us several important things about what we need from a build system.

• Convention over configuration. Maven strongly encourages developers to put source files, resources, tests, and project metadata in standard locations.  Maven make life very difficult for anyone deviating from the norms.  In general, this is good, helps teamwork and collaboration, and helps for picking up a project cold.
• Defaults just work. Similar to the last entry, if you follow maven standards, then it works fine and has a reasonable assumption of defaults.
• Dependency Management. Probably the single best feature of Maven and modern build tools.  This allows a developer to specify requirements for a project without having to directly specify the physical location of those library dependencies.  This frees the build system to resolve from the network, automatically check for updates, and manage multiple versions of libraries.
• Plugins.  Easy extensibility is key to Maven.  In fact, most of its internals actually exist as plugins, from compilation to Ant support to deployment.  The ability to add new, custom behavior and have it tie into your builds is fantastic.

What Maven does wrong

There are some pretty big gotchas in Maven.  Nothing insurmountable, but frustrating nonetheless.  And before you say “It’s open source, fix it yourself!” know that it is not that simple.  There may be resistance to a certain change, for example, from “the establishment” or some issues and Maven’s solutions are fundamentally tricky.

• Outside the ordinary is very hard. Doing anything beyond the standard use cases requires an arcane knowledge of plugins.  All of a sudden, Maven’s configuration-based approach falls down, and things become easier to do in code — ie, you have to write a plugin.  But don’t think you can use the nifty ant-scripted mojos for that plugin– for example, for compilation, there isn’t a standard way to get the classpath into the ant script.
• Dependency Management. Yep, the same benefit is a detriment.  Or, at the very least, Maven’s handling of it.  Several things fall in here.  The ones I can recall are:
  • Transitive dependencies prove very tricky and make for lots of classloading troubles.  Adding a single dependency to a POM can really change the entire dependency graph of  a project.
  • There is no way to simply say “I want the latest of library XX”.  On the one hand, Maven makes sense here; you should usually know exactly which version of software you are building against.  But there are cases, for example, due to a lack of maturity, where having to know that version number is just an annoyance.  Ivy apparently has an intermediate option here, where you can specify latest, but when a build happens, it records what that version was.
• Long-standing bugs. Maven suffers a bit from what other mature open-source products hit upon.  Whether it is loss of momentum for something that works well enough or that too many are depending on the quirks, there are lots of oddities and issues left in core plugins.  A perfect example is maven-antrun-plugin, which only sets its classpath once per maven run, regardless of how many POMs reference it.
• Documentation bugs. Some of the Maven docs are a bit light on details.  Part of it may be Maven’s built-in site creation, which leads to some sites that have lots of links but are pretty information-sparse.  Other times, many common scenarios are presented, but some of the edge cases are glossed over.  Related:  looking for a list of available Maven properties for use in your POM is tricky because of Maven’s dynamic nature.  But a simple guide describing common ones and how to construct others would be great.

Summary

Generally, what Maven teaches us is that you need to fully embrace Maven to get the full benefit of Maven.  Once you get the hang of the Maven way, things go much more smoothly.  That is not unexpected, but it is uncomfortable.  But there are times where you have to fight — deploying to multiple load-balanced servers, for example, and you end up with some nasty scripts and configuration on top of a previously elegant solution.

The important thing is, though, that Maven does work.  It’s not too much of a bear to get started with, has great defaults, and it has some compelling solutions to annoying problems.

So, can Maven be fixed?  Should it be fixed, or should a build system start from scratch?  More on that next time.

Maven Adoption Curve (Alex Miller, coworker at MetaMatrix)
Maven and Ivy (Ryan Senior, coworker at BJC)

I was trying to think of the ideal build system.  I won’t pretend that I’m going to get there in this post.  Instead, I’ll enumerate what we do every day and what we need from a build system so as to start thinking about the ideal system.

So, to the brainstorming.  Here’s what we do in the regular process of our osgi bundle development:

1. Obtain dependencies
2. Compile java source
3. Compile javafx source
4. Generate metadata / configuration data
5. Make jar files
6. Upload jar files to central repository
7. Upload jar files to deployment location
8. Update remote (OBR) xml file with information about uploaded jar
9. Generate reports about the quality of the source (clover, PMD, checkstyle, etc)
10. check out from CVS / source control
11. check in to CVS / source control
12. tag CVS / source control
13. manage versions and releases

I’m sure there are more.  Obviously, under web development, you’d deal with wars and their deployment.  We have ssh tasks to run code on the server.  We even have some install/run stuff that works pretty well.

You may have guessed we use maven.  Maven is so close, but just not there.  It is frustrating, performs a lot of magic, and just falls down in annoying ways.  We have gotten it to do all of the above, some of it is just not pretty.

Next episode, I’ll go into Maven and what it does well and what it does poorly.  Stay tuned!