maven-release-plugin with GIT

Problem with the #maven release plugin

You may recognize yourself with the following use-cases:

• We have several big modules/repositories with many branches. Per repository we have one parent pom and 50+ children poms. This is for a large JEE project (400 K lines of code) that it is delivered to several different clients.
• We create feature branches, bug fixes, integration branches, release branches etc. We have many versions of the same application deployed to different production environments.
• Every time a new branch is created, the version tag in the parent pom and all children poms need to be updated
• When these branches need to be merged all these poms cause conflicts and they need to be manually resolved, because some times there are other changes in the poms besides just the version change. It is a major problem for us. 
• We change the version on every branch because we have a continuous integration server that builds every branch and runs regression tests against every branch, therefore the versions must be different per branch.
• Being able to keep the new version isolated to one file (the parent pom) really makes sense here. (Child pom inherits version from parent property)
• We need to use SNAPSHOTS as we cannot have every developer build the whole system. Developers need to use artifacts built by #TeamCity for their feature branch and they cannot be changing to the last build version they depend on every time a new development build is created.

The #maven release plugin create a lot of issues that have been around for many many years.

Update Maven pom version on GIT checkout in TeamCity

Here is a solution to the following problems

• Deriving #Maven artifact version from #GIT branch,
• Update pom version on #GIT checkout automatically,
• Add the ability to use Pull request with Apache #Maven.

You have a workflow requirement that require you to have the artifact version of a module externally defined from the current branch in #GIT.

For example

You want to start working on a new feature branch “feature-memory-improvement”, so you branch from master a new branch named feature/feature-memory-improvement

Having unique snapshot is a something you need to share your code using a #Maven repository, so you may want to have into the branch all pom.xml version changed to

<version>FEATURE-MEMORY-IMPROVEMENT-SNAPHOTS</version>

changing all your pom.xml and doing a technical commit  will create merge conflicts when using pull request!

One solution, while not perfect is to do the following:  You can add a separate execution to run a goal which will change the version of the POM automatically in the #Maven reactor. This small script will do it¨

Keep your software stack up to date with the Apache Maven Versions Plugin

It is not unusual in a project to have a huge number of third party artifacts and Plug-in. Apache #Maven help you keep track of them, along with their transitive dependencies.

But how do you know when a new version of an artifact is available?  This is where the #Maven Versions plug-in come hand in.

The Versions Plug-in is used when you want to manage the versions of artifacts in a project’s POM.

By running

mvn versions:display-dependency-updates

in any Apache #Maven project or modules, you’ll get for example (we have a lot of 25 #Maven modules, here is only one presented as an example, the list being too long)

[INFO] -------------------------------------------------------------------------------------------------- 
[INFO] Building Unnamed - com.innoveo:skye-services-api:jar:2.2.0-M-06 
[INFO] -------------------------------------------------------------------------------------------------- 
[INFO] 
[INFO] The following dependencies in Dependency Management have newer versions: 
[INFO]   junit:junit ............................................. 4.4 -> 4.8.1 
[INFO]   log4j:log4j ......................................... 1.2.15 -> 1.2.16 
[INFO]   org.springframework:spring ...................... 2.5.6 -> 2.5.6.SEC02 
[INFO]   org.springframework:spring-test ............... 2.5.6 -> 3.0.4.RELEASE

I was fighting today against Apache Maven release plugin with multi modules projects

I was fighting today against the maven-release-plugin of #maven, solving complicated  errors in a row. As I am convince I made all possible errors,   I think it is worse to compile my findings here to help others 🙂

Maven Release Plugin

This plugin is used to release a project with #Maven, saving a lot of repetitive, manual work. Releasing a project is made in two steps: prepare and perform.

My approach to speed up things is always to define a small project (in a sandbox SVN root) that is compiling and running in 10 seconds to make some test before trying to make it run on our bigger Innoveo Skye(tm) product (35 modules)

I always have 2 projects prepared:

• One TestSimpleProject: one #Maven project with no code
• One TestComplexProject: one #maven project and 2 #Maven sub modules

For the reader that can not wait here is the running command line from TeamCity to be put in Build Runner Goals

release:clean release:prepare release:perform -Dusername=xxxxxxx -Dpassword=yyyyyy

How to check commit comments on SVN Commit

If you are using Subversion/CVS, you might have come across the issue where multiple developers working on a set of files are committing without any comments.

Subversion’s hook scripts provide a powerful way to associate actions with repository events. For example, the pre-commit hook allows you to check — and possibly abort — a transaction before it actually gets committed. I will provide you now two easy Unix bash scripts  that avoid bad developer behaviors:

• The first one “checkCommentNotEmpty.sh” is for checking that nobody submit an empty SVN commit comment
• The second one “checkCommentSyntax.sh” is able to enforce SVN commit comment pattern using regular expressions

Subversion and the importance of svn:ignore for MAVEN multi modules

Maven 2 builds can be quite slow, especially if you are not following the best practices of using Maven in Eclipse. For example if you are not removing all /target directories from version control!

If you let /target directories going into version control, the following happen:

• Each full build of the workspace,
• Each partial build of modules,

Continuous Build for Joomla

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. This article is a quick overview of Continuous Integration summarizing the technique and its current usage. Martin Fowler about continuous integration

Things get clearer in my mind…I will use a set of Open Sources software to reach this ambitious goal.

Brainstorming

Maintain a Single Source Repository.

Either Subversion running locally on windows/Linux, but I will stay with www.joomlaCode.org at the beginning

Automate the Build

• I am currently evaluation Maven for PHP but already have ANT script to build components and plugin for #Joomla 1.5 (they are not generic enough at the moment)

Make Your Build Self-Testing

• PHP UNIT tests if available with be run at this stage using PHP command line.
• Selenium to automate components testing across many platforms, as it runs in many browsers and operating systems, and can be controlled by many programming languages and testing frameworks.
• Some standard utilities to perform standard #Joomla! processes: login, logout for Apache ANT or Selenium
• I will develop either a ANT plugin or a set of Selenium test cases for deploying and removing
  • component: install, publish, remove
  • plugin: install, publish, remove
• All these utilities will be available under GPL v3 and as such free to reuse, and improve.

Everyone Commits Every Day

• This is reserved to developer having a www.joomlaCode.org account and part of the development team.

Every Commit Should Build the Mainline on an Integration Machine

I will use TeamCity continuous build server as it is free for personal use (limited to 20 concurrent build), run on Windows and #Linux but require a java VM to run (may not be wanted in a PHP environment). Anyway with ANT, it will be possible to use another build server like Cruise Control or PHP Under Control.

Keep the Build Fast

That is an objective 🙂

Test in a Clone of the Production Environment

Joomla! Build farm

• I can imagine a set of #Joomla instances, ideally 5 of each version, aka #Joomla! 1.5.3 to #Joomla! 1.5.8 and #Joomla! 1.0.10 to 1.0.15
• Joomla instances will be recreated at build time (files and databases), that mean #Joomla! will get newly installed and removed in case of successful build
• All #Joomla! instances will be running with XAMPP, ideally on port not available to the outside world for security reasons

Make it Easy for Anyone to Get the Latest Executable

Successful build (Artifacts) are only available if build is successful. Team City provide this with less effort (configuration)

Everyone can see what’s happening

• A guest account will be available or a free public area with limited access to see the result of builds.
• RSS feeds, emails and Instant messaging (Jabber) out of the box for end users or developers!

Automate Deployment

That will be, auto publish to some demo site in a configurable way. At the moment, at http://demo.waltercedric.com and http://demo2.waltercedric.com for me 🙂

Final words

• I will provide a ready to use package for Windows and #Linux and all scripts, so anybody will be able to run it also on your own.
• Critical part will be documented in my WIKI at http://wiki.waltercedric.com direct link HERE

It seem that nobody is providing such a package as I am after only one day at the top of search results in Google “continuous build joomla”

Continuous build server for Joomla!

Starting from now on, in order to

• Increase quality of my components and other (JoomlaComment :-))
• Reduce time between releases,
• Avoid subtle or recurrent issues

I will set up a continuous integration  build server at 

http://continuousbuildserver.waltercedric.com

Continuous integration describes a set of software engineering practices that speed up the delivery of software by decreasing integration times.

• Maintain a code repository, the code will stay at JoomlaCode.org subversion
• Automate the build, with Maven for PHP/Ant and either teamcity or phpundercontrol.org
• Make my build self-testing with PHP Unit and Selenium IDE
• Everyone commits every day,
• Every commit (to mainline) will be built with the help of triggers,
• The build will be fast,
• Test are done in production environment, by deploying code to  old and new version of #Joomla! 1.0 and 1.5 “not web accessible” (for obvious security reasons),
• It will be easy to get the latest deliverables by visiting http://continuousbuildserver.waltercedric.com
• Everyone will see the results of the latest build by visiting http://continuousbuildserver.waltercedric.com
• Deliverables will be automatically deployed.

In short (for all non developer), this program will

• Monitor any changes of code
• Trigger a build,
• Deploy the build to different versions of #Joomla! 1.5 (5 versions should be enough)
• Run a set of tests against these #Joomla! versions and
• Make the result available to all of you.

Soon as there is enough test cases, it will be safe to download any new release from there.