Integrating Warbler and Buildr into Scala, JRuby, Java and Rails bliss

At Vamosa we’re big fans of the Java Virtual Machine. It allows us to use the right tool for the job and deliver a high-quality consistent product for our end-users, whilst still getting the most of our developers. For years we were a .NET and Java shop. Our GUI developers would work in Visual Studio writing a C# application that via SOAP webservices would talk to the Java-backend. In June 2008 we decided to abandon our .NET Desktop GUI and redevelop and expand its functionality, delivered to the end-user’s browser using HTML+CSS+JavaScript from our Java-backend.

We spend 7months hacking away trying to get Google Web Toolkit to behave before abandoning ship a month ago and switching to Rails. We already had some success building a MRI-based RubyOnRails application called Vamosa Check and Fix. Our GUI developer pool was loving the ease of web development that comes with Rails, and really hated the total lack of productivity from GWT (worthy of a separate post).

Meanwhile I was experimenting with Scala – IMO the Java language reinvented for the 21st century. So there we were steaming ahead with JRubyOnRails, old-skool Java Spring-based code, and sexy-new Scala code. Three languages, one set of JVM byte code. So how do you build and package all this code ???

Your options are:

• Apache Maven – horrible for legacy projects that don’t build according to Maven doctrine.
• Apache Ant + Ivy – might be an option to you.
• Apache Buildr – JRuby-based build system

For us, Apache Buildr had the best fit because its a DSL based-on Rake, which happily runs on JRuby. It provided the dependency management that kept us coming back to Maven (and quickly running away again). It’s JRuby/Rake-based allowing for tight integration with Warbler, the JRubyOnRails WAR-packaging gem. And lastly there’s not a shred of XML in sight. Its a DSL, so the buildfile has a nice declarative feel to it, yet can be modified quickly using some standard Ruby-syntax to provide branching and looping. All the other build systems use XML, and then try and retrofit branching and looping, eg. using elements.

Today we have all our source code in the following folder structure:

project
src
|-- main
|   |-- java
|   |-- resources
|   |-- scala
|   `-- webapp
`-- test
|-- java
|-- resources
`-- scala

rails
|-- app
|-- config
|-- db
|-- doc
|-- lib
|-- log
|-- nbproject
|-- public
|-- script
|-- test
|-- tmp
`-- vendor

and our Apache Buildr buildfile in the root of the project tree looks like this:

require 'buildr'
require 'buildr/scala'
require 'rubygems'
require 'warbler'
 
# define the version of the Vamosa product
VERSION_NUMBER = '3.0.0'
 
# define repositories from which artifacts can be downloaded
repositories.remote << 'http://www.ibiblio.org/maven2/'
repositories.remote << 'http://scala-tools.org/repo-releases'  # define artifacts that are not available from remote repositories  artifact("javax.jms:jms:jar:1.1").from(file("libs/javax.jms.jar"))  # define the artifacts that the project depends on  SCALA         = group('scala-library', 'scala-compiler', 'axiom-dom', :under=>'org.scala-lang', :version=>'2.7.5')
SCALATEST     = [ 'org.scala-tools.testing:specs:jar:1.5.0','org.scalatest:scalatest:jar:0.9.5']
XUNIT         = ["junit:junit:jar:4.4", "org.dbunit:dbunit:jar:2.2.3", "org.mockito:mockito-all:jar:1.7" ]
JDBC_DRIVERS  = ["mysql:mysql-connector-java:jar:5.1.6"]
HIBERNATE     = [ "org.hibernate:hibernate-core:jar:3.3.2.GA",
  "org.hibernate:hibernate-annotations:jar:3.4.0.GA",
  "org.hibernate:hibernate-commons-annotations:jar:3.3.0.ga",
  "org.hibernate:hibernate-search:jar:3.1.0.GA",
  "org.hibernate:hibernate-ehcache:jar:3.3.2.GA",
  "org.hibernate:jtidy-r8:jar:20060801",
  'c3p0:c3p0:jar:0.9.1.2',
  'commons-collections:commons-collections:jar:3.2.1',
  'commons-lang:commons-lang:jar:2.4',
  'net.sf.ehcache:ehcache:jar:1.6.2',
'javax.persistence:persistence-api:jar:1.0']
# DELETED FURTHER ARTIFACTS FOR SAKE OF BREVITY...
 
# now lets do some work
platforms = ["mysql", "oracle", "mssql", "db2"]
platform = "mysql"
desc 'Enterprise Content Governance Platform'
define 'ContentMigrator' do
  project.version = VERSION_NUMBER
  project.group = 'com.vamosa'
  manifest['Copyright'] = 'Vamosa Ltd. (C) 2003-2009'
  compile.options.target = '1.5'
 
  compile.with HIBERNATE, SPRING, COMMONS, LOGGING, CONTENT_PARSER, QUARTZ, J2EE_API, SCRIPTING, SOAP, JFREE_CHART, JAVASSIST, LUCENE, XALAN
  test.with XUNIT, SCALATEST
  test.using :scalatest
 
  # get all the important components from the Rails GUI into the staging directory
  Dir.chdir("rails") do
    puts "Changed current directory to: #{Dir.pwd}"
    warble_cfg = eval(File.open("config/warble.rb") {|f| f.read})
    Warbler::Task.new(:war, warble_cfg)
    Rake::Task['war:app'].invoke
    Rake::Task['war:public'].invoke
  end
  puts "Changed current directory to: #{Dir.pwd}"
 
  # package it up
  package(:war, :file => _("target/#{id}-#{VERSION_NUMBER}-#{platform}.war")).tap do |task|
    task.include 'war/*'
    task.include "src/main/resources/#{platform}.session-factory.xml", :as=>'WEB-INF/session-factory.xml'
    task.include 'src/main/resources/jboss.jms-context.xml', :as=>'WEB-INF/jms-context.xml'
  end
end

The key things we like about this setup are:

1. Easily handling dependency artifacts like the Sun API jars locally. For example we store javax.jms.jar in our Git source repo, in the projects libs/ folder and then point to it using artifact(”javax.jms:jms:jar:1.1″).from(file(”libs/javax.jms.jar”)).
2. Integrate Warbler tasks and cherry-pick the ones you want to run, such as in our case just war:app & war:public but e.g. not war:xml because our web.xml is stored in src/main/webapp/WEB-INF instead.
3. Its Ruby so we can use loops & branching such as:

%w(mssql mysql oracle db2).each do |platform|
  package(:war, :file => _("target/#{id}-#{VERSION_NUMBER}-#{platform}.war")).tap do |task|
    task.include 'war/*'
    task.include "src/main/resources/#{platform}.session-factory.xml", :as=>'WEB-INF/session-factory.xml'
    task.include 'src/main/resources/jboss.jms-context.xml', :as=>'WEB-INF/jms-context.xml'
  end
end

Apache Buildr isn’t perfect. There are still some weird annoyances around resolving transitive dependencies, i.e. when hibernate.jar in turn depends on commons-logging.jar. But if you find yourself missing commons-logging.jar its easily added.
If something doesn’t work they way you think it ought to, you can easily dig into Buildr’s very readable Ruby code, something I couldn’t do with either Maven or Ant, and either customise it or find a quick workaround. You don’t have this black-box barrier between your buildscript and its output.

UPDATE: A nicer way of integrating Warbler and Buildr can be achieved using my Buildr extension, Barbler.