Barbler integrates itself between the build and packaging stages of the Apache Builder lifecycle and makes calls into Warbler to automate WAR-file creation. Now Warbler does a really good job for packaging standalone Rails apps. Unfortunately I needed something more integrated into our application build process, that pulls in our Spring Framework-based Java code, Scala code, and Rails application and produces a single WAR-file containing all dependent libraries, Rails code, XML deployment descriptors and Java class files. Apache Builder does everything other than the Rails-packaging. Barbler steps to provide that missing step.

Create a barbler.rb file in your project folder, which also contains your buildfile and copy the following contents into it:

# Barbler 
# is an Apache Builder extension to integrate the JRuby Warbler gem.
# For tips on how to use Barbler checkout http://www.denofubiquity.com/ruby/barbler/
# 
# This code is licensed under the Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
# Please distribute Barbler code with this code intact.
# (c) Ijonas Kisselbach 2009
 
require 'warbler'
 
module Barbler
  include Extension
 
  first_time do
    # Define task not specific to any projet.
    desc 'Warbles Rails sourcecode tree into a staging folder'
    Project.local_task('warble' => 'build') { |name| "Warbling #{name}" }   
  end
 
  before_define do |project|    
    project.task('warble'=>project.task('build'))
    project.group ||= project.parent && project.parent.group || project.name
    project.version ||= project.parent && project.parent.version    
  end
 
  # To use this method in your project:
  #   warble(:rails => path_to(:rails), :tasks => [:app, :public])
  def warble(*args)
    options = args.pop
    rails_path = options[:rails]
    warble_tasks = options[:tasks]
 
    # Define the warble task for this particular project.
    Rake::Task.define_task 'warble' do |task|
      # get all the important components from the Rails GUI into the staging directory
      puts "Warbling #{rails_path}"
      Dir.chdir(rails_path) do
        warble_cfg = eval(File.open("config/warble.rb") {|f| f.read})
        Warbler::Task.new(:war, warble_cfg)
        warble_tasks.each {|task| Rake::Task["war:#{task}"].invoke}
      end
    end
  end
 
end
 
class Buildr::Project
  include Barbler
end

Add the folllowing line to the top of your buildfile:

warble(:rails => path_to(:rails), :tasks => [:app, :public])

You can then define your warble task using the following line

warble(:rails => path_to(:rails), :tasks => [:app, :public])

whereby the first parameter is a path string to where your Rails code is located. You may locate your code in src/main/rails in which case you’d use

warble(:rails => path_to(:source, :main, :rails), :tasks => [:app, :public])

The second parameter is the list of Warbler tasks that you’d like to have executed. See the Warbler documentation for more help, or check out the Warbler source code – it’s very readable.

We spend 7months hacking away trying to get Google Web Toolkit to behave before abandoning ship a month ago and switching to Rails. We already had some success building a MRI-based RubyOnRails application called Vamosa Check and Fix. Our GUI developer pool was loving the ease of web development that comes with Rails, and really hated the total lack of productivity from GWT (worthy of a separate post).

Meanwhile I was experimenting with Scala – IMO the Java language reinvented for the 21st century. So there we were steaming ahead with JRubyOnRails, old-skool Java Spring-based code, and sexy-new Scala code. Three languages, one set of JVM byte code. So how do you build and package all this code ???

Your options are:

• Apache Maven – horrible for legacy projects that don’t build according to Maven doctrine.
• Apache Ant + Ivy – might be an option to you.
• Apache Buildr – JRuby-based build system

For us, Apache Buildr had the best fit because its a DSL based-on Rake, which happily runs on JRuby. It provided the dependency management that kept us coming back to Maven (and quickly running away again). It’s JRuby/Rake-based allowing for tight integration with Warbler, the JRubyOnRails WAR-packaging gem. And lastly there’s not a shred of XML in sight. Its a DSL, so the buildfile has a nice declarative feel to it, yet can be modified quickly using some standard Ruby-syntax to provide branching and looping. All the other build systems use XML, and then try and retrofit branching and looping, eg. using elements.

Today we have all our source code in the following folder structure:

project
src
|-- main
|   |-- java
|   |-- resources
|   |-- scala
|   `-- webapp
`-- test
|-- java
|-- resources
`-- scala

rails
|-- app
|-- config
|-- db
|-- doc
|-- lib
|-- log
|-- nbproject
|-- public
|-- script
|-- test
|-- tmp
`-- vendor

and our Apache Buildr buildfile in the root of the project tree looks like this:

require 'buildr'
require 'buildr/scala'
require 'rubygems'
require 'warbler'
 
# define the version of the Vamosa product
VERSION_NUMBER = '3.0.0'
 
# define repositories from which artifacts can be downloaded
repositories.remote << 'http://www.ibiblio.org/maven2/'
repositories.remote << 'http://scala-tools.org/repo-releases'  # define artifacts that are not available from remote repositories  artifact("javax.jms:jms:jar:1.1").from(file("libs/javax.jms.jar"))  # define the artifacts that the project depends on  SCALA         = group('scala-library', 'scala-compiler', 'axiom-dom', :under=>'org.scala-lang', :version=>'2.7.5')
SCALATEST     = [ 'org.scala-tools.testing:specs:jar:1.5.0','org.scalatest:scalatest:jar:0.9.5']
XUNIT         = ["junit:junit:jar:4.4", "org.dbunit:dbunit:jar:2.2.3", "org.mockito:mockito-all:jar:1.7" ]
JDBC_DRIVERS  = ["mysql:mysql-connector-java:jar:5.1.6"]
HIBERNATE     = [ "org.hibernate:hibernate-core:jar:3.3.2.GA",
  "org.hibernate:hibernate-annotations:jar:3.4.0.GA",
  "org.hibernate:hibernate-commons-annotations:jar:3.3.0.ga",
  "org.hibernate:hibernate-search:jar:3.1.0.GA",
  "org.hibernate:hibernate-ehcache:jar:3.3.2.GA",
  "org.hibernate:jtidy-r8:jar:20060801",
  'c3p0:c3p0:jar:0.9.1.2',
  'commons-collections:commons-collections:jar:3.2.1',
  'commons-lang:commons-lang:jar:2.4',
  'net.sf.ehcache:ehcache:jar:1.6.2',
'javax.persistence:persistence-api:jar:1.0']
# DELETED FURTHER ARTIFACTS FOR SAKE OF BREVITY...
 
# now lets do some work
platforms = ["mysql", "oracle", "mssql", "db2"]
platform = "mysql"
desc 'Enterprise Content Governance Platform'
define 'ContentMigrator' do
  project.version = VERSION_NUMBER
  project.group = 'com.vamosa'
  manifest['Copyright'] = 'Vamosa Ltd. (C) 2003-2009'
  compile.options.target = '1.5'
 
  compile.with HIBERNATE, SPRING, COMMONS, LOGGING, CONTENT_PARSER, QUARTZ, J2EE_API, SCRIPTING, SOAP, JFREE_CHART, JAVASSIST, LUCENE, XALAN
  test.with XUNIT, SCALATEST
  test.using :scalatest
 
  # get all the important components from the Rails GUI into the staging directory
  Dir.chdir("rails") do
    puts "Changed current directory to: #{Dir.pwd}"
    warble_cfg = eval(File.open("config/warble.rb") {|f| f.read})
    Warbler::Task.new(:war, warble_cfg)
    Rake::Task['war:app'].invoke
    Rake::Task['war:public'].invoke
  end
  puts "Changed current directory to: #{Dir.pwd}"
 
  # package it up
  package(:war, :file => _("target/#{id}-#{VERSION_NUMBER}-#{platform}.war")).tap do |task|
    task.include 'war/*'
    task.include "src/main/resources/#{platform}.session-factory.xml", :as=>'WEB-INF/session-factory.xml'
    task.include 'src/main/resources/jboss.jms-context.xml', :as=>'WEB-INF/jms-context.xml'
  end
end

The key things we like about this setup are:

1. Easily handling dependency artifacts like the Sun API jars locally. For example we store javax.jms.jar in our Git source repo, in the projects libs/ folder and then point to it using artifact(”javax.jms:jms:jar:1.1″).from(file(”libs/javax.jms.jar”)).
2. Integrate Warbler tasks and cherry-pick the ones you want to run, such as in our case just war:app & war:public but e.g. not war:xml because our web.xml is stored in src/main/webapp/WEB-INF instead.
3. Its Ruby so we can use loops & branching such as:

%w(mssql mysql oracle db2).each do |platform|
  package(:war, :file => _("target/#{id}-#{VERSION_NUMBER}-#{platform}.war")).tap do |task|
    task.include 'war/*'
    task.include "src/main/resources/#{platform}.session-factory.xml", :as=>'WEB-INF/session-factory.xml'
    task.include 'src/main/resources/jboss.jms-context.xml', :as=>'WEB-INF/jms-context.xml'
  end
end

Apache Buildr isn’t perfect. There are still some weird annoyances around resolving transitive dependencies, i.e. when hibernate.jar in turn depends on commons-logging.jar. But if you find yourself missing commons-logging.jar its easily added.
If something doesn’t work they way you think it ought to, you can easily dig into Buildr’s very readable Ruby code, something I couldn’t do with either Maven or Ant, and either customise it or find a quick workaround. You don’t have this black-box barrier between your buildscript and its output.

UPDATE: A nicer way of integrating Warbler and Buildr can be achieved using my Buildr extension, Barbler.

Getting started with JRuby+Terracotta requires a bit of trial and error as its not as well documented as good old Java+Terracotta. The only post you’re likely to find is one by Jonas Boner (see below). During subsequent revisions of both Terracotta as well as JRuby, the example had stopped working. These files bring that example update to date for JRuby 1.3.1 and Terracotta 3.0.1.

You can download the revised source code from my github account. You will need installs of both JRuby and Terracotta with JRUBY_HOME and TC_HOME pointing to the base folders of both products respectively, e.g.

export JRUBY_HOME=$HOME/java/jruby-1.3.1
export TC_HOME=$HOME/java/terracotta-3.0.1

Once these environment variables have been setup you can start a Terracotta server, followed launching multiple clients by typing:

./chat.sh

Background

The key to fixing the example was fixing the java.lang.NoClassDefFoundError: com/tc/object/event/DmiManager, caused by the references to com.tc.object.bytecode.Manager in the terracotta.rb file:

WRITE_LOCK = com.tc.object.bytecode.Manager::LOCK_TYPE_WRITE
READ_LOCK = com.tc.object.bytecode.Manager::LOCK_TYPE_READ
CONCURRENT_LOCK = com.tc.object.bytecode.Manager::LOCK_TYPE_CONCURRENT

Replacing the above fragment with:

WRITE_LOCK = 2
READ_LOCK = 1
CONCURRENT_LOCK = 4

and the whole example springs to life. My next problem to solve is that of Rubifying the Workmanager examples from chapter 11 of the “Definitive Guide to Terracotta” book.

Useful Links

• Clustering JRuby with Terracotta
• Incoming Revolution: Clojure + Terracotta
• My github pages

The first TechMeetup exceeded our expectations. Three great speakers created some vigorous debates. When we asked you to “bring yourself, your experiences, and your opinions”, you certainly heeded our call.

We also got some great feedback and with that in mind we’re making some small changes. We’re going to have one less speaker, allowing for more debate and more informal chat in between speaker presentations. We’ll also have more bottles of water available.

This month’s speaker’s topics are as follows:

Dave Sapien - New challenges in UI. A look at new technologies, the User Interfaces that control them and what challenges we can face with new thinking.

John Gallagher - Smarter than the Average Application. Users aren’t stupid and lazy, but much modern software is. That’s John’s opinion. He’ll be telling us why and the AI techniques he’s using to make Lapsus, his time tracker, smarter than the average application.

Like last month, we’re holding this month’s meetup in room M329 of James Weir Building at Strathclyde Uni. So bring yourself, your experiences, and your opinions. We’ll bring the drinks, the pizza, and the speakers.

See you then & there,

Sam, Ijonas & Heidi.

There are a ton of events for entrepreneurs to meet investors and bankers and VC’s but let’s face it, we’re tech, so let’s talk about tech. No name badges, no business models, just the growth of our eco-system and the advance of our tech – be it web apps, software, mobile apps, hardware, games… There’s an overlap across many of these and it’s always interesting to hear what people are hacking together, know about or have worked on before.

TechMeetup is just that – a space for everyone to meet up, talk about some of the cool stuff we’re doing, or get help on our projects from others.  It’s been running in Edinburgh for several months, with a good turn out, nice folks and great speakers – held together with the magic of pizza and beer.

On the 22nd April, Tech Meetup Glasgow will start and it’d be great to have you along.  Three excellent speakers are lined up, covering a variety of topics – from running a ticketing startup, to an introduction to developing for Android. Kick-off is at 7pm in Room M329 of the James Weir building on Stratchlyde University’s campus.

With your help and feedback, we are going to work hard to make this functional for everyone around us – a monthly event to allow you to share news, demo the technologies and projects that excite you, meet people with complimentary skillsets that may want to help out, and generally get to know others in the tech community.

So if that sounds good to you, be great to see you there.  Bring yourself, your experiences and your opinions with you.

Sam, Ijonas & Heidi.

Lastly, visit our site http://www.techmeetup.co.uk for a backgrounder on TechMeetup including past talks.

Amazon EC2 uses private/public keys files for root user authentication but you want to use those credentials as infrequently as possible. As RubyOnRails users, we are used to setting up a deploy user which we use to run Apache 2 and Phusion Passenger under. We use that same deploy user to connect to our github repositories and pull in code updates. All pretty much standard stuff.

When we using off-the-shelf Ubuntu 8.0.4 AMIs we couldn’t find any nice Capistrano recipes to setup that initial deploy user and enabling key-based login to that deploy user using our own personal keys.

The following recipe does just that:

1
2
3
4
5
6
7
8
9
10
11
12
13

  desc "uploads id_rsa.pub to the EC2 instance's deploy users authorized_keys2 file"
  task :bootstrap_deploy_user do
    system "ssh -i #{aws_private_key_path} root@#{domain} \"groupadd admin\""
    system "ssh -i #{aws_private_key_path} root@#{domain} \"useradd -d /home/#{user} -s /bin/bash -m #{user}\""
    system "ssh -i #{aws_private_key_path} root@#{domain} \"echo #{user}:#{password} | chpasswd\""
    system "ssh -i #{aws_private_key_path} root@#{domain} \"usermod -a -G admin deploy\""
    system "ssh -i #{aws_private_key_path} root@#{domain} \"mkdir /home/#{user}/.ssh\""
    for key in ssh_options[:keys]
      system "cat  #{key}.pub | ssh -i #{aws_private_key_path} root@#{domain} \"cat >> /home/#{user}/.ssh/authorized_keys2\""
    end
    system "ssh -i #{aws_private_key_path} root@#{domain} \"chown -R #{user}:#{user} /home/#{user}/.ssh\""
    system "scp -i #{aws_private_key_path} config/deploy_sudoers root@#{domain}:/etc/sudoers"
  end

You’ll need to setup Capistrano variables as you can see to make this work. My .caprc file contains the following definition:

set :aws_private_key_path, "/Users/ijonas/.ec2/ec2keypair.pem"

Our config/deploy.rb contains the following variables:

set :use_sudo, false
set :user, 'deploy'
set :password, 'xxxxx'
set :application, "yyyyy.vamosa.com"
set :domain, "yyyyy.vamosa.com"
set :deploy_to, "/var/www/apps/#{application}"

The approach is a little bit ‘hackish’ but it works for EC2, and keeps your Capistrano setup as close as possible to the best-practice.

At the time of writing, the current versions of MySQL is 5.1 and the MySQL gem is currently at version 2.7.3.

We were unable to get this combination to work together and ended up reverting to MySQL 5.0 and MySQL gem 2.7.1.

gem install -v "2.7.1" mysql

I don’t think you’re missing anything substantial from either MySQL 5.1 or the 2.7.3 gem.

apache, opensource, documents, metadata, database, json, javascript, REST, addressability, horizontal scaling, erlang.

So I found myself at Railscamp UK on the weekend without a ruby+rails topic to really talk about during the presentations part of the camp. I’m still a noob wrt. Rails, you see. Realising that CouchDB is relevant to most web developers because most developers deal with document-based content at some point and that Ruby, Rails, and Erlang are kindda like the current bratpack (in a John Hughes-sense) of computer languages, I thought it would be a great little topic for a 15-minute presentation.

So I thought I share the slides and some of the code I hacked up on the weekend. First of all the slides…

(click me)

You can play about with some sample code that was expanded upon after the presentation. Here’s a little helper class to interface into CouchDB using JSON and HTTP:

require 'rubygems'
require 'httparty'
require 'json/pure'
require 'find'
 
class CouchHack
  include HTTParty
  base_uri 'http://127.0.0.1:5984'
  format :json
 
  def update( path, document )
    options = { :body => JSON.generate(document) }
    puts self.class.put(path, options )
  end
 
  def get(path)
    self.class.get(path)
  end
 
  def create(db_path, document )
    options = { :body => JSON.generate(document) }
    self.class.post(db_path, options )
  end
 
  def define_view(db_path, design_doc_name, views )
    document = { "views" => views }
    options = { :body => JSON.generate(document) }
    self.class.put("#{db_path}/_design/#{design_doc_name}", options)
  end
end

Once that’s in place you can use the CouchHack class by defining a view like so:

ch = CouchHack.new
gem_views = {
  "small_gems" =>
    {
      "map" => "function(doc) { if (doc.length < 3000) emit(doc.path, doc) }",
    },
  "large_gems" =>
    {
      "map" => "function(doc) { if (doc.length > 3000) emit(doc.path, doc) }",
    }
}
ch.define_view("/railscamp", "gems", gem_views)

This design document, as is its known in CouchDB parleance, defines two views (think ‘baked queries’), to return a list of small gems – those less than 3000 bytes – and a list of large gems – those larger than 3000 bytes.

So where do these gems come from and how do they end up in CouchDB ? That’s what the next and last little fragment of Ruby code does.

Find.find('/Library/Ruby/Gems/1.8/gems/' ) do |f|
  if File.file?(f) and f.ends_with?('.rb' )
    begin
      gem_contents = IO.read(f)
      gem_document = { "path" => f, "length" => gem_contents.length, "content" => gem_contents }
      ch.create( "/railscamp", gem_document )
    rescue Exception => bang
      puts "Skipping #{f}"
    end
  end
end

So what happens is that the last little fragment traverses the filesystem starting at /Library/Ruby/Gems/1.8/gems/ and looks for *.rb files. When it finds one it POSTs that whole document including the file length to CouchDB, where it becomes part of the resultset of one of the pre-defined views. These views can be accessed through the following two URLs (assuming the ‘railscamp’ DB was used).

• http://localhost:5984/railscamp/_view/gems/large_gems
• http://localhost:5984/railscamp/_view/gems/small_gems

Feel free to post comments and ask questions.

Enjoy.

P.S. and if you get a chance to go to a *camp conference, GO! They’re an amazing format and you’ll learn soooo much!.

In that period I’ve been experimenting and pondering about Ruby and of course RubyOnRails. During such ponderings I’ve pondered why so many experienced Java programmers are jumping ship onto duck-typed languaged such as Ruby and Python (of which I’m a massive fan).

I’m going to write a bunch of posts about Ruby and the impact on solving certain class of computing science problems, that I believe are now more readily or more elegantly solvable – hence my interest in the Ruby language.

But first I thought I’d share with you, what I think is possibly the easiest, quickest, and dirtiest way of getting a taste of Ruby and Rails.

First of all… tooling.

• If you’re on Mac OS X, buy a copy of TextMate and download Rails 2.1.
• If you’re on Linux, download NetBeans IDE for Ruby on Rails, and download 2.1.
• If you’re on Windows, get InstantRails, followed by NetBeans again… if you have to. But I would recommend getting an Ubuntu VM with Ruby 1.8.6, MySQL 5, Rails 2.1 and NetBeans IDE installed.

Now… I wouldn’t bother buying any books until later…  I started off with books, but I find screencasts far better in conveying relevant & concise information.

With that in mind I recommend you purchase the following screencasts:

• Rails from Scratch – Part I
• Rails from Scratch – Part II

Both are from PeepCode, last about an hour, and cost $9 each. Well worth it. I recommend starting with Rails as opposed to “Pure Ruby”, because that way you get to see some of Ruby’s magic applied in the familiar setting of building a webapp. Coming from Java, it’ll be quiet a refreshing experience – no more xml deployment descriptors.

Having watched both screencasts, you can start building Rails websites with some competence. However, you’ll run into quiet a few “How do I do that ?” questions early on, such as “How do I authenicate visitors to my site?”. And before you head off to Amazon or your local bookstore to buy a Rails Cookbook… I recommend you take a look at Railscasts, which is a free (donations welcome) video podcast providing really helpful solutions to common probems.

Next… you may want to take a look at how Ruby works underneath the covers and how some of its magic manifests itself in such advancements as domain-specific languages (DSL).

Having spent many years programming Java and C#, I found that the recent advancements in test-driven development, mock objects, continuous integration are all worthy pursuits but often these advancements fought with the friction caused by stronly typed languages, i.e. all that test code I was writing was pretty verbose and clunky. Ruby’s take on the aforementioned pursuits and approaches, is positively refreshing and enjoyable.

If you’ve ever questioned why the hell you should write a unit test before the implementation itself, take a look at RSpec. RSpec is another DSL, like Rails is a DSL, this time to facilitate the writing of specification, i.e. the expected behaviour of a software programme. First you describe how your app should behave, then you write the behaviour… It takes a few moments to get your head around why you would want to do so… but then the penny drops… and once again its magical! For a great intro into RSpec I can recommend another Peepcode screencast:

• RSpec Basics

Now if you’re anything like me… Java programmer or C# programmer… many years writing Enterprise applications… lots of XML… You probably know most of what there is to know about OO programming. Classes, interfaces, and object instances are your staple day-in day-out building blocks. You know about inner classes and anonymous methods but only use those if the moon is particularly blue.

Ruby’s take on “OO” borrows from many different OO languages and to fully appreciate some of the magic and how to cast your own Ruby spells, you need to look at Ruby’s object-oriented basics. To that end, I recommend downloading all five episodes (2.5 hours) of:

• Ruby Object Model and Metaprogramming

from the guys at Pragmantic Bookshelf, which will set you back a grant total of $25.

Consuming all this information will take some time… If you’ve got a “day job” that needs to pay the bills… you’ll probably take a week or two to really sift through and appreciate what amounts to a couple of days of screencasts.

All the screencasts I’ve mentioned are of the highest quality and provide amazing value for money… Far more so than the books I’ve purchased.

I’m the CTO for Vamosa, a software company headquartered in Glasgow, with an rapidly expanding office in Boston (across the pond). We’re a 30-man strong company with a small development team in Glasgow that needs expansion. Our software allows clients to migrate and transform huge websites, from one content management system to another.

The software is built on both J2EE and .NET, using the usual suspects in componentry: Spring, xUnit, Hibernate, etc. etc. Woven through all of this is a big helping of both Jython and IronPython.

I’m looking for an opinionated, technology-obsessed, fun-loving, and inspiring team-leader, who knows his way around the Java and .NET spaces.

Working at Vamosa, you’ll occupy a fundemental role at Vamosa. We’re a product company, and you’ll be in charge of the products development. So we want you to bring ideas, be opinionated, tell us where we’re going wrong, but also be amazed by some of the stuff we’ve already done, because we think we pretty clever.

So if you are that person, contact me through email at ijonas.kisselbach@vamosa.com.

Cheers,
Ijonas.

P.S. Please, no agencies!