Scriptable dataset 1.0 released

Finally I found some time to continue working on a little project of mine, scriptable dataset, which makes it possible to use script snippets (in languages such as JRuby, Groovy etc.) in your DBUnit dataset files. Imagine for instance, you were building a web shop application and wanted to insert an order issued 14 days ago into your order table. Using the scriptable dataset, you can do exactly that:

1 2 3 4

<?xml version="1.0" encoding="UTF-8"?> <dataset> <order product="CD player" orderdate="jruby:DateTime::now() - 14"/> </dataset>

In comparison to the original release the 1.0 version contains basically some refactorings: ScriptableDataSetConfig got a new constructor, the Java 6 ServiceLoader mechanism is used to detect standard script invocation handlers, all dependencies were updated to the current versions and the entire code base was cleaned up a little bit.

Many thanks to Kevin Hutson, who contributed a test case for using Groovy as scripting language in a dataset file. It's really great to see, how GitHub is encouraging people to contribute to open source projects by making forking and merging that easy.

To allow for using the scriptable dataset in Maven based applications, I set up a Maven repository at Google code, which hosts the project's artifacts. Just add this repo to your pom.xml or settings.xml as shown below:

1 2 3 4 5 6 7 8

... <repositories> <repository> <id>http://gunnarmorling-maven-repo.googlecode.com/svn/repo/</id> <url>http://gunnarmorling-maven-repo.googlecode.com/svn/repo/</url> </repository> </repositories> ...

Then add scriptable dataset as dependency as well as a binding for sl4j, which is used for logging purposes:

1 2 3 4 5 6 7 8 9 10 11 12

... <dependency> <groupId>de.gmorling</groupId> <artifactId>scriptable-dataset</artifactId> <version>1.0</version> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-jdk14</artifactId> <version>1.5.8</version> </dependency> ...

An example for using the scriptable dataset is shown in the project's unit test ScriptableDataSetTest.

So feel free to give it a try. I'd be glad on any comments: do you like the idea of scripting in dataset files in general, what use cases could you think of, what place for improvement do you see?

A scriptable data set for DBUnit

DBUnit is a great tool when it comes to the management of test data. When doing data driven unit tests, database tables have to be populated with predefined records, that can be accessed by the code under test. DBUnit allows the specification of such test records by the means of data set files as in the following example:

1 2 3 4

<?xml version="1.0" encoding="UTF-8"?> <dataset> <location num="6" addr="Webster Street" date="2009-02-16"/> </dataset>

The data that can be inserted with such data set files is pretty static by nature. DBUnit offers a bit of dynamic with its ReplacementDataSet, which allows to replace custom-defined placeholders (e.g. [SYSDATE]) with other values (e.g. new Date()). But I am aware of no possibility to use more flexible expressions to insert things like "now - 14 days" into a date field for instance.

So I set out to create a data set implementation, that allows to use scripting language expressions in its fields. Using this ScriptableDataSet, data set files like the following can be loaded and processed:

1 2 3 4

<?xml version="1.0" encoding="UTF-8"?> <dataset> <location num="jruby:12/2" addr="jruby:'Webster Street'.reverse" date="jruby:DateTime::now() - 14"/> </dataset>

Having the power of a scripting language at your hands any dynamic expressions thinkable can be used in the fields of a data set – also "now - 14 days" is no problem any more.

How does it work? The scriptable data set leverages the scripting language integration API as defined by JSR 223 ("Scripting for the Java Platform"). An implementation of that API is part of Java 6, but it generally can be used with previous Java versions as well.

To use a certain language in a data set, a JSR 223 compatible engine for that language has to be added to the class path. Loads of such engines can be found at the scripting project at java.net. Having added the engine for your preferred scripting language (or by using the Rhino engine for JavaScript, which is delivered with the Java 6 JDK), a ScriptableDataSet can be created as follows:

1 2 3 4 5 6 7 8

List<Class<? extends ScriptInvocationHandler>> handlers = new ArrayList<Class<? extends ScriptInvocationHandler>>(); handlers.add(TestInvocationHandler.class); IDataSet dataSet = new ScriptableDataSet( new FlatXmlDataSet(new File("dataset.xml")), new ScriptableDataSetConfig("jruby", "jruby:", handlers));

Sporting DBUnit's decorator scheme, a ScriptableDataSet is created wrapping another IDataSet. Additionally a ScriptableDataSetConfig object has to be provided, that specifies

• the name of the scripting language to be used as understood by the JSR 223 ScriptEngineManager ("jruby")
• a prefix that shall precede all fields containing scripts in that language ("jruby:")
• an optional list of ScriptInvocationHandlers, that can be used to pre-process (e.g. to add common imports) and post-process scripts (e.g. to convert results into data types understood by DBUnit)

So if you like the idea of the scriptable data set, don't hesistate and get your hands on it at its git repo over at github. Do you think, it is useful at all, and if so, which scenarios for its usage could you think of?

Using JRuby with Maven

One of the features new to Java 6 is its built-in support for scripting languages (an introductory article on that topic can be found here), which enables you to load and execute programs written in a scripting language directly from within your Java program.

The scripting support in Java 6 is realized by providing an implementation of JSR 223 ("Scripting for the Java Platform"). While JavaScript is directly supported by the JDK itself, any other scripting language can be integrated as well by simply adding a JSR 223 compatible scripting engine to the class path.

The first place to look for JSR 223 scripting engines is https://scripting.dev.java.net/, where engines for Ruby, Python, Groovy and a lot of other languages can be found.

To give it a try, I wanted to include the JRuby engine into a Maven based project. That engine can be found in the Maven repo at java.net, but unfortunetaly its pom.xml is somewhat defective, as it contains the dependency script-api:javax.script, which neither exists in the java.net repository nor any other one I am aware of.

But when running on Java 6, it isn't required at all – as the scripting API is part of the JDK. So I excluded the dependency in the pom.xml of my own project, which reads as follows:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>org.gm</groupId> <artifactId>jruby-scripting</artifactId> <packaging>jar</packaging> <version>1.0-SNAPSHOT</version> <name>jruby-scripting</name> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.5</version> <scope>test</scope> </dependency> <dependency> <groupId>org.jruby</groupId> <artifactId>jruby</artifactId> <version>1.1.6</version> </dependency> <dependency> <groupId>com.sun.script.jruby</groupId> <artifactId>jruby-engine</artifactId> <version>1.1.6</version> <exclusions> <exclusion> <groupId>script-api</groupId> <artifactId>javax.script</artifactId> </exclusion> </exclusions> </dependency> </dependencies> <repositories> <repository> <id>maven2-repository.dev.java.net</id> <name>Java.net Repository for Maven 2</name> <url>http://download.java.net/maven/2/</url> </repository> </repositories> <build> <plugins> <plugin> <artifactId>maven-compiler-plugin</artifactId> <version>RELEASE</version> <configuration> <source>1.5</source> <target>1.5</target> </configuration> </plugin> </plugins> </build> </project>

Having fixed that, we can try the scripting API by evaluating a simple Ruby script, that receives a variable provided by the hosting Java program and returns the obligatory String "Hello, jruby!" ;-):

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

package org.gm.jrubyscripting; import static org.junit.Assert.*; import javax.script.ScriptEngine; import javax.script.ScriptEngineManager; import org.junit.Test; public class HelloJRuby { @Test public void helloJRuby() throws Exception { String engineName = "jruby"; ScriptEngineManager manager = new ScriptEngineManager(); ScriptEngine jRubyEngine = manager.getEngineByName(engineName); assertNotNull(jRubyEngine); jRubyEngine.put("engine", engineName); assertEquals( "Hello, jruby!", jRubyEngine.eval("return 'Hello, ' + $engine + '!'")); } }

But what to do, if you are not running on Java 6? After some more searching I finally managed to find the missing dependency in the repo of the Mule project, but with groupId and artifactId interchanged.

From there it can be added to the project, for example using a separate Maven build profile to be activated on JDK versions < 1.6:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

... <profiles> <profile> <activation> <jdk>[1.3,1.6)</jdk> </activation> <dependencies> <dependency> <groupId>javax.script</groupId> <artifactId>script-api</artifactId> <version>1.0</version> </dependency> </dependencies> <repositories> <repository> <id>dist.codehaus.org/mule</id> <name>Mule Repository for Maven 2</name> <url>http://dist.codehaus.org/mule/dependencies/maven2/</url> </repository> </repositories> </profile> </profiles> ...

Redmine on JRuby and Glassfish

Redmine is a project management software written in Ruby. It offers a lot of interesting features such as an issue tracker, wiki functionality and news feeds. This post will show up the steps required to install Redmine and run in it on JRuby using Glassfish application server.

This tutorial is based on the following assumptions:

• Ubuntu is being used as OS
• Java is installed on your system
• Glassfish is installed (%GLASSFISH_HOME% will be used in the following to refer to the installation directory)
• All downloads go to /tmp

Install JRuby

In order to run Redmine, a Ruby runtime has to be installed first. We will use JRuby 1.1.4 (the current version at the time of this writing - 1.1.5 - didn't work, probably due to this bug). Download it and add JRuby to your path as follows:

$ wget dist.codehaus.org/jruby/jruby-bin-1.1.4.tar.gz
$ cd /opt
$ sudo tar -xzvf /tmp/jruby-bin-1.1.4.tar.gz
$ sudo ln -s jruby-1.1.4 jruby
$ export PATH=$PATH:/opt/jruby/bin

Verify that JRuby is set up properly by issuing:

$ jruby -v

This should yield in:

jruby 1.1.4 (ruby 1.8.6 patchlevel 114) (2008-08-28 rev 7570) [i386-java]

Download required gems

Having JRuby up and running, we need to install the ActiveRecord JDBC MySQL adapter for Rails. Furthermore it is recommended to install JRuby's Open SSL support as well. Finally, we need Warbler, which will allow us to package the Redmine application as a WAR archive. Using the RubyGems package manager, this is easy:

$ sudo gem install jruby-openssl activerecord-jdbcmysql-adapter warbler

All required dependent gems (such as activerecord-jdbc-adapter) will automatically be downloaded by RubyGems.

Install and set up MySQL database server

Next, the database to be used by Redmine needs to be set up. We will be using MySQL, as it is a very common companion for RoR apps, though other database servers should do the trick as well.

Install the server:

$ sudo apt-get install mysql-server

Login into MySQL:

$ mysql -u root -p %YOUR_PASSWORD%

Create a database:

mysql> CREATE DATABASE redmine_production character set utf8;

Create a database user:

mysql> GRANT ALL ON redmine_production.* TO 'redmine'@'localhost' IDENTIFIED BY 'redmine';

Repeat the latter two steps for schemas redmine_test and redmine_development and leave the MySQL shell.

mysql> exit

Create data source in Glassfish

In order to set up a MySQL based data source in Glassfish, the server has to be provided with the MySQL JDBC driver. Ensure that Glassfish is stopped for the following. So let's download the driver, un-tar it and copy it into the server's lib dir:

$ wget
dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.7.tar.gz/from/http://sunsite.informatik.rwth-aachen.de/mysql/
$ tar -xzvf mysql-connector-java-5.1.7.tar.gz
$ sudo cp /mysql-connector-java-5.1.7/mysql-connector-java-5.1.7-bin.jar %GLASSISH_HOME/lib/

Now start up Glassfish:

$ %GLASSISH_HOME/bin/asadmin start-domain domain1

Create a connection pool and a JDBC resource (if you prefer a more visual way of doing such things, help can be found here):

$ %GLASSISH_HOME/bin/asadmin create-jdbc-connection-pool
--datasourceclassname
com.mysql.jdbc.jdbc2.optional.MysqlConnectionPoolDataSource --restype
javax.sql.DataSource --property
"User=redmine:Password=redmine:URL=jdbc\:mysql\://localhost\:3306/redmine_production"
jdbc/RedminePool
$ %GLASSISH_HOME/bin/asadmin create-jdbc-resource --connectionpoolid jdbc/RedminePool jdbc/Redmine

Install and set up Redmine

Now it's time to install Redmine itself (refer to the official installation guide for further information). So download and extract the archive:

$ wget rubyforge.org/frs/download.php/39477/redmine-0.7.3.tar.gz
$ tar -xzvf redmine-0.7.3.tar.gz

Create a new database environment "production_setup" and the database config file:

$ cp config/environments/production.rb config/environments/production_setup.rb
$ cp config/database.yml.example config/database.yml

Edit config/database.yml:

production:
  adapter: jdbc
  jndi: jdbc/Redmine

production_setup:
  adapter: jdbcmysql
  database: redmine_production
  host: localhost
  username: redmine
  password: redmine
  encoding: utf8

development:
  adapter: jdbcmysql
  database: redmine_development
  host: localhost
  username: redmine
  password: redmine
  encoding: utf8

test:
  adapter: jdbcmysql
  database: redmine_test
  host: localhost
  username: redmine
  password: redmine
  encoding: utf8

The "production" environment is configured to use the data source, which we previously created within the Glassfish server. No credentials are stored here, as they are part of the connection pool's configuration. Note the special environment "production_setup" which we'll use now to initialize the database:

$ rake db:migrate RAILS_ENV="production_setup"
$ rake redmine:load_default_data RAILS_ENV="production_setup"

Now we can test the application using Webrick:

$ jruby script/server -e production_setup

Package and deploy Redmine

Running the application on Webrick is fine for testing purposes, but now let's deploy Redmine on Glassfish. For this target, the Warbler gem comes into play. It takes a Rails application and creates a WAR archive from it. Besides the application itself, JRuby and all required gems are packaged into the archive as well. Therefore, this WAR is fully self-contained and can be deployed on every web container or application server. First, we have to create the Warbler configuration:

$ warble config

This will create the file config/warble.rb. Edit this file, find the line beginning with "config.dirs ..." and replace it with the following:

config.dirs = %w(app config lib log vendor tmp extra files lang)

This will cause the named directories to be packaged into the resulting WAR. Find the line beginning with "#config.gems += " and uncomment it, allowing for the gems active-record-jdbcmysql-adapter and jruby-openssl to be packaged into the WAR as well.

Having configured warbler, the web archive can be created. Go to the application's root dir and enter:

$ warble

The resulting WAR file can now be deployed to the app server. Either do this within the admin console or just by copying it to Glassfish's auto deploy folder:

$ cp redmine-0.7.3.war %GLASSFISH_HOME%/domains/domain1/auto-deploy

Upon success, Redmine can be accessed at http://localhost:8080/redmine-0.7.3. Click "Sign in" in the upper right, log in using "admin"/"admin" as user/password and you should see your personal Redmine page as depicted below.

Typically, you would now create a project or other user accounts, which can be done under "Administration".