Wednesday, January 15, 2014

rubygems with java extensions

this blog tries to show ways how to create gems which have a jar file as part of their gem. sometimes that is called gem with java-extension. but those java extentions have are vendored jar files which is very different from rubygems native extensions.

the tool I use here is the ruby-maven gem which is mainly a proper maven-3.1.x  as a gem and which comes with some ruby bin stub to start it and a small set of rake tasks. it also adds a ruby pom DSL to maven which is part of github.com/tesla/tesla-polyglot. the rake tasks along with some ruby DSL is the topic of this blog post.

for all examples from below you can find a working version in github.com/mkristian/ruby-maven-demo.

simple gem

a minimal gemspec looks like this
Gem::Specification.new do |s|
  s.name = 'simple'
  s.version = "0"
  s.author = 'sample person'
  s.email = [ 'mail@example.com' ]
  s.summary = 'simple gem'
  s.description = 'simple gem with empty jar'
end
assume our ruby files are in ./lib and the java files in ./src/main/java all you need to compile the java sources and add a jar file in lib is Rakefile
require 'maven/ruby/tasks'
now with
$ rake build
we compile the java sources and create lib/simple.jar and pack the gem.

gem with jar dependencies

usually you need some jar dependencies for your java code to work. Gem::Specification does not allow you to declare jar dependencies but it has a requirements list which is a free text to describe requirements for that gem. ruby-maven does 'use' this to declare jar dependencies. the notation used for this is the same used by github.com/mkristian/jbundler. so all we need to change is the gemspec which now looks as such simple.gemspec:
Gem::Specification.new do |s|
  s.name = 'simple'
  s.version = "0"
  s.author = 'sample person'
  s.email = [ 'mail@example.com' ]
  s.summary = 'gem with jar'
  s.description = 'gem with empty jar and jar dependencies'
  s.requirements << "jar org.bouncycastle:bcpkix-jdk15on, 1.49"
  s.requirements << "jar org.bouncycastle:bcprov-jdk15on, 1.49"
end
nothing else is needed and
$ rake build
the will include them into the classpath for compilation. unfortunately rubygems has no way (yet) to provide those jar during runtime. jbundler does include them !

gem with vendored jar dependencies

now we need a way to tell the ruby-maven to include those jar into the gem. for this we need another configuration file Mavenfile:
gemspec :include_jars => true
that looks similiar to a Gemfile. now
$ rake build
will put the dependent jar files into the ./lib directory of the gem. with that the gem is self-contained and can be used is the same manner as all other gems from rubygems.org.

gem depending on jars from other gems

having those jar dependencies declared in the gemspec allows to use those jar dependencies to compile your java code. let's have a look at a gem which depends on the simple gem from above inherited.gemspec:
Gem::Specification.new do |s|
  s.name = 'simple'
  s.version = "0"
  s.author = 'sample person'
  s.email = [ 'mail@example.com' ]
  s.summary = 'gem depending on deps of other gems'
  s.description = 'gem depending on deps of other gems'
  s.add_runtime_dependency 'simple', '=0'
end
now your java code can use the bouncy-castle jars from the simple.gem.

customizations

all customizations go into the Mavenfile like those few example below. the Mavenfile allows you to use the full pom DSL from maven, i.e. you can use any plugin you need for your jar file, define a parent pom, running java unit tests, etc. you also can tell ruby-maven to dump a pom.xml which again can be used in multi module maven setup.

the rake tasks also have a few more tasks
$ rake -T
rake build    # Build gem into the pkg directory
rake clean    # Clean up the build directory
rake compile  # Compile any java source configured - default java files are in src/main/java
rake jar      # Package jar-file with the compiled classes - default jar-file lib/{name}.jar
rake junit    # Run the java unit tests from src/test/java directory
rake maven    # Setup Maven instance
rake push     # Push gem to rubygems.org
there is plenty of space to customize the rake side of things as well.

more

the current situation with vendored jars is probably still doable but there is no control over it and conflicts with those jars will arise sooner or later.

there is no way to find out which gem adds which jar to the jruby classloader and when and which version of the jar. using current jruby (1.7.x) and the scripting container you can add create all kind of classloader issues.

once the gem do declare their vendored jar as dependencies within the gemspec then at least it is possible to have a look the dependency graph/tree for both gems as well for jars.

enjoy !!!

12 comments:

  1. Kristian,

    I followed the directions in the "gems with vendored jars" section, however the dependencies aren't being copied to the lib directory. Please review my project here and let me know if you are able to see if I am making any mistakes: https://github.com/arielvalentin/vtd-xml-ruby/tree/ruby_maven_spike/vtd_xml

    ReplyDelete
    Replies
    1. it works for me: https://gist.github.com/mkristian/8518837#file-gistfile1-txt-L20

      more output would be helpful and to add in the Mavenfile

      properties 'tesla.dump.pom' => 'pom.xml

      which dumps the pom (used to build the gem) as pom.xml - that would be great to see as well.

      Delete
    2. I also needed to add ruby-maven to the Gemfile so bundler does update it to the latest version.

      Delete
    3. Kristian,
      Hmm. Is it perhaps copying the jar to a directory other than ./lib? I do not see the dependent jar (vtd-XML) being copied into the lib directory.

      Delete
    4. Kristian,

      I added the pom.xml here: https://gist.github.com/arielvalentin/8534621

      Delete
    5. the pom.xml is the same as mine - so I wonder if you do see that include
      https://gist.github.com/arielvalentin/8534621#file-gistfile1-txt-L21
      when you run it.

      actually it copies it only to lib directories though you can change the lib directory but that is not configured (as your pom.xml shows)

      when I make the lib directory read-only I see an error when running the build.

      Delete
    6. I added the output to the gist https://gist.github.com/arielvalentin/8534621#file-output-log

      Looks the same to me at https://gist.github.com/arielvalentin/8534621#file-output-log-L49...

      however the jar is not copied to the lib directory https://gist.github.com/arielvalentin/8534621#file-output-log-L66-L67

      Delete
    7. that is strange indeed ;)

      anyways I added some debug output in the gem-maven-plugin to get to the bottom of this. please use a Mavenfile like https://gist.github.com/mkristian/8518837#file-mavenfile

      Delete
    8. I tried your change and ran into an error. I posted it here: https://gist.github.com/arielvalentin/8534621#file-using-extra-debug

      Delete
    9. did you add the snapshot_repository as well ?

      snapshot_repository 'https://oss.sonatype.org/content/repositories/snapshots', :id => 'oss'

      the missing plugin is online https://oss.sonatype.org/content/repositories/snapshots/de/saumya/mojo/gem-extension/

      btw. are you familiar with maven ?

      Delete
    10. Yes, snapshot_repository is there:
      snapshot_repository 'https://oss.sonatype.org/content/repositories/snapshots', :id => 'oss'

      Delete
  2. This comment has been removed by the author.

    ReplyDelete