Monday, February 18, 2013

getting up and running with OAuth using signet gem (ruby)

since the readme of github.com/google/signet has a straight forward example of how to use OAuth version 1, it misses the OAuth version 2 completely.being new to OAuth I did havesome problems to get the example from the readme working at first. so here is the slightly modified version of the example

now I was able to use the same for twitter after creating an application on dev.twtter.com

now I wanted to use OAuth2 with my google account so first I created a clientId on code.google.com/apis/console. with that it was possible to get data from my google acount

maybe these scripts might be of any help . . .

Thursday, March 8, 2012

simple portknocking on debian/ubuntu

since I needed some time to get it all nicely working here how I did to hide the ssh port.

knockd setup

install knockd package
apt-get install knockd
added following config /etc/knockd.conf
[options]
  UseSyslog
[openCloseSSH]
  sequence      = 7000,9000,8000
  seq_timeout   = 15
  tcpflags      = syn
  start_command = /sbin/iptables -I INPUT -s %IP% -p tcp --dport 22 -j ACCEPT
  cmd_timeout   = 15
  stop_command  = /sbin/iptables -D INPUT -s %IP% -p tcp --dport 22 -j ACCEPT
to get the daemon running on reboot, edit /etc/default/knockd and change to
START_KNOCKD=1
now you can start the daemon
service knockd start
basically I followed ubuntu help PortKnocking but did use -I INPUT to insert the rule to open the ssh port at the beginning of the chain.

iptables setup

so the firewall shall keep existing ports open and reject any connection to the ssh port. the knoch daemon will open the port for the knocking IP and close it after 15s (see config above).
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp --dport ssh -j REJECT
now to make we want this after reboot as well (see IptablesHowTo):
iptables-save > /etc/iptables.rules
create file /etc/network/if-pre-up.d/iptablesload
#!/bin/sh
iptables-restore < /etc/iptables.rules
exit 0
change the permissions
chmod +x /etc/network/if-pre-up.d/iptablesload
now with nmap you will see no trace of ssh:
$ nmap 192.168.56.101
Nmap scan report for example.net (192.168.56.101)
Host is up (0.0088s latency).
Not shown: 998 closed ports
PORT    STATE SERVICE
80/tcp  open  http

Nmap done: 1 IP address (1 host up) scanned in 0.18 seconds
hope that help !

simple portknocking on debian/ubuntu

since I needed some time to get it all nicely working here how I did to hide the ssh port.

knockd setup

install knockd package
sudo apt-get install knockd
added following config /etc/knockd.conf
[options]
  UseSyslog
[openCloseSSH]
  sequence      = 7000,9000,8000
  seq_timeout   = 15
  tcpflags      = syn
  start_command = /sbin/iptables -I INPUT -s %IP% -p tcp --dport 22 -j ACCEPT
  cmd_timeout   = 15
  stop_command  = /sbin/iptables -D INPUT -s %IP% -p tcp --dport 22 -j ACCEPT
to get the daemon running on reboot, edit /etc/default/knockd and change to
START_KNOCKD=1
basically I followed ubuntu help PortKnocking but did use -I INPUT to insert the rule to open the ssh port at the beginning of the chain.

iptables setup

so the firewall shall keep existing ports open and reject any connection to the ssh port. the knoch daemon will open the port for the knocking IP and close it after 15s (see config above).
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp --dport ssh -j REJECT
now to make we want this after reboot as well (see IptablesHowTo):
iptables-save > /etc/iptables.rules
create file /etc/network/if-pre-up.d/iptablesload
#!/bin/sh
iptables-restore < /etc/iptables.rules
exit 0
change the permissions
chmod +x /etc/network/if-pre-up.d/iptablesload
now with nmap you will see no trace of ssh:
$ nmap 192.168.56.101
Nmap scan report for example.net (192.168.56.101)
Host is up (0.0088s latency).
Not shown: 998 closed ports
PORT    STATE SERVICE
80/tcp  open  http

Nmap done: 1 IP address (1 host up) scanned in 0.18 seconds
hope that help !

Saturday, February 18, 2012

simple guard for rails authorization

due to some recent dicussions about authorization I feel obliged to show how to guard your controller actions with a simple Hash per controller and a single before fitler. the whole will be orthogonal to the application, i.e. no coding required beside setting up this Hash.

the only assumption is that the permissions are organized with user/group model or user/role model, similar what you find on linux/unix systems or with PosixAccounts/PosixGroups on LDAP systems.

in case you need a full fledged authroization framework with DSL and protecting the models instead of the controllers probably github.com/brayn/cancan

setup the permissions per controller

let's have a scaffolded account resource (rails g scaffold accounts name:string) and for each action for the acounts controller you configure a list of groups which are allowed to perform that action.

PERMISSIONS = {
 :accounts => {
   :index => [:root, :supervisor, :accountant],
   :show =>[:root, :supervisor, :accountant],
   :new =>[:root, :accountant],
   :create =>[:root, :accountant],
   :update =>[:root, :accountant],
   :edit =>[:root, :accountant],
   :destroy =>[:root]
 }
}

to check the permissions you need some method like this

def allowed?(action = param[:action], controller = params[:controller])
  perm = PERMISSIONS[controller.to_sym] || {}
  allowed = perm[action.to_sym]
  if allowed
    found = allowed & (current_user_groups || [])
    found.size > 0
  end
end

now the guard is almost in done. just add a before filter and a method which returns the an array of groups (here these are group-names) of the current-user.

before_fitler :guard
def guard
  if allowed?
    raise "permission denied"
  else
    raise "no permission for #{params[:action]}"
  end
end
def current_user_groups
  [:root] if current_user # change this to something real !!
end

the allowed? method from above can be put into the application_helper.rb to have it available in the view templates.

def allowed?(action, controller = params[:controller])
  controller.allowed?(action, controller)
end

the controller defaults to the current controller but the action is compulsory for the view.

add some default permissions

with this allowed? method

def allowed?(action = param[:action], controller = params[:controller])
  perm = PERMISSIONS[controller.to_sym] || {}
  allowed = perm[action.to_sym] || perm[:defaults]
  if allowed
    allowed & (current_user_groups || [])
  end
end

the config is reduced to

PERMISSIONS = {
  :accounts => {
    :defaults =>[:root, :accountant],
    :index => [:root, :supervisor, :accountant],
    :show =>[:root, :supervisor, :accountant],
    :destroy =>[:root]
  }
}

add a superuser

the superuser group is :root so we can change the allowed? method to

def allowed?(action = param[:action], controller = params[:controller])
  perm = PERMISSIONS[controller.to_sym] || {}
  allowed = perm[action.to_sym] || perm[:defaults]
  if allowed
    found = (allowed | [:root]) & (current_user_groups || [])
    found.size > 0
  end
end

this reduces the PERMISSIONS setup to this

PERMISSIONS = {
  :accounts_with_defaults_and_root => {
    :defaults =>[:accountant],
    :index => [:supervisor, :accountant],
    :show =>[:supervisor, :accountant]
  }
}

this is not perfect but for most cases sufficiently compact and readable.

groups

looking at the code then the groups do pop up at two places
  • the PERMISSIONS
  • the current_user_groups method
with such a setup you can make the users and groups totally orthogonal to the application. for example they can be stored as PosixAccounts/PosixGroups in an LDAP system. the allowed? method uses only the controller and action name as parameter. the current_user_groups method can be generic depending on your user/group model and the PERMISSIONS could be a database table as well, allowing to configure permissions on the fly without changing the code !

one advanced example

now we assume a group can be associated with an organization (which is part of domain model).

first we change the allowed? method to check the found allowed groups for agiven controller-action-pair if any if it is allowed to access the current organization. this check is done by a given block.

def allowed?(action = param[:action], controller = params[:controller], &block)
  perm = PERMISSIONS[controller.to_sym] || {}
  allowed = perm[action.to_sym] || perm[:defaults]
  if allowed
    found = (allowed | [:root]) & (current_user_groups || [])
    if block
      permitted = found.detect do |group|
        block.call(group)
      end
      permitted.size > 0
    else
      found.size > 0
    end
  end
end

now we add some extra methods to the application_controller.rb

def guard_organization
  guard do
    GroupsOrganziationsUser.first(:conditions => ["group_name = ? or org_id = ? and user_id = ?", group, current_organization.id, current_user.id]).size > 0
  end
end

def current_organization
  @org ||= Organization.find(params[:org_id])
end

to change the account_controller.rb to use the restricted group the before filters need to reflect this

skip_before_filter :guard
before_filter :guard_organization

more

there is also a gem available which wraps this approach and adds some yaml configuration for the permissions: ixtlan-guard

Sunday, September 25, 2011

JRuby and RubyGems and JavaClassloader

jruby and classloading

rubygems from MRI point if view has nothing do with java classloader. but for jruby you need to deal with this from ruby side. jruby comes with a neat feature(s) that you can 'require "myjar.jar"' and use the java classes more or less in ruby manner. that jar just gets added to the jruby classloader during runtime.

now as a gem developer I might to use certain java libraries and a common way to do it, is to put the jar into the lib directory and just require it when needed. as java developer I first needed to think about: duplicated jar with different version is a source for classloader problems. and if any gem just can add any time a new jar to classloader, there is basically no control over what is going on, and when !

duplicated libraries in the LOAD_PATH was also a common source of annoying errors with rubygems in the time before bundler. bundler works great but for jar dependencies there is no(?) way to do things right now.

example

I set up a a gem and jar which is used inside a java-servlet with embedded jruby. so it is a java application using ruby and rubygems: http://github.com/mkristian/examples/tree/webapp_with_nokogiri_without_maven

the web archive has the following dependencies:
 com.example:web:war:1.0-SNAPSHOT
 +- com.example:app:jar:1.0-SNAPSHOT:compile
 |  +- xerces:xercesImpl:jar:2.10.0:compile
 |  |  \- xml-apis:xml-apis:jar:1.4.01:compile
 |  +- rubygems:gem:gem:0.0.0:compile
 |  |  \- rubygems:nokogiri:gem:1.5.0:compile
 |  \- org.jruby:jruby-complete:jar:1.6.4:compile
 +- javax.servlet:servlet-api:jar:2.5:provided
 +- junit:junit:jar:3.8.1:test
 \- rubygems:jruby-openssl:gem:0.7.4:compile
    \- rubygems:bouncy-castle-java:gem:1.5.0146.1:compile
this info got extracted by the package descriptors (gemspec) and there there are lots of hidden jars:
bouncycastle:bcmail-jdk15:jar:140
bouncycastle:bcprov-jdk15:jar:140
joda-time:joda-time:jar:1.6
org.yaml:snakeyaml:jar:1.8
msv:isorelax:jar:20050913
thaiopensource:jing:jar:20030619
nekohtml:nekodtd:jar:0.1.11
net.sourceforge.nekohtml:nekohtml:jar:1.9.15
xerces:xercesImpl:jar:2.9.1
the last line even shows a jar which is twice in the classloader with different versions xerces:xercesImpl:jar with 2.9.1 and 2.10.0. this is very similar to the pre-bunlder times.

'jar as a gem' approach

one approach was to wrap a jar into gem and use the gem as normal gem. if you install jruby-1.6.x you will get a "experimental" feature which allows you to install any maven artifact (mainly jar files) as gem. great idea BUT the moment you upgrade your rubygems installation on your system that rubygems maven support is gone. and it does not look like this will improve in the near future. so there is no way to include maven-gem dependencies in your gems and publish these gems on rubygems.org

there was/is the attempt to launch a gems repository with all the maven-gem from central maven repository. even if this goes online I do not see how to use it for your regular gems, unless any jruby installation makes sure you are using both rubygems.org and maven-gem repository as source for you gem installations. and it has some small short comings due to the difference rubygems and maven resolve dependencies.

even if one of those work it is only a partial solution of the problem. if you want to run your rails or sinatra application on java application server where you have a more complex classloader hierarchy you end up with classloader having "hidden" jars. the latest nokogiri release recommends to "comment out" some require statement in certain environments (if I remember the mail right).

bundler for jar files ?

one possible solution is to deal with the jar dependencies "outside" of ruby. a bit like have a tool setting up your environment variable CLASSPATH and you just execute jruby as usual but have all the jar you need ready to use. I will show a slightly different way of doing it: ruby-maven gem.

just define you jar dependencies in a Mavenfile like (taken from http://github.com/mkristian/jibernate/blob/master/Mavenfile):
jar 'org.hibernate:hibernate-core', '3.3.2.GA'
jar 'org.hibernate:hibernate-annotations', '3.4.0.GA'
jar 'org.hibernate:hibernate-tools', '3.2.4.GA'
jar 'javax.transaction:jta', '1.1'
jar 'javassist:javassist', '3.8.0.GA'
jar 'mysql:mysql-connector-java', '5.1.9'
jar 'postgresql:postgresql', '8.4-701.jdbc4'
jar 'com.h2database:h2', '1.2.138'
jar 'org.apache.derby:derby' ,'10.5.3.0_1'
jar 'org.hsqldb:hsqldb', '2.0.0'
this almost looks like a Gemfile for jars, but the version is compulsory.

now we need to get that info into the gemspec of gem, then ruby-maven can use both the usual part of gemspec to setup gems (via bundler) and the jar dependencies to setup the classpath. there is a little feature in gemspec which allows to add requirements. jar dependencies are a sort of requirements - just add in your gemspec:
s.requirements.put File.read('Mavenfile')
that basically adds human readable requirements to you gemspec and do it in a way that it can be user/parsed by ruby-maven.

back to example using ruby-maven

this example uses nokogiri-maven gem instead of nokogiri and jruby-openssl-maven instead of jruby-openssl and instead of jruby-complete it uses jruby-core and jruby-stdlib. the later pulls in all its  dependencies via maven and the "maven" gems do not come with the jar files and do not require them instead using the requirements declaration of the gemspec to specify the jar dependencies. with this you get following overview (rmvn dependency:tree)
 com.example:web:war:1.0-SNAPSHOT
 +- com.example:app:jar:1.0-SNAPSHOT:compile
 |  +- xerces:xercesImpl:jar:2.10.0:compile
 |  |  \- xml-apis:xml-apis:jar:1.4.01:compile
 |  +- rubygems:gem:gem:0.0.0:compile
 |  |  \- rubygems:nokogiri-maven:gem:1.5.0:compile
 |  |     +- msv:isorelax:jar:20050913:compile
 |  |     +- thaiopensource:jing:jar:20030619:compile
 |  |     +- nekohtml:nekodtd:jar:0.1.11:compile
 |  |     \- net.sourceforge.nekohtml:nekohtml:jar:1.9.15:compile
 |  +- org.jruby:jruby-core:jar:1.6.4:compile
 |  |  +- org.jruby.joni:joni:jar:1.1.6:compile
 |  |  +- org.jruby.extras:jnr-netdb:jar:1.0.3:compile
 |  |  +- org.jruby.ext.posix:jnr-posix:jar:1.1.8:compile
 |  |  +- org.jruby.extras:bytelist:jar:1.0.8:compile
 |  |  +- org.jruby.extras:constantine:jar:0.6:compile
 |  |  +- org.jruby.jcodings:jcodings:jar:1.0.5:compile
 |  |  +- org.jruby.extras:jffi:jar:1.0.8:compile
 |  |  +- org.jruby.extras:jaffl:jar:0.5.11:compile
 |  |  |  +- org.jruby.extras:jffi:jar:native:1.0.6:runtime
 |  |  |  +- asm:asm:jar:3.2:compile
 |  |  |  +- asm:asm-commons:jar:3.2:compile
 |  |  |  |  \- asm:asm-tree:jar:3.2:compile
 |  |  |  \- org.jruby.extras:jnr-x86asm:jar:1.0.0:compile
 |  |  +- org.yaml:snakeyaml:jar:1.8:compile
 |  |  +- jline:jline:jar:1.0:compile
 |  |  \- joda-time:joda-time:jar:1.6:compile
 |  \- org.jruby:jruby-stdlib:jar:1.6.4:compile
 +- javax.servlet:servlet-api:jar:2.5:provided
 +- junit:junit:jar:3.8.1:test
 \- rubygems:jruby-openssl-maven:gem:0.7.4:compile
    \- bouncycastle:bcmail-jdk15:jar:140:compile
       \- bouncycastle:bcprov-jdk15:jar:140:compile
now you have the control of your jars back. there is only one xercesImpl and you can "update" joda-time, snackyaml, bouncycastle, etc with backward compatible versions and use them instead inside your warfile.

since the whole thing was born within the jibernate project see the demo https://github.com/mkristian/dm-hibernate-adapter/tree/master/demo there (not sure if it is in a working state) and look the Gemfile which just adds dm-hibernate-adapter and you have all the jars in place if you use ruby-maven (rmvn rails server) to start the server:

 . . . .
 +- rubygems:dm-hibernate-adapter:gem:0.1pre:compile
 |  +- org.hibernate:hibernate-core:jar:3.3.2.GA:compile
 |  |  +- antlr:antlr:jar:2.7.6:compile
 |  |  +- commons-collections:commons-collections:jar:3.1:compile
 |  |  \- dom4j:dom4j:jar:1.6.1:compile
 |  |     \- xml-apis:xml-apis:jar:1.0.b2:compile
 |  +- org.hibernate:hibernate-annotations:jar:3.4.0.GA:compile
 |  |  +- org.hibernate:ejb3-persistence:jar:1.0.2.GA:compile
 |  |  \- org.hibernate:hibernate-commons-annotations:jar:3.1.0.GA:compile
 |  +- org.hibernate:hibernate-tools:jar:3.2.4.GA:compile
 |  |  +- org.beanshell:bsh:jar:2.0b4:compile
 |  |  +- freemarker:freemarker:jar:2.3.8:compile
 |  |  \- org.hibernate:jtidy:jar:r8-20060801:compile
 |  +- javax.transaction:jta:jar:1.1:compile
 |  +- javassist:javassist:jar:3.8.0.GA:compile
 |  +- mysql:mysql-connector-java:jar:5.1.9:compile
 |  +- postgresql:postgresql:jar:8.4-701.jdbc4:compile
 |  +- org.apache.derby:derby:jar:10.5.3.0_1:compile
 |  \- org.hsqldb:hsqldb:jar:2.0.0:compile
 . . . .

so basically you get "normal" workable gems with some extra info to setup a classloader/classpath. you can do that setup with ruby-maven or any other possible way. with this you regain control over your classloader, can exclude certain jars or replace them with a different version (maven allows all this), build war-files suitable for package them into an enterprise archive (ear-file), etc.

things to come

bundler offers the binstubs which is a convenient way of calling ruby commands in bundler context, the next release of ruby-maven will create binstubs which uses bundler for the gems and sets up a classpath for the jars and execute jruby in that context.

finally

I published the nokogiri-maven and jruby-openssl-maven gems to rubygems.org download and have a look. there are still more things to consider like sometimes the JRubyClassLoader is the ContextClassLoader some times not (standalone jruby vs. war-context), etc.

feedback is welcome.

Saturday, April 30, 2011

a dragon

(  )   /\   _                 (     
    \ |  (  \ ( \.(               )                      _____
  \  \ \  `  `   ) \             (  ___                 / _   \
 (_`    \+   . x  ( .\            \/   \____-----------/ (o)   \_
- .-               \+  ;          (  O                           \____
                          )        \_____________  `              \  /
(__                +- .( -'.- <. - _  VVVVVVV VV V\                 \/
(_____            ._._: <_ - <- _  (--  _AAAAAAA__A_/                |
  .    /./.+-  . .- /  +--  - .     \______________//_              \_______
  (__ ' /x  / x _/ (                                  \___'          \     /
 , x / ( '  . / .  /                                      |           \   /
    /  /  _/ /    +                                      /              \/
   '  (__/                                             /                  \


I found this somewhere with nmap and it is to remember of an old friend of mine.

Tuesday, April 5, 2011

GWT with RubyOnRails backend glued together with Maven/JRuby

let's first look a the different parts involved:
  • GWT (Google Webtoolkit) comes with "generators" which focus on ANT as build system
  • Ruby on Rails has its way of starting a new application and scaffold models and their views
  • JRuby allows to use java side by side with ruby and allows to run a rails application in a java servlet engine
  • Maven is a build tool mainly for java project. but with a set of jruby maven plugins (j)ruby project can be managed almost as well. with the advent of Roo the maven support for GWT drastically improved.
  • Ruby-Maven Gem tries to bridge the ruby and maven world by offering a more concise commandline interface. as gem it is easy to install from ruby point of view.
  • RestyGWT is a GWT generator for REST services and JSON encoded data transfer objects.
walkthrough of an example
first you need ruby-maven installed. when you use it with ruby (MRI) then you also need java >1.5 installed or use jruby. please update the gem if already installed:

jruby -S gem install ruby-maven

or
ruby -S gem install ruby-maven
using MRI will improve startup time drastically.

create a rails application
with ruby-maven there are new commands rmvn, gwt and jetty-run with this you can create a new rails application with GWT setup like this
gwt new my_app com.example
with this you have already a GWT application with EntryPoint and could be started but does not do much. the class layout within the GWT client package:
src/main/java/com/example/client/
  ├── client
  │   ├── ActivityPlaceActivityMapper.java
  │   ├── ActivityPlace.java
  │   ├── managed
  │   │   ├── ActivityFactory.java
  │   │   ├── MyAppModule.java
  │   │   └── MyAppPlaceHistoryMapper.java
  │   └── MyAppEntryPoint.java
  └── MyApp.gwt.xml

scaffold a resource with model, controller, gwt-model, etc
go into the created application (or any existing rails app)
cd my_app
now you can scaffold a model (currently two steps are required for this)
rmvn rails generate scaffold user name:string
this creates a rails like structure following the GWT activity/place/view pattern:
src/main/java/com/example/client/
├── activities
│   └── UserActivity.java
├── models
│   └── User.java
├── places
│   ├── UserPlace.java
│   └── UserPlaceTokenizer.java
├── restservices
│   └── UsersRestService.java
└── views
    ├── UserViewImpl.java
    ├── UserView.java
    └── UserView.ui.xml
finally you can run the application.

start the GWT development shell

before running the application you need to migrate the database
rmvn rake db:migrate
and start the GWT development shell
gwt run
the start url for the new resource is

to run the application with default webrick you need first to compile the GWT part
gwt compile
the compiler will output the GWT app in public/my_app and then start the webrick. the start url is http://localhost:3000/MyApp.html#users
rmvn rails server
or if you prefer the jetty server with http://localhost:8080/MyApp.html#users
rmvn jetty:run
build the war file
rmvn package 
the war file can be found in the target directory

more
you can also use proper maven3 instead of ruby-maven but the command-line-interface of maven is much more clumsy though the functionality is the same.

more up to date infos are here: https://github.com/mkristian/rails-resty-gwt

enjoy !
any comment and/or suggestion is welcome.