A Journeyman's Journal

Just released Griffon Validation plugin v0.9. This is the release corresponding to the 0.9.5 Griffon core. Since there were significant amount of changes introduced in Griffon 0.9.5, regarding plugin framework, therefore this release is not backward compatible with older Griffon versions.

Bug Fixes:

Other than some minor improvements in AST transformation logic, this release fixes one major defect which prevents validator from being properly executed if a mvc group has been destroyed and recreated later.

New Project Repository:

Griffon validation plugin has been moved to GitHub.

https://github.com/NickQiZhu/Griffon-Validation-Plugin


Enjoy the new release, and feel free to send me a pull request if you see missing features in this plugin.

Labels: griffon, groovy, open source, validation

About 2 months ago we setup a experimental internal cloud environment for one of our client using pure commodity servers and open source technology (OpenNebula 2.2.1 on Ubuntu 11.04 Enterprise Server). Since then the proof-of-concept environment has been working pretty well for our client hosting internal QA, Development, as well as some IT virtualized appliances. However the initial setup was based on SSH transport which works well in the small environment but struggles with large VMs. The main disadvantages of SSH based configuration are:

• Performance with large VM image - With ssh transport each VM image will have to be transferred to the cloud node from the repository by the controller, and then retrieved back from the node (if save flag is on). Obviously this process takes a lot of bandwidth with large images.
• No live migration - Live migration requires shared VM image storage between cloud nodes. Live migration is the key feature allowing any cloud operator to maintain the no downtime illusion and elasticity.
• Vulnerable to data loss - If you are like our client who is running a cloud on top of commodity servers or even PCs, then most likely your cloud node server does not have RAID disks or much of other hardware redundancies. Since ssh based solution transfers the image to the node then it lets the node to run the VM using local hard disk, thus if you suffer a hard drive failure while running the VM you might loose all or part of your VM data.

To solve these problems OpenNebula and Linux community have already provided many solutions, in this post I would like to introduce the most straightforward alternative solution with NFSv4. With NFS a server can export a part of it's own file system to client machines whom in turn can then mount the exported file system locally and use it just as any other local file system. Originally we expected the change to be quite simple and straightforward however it turned out there were a few interesting hurdles some OpenNebula specific some Ubuntu. I hope this post can help you implementing similar solution with little effort.


First: NFS Server

Since this NFS server will be your centralized storage server therefore a little bit extra investment in this server would not hurt. We recommend to at least have 2 physical NICs, dual power supply, and RAID 1/5/10 disks.

(In our environment we chose software RAID 5 with 3 disks and 1 hot spare)

Install NFS4 Server

Labels: daylog, nfs, opennebula, ubuntu

Just released v0.8.1 for Griffon Validation Plugin, a minor bug fix release to address the multi-error rendering issue recently discovered by Frank Shaw. If you are using several constraint especially custom constraints on your model, it is highly recommended to upgrade to this release immediately since the error rendering engine currently does not handle multi-error scenario correctly. To upgrade please use the following command:

griffon install-plugin validation

Labels: griffon, gvalidation, plugin, release note

I am happy to announce the release of Griffon Validation plugin v0.8. In this release the following enhancement and new features were introduced:

1. Upgraded to match Griffon core 0.9.3 release

For v0.8 release you will need Griffon 0.9.3+ to run it.


2. Upgraded to i18n plugin 0.4 release

Thanks to the new multi-bundle support in i18n plugin now when using validation plugin you no longer need to manually create default error messages. The validation plugin is now shipped with these default messages and will automatically fall back to it if no other localized messages are found.

3. Real time validation support

Now validation plugin can automatically trigger validation for your model beans if you include the realTime flag in your annotation.

@Validatable(realTime=true)

class MyModel{

  ....

}

The real time validation feature is implemented relying on the property change support therefore any property value change, for example triggered by bind{}, will invoke the validation logic associated with that particular property. The actual validation is performed in a separate thread using GriffonApplication.execAsync() before it switches back to EDT for error rendering. Currently this flag only works with Griffon MVC model beans if you apply it to a regular POGO object the flag will be ignored.

As usually you can upgrade your validation by simply issue the following command:

griffon install-plugin validation

And please refer to the plugin Wiki for detail information of the plugin.

Labels: griffon, plugin, release note, validation

I have decided today to terminate Hydra Cache project. What is Hydra Cache anyway?

About 2 and half years ago, a few of my friends and I, inspired by our experience with one of our client at that time and Amazon's Dynamo design, started an open source project to build a distributed elastic cache called Hydra Cache. However after the inception of this project I personally was involved in a number of startup business and open source projects, thus spreading myself too thin and even till today still only have the Hydra Cache project in a pre-release 1.0RC1 stage (although really close to be release ready).

Of course this slow development of a promising project only had myself to blame. When I was searching for the reason why I want to terminate the project despite it being so close to release, I had many answers such as competent alternative open source options (JBoss InfiniSpan) now available, but I think the most important reason is that I have lost my drive and obsession with this idea a long time ago. All the things I did after that point for one reason or another were not really helping the project but rather just me dragging my feet to admit the fate of the project. Nevertheless this project has taught me many lessons, both technical and non-technical, however most importantly after this experience I realized the best way to do an outstanding job with open source project is to do it with your passion and have fun. The moment you realize the flame of your passion is gone and you are no longer having fun with it, you should stop dragging it alone. Since we don't get to be paid for open source contribution, so if you are not having fun then why even bother :)

Labels: hydra cache, open source, termination note

Recently I was in a process of setting up an open source cloud computing environment. The experience I gained through this project was very interesting to say the least. The goal for me is to create a ultra reliable cloud computing environment with the potential to handle not only IaaS requirement but also PaaS as well as future SaaS goals. On top of that this environment should be able to utilize commodity servers or even desktop grade machine to form the cloud. After much research and experiment, eventually I have decided to go with OpenNebula due to its rich feature set and well thought-out integration with different hypervisors as well as public cloud vendor.

In this post, I would like to share my findings while setting up OpenNebula 2.2.1 on Ubuntu 11.04 Enterprise Server to maybe make your life a bit easier if you decided to go down a similar path. The hypervisor technology we decided to use is KVM however OpenNebula works with Xen and VMWare as well.

(In this guide I will focus on how not why. To find out more about why certain steps were performed please refer to OpenNebula document)

Cloud Controller Setup

1. Getting OpenNebula 2.2.1

I basically just downloaded the tar ball from http://downloads.dsa-research.org/opennebula/

2. Requisite Software Installation

sudo apt-get install ruby sqlite3 libxmlrpc-c3-0 openssl ssh

sudo apt-get install ruby-dev rubygems rake make libxml-parser-ruby libxml2 libxslt1.1 libxml-ruby libxslt-ruby libnokogiri-ruby1.8

3. Setup Directory Structure

$ tree /srv
/srv/
|
`-- cloud
  |-- one
  `-- images

4. Create OpenNebula User and Group Account

groupadd cloud
useradd -d /srv/cloud/one -g cloud -m oneadmin
sudo passwd oneadmin
$ id oneadmin
uid=1001(oneadmin) gid=1001(cloud) groups=1001(cloud)


5. Prepare for Build

sudo apt-get install libsqlite3-dev libxmlrpc-c3-dev scons g++ ruby libopenssl-ruby libssl-dev ruby-dev make rake rubygems libxml-parser-ruby1.8 libxslt1-dev

6. Fix bug #265 (Optional)

Currently OpenNebula automatically deletes VM image that fails to start for whatever reason. This behavior might not be desirable for some installation including ours. This is currently filed as a bug/improvement for 3.0 release, but you can apply the temporary patch following the instruction on this page

http://dev.opennebula.org/issues/265

7. Build

~$ wget
~$ tar xzf
~$ cd one-2.0
~/one-2.0$ scons -j2
....
scons: done building targets.
~/one-2.0$ ./install.sh -d /srv/cloud/one

8. Add OpenNebula environment variables

Edit ~/.profile and add the following lines (or .bashrc if you want to use bash)

export PATH=$PATH:/var/lib/gems/1.8/bin:/srv/cloud/one/bin
export ONE_LOCATION=/srv/cloud/one
export ONE_XMLRPC=http://localhost:2633/RPC2
export ONE_AUTH=/srv/cloud/one/.one/one_auth

9. Create the user's OpenNebula config directory:

~$ mkdir ~/.one

10. Configure Authentication File

# Add this one liner to one_auth file match oneadmin user's password
~$ vim /srv/cloud/one/.one/one_auth

oneadmin:oneadmin

11. Prepare to install Sunstone (WebUI for OpenNebula)

sudo gem install json sinatra thin rack sequel

* Ubuntu does not add gem to your path so add the following directory to your path /var/lib/gems/1.8/bin

At this point your controller server is pretty much setup now. Next we will create a cloud node server.


Cloud Node Setup

1. Verify CPU Virtualization Support

Since KVM relies on CPU virtualization support before we install KVM we need to make sure it will work with your hardware.

egrep '(vmx|svm)' /proc/cpuinfo

If there is any match, it means you CPU supports KVM otherwise you can try VMWare hypervisor or Xen.

2. Install KVM

If you have already selected KVM during Ubuntu server installation then you really don't have to do anything here, otherwise:

sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder virt-manager virtinst bridge-utils


3. Remove default NAT bridge

By default if you selected KVM during Ubuntu installation it will also install the default NAT bridge. But usually this NAT bridge is not needed for server type setup and compromise performance, so I will remove the bridge here.

# virsh net-destroy default
# virsh net-undefine default
# service libvirt-bin restart
# ifconfig


4. Setup bridge for VM network

In a server type environment you want your VM to be accessible from anyone on the network, to achieve that you need to bridge the network from the host. Modify the /etc/network/interfaces file.

DHCP

auto eth1
iface eth1 inet manual

auto virbr1
iface virbr1 inet dhcp
hostname 
bridge_ports eth1
bridge_fd 9
bridge_hello 2
bridge_maxage 12
bridge_stp off

Static IP

auto eth1
iface eth1 inet manual

auto virbr1
iface virbr1 inet static
   address 10.128.129.43
   network 10.128.129.0
   netmask 255.255.255.0
   broadcast 10.128.129.255
   gateway 10.128.129.1
   bridge_ports eth1
   bridge_fd 9
   bridge_hello 2
   bridge_maxage 12
   bridge_stp off

5. Setup Software required by OpenNebula

sudo apt-get install ruby ssh

6. Create Directory Structure

$ tree /srv
/srv/
|
`-- cloud
  |-- one

7. Create OpenNebula User and Group (with the same G/UID)

groupadd --gid 1001 cloud
useradd --uid 1001 -g cloud -G libvirtd -d /srv/cloud/one oneadmin
sudo passwd oneadmin – setup the password to oneadmin
chown -R oneadmin:cloud cloud

* Make sure the gid and uid match the ones on the controller

8. Setup password-less SSH access from controller to node for oneadmin user

ssh-keygen
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ cat ~/.ssh/config
Host *
StrictHostKeyChecking no

* If you are using SSH transport driver with OpenNebula, you need to make sure there is bi-directional password-less ssh access from controller to node and vice versa.

Finally now you are done setting up a basic cloud environment. Try start OpenNebula on the controller:

~/one/bin/one start

You should be able to add the host by using onehost command:

onehost create im_kvm vmm_kvm tm_ssh

Hope you have found this guide helpful and having fun with OpenNebula :) In future post I will try to cover how to run OpenNebula on distributed reliable file system MoosFS to achieve true elasticity without using expensive SAN solution.

Labels: cloud computing, opennebula, tutorial

Recently I used Installer plugin for Griffon in one of my open source project. Overall it was very easy to install and have it creating a simple installer for your Griffon application, however when it comes to customizing the izPack configuration I did not find any good documentation. Thanks to the usual helpfulness of Andres Almiray :) and going through some source code, I managed to customize the installer to meet my requirement and would like to share my findings here.

Once you install the installer plugin through:

griffon install-plugin installer

A set of izPack configuration templates will also be installed with the plugin. The template works pretty well for any Griffon example application but probably does not make too much sense for anything else. The easiest way to provide your own customization is by creating your custom template and override the default ones by hooking into Griffon build time event notification. I will show you how to do that here step-by-step.

First: Create your installer source directory to store your configuration and resources such as icons. In this tutorial we will create a folder structure as the following:

/src/installer/izpack/resources

Second: Create your own izPack configuration. Copy the default configuration to the resources folder you just created. The default configuration files can be found under ~/.griffon/projects/installer/izpack/resources.

Third: Create event handler to listen on packaging event and override the default configurations with yours. Open _Events.groovy file under /scripts (Create it if it does not exist yet). Add the following lines:

eventPreparePackageEnd = {installers ->
 ant.copy( todir: "${projectWorkDir}/installer/izpack/resources", overwrite: true ) {
     fileset( dir: "${basedir}/src/installer/izpack/resources", includes: "**" )
 }

 ant.replace( dir: "${projectWorkDir}/installer/izpack/resources" ) {
     replacefilter(token: "@app.name@", value: griffonAppName)
     replacefilter(token: "@app.version@", value: griffonAppVersion)
 }
}

Now run the izPack packaging command

griffon package izpack

You should see the customized installer based on your configuration being generated. Hope you have found this tutorial helpful and again you are always welcome to provide your feedback and comment here.

Labels: griffon, installer, izpack, tutorial

Just released JNDI Warrior 0.3 yesterday. In this release the following improvement and new features have been implemented:

• Better UI look and feel to give it more mature look rather than the original demo-like appearance
  
• Better ability to manage classpath per JNDI connection session
• A izPack powered cross-platform installer to make it easy to install the application on any platform
  
• New feature implemented to create, edit, and execute Groovy script from the application directly making prototyping and debugging with JNDI even easier.
  

Two built-in variables are available currently within the script.

1. context - IntitialContext of connected JNDI provider
2. out - a PrintWriter instance which allows you to print directly to script output area

Here is a screenshot of the 0.3 release:

Labels: jndi warrior, open source, release note