Zen in the art of IT

Spring 3, Spring Web Services 2 & LDAP Security.

2012-01-29T20:18:00.000+02:00

This year started on a good note, another one of those "the deadline won't change" / "skip all the red tape" / "Wild West" type of projects in which I got to figure out and implement some functionality using some relatively new libraries and tech for a change, well Spring 3 ain't new but in the Java 5, weblogic 10(.01), Spring 2.5.6 slow corporate kind of world it is all relative.
Due to general time constraints I am not including too much "fluff" in this post, just the nitty gritty of creating and securing a Spring 3 , Spring WS 2 web service using multiple XSDs and LDAP security.
The Code:

The Service Endpoint: ExampleServiceEndpoint
This is the class that will be exposed as web service using the configuration later in the post. The Code: CustomValidationCallbackHandler

This was my bit of custom code I wrote to extend the AbstactCallbackHandler allowing us to use LDAP.As per the comments in the CallbackHandler below, it's probably a good idea to have a cache manager, something like Hazelcast or Ehcache to cache authenticated users, depending on security / performance considerations.
The Digest Validator below can just be used directly from the Sun library, I was just wanted to see how it worked.

The service config:
The configuration for the Endpoint, CallbackHandler and the LDAP Authentication manager.
The Application Context - Server Side:
The Security Context - Server Side:

The Web XML:
Nothing really special here, just the Spring WS MessageDispatcherServlet.
The client config:
To test or use the service you'll need the following:
The Application Context - Client Side Test:

The Security Context - Client Side:

As usual with Java there can be a couple little nuances when it comes to jars and versions so below is part of the pom I used.
The Dependencies:

Wordballs Release

2012-01-28T07:30:00.000+02:00

Finally, Wordballs is live!
My latest physics based word game. Please check it out if you have an iOS device, and let your friends know.

It's Free (with ads) and 16mb to download.
Please take a couple seconds to rate it as well, it will be much appreciated

This game actually took me some time to do. Crazy deadlines and some nasty hours at work meant for some slow going and some very late nights at times.

I would just like to thank the teams from:

Chipmunk Physics
Cocos 2D

Without their awesome libraries / code this Wordballs would have been impossible.

Free Universal Version - In App Purchase available to remove ads
http://www.wordballs.com/

ccColor3B constants.... A whole bunch of them.

2011-12-17T08:30:00.000+02:00

This is bound to save everyone some time when looking at defining custom colors for cocos2d.Found this list on the cocos2d forums (thanks to a member there called Yeti):

Cached Property File Enum

2011-12-06T19:05:00.001+02:00

We had a little need to dynamically enable and disable features on our current project and I sorta left it for the last day, came up with a simple little solution that ended up quite neat.

A cached property file enum:

How it looks being used:
Code:
Test:
The .properties file:
FEATUREA=true
FEATUREB=false
FEATUREC=true

How it works:

The Pragmatic Programmer - Review / Summary Notes.

2011-12-06T14:34:00.001+02:00

I recently finished The Pragmatic Programmer, to be completely honest this had been the 3rd attempt to read it, although the book is good and well worth the read, I had unfortunetly learnt most of the lessons the hard way on my own over the last decade or so of being a software developer, so I found often myself easily distracted.

I have to add, had I read this when it was published it would have undoubtedly saved me some pain, but even with that it's always good to reaffirm some of your good habits, and keep your bad ones in check. It covers 46 sections with 70 different tips that are generally useful to any programmer.Jeff Atwood - coding horror made a list of all 70 tips.

As with most things some tips are "more equal" than others, the ones that really stick out and probably can't be emphasized enough are:

Care About Your Craft
Why spend your life developing software unless you care about doing it well?
Think! About Your Work
Turn off the autopilot and take control. Constantly critique and appraise your work.
Don't Live with Broken Windows
Fix bad designs, wrong decisions, and poor code when you see them.
Invest Regularly in Your Knowledge Portfolio
Make learning a habit.
DRY - Don't Repeat Yourself
Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.
Make It Easy to Reuse
If it's easy to reuse, people will. Create an environment that supports reuse.
Prototype to Learn
Prototyping is a learning experience. Its value lies not in the code you produce, but in the lessons you learn.
Don't Panic When Debugging
Take a deep breath and THINK! about what could be causing the bug.
"select" Isn't Broken.
It is rare to find a bug in the OS or the compiler, or even a third-party product or library. The bug is most likely in the application.
Design with Contracts
Use contracts to document and verify that code does no more and no less than it claims to do.
Crash Early
A dead program normally does a lot less damage than a crippled one.
Minimize Coupling Between Modules
Avoid coupling by writing "shy" code and applying the Law of Demeter.
Don't Program by Coincidence
Rely only on reliable things. Beware of accidental complexity, and don't confuse a happy coincidence with a purposeful plan.
Refactor Early, Refactor Often
Just as you might weed and rearrange a garden, rewrite, rework, and re-architect code when it needs it. Fix the root of the problem.
Design to Test
Start thinking about testing before you write a line of code.
Some Things Are Better Done than Described
Don't fall into the specification spiral at some point you need to start coding.
Don't Be a Slave to Formal Methods.
Don't blindly adopt any technique without putting it into the context of your development practices and capabilities.
Don't Use Manual Procedures
A shell script or batch file will execute the same instructions, in the same order, time after time.
Test Early. Test Often. Test Automatically
Tests that run with every build are much more effective than test plans that sit on a shelf.
Coding Ain't Done 'Til All the Tests Run
'Nuff said.
Test State Coverage, Not Code Coverage
Identify and test significant program states. Just testing lines of code isn't enough.
Find Bugs Once
Once a human tester finds a bug, it should be the last time a human tester finds that bug. Automatic tests should check for it from then on.

Determining iOS Hardware / Device

2011-11-25T19:19:00.001+02:00

I am currently working on my second iOS game in which I am creating dynamic Open GL ES reflections, and noticed on the older devices it would cause a drop in frames per second.It doesn't on the iPad 2, I do not have iPhone 4S yet, but I assume it won't have a visible effect there either with the new CPU.

So to not diminish the user experience I decided to remove those reflections, on the interactive CCSprite, on the older devices.

I went searching and found : UIDeviceHardware - Determine iOS device being used

I ended up just using:(platformString wasn't useful to me)I then just created a convenience method for myself to check for the version that have the Dual-core Apple A5 processor.

Cocos2d CCSprite Reflection

2011-10-16T08:40:00.000+02:00

With yet another 2 months of Wild West Development behind me, I can hopefully get back to a somewhat normal life again, blogging, learning and a little iOS game development.

What I want to share today is my little hack to reflect a CCSprite used in to cocos2d. I say hack because it does feel a little dirty, i didn't go and learn about the details of Open GL and the like, simply patched things together to get a working solution.

Below is the code for my ImageReflector, the basic idea and how it got it work was simple:

The best looking reflection example I found was Apples' example.So the methods createBitmapContext, createGradientImage and reflectedImage are directly from the above example.I wanted to use it in cocos2d so I used CCRenderTexture to create a UIImage, passed that to Apples' example code and then simply converted back to a CCSprite with the help of the CCTexture2D class.

Usage:Class:

iPhone Gmail Username / Password Error

2011-09-02T08:54:00.001+02:00

This morning, I all of a sudden could not get my gmail mails on my iPhone, constantly got:
Cannot Get Mail - The user name or password for imap.gmail.com is incorrect.

After a bit of searching, I found the solution to be simple, logon and input the Captcha phrase.

Link Below:
https://www.google.com/accounts/UnlockCaptcha

Mac OSX Global Git Ignore - XCode Project and State Files.

2011-08-31T22:44:00.000+02:00

With my recent switch to Mac, Objective C and XCode and XCode using Git as it's default VCS (Version control system), I figured it would probably be a good time to switch over to using Git. I registered with GitHub purchased the base 5 private repo plan and uploaded my projects and images. I didn't realize at the time then what pain I would cause myself.

A little later, my wife joined me in working with those projects only to discover that under certain scenarios we could never actually merge or sync the branches from XCode. There were always couple files that would be "uncommited" in XCode. Nasty XCode project state files.

I had not worked with Git before this actually, so after reading up a bit about project files and the like and how a ".ignore" file would do the trick.
After I little bit of struggling, I realized that for ignore to work, the files needed to be removed from the repository first. Thankfully I did not have any important history so I could simply recreate them. You can also remove them from git with: git rm --cached

To go about creating, editing configuring a global ignore file on OSX I did the following:

First problem on a Mac, any "." files are seen as system files and those are automatically hidden from you.

To get around that open "Terminal" and use the commands below to show and hide system files (you'll need to either restart or force quit Finder for the changes to take effect).

Show:
defaults write com.apple.finder AppleShowAllFiles -bool true

Hide:
defaults write com.apple.finder AppleShowAllFiles -bool false

Once I had done that I simply went searching for a couple example ignore files and combined them into the example below.

Create a file in your user directory called ".gitignore_global", populate it with the contents required, (below is mine, that does take care of the XCode problem files)
Then in Terminal run:
git config --global core.excludesfile ~/.gitignore_global

My global ignore file:

Java / XML UTF-8 marshalling and validating pains

2011-08-24T10:59:00.002+02:00

This is a post on the little pains I am sure most people have endured while working with Java and XML from different sources, with 'funny' characters...

If you use a FileWriter instead of a FileOutputStream it will use your OS encoding (So cp1252, ISO-8859-1 or US-ASCII and not UTF-8... ):

A way to ensure that you are using UTF-8 is:

If you are reading XML from a stream, it is probably safer to do something like:

When validating a schema note the .getBytes("UTF-8"):

When marshalling (Use a stream rather than a writer):

Lastly if you are getting XML from some unknown source there are some chars that are outside the legal XML Unicode limits that you can not encode. To remove those:
Code below from another blog :

Mac Attack...

2011-07-26T22:38:00.000+02:00

I have not created a blog entry in some time, for the last 2 months or so all my weekends and evenings have been spent on my latest addiction, iPhone development.

A couple months back I spent some time deciding what programming language to learn, blogged about it here. At the time I based on Tiobe, job stats, current interest and not taking the final step to actually buy a Mac, I choose C#. I spent a couple weeks messing with it, nothing serious, started a WPF clone for one of my favorite tools of all time. Baretail, but it just didn't catch my imagination, so I lacked the drive to finish.

I then got my iPhone, and that was all she wrote... I was hooked, coming from a lame excuse of a "smart phone" The HTC-Touch-Diamond-2-windows-mobile-thing, to the iPhone was a wonderful experience. It didn't take long before I could not resist the desire to program of my new toy... off to the iStore, out with the credit card, returning triumphantly with a single box, (iMac i5 quadcore 4gb 21.5"), inside the box there is a single cable, plug in the power... (spend a couple minutes looking for the on switch)... basic OS config, user and such.... done.

I had heard something about xCode, off go App Store... wait a day for the monster 4.5gb download. Stumble around hello world, and 2 or 3 other tutorials and that was that. I was hooked and with that 2 months of weekends and evenings were lost infront of the iMac. This last weekend my wife and I eventually submitted the game, and now we have to wait for probably 2 nerve wracking weeks for approval or rejection.

I'll will discuss more specific details once it is released but For now I'll share some experiences and thoughts from the last 2 months.

Objective C
I have to say, I like the language although coming for a Java / .Net background some of the intricacies of the syntax, lack of type enforcement and actually having to do your own memory management can be a little tedious when starting.
All that aside but both my wife (a C# developer) and I (Java) didn't feel too uncomfortable picking it up.

Open Source and Other resources
No. 1 most useful thing I found when writing a game for an iOS platform is:
Cocos2D
These guys have done an awesome job, and there is a decent size community / forum to help out if you do get stuck.
I haven't looked at Sparrow as I only found it a couple days after implementing Cocos2D and decided to stick with that. Another blogger did a comparison here.

Physics Engines, there are a couple out there, I figured I would gave it a skip for the first app, there would be enough to worry about without having to learn one of them either.
The 2 Main contenders I can see are:
Chipmunk
Box2D

XCode 4...
ouch, XCode just feels behind the likes of Eclipse, Visual Studio and IntelliJ. Little things we take for granted, are really missed. I am not going to go into the details, just know if you already use Eclipse, Visual Studio (with Resharper) or IntelliJ and start with XCode, you are going to be a little frustrated. To be fair, some of the frustrations were probably from a lack knowledge on my part and to top it off never having coded in the language before.

After submitting my app I decided to try App Code and loved it straight away. It's still in "beta", but using AppCode after coming from IntelliJ fells very natural. So thankfully there will be a real alternative once they release it officially.

General
I will try post useful Objective C/Game/Apple code and examples as I learn. The first 2 quick things to start with:

1. When Using Cocos2D.... and trying to incorporate UIScrollView from the SDK, prepare yourself for some pain. The CCScrollLayer however, is completely painless. We extended it and completely customized it to our game look and feel, it was very simple to do.

2. If you want to sort an array of integers in descending order use a NSSortDescriptor, very hand class actually:

1 Year and 70 000 views later, the stats...

2011-05-23T06:24:00.000+02:00

So it has been just over a year, since my first blog post. Even though the whole work / life thing has kept me overly busy the last couple months, reducing me to 1-2 posts a month, it has been quite a pleasing experience blogging. I would recommend it to all developer types, it's an good way to keep you investigating, learning and having your work out there and visible to people makes you put in that little extra effort.

The actual reason for this post is for some discussion on the stats... There were 1 or 2 interesting percentages I would not have expected.

Views By Country

No big surprise that the US has the most views, this being a English technology based blog, it is however interesting that Germany is second, beating out the UK and India, both with larger populations. South Africa on the list is probably only there because I force my colleagues and friends to read it :)

Views By Browser

With this being an Java-technical-open source oriented blog makes FireFox the obvious choice to lead the browser war. However with Chrome in 2nd by a such large percentage was a bit unexpected.

Views By OS

On OS Windows is still number one. Apple is competing nicely with Linux / Unix combined, this is probably due the large apple following in the US and the boom of the iPad.

Java Compression.

2011-05-16T05:58:00.001+02:00

In a recent project, we had to do something I had personally never really had to look at; Compression. We needed to take a couple files and images, zip them up and make them available for FTP, and yes somedays it does feel like we are back in the 90's. Besides the FTP trip into the past its was good opportunity to spend a little bit of time on the subject.

Compressing Files
So above the usual IO classes BufferedInputStream, FileOutputStream and File there are:
ZipInputStream - An input stream for reading files in the ZIP file format. Zip entries are not cached, unlike ZipFile.
ZipOutputStream - An output stream for writing files in the ZIP file format. This has a default internal buffer of 512, a BufferedOutputStream can be used to increase this.
ZipEntry - Represents an entry int a zip file.
ZipFile - Used to read entries from a zip file. The entries are cached.
CRC32 - Used to compute the CRC-32 of a data stream.

Below is an example showing how to compress and decompress files in a folder, with and without a checksum:

Compressing Objects
We didn't end up using object compression but I had a look at it anyways. I did a little generic compress / expand util, don't know if it will ever be useful. I left the input params as OutputStream and InputStream as this could theoretically be used with any stream implementation from socket communication to string manipulation.

The compression related classes being used here:
GZIPInputStream - An input stream filter for reading compressed data in the GZIP file format.
GZIPOutputStream - An output stream filter for writing compressed data in the GZIP file format. Default internal buffer of 512, use BufferedOutputStream if you require more.

Little Spring Gem: ReflectionTestUtils

2011-04-05T14:34:00.000+02:00

I have recently been slacking on content on my blog, between long stressful hours at work and to the wonderful toy that is an iPhone, I have taken a little break from anything "development" related after leaving the office to maintain my sanity. Now with the project delivered, and things quieting down again I can re-focus my excess neuron energy back to processing more IT related information.

One little thing I discovered in my last project, which I am a little embarrassed about, being a Spring nut, is the little gem that is : ReflectionTestUtils.

I mostly try write unit tests that do not include Spring, for the reason, that you should be testing your code and not Spring config. However sometimes it's really useful and quite beneficial to have your test code wired up with Spring, be that for integration tests or just to extend your test suite.
One issue was, I always found myself adding "setters" to my component interfaces, I always hated doing it, but I would con myself by saying: "It was for more testing, and more testing is always a good thing", and move along. I eventually (while procrastinating on work I didn't really want to do) went searching for a cleaner solution.

1 search and 1 minute later, a little red faced, I kicked myself. I should have guessed straight away that Spring would have thought of this scenario From 2.5 Spring had the ReflectionTestUtils class, which simply lets you set your dependencies / mocks via reflection.

So easy, tell it which object, the name of the field and set the mock / value you want to set. Neater interfaces, good times.
Below is an example using EasyMock interfaces and inject them for my test.

The simple Big-O Notation Post.

2011-03-13T16:19:00.003+02:00

I make no claim to be a "computer scientist" or a software "engineer", those titles alone can spark some debate, I regard myself as a software developer and I generally don't study the math and science behind everything I do. I generally learn what is relevant and useful to my day to day functioning and only rarely go deeper and dabble in the theory behind it. This is one of those occasions, so I decided to scour the internet and see what I could pick up. I hope to keep this simple, practical and to the point.

Big-O:

Describes how the algorithm scales and performs, in terms of either the execution time required or the space used.
Is relative representation of complexity. This allows you to reduce an algorithm to a variable which in turn allows you to easily compare it to another.
Describes an upper limit on the growth of a function, in the other words the "worst case scenario".

There is also Big-Omega notation which looks at the lower bound / "best case scenario" stating that the algorithm will take at least X amount of time and Big-Theta which is tight bound to both lower and upper / "average".

Some quick observations in determining Big-O:

A Sequence of statements, or things like conditional checks are constant: O(1)
A loop of statements result in : O(n) n being the number of loop executions.
Nested loops are multiplied together: O(n²) where n is the times the outer loop executes and m is the times the inner loop executes.

Comparing the common notation examples:
(Thanks to Algorithms: Big-Oh Notation.)

Constant
O(1)

Logarithmic
O(log n)

Linear
O(n)

Linear Logarithmic
O(n log n)

Quadractic
O(n²)

Cubic
O(n³)

512

256

4,096

1,024

10,240

1,048,576

1,073,741,824

1,048,576

20,971,520

10¹²

10¹⁶

Java code example:
Show examples of notations in the table above.

Common Data Structures and Relative functions:Lists and Sets:

Structure	get	add	remove	contains
ArrayList	O(1)	O(1)*	O(n)	O(n)
LinkedList	O(n)	O(1)	O(1)	O(n)
HashSet	O(1)	O(1)	O(1)	O(1)
LinkedHashSet	O(1)	O(1)	O(1)	O(1)
TreeSet	O(log n)	O(log n)	O(log n)	O(log n)

* ArrayList Notes:

Thanks to reader comments, the add method on an ArrayList should be:O(1) amortized (and O(n) in worse case).Useful reference links:

Constant Amortized Time

Linked List vs ArrayList

Maps:

Structure	get	put	remove	containsKey
HashMap	O(1)	O(1)	O(1)	O(1)
LinkedHashMap	O(1)	O(1)	O(1)	O(1)
TreeMap	O(log n)	O(log n)	O(log n)	O(log n)

References:

Algorithms: Big-Oh Notation.

Algorithmic Complexity and Big-O Notation.

Determining Big O Notation An easier way .

Wikipedia.

Selecting a new programming language to learn.

2011-02-20T23:46:00.002+02:00

I have been itching to learn a new language, but being a Java freak, I always end up convincing myself to spend the time and effort discovering, investigating or playing with something in the Java open source stable, Spring, Hadoop, Joda Time, Hibernate, Maven, Hazelcast, EhCache etc etc. Developing in Java these days is almost purely about knowing and wiring together frameworks, which is both a good and a bad thing. (as well as a topic for another day).

Now to get myself to not redirect the "new language" energy into Y.A.F (yet another framework) I decided to give the languages out there a proper look and see which would be the best fit and most beneficial to my work, marketability and just general 'IT Zen'.

So what do I require from a language:
IDE... my number 1 thing is an IDE, if there isn't a decent IDE for a language it is frankly not worth the time and effort. I don't see myself as a "scientist" where I feel the need to cause myself pain and inconvenience to be "pure". I want a comfortable productive working environment, and VI or Notepad with a command line utility ain't it.

Established... Every couple years someone somewhere tries to define some new language, and most of those die in obscurity for example brainf*** or anything listed on Esolang.

Popular / In Demand... As with most things, popularity is good, it means:
open source community, support and most importantly jobs. If you ever want to see the current popularity of a language Tiobe is the site to visit.

So who are the contenders out there?
Based on Feb 2011 Tiobe index:
Java is still the no.1 most popular, it has awesome IDEs and it's been around for just more than 15 years (January 23, 1996), but thankfully I know Java reasonably well :)... so moving right along... To narrow down the list quickly, I won't look at any languages that are losing popularity, for obvious reasons, so from the top 20 on the Tiobe list that excludes: C, C++, PHP, VB, JS, Perl, Ruby, Delphi, Go.
(~~C, C++, PHP, VB, JS, Perl, Ruby, Delphi, Go.~~)

Which leaves behind:
Python, C#, Objective-C, Lisp, NXT-G, Ada, Pascal, Lua, RPG

Now there is a line between established and old, I am going to make a call that could offend some people and say Pascal and RPG are just old. (~~Pascal~~, ~~RPG~~)

Ada, don't know much about it, after reading the ADA overview, it seems okay, going to exclude it based on popularity. (~~Ada~~)

Lua, from a quick read it is a scripting language. (~~Lua~~)

NXT-G has something to do with lego or some robotics, not very mainstream. (~~NXT-G~~)

Lisp again like Ada, at first glace seems fine, just not popular enough. (~~Lisp~~)

Then there are the "New, built on other platforms" functional languages: Scala, F#, Clojure. Although very temping being on the bleeding edge, it's not all that profitable or marketable yet. I'll give them some time to standardize, settle down and see if they are widely adopted. They do appeal greatly to my inner geek, so will always be keeping an eye on them.

So this leaves me with:
Python, C#, Objective-C, (and Java).

Straight away based on the above list we can Tick: IDE, Established and Popular / In demand. We all know they have decent IDEs: Eclipse, XCode, Visual Studio, (IntelliJ and Netbeans). They have also been around and are well known.

Now looking at number of jobs:
Found a site (Simply hired) with a graph displays the percentage of jobs with your search terms anywhere in the job listing. Since June 2009, the following has occurred:

Python jobs increased 72%
C# jobs increased 77%
Objective-c jobs increased 268%
Java jobs increased 76%

Python, C#, Objective-c, Java trends

Python jobs | C# jobs | Objective-c jobs | Java jobs

With the recent boom of iPads and iPhones the Objective-C percentage is not all that surprising. I do have a problem with Apple, Objective-C and XCode and that problem is you need a Mac to run it. Once you start down that road you end up having to change everything to Apple, and I am not ready to do that. So for now I am going to drop ~~Objective-C~~ out of the running. Although if I ever do buy into the whole Apple thing, this will got back to the list.

Leaving me with Python and C#, looking at their salaries compared with Java:
(Data from Payscale).
US Data
Java
PayScale - Java Skill Salary, Average Salaries by Years Experience

Median Salary by Years Experience - Skill: Java (United States)

CAREER TOOLS: Salary Calculator, Career Path Tool, Cost of Living Calculator, Meeting Miser

Python
PayScale - Python Skill Salary, Average Salaries by Years Experience

Median Salary by Years Experience - Skill: Python (United States)

CAREER TOOLS: Salary Calculator, Career Path Tool, Cost of Living Calculator, Meeting Miser

C#
PayScale - C# Skill Salary, Average Salaries by Years Experience

Median Salary by Years Experience - Skill: C# (United States)

CAREER TOOLS: Salary Calculator, Career Path Tool, Cost of Living Calculator, Meeting Miser

South Africa Data

Java
PayScale - Java Skill Salary, Average Salaries by Years Experience

Median Salary by Years Experience - Skill: Java (South Africa)

CAREER TOOLS: Salary Calculator, Career Path Tool, Cost of Living Calculator, Meeting Miser

Python
PayScale - Python Skill Salary, Average Salaries by Years Experience

Median Salary by Years Experience - Skill: Python (South Africa)

CAREER TOOLS: Salary Calculator, Career Path Tool, Cost of Living Calculator, Meeting Miser

C#
PayScale - C# Skill Salary, Average Salaries by Years Experience

Median Salary by Years Experience - Skill: C# (South Africa)

CAREER TOOLS: Salary Calculator, Career Path Tool, Cost of Living Calculator, Meeting Miser

Based on the US data, I would have gone with Python, it's not as popular as C# but the pay is slightly better, I would also get to keep using Eclipse (PyDev) and Spring, but as soon as I looked at the South African data, I realized something, Python is really not big here. I manually went searching for Python positions advertised.. and found a grand total of 2 and the salaries were not good.

(~~Python~~)

Leaving C# as the last language standing.

It's got Visual Studio (even a free version Visual Studio Express), It has proven itself over the last couple years, it's out innovating Java at the moment, there's a ton of jobs, a whole range of certifications and the salaries have closed the gap on Java.
Seems quite a logical choice to me.

To top it off, I have also used C# many years back, so it won't not entirely new. Most of the successful Java open source projects (Spring, Hibernate etc etc) have been ported so all that knowledge is reusable, which also counted a little in my decision. Now I just need to stop working 12-14 hours a day, and I can focus on getting back to my Microsoft roots with little C# as a Java developer. Hopefully a couple months after that I can go through this process again, looking at Python, Objective-C, the mobile platforms (iOS, Android, windows) or maybe rather a concept change to functional with the likes of Clojure or Scala.

Validate XML against its XSD

2011-02-14T07:09:00.000+02:00

Just a quick code snippet for possible future reference. How to validate a XML against it's XSD.

Surviving the Wild West Development Process

2011-02-07T07:32:00.001+02:00

Now in a perfect world, we all strive for a some kind of development process, be that: Waterfall development, Prototyping, all things Agile or whatever process the CIO / CTO /CEO got sold on by some consultancy.

But in the real world what sometimes ("sometimes" being quite often actually) happens is:
What I like to call the Wild West Development Process (WWDP)... the main principle is basically “promise first and ask questions later”.

Getting started with WWDP is pretty simple:
1. You need someone, somewhere in a place of power and influence in your organization to promise software by some unachievable date.
2. Add in a eager marketing department and some press releases…
3. stir and leave to simmer.
4. After simmering for a while inform the software development department that they must have something, that does "stuff" and the release date can not change.

I am not sure if this is a result of a immature IT industry in my country, but I have spent many years in environments like this. It even seems to find me in the most unlikely employers, the large corporate organizations like insurance and banking institutions, where general red tape and corporate processes should limit this. I started my career in one of those “startup-sell-information-tech-and-business-processes-to-anyone-cheaper-than -anyone-else-type-consultancies". Which means every project was driven by WWDP so I started in the deep end, and for about 4 years I knew no other way of developing software. I currently find myself in another one of those projects. I'd like to share some points that I found helpful in actually delivering some software in unimaginable unmovable deadlines.
(to be fair some of these are in the standard development processes mentioned earlier, maybe with just different priorities)

Pick the right developers
Not everyone can cope in this environment, knowing your staff / team is very important: late nights, constant crisis, constant changes and bucket loads of pressure and stress aren't for everyone. Some people thrive, some survive and run for the hills after, some just end up whimpering in the corner and need to be replaced (which is a disaster).

Have the developers involved.
Projects like these don't allow for the usual requirements / design / develop / test paradigm, all of these functions run concurrently. Having the developers know as much detail as the architect, analysts and testers is vital. Developers will have to make decisions affecting the architecture and business functionality constantly, if everyone has the same picture you'll save time.

"Prototype" early
I say "Prototype" in quotes because unlike an actual prototype there won't be time to throw this code away. This "prototype" is to have the system in it's entirety up and running very early in the project. It can be a "shell" but having it up means testing can start early even if it's just the basics and no business functionality. Since there are no real requirements, make sure the design caters for adding them as they are defined.

Integrate early
If the system has any integration points, between systems, teams or external parties, make sure this is also in the "Prototype Shell", integration always takes longer than you think, having the interfaces up and available to test is crucial.

Keep track of communication
When the shock of not meeting an unachievable deadline kicks in, someone will always look for a scapegoat, always keep at least an email trail making sure you aren't that goat.

Automated builds and deploys
The standard agile practice of continuous integration is as important as ever, and even if you skip other parts of you software development cycle this is something that should not be dismissed.

Focus
With hundreds of things that need to happen a matter of days or weeks, it's very easy to try do 6 things at a time. Don't. Micro manage your time and tasks and focus. It's easy for us developers to start a task, see something, change something and lose a day which you don't have in the first place. If you have to, have someone keep an extensive list of "TODOs" that can be used in the cleanup phase.

Clean up
Once the unachievable deadline has been met, before starting the next phase, have at least one "cleanup" release. This cleanup phase should add very little or preferably no new functionality, but will allow for design and development decisions made in haste to be reviewed and refactored.

Pace Yourself
No matter how super human you may think you are, all of us have a limit to how many hours we can keep going. At some point you are actually being counter productive and a couple more hours sleep will actually help the project a lot more.
Know your limits.

I am sure there are a couple more, but currently being on a WWDP project I need to re-focus and leave that for the clean up phase later :)

Joda Time == Good Times

2011-01-09T23:31:00.000+02:00

I heard about Joda Time a couple months back, but finally got to use and implement it in a project.

We all know the whole Java date thing is ugly, and I sure most people have the own "implementations" and extensions like Period objects and utility Date Diff classes. I know I've written my fair share over the years. I won't be doing that again...

I am not going to go into too much detail as the Joda documentation is pretty good.
I just wanna highlight some of the very usual functionality that won me over:

Setting the "System" time, very useful. The financial systems I have worked on throughout my career always had some concept that required the "Current Date" mostly for data and rules with effective date periods.

Joda lets you statically set the "Current Date" making it a pleasure to test with, ensuring that any new DateTime objects used will be based off the set date:

Intervals:

The classic Date diff:

Working with "parts", just the date or the time:

The "all you need to know link" for tuning Hibernate

2010-12-31T08:48:00.000+02:00

I stumbled across this awesome article today: Revving Up Your Hibernate Engine.
Well worth the read if you are or are planning to use Hibernate in the near future.

To quote the summary of the article:
"This article covers most of the tuning skills you’ll find helpful for your Hibernate application tuning. It allocates more time to tuning topics that are very efficient but poorly documented, such as inheritance mapping, second level cache and enhanced sequence identifier generators.

It also mentions some database insights which are essential for tuning Hibernate.
Some examples also contain practical solutions to problems you may encounter."

Find all the classes in the same package

2010-12-24T09:38:00.000+02:00

It's been a slow month for blogging, due to the release of a new World of Warcraft expansion, crazy year end software releases at work and the general retardation of the human collective consciousness that happens this time of year.

With all of that I did however have some fun writing the following bit of code. We have a "generic" getter and setter test utility used to ensure a couple extra %'s of code coverage with minimal effort. That utility did however requires that you call it for every class you want to test, being lazy I wanted to just give it a class from a specific package and have that check all the classes.

I initially thought there would have to be just a simple way to do it with the Java reflection API. Unfortunately I didn't find any, it seems a "Package" does not keep track of it's classes, so after a bit of digging on the net and feverish typing (ignoring the actual project I am currently working on for a little) here is what I came up with. The tip is actually the just methods from the java.lang.Class object: "getProtectionDomain().getCodeSource().getLocation().toURI()", giving you the base to work from and I found that somewhere on Stackoverflow.

There are 2 public methods: findAllClassesInSamePackage and findAllInstantiableClassesInSamePackage. (For my purposes of code coverage I just needed the instantiable classes.)

Usage:

PackageClassInformationFinder:

Enabling enterprise log searching - Playing with Hadoop

2010-12-05T22:29:00.001+02:00

Having a bunch of servers spewing out tons of logs is really a pain when trying to investigate an issue. A custom enterprise wide search would just be one of those awesome little things to have and literally save developers days of their lives not to mention their sanity. The "corporate architecture and management gestapo" will obviously be hard to convince, but the chance to write and setup my own "mini Google wannabe" MapReduce indexing is just too tempting. So this will be a personal little project for the next while. The final goal will be distributed log search, using Hadoop and Apache Solr. My environment this mostly consists of log files from:
Weblogic Application server, legacy CORBA components, Apache, Tibco and then a mixture of JCAPS / OpenESB / Glasshfish as well.

Setting Up Hadoop and Cygwin
First thing to get up and running will be Hadoop, our environment runs on windows and there in lies the first problem. To run Hadoop on Windows you are going to need Cygwin.

Download: Hadoop and Cygwin.

Install Cygwin, just make sure to include the Openssh package.

Once installed, using the Cygwin command prompt: ssh-host-config
This is to setup the ssh configuration, reply yes to everything except if it asks
"This script plans to use cyg_server, Do you want to use a different name?" Then answer no.

There seems to be a couple issues with regards to permissions between Windows (Vista in my case), Cygwin and sshd.
Note: Be sure to add your Cygwin "\bin" folder to your windows path (else it will come back and bite you when trying to run your first map reduce job)
and typical to Windows, a reboot is required to get it all working.

So once that is done you should be able to start the ssh server: cygrunsrv -S sshd
Check that you can ssh to the localhost without a passphrase: ssh localhost
If that requires passphrase, run the following:

ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

Now back to the Hadoop configuration:
Assuming that Hadoop was downloaded and unzipped into a working folder, ensure that the JAVA_HOME is set. Edit the [working folder]/conf/hadoop-env.sh.

The go into [working folder]/conf, and add the following to core-site.xml:

To mapred-site.xml add:

Go to the hadoop folder: cd /cygdrive/[drive]/[working folder]
format the dfs: bin/hadoop namenode -format
Execute the following: bin/start-all.sh

You should then have the following URLs available:
http://localhost:50070/
http://localhost:50030/

A Hadoop application is made up of one or more jobs. A job
consists of a configuration file and one or more Java classes, these will interact with the data that exists on the Hadoop distributed file system (HDFS).

Now to get those pesky log files into the HDFS. I created a little HDFS Wrapper class to allow me to interact with the file system. I have defaulted to my values (in core-site.xml).

HDFS Wrapper:
I also found a quick way to start searching the log file uploaded, is the Grep example included with Hadoop, and included it my HDFS test case below.Simple Wrapper Test:

Design Patterns in the JDK.

2010-11-22T07:12:00.002+02:00

I saw an article (well more of a rant) the other day, by Rob Williams
Brain Drain in enterprise Dev. I have to say, I do agree with some of what he is saying. I know from my personal experience, I had spent a good 2 or so years just wallowing in the enterprise development world, not learning anything and actually losing my skills I developed before. The corporate confront zone is not conducive to eager technologists.

In this article he also stated:
"1 in 10 cant even pass a simple test like ‘which design pattern is used in the streams library that makes BufferedFileReader interchangeable with a FileReader?'"

I also tested it at work and I only had 1 out of the 8 people asked that got it right

Without much confidence, I had guessed Decorator based on "interchangeable". I then decided that was actually some worth sneaking into future interviews, and probably a good time to revise a little.

So I went scouring the internet to find all I could on the topic and I didn't actually find as much as I thought I would. Most of it came from BalusC at Stackoverflow, the rest was very scattered between blog posts, java ranch, some old pdf's and articles I had. I didn't take every single example of every single pattern I found, but rather the common ones.
This may be a good way for people to learn about patterns, quite often they are using them everyday without realizing.

Structural
Adapter:
This is used to convert the programming interface/class into that of another.

-java.util.Arrays#asList()
-javax.swing.JTable(TableModel)
-java.io.InputStreamReader(InputStream)
-java.io.OutputStreamWriter(OutputStream)
-javax.xml.bind.annotation.adapters.XmlAdapter#marshal()
-javax.xml.bind.annotation.adapters.XmlAdapter#unmarshal()

Bridge:
This decouples an abstraction from the implementation of its abstract operations, so that the abstraction and its implementation can vary independently.

-AWT (It provides an abstraction layer which maps onto the native OS the windowing support.)
-JDBC

Composite:
Lets clients treat individual objects and compositions of objects uniformly. So in other words methods on a type accepting the same type.

-javax.swing.JComponent#add(Component)
-java.awt.Container#add(Component)
-java.util.Map#putAll(Map)
-java.util.List#addAll(Collection)
-java.util.Set#addAll(Collection)

Decorator:
Attach additional responsibilities to an object dynamically and therefore it is also an alternative to subclassing. Can be seen when creating a type passes in the same type. This is actually used all over the JDK, the more you look the more you find, so the list below is definitely not complete.

-java.io.BufferedInputStream(InputStream)
-java.io.DataInputStream(InputStream)
-java.io.BufferedOutputStream(OutputStream)
-java.util.zip.ZipOutputStream(OutputStream)
-java.util.Collections#checked[List|Map|Set|SortedSet|SortedMap]()

Facade:
To provide a simplified interface to a group of components, interfaces, abstractions or subsystems.

-java.lang.Class
-javax.faces.webapp.FacesServlet

Flyweight:
Caching to support large numbers of smaller objects efficiently. I stumbled apon this a couple months back.

-java.lang.Integer#valueOf(int)
-java.lang.Boolean#valueOf(boolean)
-java.lang.Byte#valueOf(byte)
-java.lang.Character#valueOf(char)

Proxy:
The Proxy pattern is used to represent with a simpler object an object that is complex or time consuming to create.

-java.lang.reflect.Proxy
-RMI

Creational
Abstract factory:
To provide a contract for creating families of related or dependent objects without having to specify their concrete classes. It enables one to decouple an application from the concrete implementation of an entire framework one is using. This is also found all over the JDK and a lot of frameworks like Spring. They are simple to spot, any method that is used to create an object but still returns a interface or abstract class.

-java.util.Calendar#getInstance()
-java.util.Arrays#asList()
-java.util.ResourceBundle#getBundle()
-java.sql.DriverManager#getConnection()
-java.sql.Connection#createStatement()
-java.sql.Statement#executeQuery()
-java.text.NumberFormat#getInstance()
-javax.xml.transform.TransformerFactory#newInstance()

Builder:
Used simplify complex object creation by defining a class whose purpose is to build instances of another class. The builder pattern also allows for the implementation of a Fluent Interface.

-java.lang.StringBuilder#append()
-java.lang.StringBuffer#append()
-java.sql.PreparedStatement
-javax.swing.GroupLayout.Group#addComponent()

Factory method:
Simply a method that returns an actual type.

-java.lang.Proxy#newProxyInstance()
-java.lang.Object#toString()
-java.lang.Class#newInstance()
-java.lang.reflect.Array#newInstance()
-java.lang.reflect.Constructor#newInstance()
-java.lang.Boolean#valueOf(String)
-java.lang.Class#forName()

Prototype:
Allows for classes whose instances can create duplicates of themselves. This can be used when creating an instance of a class is very time-consuming or complex in some way, rather than creating new instances, you can make copies of the original instance and modify it.

-java.lang.Object#clone()
-java.lang.Cloneable

Singleton:
This tries to ensure that there is only a single instance of a class. I didn't find an example but another solution would be to use an Enum as Joshua Bloch suggests in Effective Java.

-java.lang.Runtime#getRuntime()
-java.awt.Toolkit#getDefaultToolkit()
-java.awt.GraphicsEnvironment#getLocalGraphicsEnvironment()
-java.awt.Desktop#getDesktop()

Behavioral

Chain of responsibility:
Allows for the decoupling between objects by passing a request from one object to the next in a chain until the request is recognized. The objects in the chain are different implementations of the same interface or abstract class.

-java.util.logging.Logger#log()
-javax.servlet.Filter#doFilter()

Command:
To wrap a command in an object so that it can be stored, passed into methods, and returned like any other object.

-java.lang.Runnable
-javax.swing.Action

Interpreter:
This pattern generally describes defining a grammar for that language and using that grammar to interpret statements in that format.

-java.util.Pattern
-java.text.Normalizer
-java.text.Format

Iterator:
To provide a consistent way to sequentially access items in a collection that is independent of and separate from the underlying collection.

-java.util.Iterator
-java.util.Enumeration

Mediator:
Used to reduce the number of direct dependencies between classes by introducing a single object that manages message distribution.

-java.util.Timer
-java.util.concurrent.Executor#execute()
-java.util.concurrent.ExecutorService#submit()
-java.lang.reflect.Method#invoke()

Memento:
This is a snapshot of an object’s state, so that the object can return to its original state without having to reveal it's content. Date does this by actually having a long value internally.

-java.util.Date
-java.io.Serializable

Null Object:
This can be used encapsulate the absence of an object by providing an alternative 'do nothing' behavior. It allows you to abstract the handling of null objects.

-java.util.Collections#emptyList()
-java.util.Collections#emptyMap()
-java.util.Collections#emptySet()

Observer:
Used to provide a way for a component to flexibly broadcast messages to interested receivers.

-java.util.EventListener
-javax.servlet.http.HttpSessionBindingListener
-javax.servlet.http.HttpSessionAttributeListener
-javax.faces.event.PhaseListener

State:
This allows you easily change an object’s behavior at runtime based on internal state.

-java.util.Iterator
-javax.faces.lifecycle.LifeCycle#execute()

Strategy:
Is intended to provide a means to define a family of algorithms, encapsulate each one as an object. These can then be flexibly passed in to change the functionality.

-java.util.Comparator#compare()
-javax.servlet.http.HttpServlet
-javax.servlet.Filter#doFilter()

Template method:
Allows subclasses to override parts of the method without rewriting it, also allows you to control which operations subclasses are required to override.

-java.util.Collections#sort()
-java.io.InputStream#skip()
-java.io.InputStream#read()
-java.util.AbstractList#indexOf()

Visitor:
To provide a maintainable, easy way to perform actions for a family of classes. Visitor centralizes the behaviors and allows them to be modified or extended without changing the classes they operate on.

-javax.lang.model.element.Element and javax.lang.model.element.ElementVisitor
-javax.lang.model.type.TypeMirror and javax.lang.model.type.TypeVisitor

Now with a new cleaner look...

2010-11-19T19:21:00.000+02:00

I had a couple mails and comments stating that my blog theme made it hard to read, so for a complete reversal ... I went as clean as I could.

Learning to Drool... Part 2

2010-11-15T07:14:00.000+02:00

In Part 1 I went through the basic syntax and requirements to get a rule developed and tested.

Now to extend that, the Drools documentation is actually quite good, there is just a ton of it, so I will try to just focus on some of the main topics.

First a little thing I had to do to get the rules to run from your tests using maven, the .drls are not in the classpath by default, a simple way around that was to add the following to the POM:

Now more rules scenarios and usages:

Collections:
Querying the contents of a can be done in 2 ways, contains and memberOf, the difference is that the collection used in conjunction with memberOf must be a variable.

The drl:

Regular Expressions:
You can use a regex as a selection criteria as well with the key words Matches and Not Matches.

The drl:

Global Variables:
You can define global variables, they should not be used as the are sometimes in code, to pass information between methods or in this case rules. They should rather be used to provide data or services that the rules use. An example would be something like an application specific logger, or perhaps constant lookup data loaded when the application starts.

The test:

The drl:

Rule Attributes:
On a rule you can specify attribute, there are a number of these. I'll just mention a couple handy ones (quoting the official documentation):

no-loop
default value: false
type: Boolean
When the rule's consequence modifies a fact it may cause the Rule to activate again, causing recursion. Setting no-loop to true means the attempt to create the Activation for the current set of data will be ignored.

salience
default value : 0
type : integer
Salience is a form of priority where rules with higher salience values are given higher priority when ordered in the Activation queue.

dialect
default value: as specified by the package
type: String
possible values: "java" or "mvel"
The dialect species the language to be used for any code expressions in the LHS or the RHS code block. Currently two dialects are available, Java and MVEL. While the dialect can be specified at the package level, this attribute allows the package definition to be overridden for a rule.

date-effective
default value: N/A
type: String, containing a date and time definition
A rule can only activate if the current date and time is after date-effective attribute.

date-expires
default value: N/A
type: String, containing a date and time definition
A rule cannot activate if the current date and time is after the date-expires attribute.

Guided Rule:
The guided rule editor seems to allow for everything you can do in code, just visually, and potentially more intuitively for those non developers. Only thing you need to do is make sure to have the objects you want to use imported in a .package in the same location as the created .brl file.

Decision Tables:
In my opinion the world actually runs on spreadsheets. We all like to think that it only functions because of us and our fancy applications, but truth be told the world would miss spreadsheets more than any other single application.

The business people giving us our requirements understand spread sheets, some of them better than us developers and this is the biggest single bonus on decision tables. The Drools decision table at first glance did look as if it would not be as simple to hand over to business users as the Quickrules one, but it is actually split quite clearly into "code" and "data"

Now to breakdown a decision table into its parts...
When creating a decision table, the eclipse plugin gives you an example, I am going to work through that.

C2: Keyword ruleset, just to state that this spread sheet is a ruleset (package).
D2: The ruleset (package) name.
Under this row you can specify the following optional keywords:
Sequential - The cell to the right of this can be true or false. If true, then salience is used to ensure that rules fire from the top down.
Import - The cell to the right contains a comma separated list of classes to import.
Variables - The cell immediately to the right can contain global declarations which Drools supports. This is a type, followed by a variable name. (if multiple variables are needed, comma separate them).

C3/D4: Notes heading and the actual notes.

C5: The RuleTable keyword and name. The "RuleTable" keyword must appear on top of the first condition column. You can have multiple rule tables on a sheet, they must just be separated by a line.

C6: Marks the column as a CONDITION column, G6, does this for the ACTION. You need at least one of each for a table to be valid. If there is no data in a CONDITION column, then that condition does not apply.
Other column optional keywords are:
PRIORITY - This states that this column's values will set the salience
DURATION - This will set the duration values for the rule row.
NO-LOOP - Same as in the drl, this specifies if rule is not allowed to loop.

C7: This and subsequent columns in the row, define the actual variables referenced in the rule table.

C8: This and subsequent columns in the row specifies where we are getting the data from.

Row 9 and Column B are just labels / headings making the data simpler to understand, all the other fields and columns before there can be hidden as not to scare the "less technical". The table inside B9 is then were the specific rule data is defined, hopefully directly from the specification, by a non developer.

I have uploaded this project to my Google code project, incase anyone wants it. I had a little issue with the decision table because I run OpenOffice at home and not Microsoft Office, the plugin expects Excel, so it leaves a ugly little red X in my project, but it opens fine outside the IDE and still compiles in maven.

Part 3, I'll have a look at the Rule Flow, and start tackling the monster that is Guvnor

Learning to Drool... Part 1

2010-11-08T07:24:00.001+02:00

This series of posts will be about me getting to grips with JBoss Drools. The reasoning behind it is: SAP bought out my company's current rules engine and Drools is one alternative we will be looking into as soon as someone has the skills to get a proof of concept up.

Although there seems to be a fair amount of documentation, I always find it helps having walked through examples, which is what I am going to do here.Drools on first glance can be quite daunting, it is made up of :

Drools Expert (rule engine)
Being a developer this is where I will begin, the actual rules and implementation of them.

The other parts I'll get to later are:
Drools Guvnor (BRMS/BPMS)
Drools Flow (process/workflow)
Drools Fusion (event processing/temporal reasoning)
Drools Planner (automated planning)

So to begin.
For part 1, I just want to get my feet wet, I download only the Eclipse plugin and the binaries

Need to install the Eclipse plugin, used the update site. Unzip the binaries to a directory and withing the Eclipse plugin settings point to that directory.

The eclipse plugin will allow you to create a Drools Project and that includes the "Drools Library", but if you are using maven you need to point to the JBoss release repo for the Drools dependencies, The KnowledgeRuntimeLoggerFactory needs XStream which you can just get from the standard maven repo. Below is my POM:

To add a rule, on src/main/rules, Right click -> New -> Other... Drools/Rule Resource, be sure to choose the "individual rule". This leaves you with an empty drl file:

To understand and use the rule language, I read the Drools Documentation and the examples project.
What actually took me a little to grasp, was the basic syntax and how objects were handled in the rules and I did struggle to find anything that actually explains it simply so I will give it a shot.

A note on variable names.. they do not need to have the '$' but it was used in the example and without it quickly gets quite confusing.

Now to step through the parts of a rule:

The package, and import keyword are self explanatory, what happens after the when is not.
What is happening in "$vo : RoolVO( stringValue == "Learning to drool", $booleanVal : booleanValue )" broken down actually is:

stringValue == "Learning to drool" - This is a constraint that allows us to find all RoolVO objects known to the knowledge base that have the getStringValue() value equals to "Learning to drool". If there are multiple instances of RoolVO that comply we will run this rule more than once, these are also referred to as matched objects. You can also have multiple constraints separated by a ",".

$booleanVal : booleanValue - We are declaring a new local variable of type boolean called $booleanVal and gets it value from isBooleanValue.

$vo : RoolVO - We are declaring a new local variable of type RoolVO called $vo.

The next line:
"eval( $booleanVal )" - Evaluates the boolean variable, this needs to evaluate to true for the 'then' part of the rule to be called.
Then:
System.out.println( "First Rule" ); - Standard sys out.
$vo.setStringValue("Done."); - Sets the String value on the current RoolVO object that matched the constraints to Done.

The main classes / interfaces needed for a basic rule execution seem to be the following:
org.drools.KnowledgeBase and it's factory
org.drools.KnowledgeBaseFactory:
This is the repository of all the relevant knowledge definitions; it contains rules, processes, functions, type models.

org.drools.builder.KnowledgeBuilder and it's factory org.drools.builder.KnowledgeBuilderFactory:
Transforms / parses a source file (.drl, .xsl) into a KnowledgePackage that a KnowledgeBase can understand.

StatefulKnowledgeSession created by the KnowledgeBase .newStatefulKnowledgeSession();
This session is used to communicate with the actual rules engine.

To quote the drools JavaDocs:
StatefulKnowledgeSession is the most common way to interact with a rules engine. A StatefulKnowledgeSession allows the application to establish an iterative conversation with the engine, where the reasoning process may be triggered multiple times for the same set of data.

I wrote a simple test for the rule I described earlier.

To be honest, Drools is not nearly as intuitive as Quickrules was from what I have seen in this first task. However, in Part 2, I will start looking at using the rule flows, decision table functionality as well as the guided rules:

Those in partnership with the DSL (Domain Specific Language) construct hopefully do allow for more intuitive rules creation.

Less code - More rules

2010-11-03T05:47:00.001+02:00

Less code - More rules...

There I said it...Now before condemning me to the pits of hell let me add some context.

In large corporate financial development where systems are comprised of thousands of business rules and run in production for many years... rules, rules, rules are the key... more specifically the management of rules with a Rules engine and/or a BRMS(Business Rule Management System). Now I know, some people love them, some people hate them and some "wrote their own"... personally I think they add a huge amount of benefit when used within the correct context.

I have worked in an environment where we have both successfully and unsuccessfully used a BRMS(Yasutech Quickrules) in implementing countless rules, flows, decision tables over a 5 year period. Unfortunately Quickrules is no longer actively supported, SAP has squished them into SAP Netweaver BRM.

I work in the health insurance industry, and there is a vast amount of analysis, risk management and business rules that go into the "simple" act of getting a stay in hospital paid. What makes it even worse is that these rules change, and they change often, there a constant stream of changes due to legislation changes, overly smart actuaries or overly zealous marketing department announcements. All this creates an environment where there are literally thousands of volitile rules, calculations, scenarios. Now add the fact that large corporate applications like these sometimes live for a decade or more and having business rules spread between code, store procedures, data tables and web pages and you have a recipe for disaster. (These are the systems we fortunately get to rewrite them from time to time.)

Based on my experience over the last couple years; the biggest single benefit of a BRMS is simply "separation of concerns".
What this separation allows for is:
1. Rules to be developed, tested, and possibly even deployed on a different time
line from the application itself.
2. The use of different resources. We managed have both the architect and business analysts define, develop and test rules with very little effort from the developers. I know this is a scary concept, and we only did it after years of having everything in place but with the correct standards, peer review, testing and training it is a possibility.
3. More business involvement and ownership, having the ability to simply let the business owners work with their own spreadsheets and import them directly (into development environment) helped cut down on the red tape and actually made them think about their requirements for a change.
4. More quantifiable delivery segments and estimates for those pesky project managers.

One very important thing to note is that even though rules can be handled separately, they still need to be part of the development and quality processes. After all, the business rules (depending on system) are what it's all about in the end. Most of what we do in these financial corporate application is to facilitate the running of business rules. If anything there should be stricter control and quality assurance on the rules than on the code.

The second thing worth mentioning (which is also actually a by-product of the separation) is visibility of business rules to people that understand them. To be able to print out and sit with either the business analysts or the actual business representatives and visually discuss the flow and structure of the rules proved to be very helpful.

There are of course many other benefits to BRMSs auditing, performance and management, but to me the separation and visibility is were the real value lies.

So with Quickrules gone and with Ilog JRules and Blaze Advisor not available to the common man, I turn to Drools.
Over the next couple weeks I need to spend some time ... Learning to Drool...

On first glace Drools does seem a little daunting, split into :
Drools Guvnor (BRMS/BPMS)
Drools Expert (rule engine)
Drools Flow (process/workflow)
Drools Fusion (event processing/temporal reasoning)
Drools Planner..

Hopefully there is a simple, quick way to get to grips with it all as I would actually like to run a concurrent proof of concept on my next project to see how it compares.

Top 7 of 97 Things Every Software Architect Should Know

2010-10-25T08:59:00.001+02:00

Last week I commented on the 97 Things every programmer should know. I figured I would do the same for the 97 Things Every Software Architect Should Know. Initially I thought there may be some overlap between the 2, but they have done very well to focus each book on the targeted readers. I did however feel that quite a few of this "97 Things" were very similar in the principle they were trying to explain and so I grouped them under the relevant principle.

Before I start I just need to comment on the first chapter of the book:
Don't put your resume ahead of the requirements - Nitin Borwankar

I am not sure how I feel about this one. I understand the point and I do partially agree, but it also doesn't mean that the "latest shiny object in the latest shiny language or the latest shiny paradigm" isn't the best thing for the company or project. The choice of some new technology for a solution is a very difficult one and should not be taken lightly. Staying with what is already known is definitely the safer option and possibly the correct choice for a project or company in the short term, but when does this become something that actually does more harm than good?

Moving along to my top 7 things:

1. Understand The Business Domain - Mark Richards
Us "systems" folk, often experts in our own field don't always spend the time to understand the actual business we are involved in e.g. insurance, finance, marketing. To be a more effective software architect appropriate industry knowledge is as important as staying up to date with tech.

2. Before anything, an architect is a developer - Mike Brown
"If you design it, you should be able to code it."
I am a firm believer in the above statement. Architects sometimes don't have the time or the will to keep their developer skills current and over time let these skills atrophy. This can then lead to grand designs, with ugly implementations with unhappy developers

3. Dealing with Developers
There were a couple topics on interaction with developers, I decided to group them into 1 point:
Find and retain passionate problem solvers - Chad LaVigne
Give developers autonomy - Philip Nelson
Empower developers - Timothy High
An architect may be a developer, but he also needs to lead them, like it or not, architects by default have a leadership role to play when it comes to the developers on their project. Developers left to their own devices will run wild (we always do ;) ). So as with most things in software architecture it's all about balance:

if (Too much control)
developers = unhappy
else if (Too little control)
customers = unhappy

if(developers or customers are unhappy)
result = bad

I believe the relationship between developers and architects is one of the many crucial factors in the success of a project.

4. Performance
There are 2 topics about performance:
It's never too early to think about performance - Rebecca Parsons
Application architecture determines application performance - Randy Stafford
In my current environment where transaction processing times are tightly monitored and contracted with SLAs that have financial implications, performance is always top priority. However in many other teams / projects / companies I have worked in the emphasis on system performance is almost always left to the end, and sometimes no amount of hardware is going to solve the problem.

5. Record your rationale - Timothy High
This is something that I feel is often neglected. Quite recently a project that I had been involved in for a long time hit the spotlight for all the wrong reasons: customer, user and management dissatisfaction. The first thing to be questioned was not the analysis, requirements, testing, management or expectations, but rather the architecture. Documentation discussing all the decisions, options looked at and reason for options taken would have been valuable. When things go fine, no one will even know about the document, but when things turn bad as they sometimes do having justification and documentation for all the major decisions will be a lifesaver.

6. Stand Up! - Udi Dahan
"The easiest way to more than double your effectiveness when communicating ideas is quite simply to stand up."
This is more of a general point rather than one just for architects, anyone attempting to influence people and situations can benefit from this principle.

7. Great software is not built, it is grown - Bill de hora
"The single biggest predictor of software failure is size; on reflection there's almost no benefit to be had from starting with a large system design."
"Have a grand vision, but not a grand design."
It is all too tempting to jump in and design the total solution. By starting with a basic, small, solid system implementation and growing from that base over the life of the project always with a grand vision in mind, you allow the software to evolve rather than just stuck together. Evolution turned out ok for us, no reason it shouldn't be the same for software.
This approach should not be confused with prototyping, there shouldn't be any quality shortcuts taken and it shouldn't be thrown away.

All the contributions for both books are actually available on the O'reilly site:
Programmer
Architect

There are also a number of "things" not included in the books available on the wiki.

//ignore the next bit
Technorati claim:987BEU5BKS8H

Top 9 of 97 Things every programmer should know

2010-10-18T08:28:00.002+02:00

I recently finished 97 Things every programmer should know. Well to be completely honest I did skim over a couple of the 97, but all and all this was a very nice compilation of thoughts and topics about software development from very experienced authors. Well worth a read.

A couple of the "97 Things" discussed, stood out more than the others, the following would be the ones I rank as my top 9.

1. The Boy Scout Rule - Robert C. Martin (Uncle Bob)
"You don’t have to make every module perfect before you check it in. You simply have to make it a little bit better than when you checked it out."

To be honest this is not something I have followed throughout my career, and although I certainly try improve code where I can, I never did it per check-in. I do however feel that it is an awesome principle and should be something that is actually part of a code review process. It is all to easy to just say:
"It was like that already"
"that nasty code was there for years, I am not going to touch it."
"It never had any tests"

I work in a corporate environment were applications often last for 4-10 years. If part of the process is always to just make something a little better, everything from deleting unused code to writing a single extra unit test, year after year... it will end up with saving a lot of people a lot of time and money.

2. Beauty Is in Simplicity - Jørn Ølmheim
"The bottom line is that beautiful code is simple code."

Over the years this has become particularly important to me. Earlier in my career, especially when starting somewhere new I would design the crap out of something.
It would be: "Welcome to the school of over-engineering", everything would be abstracted to the nth degree, there would be patterns for the patterns, interfaces on interfaces for the abstractions and a huge amount of code and components catering for every "what if" that ever existed, that only I understood. All of this would lead to very "fancy" code, and me stroking my ego, but at what cost?
Only the minor little one: an utter nightmare to maintain. I have for a while now favored the simplest implementation of almost any solution, even if it is not necessarily the prefect technical implementation. In the world of long living software, maintainability should quite often outweigh other concerns.

3. Step Back and Automate, Automate, Automate - Cay Horstmann
I believe in automating everything possible: builds, deploys, code analysis, unit testing, functional testing, integration testing. None of us actually want to look at that stuff everyday and automation is the only way to get away from that. Can't say automate enough. Automate, Automate, Automate.

4. Continuous Learning - Clint Shank
This is a very important topic, we are in a industry that is constantly growing, changing, shifting and as a programmer you need to be learning and improving yourself wherever you can. It's very easy to get into a comfort zone and just rest on your laurels, I did that for a couple years, and I do regret it now.
Things I am trying to do to keep up and would recommend:
1. Get a Kindle... then buy & read books.
2. Use Google Reader add the popular blogs and website RSS feeds for your specific field as well as a couple outside your field that interest you.
3. Start a blog, by putting my code and thoughts out there, I put in more effort knowing that it's going to visible than if I just wrote the code/article for myself. I also force myself to do 1 - 2 posts a week, ensuring that I must always find new content to learn about.
4. Join an open source community, we generally don't get to do enough "technical" development in our corporate environments.

5. Check Your Code First Before Looking to Blame Others - Allan Kelly
Exactly what it the title says, we all look to put the blame on anything other than our "perfect" code. Everything from the OS to the network guys, the DBAs to the JVM, 3rd party libraries to other teams' interfaces are always blamed first. This often leads to countless hours of wasted effort, or completely avoiding the issue that eventually comes back to bite you in the butt. Most of us can mention moments where we have done this, it's frighteningly common.

6. Hard Work Does Not Pay Off - Olve Maudal
"truth is that by working less, you might achieve more"

"If you are trying to be focused and “productive” for more than 30 hours a week, you
are probably working too hard."

I couldn't agree more with Olve Maudal. A lot of us have been there, we have spent the days, weeks, months at work, but we often don't see the negative sides of what happens when you work 50-70 hours a week. Everything from common logic to motivation and team dynamics go out the window. Even if the short term goals are achieved, quite often the long term repercussions are a lot worse. No one goes back and see's why they have to re-write a system 4 years down the line because the architecture or code is failing. I'd love to know how many of those were because of ludicrous hours and impossible time lines. I can think of at least 2 "re-writes" that I was involved in where you could see it was the result of long hours. People you know to write great code end up hacking solutions at 1am in the morning.
There are times and there always will be, where a couple extra hours are required, just try keep them down to a minimum.

7. Comment Only What the Code Cannot Say - Kevlin Henney
Exactly that... Don't tell another programmer what he can see in the code. Comments stating the business reason or requirement behind a particular complex algorithm is a lot more useful than //loop through the outcomes and add 1.

8. Know Your IDE - Heinz Kabutz
Something a lot of us are guilty of, we spend a couple hours a day in our IDE, learn the shortcuts, learn the features, we probably couldn't begin to calculate the time that would be saved if everyone used their IDE to it's full potential.

9. Learn to Estimate - Giovanni Asproni
I feel this is something that comes with experience, I pride myself on my estimates and over the years I have tried to nurture this "skill". I have a couple points I would like to share on how I get better estimates:
1. Be honest with yourself, make sure you know what you think you know, and openly admit what you don't. This is the quickest way to end up in trouble.
2. Keep track of what you do and how long it, not necessarily for the project manager, but for yourself, see point 1.
3. Don't rely on other peoples perceived skills and timing, see point 1.

Will try get through the rest of 97 Things Every Software Architect Should Know in the next week or so, will be interesting to see how it compares to the programmer one and how much overlaps.

Creating your own custom Spring configuration.

2010-10-11T00:19:00.001+02:00

Recently I volunteered myself to do some of the Spring integration for Hazelcast, for one main reason actually.. I had always wondered how to actually do that. It is something you don't get to do often in a corporate working environment. As with most things Spring related it actually turned out to be very simple.

First thing I needed to do was define a schema for the Hazelcast xml config. If you want to add your custom XML configuration into Springs', it will require that you have a schema for the specific XML.

So what is the quickest way to get started on a schema? Altova XML Spy, it generates a basic XSD form a sample XML which takes care of about 40% of the actual work. Then it's all about the details, min / max constraints, enumerations, defaults, attributes and documentation. I believe XSDs should be verbose, so the version below is not 100% complete but it is a good starting point.
I am not going to add the XSD to this post as is quite long and doesn't add much with regards to the actual Spring integration. If interested, it is available here...

On the Spring side you need the following:
1. An implementation of: NamespaceHandlerSupport, this basically tells Spring when it finds "hazelcast" use the "HazelcastBeanDefinitionParser".

2. An implementation of: AbstractSimpleBeanDefinitionParser. This implementation actually processes the XML from the Spring application config. You need to override
getBeanClass and doParse methods, to specify what type of object and how to create it respectively.

3. 2 files in the META-INF
3.1. A spring.handlers file containing the namespace and the NamespaceHandlerSupport implementation (in my case):
http\://www.hazelcast.com/schema/config=com.hazelcast.spring.HazelcastNamespaceHandler

3.2. A spring.schemas file containing the location of the schema:
http\://www.hazelcast.com/schema/config/hazelcast-basic.xsd=hazelcast-basic.xsd

and that is it...

The XML you described in your XSD is now available to be used within your the Spring config.

The example below is a Hazelcast map cache config, which can then be injected where required as ref="config".

The source and tests for this are available in the Hazelcast code project here.

Spring 3: Spring Expression Language - SpEL

2010-10-05T00:21:00.002+02:00

I realized that I have covered most of the major functions in the Spring 3 release, but completely overlooked SpEL. Now even though I am a huge Spring fan, all this functionality does actually scare me a little bit. To often have I seen people hang themselves with awesome, innocent looking ropes just like this one. Soon there will be some project that needs to be "dynamic" and some person/consultant/architect/developer will have this amazing idea and place code somewhere they can edit it externally. Then this SpEL can read this and "dynamically do stuff"...
(I am pretty sure that I have been that "person" somewhere in my career, it all feels horribly familiar.)
And all of that only comes at the price of some poor developers' sanity trying to maintain it. To be fair, this view is purely based on personal experiences and maybe slightly unjustified, but I have seen my share of good ideas gone bad in my time and it feels like SpEL is wide open for misuse.

Putting my concerns aside and refocusing on: it's something new , it's shiny and Spring-made so I obviously can't resist trying it out.
First I go digging for information to answer: "What is SpEL, Spring Expression Language?"
A quick summary of what I find:
It is an expression language that supports querying and manipulating objects at runtime. It can be used as part of the standard configuration, XML or annotation based, but can also be evaluated directly by the application code. It allows you to overcome the limitation of using only fixed values. It can be used for bean definitions in both XML- and annotation-based configurations. There is a facility to access system, environment and context parameters.

I have just covered thing that I would generally need or have run into... there is however quite a bit more information within the Spring documentation

So using Maven first thing I needed was to add following:

Using SpEL in Code:

Below I have examples code of: Setting a value, Calling a constructor, creating variables, calling a static function, all the basic operators, the instanceof operator and even the ternary if.

Using SpEL in XML Config:As with the Java code, here is the same functionality within you Spring application XML.

Using SpEL in Annotations:
And then again will the @Value annotation.

Guice & Hazelcast.... Method Caching Interceptor

2010-09-23T00:21:00.004+02:00

A couple days back I did Spring, AspectJ, Hazelcast Method Caching Aspect and I received an anonymous question on how to do the same thing in Guice I had heard of Guice, I knew what it was, but... I have been a Spring zealot for many years now, so my first reaction was:
What??? Guice? me?? ... Not use Spring? ... why would you ever not do this is Spring?... after a bit more whining I finally found myself saying "Oh ok, let me do it in this Spring wannabe "Juice" framework, just to prove that Spring is better."

All I can say is dammit! I hate being wrong ... the Guice is sweet. It took me about an hour to go from never using it before to having this AOP Method Cache Interceptor project done. A lot of that time was me being a typical developer and playing rather than actually reading the docs. In other words, it's simple, and that is always a good thing.
Now before someone takes this the wrong way: I know Spring does a lot more, Guice is focused on DI, Spring integrates with more, most of the popular Google apps run on Guice so it is rock solid etc. etc... I am not doing this as a comparison and definitely don't want to choose sides, but rather just showing another way of doing something.

I will only be posting the interesting bits of this project here, the full source is available here :
Source Code

First thing, Guice is available in the Maven repo and they recommend using that rather than the google.code site. The POM for this project:

Guice (as they say in their documentation) takes full advantage of annotations and generics. So for this project I created a @Cacheable interface to allow me to annotate the methods I wanted cached.

To configure Guice you need to extend an AbstractModule, this is where you configure what gets injected where by what. In the example below I use a construct called a Provider to allow me to use the Hazelcast factory to inject and instance. One other important thing to note in the code below is the "requestInjection(cache)" when injecting into interceptors you need to specify that in order to that injection before actually using the interceptor.

The AbstractModule:

And lastly for the actual interceptor. Right at the bottom of the source below you will see the @CacheName interface, this is one of a number of ways to map configuration values to be injected. The interceptor implements MethodInterceptor and you get all the required information about the method execution from the MethodInvocation parameter.

So there you go Mr Anonymous, hope it helps.

Spring, AspectJ, Hazelcast Method Caching Aspect

2010-09-20T08:52:00.000+02:00

After my recent discovery of Hazelcast, I decided to update my caching aspect project to include a Hazelcast implementation.

The source code for this project is available here

For more information please refer to the actual source or my original Ehcache post. The main difference here is that the Hazelcast implementation is by default a distributed cache, and takes advantage of all the other functionality that Hazelcast provides.
The main points of interest for this implementation are:
- The actual aspect class. See code below.
- The application context. See xml below.
- The Hazelcast configuration. I am using the default Hazelcast configuration, located in the hazelcast-version.jar. To use a custom one, you can define a hazelcast.xml and inject it via Spring.
- The CacheKeyStrategy in the code below is an interface defined to allow key generation for the cache for specific requirements. There is a default implementation that simply sorts and uses hashcodes of the values in the class. You can define a strategy per type you want to cache.

note: java docs only removed for size of post

The aspect:

The application context:

The test case:

Hazelcast - a simple distributed caching alternative

2010-09-13T00:13:00.002+02:00

I have over the last couple years used EhCache either via Spring (with AOP) or having it configured as hibernates' cache. I have never had the time or the need to look into the whole distributed caching available with Terracotta. However I recently stumbled on to Hazelcast and read the following:

"Hazelcast is pure Java. JVMs that are running Hazelcast will dynamically cluster. Although by default Hazelcast will use multicast for discovery, it can also be configured to only use TCP/IP for environments where multicast is not available or preferred. Communication among cluster members is always TCP/IP with Java NIO beauty. Default configuration comes with 1 backup so if one node fails, no data will be lost. It is as simple as using java.util.{Queue, Set, List, Map}. Just add the hazelcast.jar into your classpath and start coding."

I was instantly intrigued. So it sounds simple enough...Open IDE...jump in... being a maven fan, I figure, why download and manually "install" when you can just use maven, so I search around a little and add the following to a pom.

Edit: 1.8.4 is now outdated check http://www.hazelcast.com/downloads.jsp for the latest.

From the online documentation, I saw there was an InstanceListener interface. I create a simple Main class, that implements that and lists the instances.

So running this just adds a listener to the Hazelcast instance. According to the documentation Hazelcast allows for the following (and lots more):
# Distributed java.util.{Queue, Set, List, Map}
# Querying distributed data

I created a couple Main classes to run these separately, so that I could see if they are usable across different JVM executions.

So within minutes... I have a distributed cache across 4 applications, that I can monitor, control, query... (and btw. Hazelcast allows you to add indexes to the caches, same as a database to optimize queries).
If I was to place these 4 applications on 4 different machines, they will by default cluster (multicast for discovery) and have built in failover, monitoring and even persistence if required.

Coming from an environment where we have dozens of servers all caching data, this has got to be one of the most exciting open source products I have come across. Now I just need to convince the powers that be to prototype this and this may be the first open source project I actually try get involved in. The possibilities I am seeing for this type of functionality and then integrating it with the Spring Framework may actually make my brain explode.

Developer / Architect interviews, looking at it from both sides. Part 2

2010-09-07T20:28:00.001+02:00

Part 2, The Interviewer...

In Part 1, I covered things that would possibly aid you as an interviewee, now onto the other side. I will be the first to say, I still have lots to learn about interviewing people, and some of what I list below, I need to practice as well.

All companies have their general interview process, being certain technical tests, standard questionnaires and particular processes. Going beyond those -because they are a necessary evil- other things I feel important when interviewing a developer are:

1. Probably the most important thing I look for in a developer, being a developer, is a passion for development. Sadly this has also proven to be the hardest thing to find. There was only one candidate out of the 20 something interviewed that showed some passion for development and he ended up accepted an architecture role, where he probably won't get to develop. I feel if someone shows passion for what they do, they will learn faster, be more focused, be more positive and generally a much safer bet to employ. Judging a developers' passion in an interview can be difficult, the best way I can think of is to explore an area of their expertise. Taking myself as an example all you would need to say is: "So I see you worked with the Spring Frame..." and that would be the rest of the interview.

2. When you receive a candidate's CV, read it, make notes, add specific questions. Do this before the interview as once you are in the actual interview it is far more important to pay attention to the candidate. During this preparation look for discrepancies, these creep in, things like someone rates themselves 4/5 in say JavaScript or some application server for example... but then none of the jobs / projects they mention used that specific technology. Make sure to find out the detail, as this could be a miss communication somewhere between the candidate, recruiter, HR department and you, or actually something alluding to a "creative" CV. Always check for job hopping, in areas where there are large systems running it generally takes a developer 3-6 months to become really productive, if someone leaves within a year it's actually a huge loss.

3. Try get the candidate to relax, people do not show their actual personality under stress. So begin the interview with light personal questions, none that will get you in trouble with the HR department, but just enough to allow the candidate loosen up before jumping into the technical stuff.

4. Make sure everything required for the interview is available and ready. A clean whiteboard and markers or paper and pens if the candidate needs to draw or describe anything. Having both available is preferable as some people are really not comfortable drawing on a board.

5. People sometimes don't feel very comfortable asking the interviewer questions, specially in the case of panel interviews. Prepare a standard list of points that would answer questions the candidate didn't ask. I mentioned some in Part 1 and some examples as an interviewer may be:

The regular day /week in the life of a developer here looks like: weekly dev meetings, daily meetings close to deadline, multiple tasks, tech specs, code reviews and unit tests, etc etc.
We support 15 different application on a 24/7 basis every couple weeks.
The team size and structure is as follows: 12 people, 1 team lead, 1 architect, 6 developers...
The career path here is defined as follows Developer - Senior Developer - Architect - etc.
We supply you with a choice of laptop or desktop, duel monitors, 4gb RAM, IDE is up to you.

6. 'Go with your gut'; the general interview process is actually too short to be able to completely assess someones skills and compatibility. We work in a very high stress environment, with hundreds of different scenarios everyday and I don't think we can determine if the candidate will cope or excel within a 1 - 2 hour period. There are no hard and fast tests that will determine if the candidate will take responsibly for their work. If they will complete things in a timely manner under pressure. If they will boost the morale of those in the team, or will they turn out to be a complainer that will to jump ship at the earliest opportunity.

My final point to end off -something I heard from my wife's boss- which appealed to me greatly. This will probably get some reactions from people and unfortunately my current boss did not allow me to actually do this... but... In a perfect world, where you get a fair amount of high quality CVs, I would personally love to divide the large stack of applications in half and toss 1 half of them directly into recycle bin. If you are unlucky, it's probably better you find work somewhere else.

Developer / Architect interviews, looking at it from both sides. Part 1

2010-09-06T08:17:00.001+02:00

Part 1, The Interviewee...

Firstly let me say, I am by no means an expert interviewer, I have just recently been included in the whole interview process. I originally wanted nothing to do with it actually, but now several months and 20ish interviews later I quite enjoy it. The only frustrating thing is how people actually come to an interview, in one word: Unprepared. This may be a South African thing, where the demand way out strips the supply when it comes to software developers, but sadly the quality of the candidates coming for interviews is really disappointing. I am going to try keep this technology agnostic, as it is not only limited to Java, my wife -a dev team manager in a .Net technology company- goes through the same frustrations when looking for new staff.

So out of the 20ish candidates I have interviewed in the past months only 3 potentially had the skills we required, 1 was hired, 1 accepted an architect position at another company rather than the senior developer we were offering and the other's personality was not a match for the team. The other 17-20 people were either unprepared, unskilled or incorrectly represented by a recruitment agency.

I personally have not been for an interview in about 4 years, so my own interview skills may be a little rusty. However if I had to apply some of my experiences of being the interviewer into being the interviewee. I would do the following, above and beyond the standard interview success practices of being presentable, on time, professional, confident and having a self sales pitch.

1. If you are going through a recruitment agency, make sure that they show you the CV (Curriculum Vitae / Resume) they will be sending to their clients. Make sure you are happy with it. I would say about half the CVs we get really do not accurately describe the candidate. When this happens it immediately changes the tone of the interview and your chances diminish greatly.

2. Find out upfront what kind of tests the company makes candidates do, and prepare for those.
In the case of my current employer:
2.1 There is a general "aptitude" test, which I personally don't agree with, but it generally covers classic word sum, riddle, trick and what's the next in the sequence type questions. There seems to be a lot of companies doing this kind of test these days. So it is well worth searching the web and practising this kind of test. One site I did find quickly India Bix, would be a good way to prepare.
2.2 A Tech Check, which is a scaled down version of the official certifications and as such the same preparation can be done for those.
2.3 Finally after the initial interview there is a long Psychometric Test. Which you can't really prepare for.

3. Find out who is interviewing you, and just to a quick web search on them. They may have a blog, a book, a twitter feed, public facebook account all of which can give you vital information that can drastically alter the entire interview. Take myself for example, if you knew I was interviewing you and you found my blog, you would find this article along with a whole bunch of technical stuff about the environment / company that can give you a very nice advantage in the interview.
I Google all the candidates, only fair that they do the same.

4. Stay Current, we are in a ever changing industry, companies and your current employer may not be able to keep up with the changing technology environment, but this doesn't mean we as IT professionals shouldn't. As an interviewer I often ask the questions, "how do you stay current?", "what do you read?". How I try stay current, is this blog and Google Reader, I follow and read many Java, development and architectural related websites and blogs. I also plan to get the new Kindle in the next week or so, as there are quite a number of books that I need to get through.

5. Make sure that you read and know the common interview questions for your technical specialities. There are a lot of resources online to help with that. You don't want to show up to an interview as a Hibernate, Spring, Oracle or Asp.net 'expert' and then get put on the spot by a relatively simple well known interview question.

6. Ask questions, from the interviews I have done very few people have even asked the basic questions. It is important to ask questions, firstly there are things you should know about your working environment and you to engage the interviewer. Some example questions I would ask are in no particular order:
6.1. What does a regular day in the life of a developer or architect here look like?
6.2. What applications are you supporting? What are the support hours like?
6.3. What is the team size and structure?
6.4. What is the career path and policies on training?
6.5. What IDE, and what kind of hardware will be supplied?
6.6. Why are you hiring? Is it because the team is growing? (or are people jumping ship... don't ask that :) their answer will imply that.)
6.7. What are the greatest challenges facing the new person in this position?
6.8. What is the management style of my direct manager?
6.9. Who will evaluate my performance? When? How?

On a side note:
If I was to find new employment I would firstly research companies I want to work for and see if I could directly deal with them, as I stated before recruitment agencies can sometimes do you more harm than good. Search the top 100 companies to work for in your country / city, check the technologies they use, their culture, their location. I would also use social networks, a candidate that comes recommended from a current employee already has a huge advantage to just some random Joe Smith off the street.

In Part 2, I will discuss some of my experiences on the other side of the table as a interviewer.

Chunking a List into parts.

2010-09-01T19:04:00.001+02:00

Needed to batch up the contents of a list into groups of 4 today. Spent a little more time than I would have liked on it, but I think it's a pretty neat bit of code now, and I actually got to use java.util.concurrent.CopyOnWriteArrayList for the first time.

The result of running this:

Top level list 1 contains:
1
2
3
4
Top level list 2 contains:
5
6
7
8
Top level list 3 contains:
9

Cargo, Maven and the benefits of integrated functional testing

2010-08-28T10:01:00.000+02:00

I little while back an ex-colleague mentioned that I should have a look at an open source project called Cargo. So what is Cargo, their mission statement states "Cargo is a thin wrapper that allows you to manipulate Java EE containers in a standard way." and that is exactly what they do.

In 2 of my previous posts:
Some useful Maven configurations
and
Your POM is quite nice

I went through some maven configurations and practices along with the concept of continuous integration. This article and Cargo make a logical "Part 3". In the above 2 articles there is a lot of emphasis on continuous testing, but with some very large / legacy systems total unit test coverage is something of a luxury and quite often neglected. Depending on the size, type and deployment of the application there will be certain circumstances where to quickly gain, promote and gauge system stability, integration or functional testing will be a much faster and more beneficial practice than retro fitting unit test coverage.

Tools like Selenium, QTP (QuickTestPro) a number of others, allow you to simply record, playback and validate the application. In my current environment and I am sure in a lot of other places, these types of tests are run by people outside the development team, namely testing teams, clients or even business analysts. I personally feel to gain the most benefit out of functional testing and automation tools these need to happen at build time. Having these tests run outside of the standard development environment add extra unneeded iterations, and more external dependencies that can delay a software project. Bringing these type of tests into you build process allow you detect, maintain and repair system issues before influencing the world outside the development environment.

Like everything else surrounding the "continuous integration" philosophy, automation is key. This is were Cargo helps you out. If you already have the other tools like Maven and Hudson setup it is very quick to integrate and benefit from Cargo.

My environment runs on Weblogic, so that is what I will be using as my example, but Cargo supports about all the major web and application servers used. So this should work for anyone.

Maven has the concept of phases which can be thought of like a collection of goals. A full list of the phases of Maven is available here: Lifecycle Reference

The 3 we are interesting in with regards to Cargo are:
pre-integration-test
perform actions required before integration tests are executed. This may involve things such as setting up the required environment.
integration-test
Process and deploy the package if necessary into an environment where integration tests can be run.
post-integration-test
Perform actions required after integration tests have been executed. This may including cleaning up the environment.

So any phase higher than ’pre-integration-test’ will trigger the deployment to your application server.

Cargo can only stop and start local application servers, deployments can take place remotely, not all application servers have "deployers", detail for these are on the Cargo site. The one feature I am very happy with is the server properties. You can from within you POM configure all the required, JDBC, JMS, and JVM settings you require for your application server to function, which is awesome. In my examples I just needed JVM and JDBC settings, but theoretically you should be able to configure most things you may require.

You could integrate the Cargo plugin into your existing POMs, I prefer to keep it separate. I am not including any Selenium configuration to keep this post about Cargo. However here is a very nice Maven Selenium Guide.

So for the local server configuration, this will startup the application server, create the JDBC configuration and deploy the war.

Pimp your Eclipse, a collection of darker colour schemes

2010-08-17T07:26:00.000+02:00

Since my recent disappointment with NetBeans, I figured I would make myself feel a bit better by customizing my Eclipse to almost look a little like one of the themes that come with Netbeans. Now Eclipse doesn't really cater for "Themes", more just colour schemes that you can import. There are 2 basic ways to import bulk settings...

Firstly, make a backup! (I don't wanna be blamed for lost hours :))
To do that in Eclipse, go to File > Export > Preferences and export all the preferences. This will export all your settings so you can quickly roll back.

Now the 2 ways to update (besides changing every setting manually) are:
1. Import.
If you have or download a *.epf file you can import. In Eclipse through File > Import > Preferences.

Or
2. Manual.
The settings for the basic and java editors are stored in the following 2 files:
[workspace]\.metadata\.plugins\org.eclipse.core.runtime\.settings\org.eclipse.jdt.ui.prefs
[workspace]\.metadata\.plugins\org.eclipse.core.runtime\.settings\org.eclipse.ui.editors.prefs

The problem with the import is that it can contain ALL the settings of the person that exported it, there is no option to only export the UI bits, so I personally prefer the Manual route, but both work fine.

So to begin, my theme, Dark Blue Eclipse:

The 2 perfs files can be downloaded from, and copied the manual way:
Download List

I tried the following theme, and quite liked it, however I did miss blue:
Prabir's Blog

I also came across the following collection of themes and what is nice with the links here is that srand did it the manual way, and the other is a clean *.epf import ( I checked the files, no unrelated settings):
srand's blog
Which actually come from:
Eclipse color themes

Inkpot:

Sula:

Vibrant Ink:

Wombat:

I have not actually tried the following site but it should be very useful if you are looking to create your own colour scheme:
Edwards research Eclipse Gen

Spring, JMS, Weblogic, and FizzBuzz

2010-08-16T00:13:00.001+02:00

There are 2 reasons for me posting this example:
I have not looked at anything Spring related for about a month, and last week I got to show someone new to Spring all the joy that it can bring, I couldn't shut up for about a hour. One of the reasons he is looking into using Spring happens to involve JMS, something I had not yet covered in my blog.
The other reason is:
I was challenged by a work colleague to come up with a more complex FizzBuzz implementation than his monster rule implementation and not being a person to back down on a challenge.

Here it is. Spring, JMS, Weblogic and FizzBuzz

All the code for this example will be available --Here--

There are a couple JMS examples out there using ActiveMQ, and either JMS "client type" implementations or Message Driven POJOs but not together. For this example I will show both methods and will use Weblogic, those that have worked with Weblogic know there are always little nuances. The first thing is to have a standalone client for Weblogic, you need the weblogic.jar (Full Client), I have uploaded this into my local maven repo so that I can just place this within my POM.

In the Weblogic admin console I set up the JMS modules as follows (5 Queues and a Connection Factory):

Now for what this example does:
* It creates 100 messages...
* It reads 100 messages... while reading, it checks the content, depending on the value it creates a message to a fizz queue or a buzz queue or a fizzbuzz queue and a result queue.
* There is message driven POJO on the result queue that prints out the content.

How it does this:
The Spring application context below is pretty straight forward but it does have a couple things to note:
1. The jndiTemplate, jmsQueueConnectionFactory, jmsDestinationResolver, jmsTemplate are all used in combination to get to the application server, and connect to the actual JMS queues.
2. The "messageDrivenListener" and "queue" beans are the configuration for the message driven POJO. In that config is "concurrentConsumers", this allows you to specify how many listeners will be created to process the messages on the queue. For this example I only require 1.

I have created just a simple example message structure for this, with a generic content which looks like:

The message driven POJO is just the standard javax.jms.MessageListener interface implementation:

The client classes to read and create messages, make use of Springs JMSTemplate to simplify your interaction with the JMS server:

And the finally the Main class, that creates the 100 messages, then reads them, publishing messages to 4 different queues. The reading of the results queue is triggered by the application server and handled by Springs "DefaultMessageListenerContainer"

Note: if you constantly run this example, you will end up with a ton of messages on the Fizz, Buzz and FizzBuzz queues as there is nothing reading those messages to clean it up.

NetBeans 6.9 and 6.9.1 find usages not working.

2010-08-11T07:40:00.000+02:00

Yesterday I was investigating an issue, during my investigation I started following the code trail, I finally get to a place where the when calling "Find Usages" on a method returns me nothing... I think great... Delete, Delete, Delete... cleaning up unused code always takes me to a happy place, so after this I just check with someone that had originally worked on this bit of functionality which goes a bit pale, stating that there has to be something wrong and that code must run...

I quickly revert back to local history, do the "Find Usages" again, and yip still no reference. We go to the code where we think it should be called... and surprisingly it is there... go delete the code, do a "mvn clean install" and it fails correctly...

Off to Google... "netbeans find usages not working" leads me to the following logged bug: Bug 186314

As I commented there, I installed the latest 6.9.1 version and the issue still occurs.

This is such a nasty bug in my opinion that I will probably have to go back to Eclipse until it is sorted. I have just recently changed teams and have roughly 14000 java files that I know nothing about, life without "Find Usages" will be painful.

Maven Remote Windows Scheduled Task

2010-08-10T22:40:00.000+02:00

Just a quick Maven / Windows config I came across the other day, I have recently switched teams and they are using Maven and Hudson to control some of their server tasks. I figure it could come in handy sometime.

To start and stop a Windows Scheduled task from a Maven pom, actually using the ant run plugin

Replace the [Server Name] and [Task Name] obviously.

Domain Name Change

2010-08-07T08:41:00.004+02:00

Since I am now quite addicted to this whole blogging thing, I decided to "move" my blog to new url: www.briandupreez.net... it's just a lot simpler to tell someone my name + .net than java hyphen it hypen zen . blogspot.com... which will still work anyways.

www.briandupreez.co.za should also be routing to the .net address, I say should as I played and broke a couple DNS settings trying to get everything pointing to the right place and found a pretty useful site if you ever have to check out what your DNA settings look like to the rest of the world Pingability... There is a link to instantly check your domains relative mappings Quick Check.

Glassfish ESB, easier than you think. Part II

2010-08-03T06:21:00.000+02:00

To continue from Part I, which simply just went in and out of the BPEL process, I will now add a couple EJBs to show a nice little Glassfish ESB structured activity, namely: Flow

Since this is now getting a little larger, I have made all the demo source available ... here.

So using the EJB project Wizard I create 2 EJB projects: DemoEJB1, DemoEJB2
once you have the project, Add a webservice:

For demo purposes I am going to have 2 web services with 2 methods, gatherAppData and gatherHistoryData, these 2 functions can happen concurrently and cause you no pain as long as you change different data in your object model.
Once that is done, from the project menu select the web service, drag it over to the right hand side of the BPEL editor, on the orange circle.

Then update the Partner Link information that pops up.

To execute the 2 web service calls concurrently you will need a "flow", grab that from the structured activities.

From the web service palette grab 2 invoke objects and place them in the flow.

Edit the invoke objects, choosing the different webservices.

Create variables for both, in and out. Try stick to a naming convention of some sorts as when you get to 20ish components it can get very messy.

So now the BPEL will look something like the following.

You can add as many concurrent steps to a flow as you like or as many different values you have within your object model.
To get these 2 invokes to function we need to assign the input and output variables and then assign the result of both to the output variable of the BPEL process. For each assign, map the input variable of the BPEL to the input variable of the relevant data for the service that is being called. In the 3rd assign map the relevant part of the each output variable to the output variable of the BPEL.

just for clarity here is the last assign in the mapper view.

And that is it... 2 concurrent web service requests within a Glassfish ESB process

The final steps are to add the newly created EJBs to the "Service Assembly" in the composite application, you can just drag the EJB project directly onto the middle JBI section, build, deploy and test.

Eclipse -clean

2010-08-02T11:06:00.000+02:00

I am sure most Eclipse users have run into some "funny" (as in, this milk tastes funny, not funny haha) issues from time to time, namely:

Extremely slow start up times.
Unable to open any editor.
A resource already exists on disk when building.
etc.
etc.

These will probably be more of an issue for users that install a large amount of plugins or products like MyEclipseIDE.

I was having some of the issues I mentioned above, went searching for a bit and found and Eclipse option:
-clean

It looks like this tells Eclipse to erase and rebuild it's internal plugin cache which in turn sorts out some versioning / state issues. You only need to run it once so the option can be removed after the workspace restart.

So for MyEclipseIDE I added it to myeclipse.ini just before the -vm option.
Closed and opened Eclipse... and both issues were solved. /cheer

If you are just using standard Eclipse, you can place it in your eclipse.ini

Glassfish ESB, easier than you think. Part I

2010-07-25T09:10:00.000+02:00

Over the last 2 years I got to learn, use and enjoy OpenESB, we switched to GlassfishESB when it was released with the support of Sun. Unfortunately I am not using it anymore at work and I missed the 2.2 release in which there seems to have been a couple nice changes, but to get to the point. The concept of a ESB (Enterpise Service Bus) will seem daunting or intimidating to some people, this post is to get rid of that and show how developer friendly GlassfishESB can be. Another huge benefit people do not often mentioned is working on very large distributed applications a tool like GlassfishESB with it's visual representation of your entire deployment makes maintenance and system structural updates a lot simpler.

Anyways to begin...
Go download GlassfishESB / Netbeans / Glassfish and any other components you need here

Install it, start it up and you are probably half way done, it is that simple.

Ok so first thing you want to do is create a standard Java project.. and create some classes, for this demo I am just creating a simple "message" with a Header and Body:

Now the next bit is my way of "cheating" to actually have anything connect to the ESB process you need to have an webservice, you get that by defining a WSDL and XSD and adding them to the BPEL, but who wants to type out all that boring XML...

So now create an EJB Project:

Click next..

Click next... Finish

Add the Java project to the libs of the EJB project.

Create a new webservice...

And make sure that ... the EJBs' code looks like

Now for the cheat ...

This will generate the WSDL and XSD...

Now we create the BPEL Project

Copy the WSDL and XSD from the EJB Project..
Open the bpel... Drag the WSDL onto the little orange circle on the left, the following will popup:

Now your BPEL looks like:

Add a Receive, Assign and Reply to the BPEL, the edit the Receive:

Edit the Reply:

Click on the Assign and then on "mapper" in the bpel editor window, drag the contents of the In variable to the contents of the Out variable

Save, and done...

Now just add a composite application.

Open the Service Assmbly and drag the BPEL project onto the JBI module section:

:

Add some Endpoints from the Palette, connect the arrows:

Deploy.. and you are done.

If you like you can add a test to the Composite Application, very simple, On the Test folder, right click add, choose your main WSDL, choose the method you wanna test, it generates test xml based on the XSD. On the properties of the test set the port in ${HttpDefaultPort} in the properties of the test...and test...

In Part II , I will add some more EJBs, so that this actually becomes a functioning application and some more awesome GlassfishESB functionality.

So your POM is quite nice.. now what?

2010-07-22T07:51:00.000+02:00

Last week, I described some of the Maven lessons and configurations we had used in the last while: Some useful Maven configs...

So now that you have all these Maven projects, local repos, well managed versions, unit tests, code coverage and static code analysis what do you do next?...
You run it all the time and you let everyone know how it is going...
Continuous Integration (CI)...
I am not going to go into too much detail about the concept of CI, Martin Fowler and countless others have already done that.
Martin Fowler - Continuous Integration

I will however ramble out some personal opinions of the advantages and challenges that come with CI and mention some of the tools out there. I say "challenges" because I don't feel there are actually any disadvantages with this approach, but rather it just illuminates some difficulties that always existed in software development.

The Advantages:
CI gives you an early warning of any major broken or conflicting code changes with continuous unit testing of all the changes, it enables better project visibility. Depending on thoroughness of your CI environment there can be immediate feedback to developers on the quality, functionality and state of the application. Bugs, systems issues and the time needed to correct them gets exponentially worse the longer you they go unseen and are left unattended. Another benefit is that there is always a "current" build, that can be used for anything like testing, demos or interim releases, so not only are the unit tests running all the time, functional testing can also happen on more current code.
After the initial setup and with having all the SCM integration, building, deploying and reporting automated it allows people to focus on what they actually should be and not constant "environment" issues.

The Challenges:
If CI is something new to the environment there will be some initial setup time / cost, including maybe some extra hardware as build machines.
I feel there need to be well developed, comprehensive unit test suites these are crucial if there is to be any benefit gotten out of the automated testing. With this need for comprehensive unit test suites there is a requirement for all parties involved clients, management and developers to understand that more upfront development time will be required and there will be long term benefits. What happens all too often as soon a deadline looms... all standards, tests and processes go flying out the window, and as soon as that happens there is generally never time to go back and fix it and those holes in testing and quality will remain in the system.
There is also challenges with large changes can be hindered by the constant builds although this can be be handled in a couple ways, depending mainly on how you handle your Revision control / SCM (source configuration management), but that is a subject for another day.

The Tools:
I am only listing the ones that I feel are probably the most common (in the java environment), and obviously all are free and open source. I was tempted to create a very scaled down & up to date version of: CI Feature Matrix, but when you get down to the details, the 3 tools below all do very similar things, and have very similar features. They all can run, schedule, build, notify, integrate with SCMs (CVS, Subversion, Git, Mercurial, StarTeam, ClearCase), graph and have plugins for pretty much everything you will ever need.

Hudson
Cruise Control
Apache Continuum

We have used Hudson for the last couple years, and a couple practices that have served us well are:
Ensure that a build can always be reproduced, the build must be a clean build from scratch and by that I mean all the code must code from your SCM tool.
Set up email notifications to all the developers, the architect/s, and the project manager or development manager of the project. That way everyone has an idea of what is going on... and sometimes us developers need a little "motivation". There are also RSS feeds available for all builds, failed builds etc. I personally use those rather than the emails... but that is purely personal preference.
Hudson needs a fair amount of disk space on large projects, to avoid issues midway in a project ensure you have ample space and keep an eye on the version / archiving settings.
There are plugins for all major IDEs that allow you monitor and control Hudson without opening a browser, I personally like having the Hudson build information available within my IDE.
Something I would have liked to have set up, is integration with a issue tracking system like bugzilla or something, but alas my little corporate doesn't work that way.

MyEclipseIDE Maven4MyEclipse Null Pointer bug

2010-07-19T21:40:00.000+02:00

I use MyEclipseIDE generally as my main IDE, I recently switched development teams at work, while importing all their existing maven projects with the Maven4MyEclipse plugin into my new workspace I had quite a few of these little annoying popups:

I went and checked the Log:

java.lang.NullPointerException
at com.genuitec.eclipse.maven.project.MyEclipseProjectSupport.findWebRoot (MyEclipseProjectSupport.java:204)
at com.genuitec.eclipse.maven.project.MyEclipseProjectSupport.configureWebProject (MyEclipseProjectSupport.java:71)
at com.genuitec.eclipse.maven.project.MyEclipseProjectSupport.configure (MyEclipseProjectSupport.java:57)
at org.maven.ide.eclipse.project.configurator.AbstractLifecycleMapping.configure (AbstractLifecycleMapping.java:57)
at org.maven.ide.eclipse.internal.project.GenericLifecycleMapping.configure (GenericLifecycleMapping.java:48)
at org.maven.ide.eclipse.internal.project.ProjectConfigurationManager.updateProjectConfiguration (ProjectConfigurationManager.java:332)
at org.maven.ide.eclipse.internal.project.ProjectConfigurationManager.updateProjectConfiguration (ProjectConfigurationManager.java:318)
at org.maven.ide.eclipse.actions.UpdateSourcesAction$1.runInWorkspace (UpdateSourcesAction.java:93)
at org.eclipse.core.internal.resources.InternalWorkspaceJob.run
(InternalWorkspaceJob.java:38)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)

So I figure it is moaning about the WebRoot... let me try:

Right click on project go to MyEclipse Menu and Add Web Project Capabilities.

Worked like a charm.

Some useful Maven configurations, practices and lessons learnt

2010-07-14T08:57:00.000+02:00

After using Maven for a while, my colleagues and I have configured a couple useful things and learnt a few lessons.

Edit: as per comments
Local Repository Managers
On the maven site here: Repository Management there is a brief description of the major repo managers out there
and a very good breakdown on why they are a good idea. We had a couple issues with Artifactory, (this was a couple years back) and in the end went with Nexus, which has proven to be very stable and offers all we need.

Parent POMs
Define common things, like versions, compiler settings, reporting settings and common build plugins in a parent POM to allow this configuration to be used throughout the applications' components and sub projects. If you have multiple projects with different requirements you can create another layer of Parent. To give an example, we had a legacy project, and the new re-write. For the legacy system we could not enforce checkstyle or code coverage but we did want to enforce it for the new project. So we created a base parent pom that had some general configuration and version numbers in it and then another parent pom for the new system that had the base parent as it's parent and then all the specific code coverage and style checks.

Versioning(See update at end of post)
In a parent POM define variables for all out internal component versions, so that you would have 1 central place to change release versions / snapshots etc.
We didn't include the versions of common 3rd party libraries like Spring, The Apache commons projects, JUnit etc, which in hindsight was a mistake. This is now really quite a pain 3 years and 30ish POMs down the line when trying to upgrade a specific library, and invariably one gets missed and then you have 2 / 3 versions of Spring or Hibernate sneaking into the dependencies.

Parent POM:

Project POM:

Compiler Settings
Just to ensure that it is the same accross all the projects using the parent.

Common Build Settings
1. Unit Testing
Actually running and reporting on the unit tests and results.
1.a. In the report plugin I have included command line arguments to increase the VM size, we found that on our large projects with tons of unit tests it did occasionally ran out of memory.
1.b. In the build plugin, there is a "parallel" config, that i recently saw in John Smarts blog, that allows you to run the test classes concurrently.
2. Cobertura Code Coverge, This is great way to ensure that there are atleast a certian amount of test cases and that code is being run.
It unfortunately can't check the validility of the test cases so developers will be developers and code "useless" tests just to pass the coverage requirement, but it is still worth putting the plugin in, and "useless" tests can be caught with code reviews.

1. Surefire - Running and Reporting

2. Cobertura Code Coverage
We found that in our predetermined deadline environment going for more that 50% coverage really extended the development time and effort for components more than we could afford. There were a couple smaller components that off the bat had 80%+, but we couldn't enforce it on all due to the time constraints.We are now slowly increasing that percentage when ever there is new work on a component but that is also quite a slow and painful process.
The "totalLineRate" is the setting that determines the coverage % required.

Checkstyle
We defined our basic coding standards within Checkstyle, and to ensure that they are followed we would fail the build if the code doesn't comply. I have also snuck in the "taglist-maven-plugin" plugin under this topic as it was a pet peeve on one of the developers, and with it in the reporting section it will show up when using 'mvn site'.

Profiles
One of our .EARs needed to be deployed on both Glassfish and Weblogic and there were unfortunate compatibility issues between the 2 so we couldn't actually deploy the same ear.
We configured that specific POM with 2 profiles. The Weblogic on was set to be the default, and when we required a new deployment onto Glassfish we would run maven with -Dglassfish-build=true.

Edit 2: Dependency Management and Versioning
Again, thanks to the people that commented. I am always happy to learn something new. So a couple people pointed out that I completely missed this. When writing this I based it on personal and team experiences and we somehow missed this section of the maven docs. I state in "Versioning" it is now a bit of a pain in our lives, so to help out others, below is the link to how Maven says you should handle dependencies.

Dependency Management

Spring 3: RESTful Web service Support... Take 2 with JPA and on Glassfish 3

2010-07-13T10:43:00.000+02:00

In my original example. I just used a horrible static list in a controller as state for the CRUD methods. I couldn't leave it like that so I have now updated that version to use JPA to a MySQL DB.

This updated source code, config and POM files are available in SVN

I am not going to go through all the code like in my first example, but just go through a couple things I changed and new things picked up while making the change

1.
This thing, that actually cost me quite a bit of time...
The exception:
"A JDBC Driver or DataSource class name must be specified in the ConnectionDriverName property"

Which actually meant in my example
"I couldn't find your persistence.xml" imagine that... obviously very clear.

When you have a web application you can not just put your JPA persistence.xml in the META-INF folder like you would with a EJB application. Sad thing is I actually (after struggling for quite a while) remembered I had read this while studying for the SCBCD exam like 2 months ago. Doh!

So you need it in a META-INF in with your source, so you can either put in in a jar or if you lazy just have a META-INF in java/main/src.

2.
Trying to get Hibernate to run on Glassfish 3 was more effort that I thought it would be, and plain straight JPA was quick and simple.

3.
I used MySQL many years back, and it was alright. This latest version 5.2 Rocks... Seriously, go download the DB server, the divers and the Workbench.. The workbench is the winner... simple easy admin, almost instant EER / ERD diagrams, nice SQL query interface too, for my own home development I doubt I will look any further.

4.
For the last project I worked on we used Hibernate, and we used the generation tools to create our mappings and DAOs, which left us will a ton of Abstract and Implementation classes. I was never a fan of all those so went searching while doing this example and came across the Java Generic DAO implementation, which is very nice.

So you create a base DAO with the Key and Entity defined as generic parameters, just for ease of reading i went with "key" and "entity" instead of the usual K and E.

You then just have the following very small implementations just defining the actual Class type.

Very neat.

5.
In my original controller I had:
@RequestMapping(value="/users/", method=RequestMethod.POST) on each method.

Much nicer to have that on Controller level and just the following on the methods:
@RequestMapping(method=RequestMethod.POST)

6.
I mapped my dto purely through JPA annotations:

Gah...This was annoying... Glassfish 3.0.1 on Vista...

2010-07-12T18:03:00.000+02:00

I have some time off... So I decide to check out Glassfish 3.0.1 (yes that is very geeky), it should be a nice new toy even with and evil red logo on it now. I am quite fond of glassfish, it was a pleasure to work with as a developer. I had spent a good almost 2 years using and fighting for the use of Glassfish 2.1 / OpenESB or Glassfish ESB... and lost the month I decided to get married and go on honeymoon...
That is probably lesson in itself, with some kind of motto:Never go on leave

.... anyway ....

Back to my current annoying discovery:
I try start up Glassfish after the install... (in Netbeans 6.9) and it fails with a silly little error:
"Address family not supported by protocol family: bind: 7676"
and then proceeds to shutdown..

So off to google...
Read quite a few useless forum posts...
Eventually find the below... pay it no attention of course... and carry on reading useless posts...
"
Hi,

Try to change configuration of localhost in file
C:\Windows\System32\drivers\etc\hosts
from
::1 localhost
to
127.0.0.1 localhost
"

After another couple minutes, I decide "what the hell", and I go find the host file, make the change...big surprise... Windows doesn't let me.
Download "Unlocker", pretty neat utility if you want to overpower the Windows file locks, but it picks up no lock.
Another trip to google.... Only to find you need to run the edit application as an administrator obviously...

Make the change, and Tah Dah! Nice new Glassfish 3.0.1 server up and running.

Ehcache 2.1 Monitoring

2010-07-08T17:20:00.000+02:00

In my previous post Spring, AspectJ, Ehcache Method Caching Aspect

I demo'ed and described how and why to cache the results of methods. Now to enhance that, I did some more investigation into the latest release of ehcache available at Terracotta

With 2.1 they released a monitoring api, to allow you to monitor cache's real time within your runtime environment, this will allow to fine tune and manage any cache you may have... be that custom caching with something like my MethodCacheAspect or your applications' Hibernate caches. To setup and use the monitoring you will need to do the following (I am setting this up on weblogic 10.3)

I placed the following 4 jars into my domain/lib, this will be different for every application server, but as long as the following are in the relative classpath you should be ok.

ehcache-core-2.1.0.jar
ehcache-probe-1.0.0.jar
slf4j-api-1.5.11.jar
slf4j-jdk14-1.5.11.jar

So now for the configuration:

The ehcache.xml, needs to be in your classpath and have the following details:

Take note of the following that is added to the cache you want to monitor.

In the start up of the application server... in my case weblogic 10.3 you will see the following:
(it is a licensed product from terracotta, and I am only using in a development environment to tune my cache settings)

INFO: Initializing EHCache CacheManager
04 Jul 2010 6:18:24 PM net.sf.ehcache.distribution.RMICacheManagerPeerListener
WARNING: Explicitly setting the listener hostname to 'localhost' is not recommended. It will only work if all CacheManager peers are on the same machine.
04 Jul 2010 6:18:25 PM org.terracotta.ehcachedx.monitor.probe.ProbeDxService startBackend
INFO: Started probe at http://localhost:51115/monitor/list
04 Jul 2010 6:18:25 PM org.terracotta.ehcachedx.license.LicenseResolver resolveLicense
WARNING: No Ehcache Monitor license key found. This monitoring probe software is not licensed for production usage, and is only licensed for development usage. See LICENSE.txt for details. A temporary key will be generated for development usage. When the temporary key expires, the Ehcache monitoring probe capability will be suspended but your ability to continue to use Ehcache will not be affected. Please contact sales@terracottatech.com to request a license.
04 Jul 2010 6:18:25 PM org.terracotta.ehcachedx.license.LicenseResolver logLicense
INFO:
--------- Ehcache Monitor license key ---------
License type = DevOnly
License number = 0
Licensee = DevOnly User
Product = Ehcache Monitor
Capabilities = monitor
Expiration date = 2010-07-07
-----------------------------------------------
04 Jul 2010 6:18:25 PM org.terracotta.ehcachedx.monitor.probe.ProbeDxService$RegisterWithServer run
INFO: Null license registered

Now that it is up and running, you can browse to:
http://localhost:51115/monitor/list

There you will get a list of methods that allow you to query you live caches, or even check the status of the CacheManager with, "probeGetCacheManagerStatus( )" very handy if you want to monitor the health of your live application. We have had a couple occasions where we we getting exceptions calling another teams' services and it took a while to discover that the "Cache Manager is not alive" exception.

The Ehcache API is REST based service. You Should be able to to use a simple RestTemplate like I used in my Spring 3 RESTFul example to manipulate and query the API, or you can just use a standard web browser:

http://localhost:51115/monitor/probeGetCacheManagerStatus
Result:
STATUS_ALIVE

http://localhost:51115/monitor/probeGetCacheNames
Result:
testCache
restCache

http://localhost:51115/monitor/probeGetCacheElementMemorySize?cache=testCache
Result:
584

The full list of functions are:

AmountOfHistoryStoredOnMonitor( )
getSampledStatisticNames( )
getSamplingHistory( )
getSamplingSeconds( )
getVersion( )
probeEvictExpiredElements( cache )
probeFlushCache( cache )
probeGetAllCachesConfigAllValues( )
probeGetAllCachesSampledStatisticValues( name )
probeGetCacheConfigAllValues( cache )
probeGetCacheConfigNames( )
probeGetCacheConfigValue( cache, name )
probeGetCacheConfigValues( cache, name )
probeGetCacheCount( )
probeGetCacheElementCount( cache )
probeGetCacheElementCountDisk( cache )
probeGetCacheElementCountMemory( cache )
probeGetCacheElementMemorySize( cache )
probeGetCacheElementMetaDataAllValues( cache, key )
probeGetCacheElementMetaDataNames( )
probeGetCacheElementMetaDataValue( cache, key, name )
probeGetCacheElementMetaDataValues( cache, key, name )
probeGetCacheKeys( cache, count, [query] )
probeGetCacheManagerName( )
probeGetCacheManagerStatus( )
probeGetCacheNames( )
probeGetReplicationMode( )
probeGetSampledStatisticAllValues( cache )
probeGetSampledStatisticHistory( cache, name, [fromDateTime], [timeInterval], [offset] )
probeGetSampledStatisticValue( cache, name )
probeGetSampledStatisticValues( cache, name )
probeRemoveAllFromCache( cache )
probeRemoveFromCache( cache, key )
probeRemoveQueryFromCache( cache, query )

Some of these could be very valuable to fine tune you caching strategy. Knowing things like "probeGetCacheElementCountDisk", "probeGetCacheElementMemorySize" and being able to monitor them may allow you to tailor your Ehcache config for optimal usage and consequently better application performance.

Also "probeFlushCache" or "probeRemoveFromCache" allow you to clear out the cache in the case of a data change without having to reinitialize / restart your application.

The tale of a Calendar... a Date... and a Object reference bug..

2010-07-06T22:11:00.000+02:00

To set the scene, there is a software release 4 days away and everyone is in mid-pre-release-panic... There is one of those nasty bug's only happens sometimes, and was missed by testing until 2 hours before final signoff, and is "absolutely necessary" for the release.

I rush off go through the date calculations done by our system, adding and changing To and From dates into a sequence depending on some items, not overly complicated but just enough to have quite a few Calendar and Date objects flying around.

So I figure it has to be an object reference being used of one the Date objects, because 2 of the dates set are to the same value. This obviously only happens on the actual deployment and my unit tests pass, and debugs print out the right values..

First thing I do is look for shared Date objects (all the Date objects get created via Calendars or set back to the Calendar like below:

The shared Date objects are then used on a internal Period class that has From and To dates and some convenience methods that is used all around our system and it was those date values that were incorrect.

So I think... easy.. I'll just add ".clone()" the Date because it must be their object references.... I put in 2x .clone() compile, deploy ... test... fail... 2 Dates still the same.

Go back to the code, get a second pair of eyes, a work colleague. Together we slap together this completely immutable class, that makes the calculations less complicated and ensures that there can't be any object reference sharing between that dates and the calendars or anything, and at the same time create a new Period object to hold these new dates from our new class that we made sure that could not be messed with by reference.

build, deploy ... test ... works like a charm.. notify the testers, they sign off ... all is good.

A hour or so later, the same work colleague pops me an IM on MSN Messenger... says that this could happen somewhere else... we get the code... at roughly the same time we both happen to go into Calendar.getTime() source code and to our surprise (and amusement) we discover a little gem... both can't believe that we had not actually seen it before..

Calendar source code below:

The standard java Calendar had ensured a long time ago that you would not have this object reference scenario and that it is safe by using the "TimeInMillis". So we had wasted a good 30 minutes of our lives, when the only issue was our own Period object that was the bad reference.

So I guess the point of this is....

1. Using the Calendar.getTime() and setTime() is safe and won't let you pollute object references.. and...
2. I had a similar discovery with Spring a little while back (...) So it is fair to say if you have the source code of classes you are using, its probably an excellent idea to have a look at it from time to time.

Spring 3: RESTful Web service Support

2010-07-05T00:28:00.000+02:00

UPDATE: This has been updated to use JPA and a more real life type object with Id's and the like... Spring 3: RESTful Web service Support Take 2 with JPA

So in between some really long days at work, the usual before release panic and regression defects. I finally completed a simple RESTFul example, before getting into the details, here are a couple facts about RESTFul services.

REST stands for Representational State Transfer.
Each unique URL is a representation of some object.
You can get the contents of that object using an HTTP GET.
You may use a POST, PUT, or DELETE to modify the object.
It is not necessary to use XML as a data format, so for small amounts of data it can be significantly lighter on bandwidth.
REST as a protocol does not define any form of message envelope.
REST takes advantage of HTTP caches. (A cache can't to do anything a POST; but they can cache GETs and expire those entries based on PUTs and DELETEs.).
The Sun specification for RESTFul webservices can be found atJAX-RS
Spring does not directly support or implement the JAX-RS spec but instead RESTful functionality is added to feature Spring MVC.

As with my other examples all the source code, config and POM files are available in SVN

As we are exposing information via HTTP this is going to be a web project. I won't be going into details of compiling and deploying a web application, but will go through all the required parts
Starting with the web.xml:

The DispatcherServlet will receive the request it will then, based on the servlet mapping
go looking for "Spring3RESTFul-servlet.xml" which in turn defines the controller and view (which are dynamically searched for and created):

The component-scan in the application context above will be used to automatically pick up our @Controller. If you go to /app/users/Bob, the getUser method is executed and their respective name is passed through as a parameter. The Spring JaxB 2 marshaller will be used to marshall the User object in and out of XML. Any of the OXM marshallers can be used, I just simply prefer JaxB.

Note: You really should not have a horrible static map containing state within a controller (like I do below), I am only doing it now because this is an example, and I want to keep it simple and not digress into a full blown application. If I have time over the next week or so, I'll remove that map of data and just create an embedded DB or something.

You can view the results in your browser by simply going to the url. I am using weblogic, so mine looks like "http://localhost:7001/Spring3RESTFul-1/app/users/Mary" but depending on your deployment it could differ as long as you add "/app/users/Mary" to your context root you should get the following displayed. (if you are using firefox)

However, chances are you are not writing this to view some XML in a browser but rather to get or manipulate information from one system or component to another. For this I created a little client, this client uses Springs' RestTemplate which is autowired and lets me access the HTTP functions:

Quote from Spring blog:

DELETE delete(String, String...)
GET getForObject(String, Class, String...)
HEAD headForHeaders(String, String...)
OPTIONS optionsForAllow(String, String...)
POST postForLocation(String, Object, String...)
PUT put(String, Object, String...)

This example only uses getForObject, postForObject, put and delete for the moment, so just the normal CRUD operations.

Now to use the above client I have a test application context injecting the RestTemplate and a test case preforming the create, retrieve, update and delete.

Note:Because I just wanted to create 1 project for this example I have included the following, just so that you can compile and deploy and then run the test case... but ever adding this to a actual project POM file should be grounds for a good beating.

Lastly actual test case. Deploy the war file created by the POM to the web / application server of your choice before running the below:

And that is it... a Simple CRUD RESTFul Webservice example with Spring 3.

Update: Spring 3: OXM Blog entry

2010-06-27T10:44:00.000+02:00

While doing my RESTful webservice example I came across a marshaller that I had missed in my original Spring 3: OXM post.

XStream

To the maven pom I added:


 com.thoughtworks.xstream
 xstream
 1.3.1

To the class being marshalled:

@XStreamAlias( "MyXMLName" )

And to the application context:

I have also uploaded the example project into SVN

No REST for the wicked or the Corporate Developer

2010-06-25T18:42:00.000+02:00

REST vs. Other Web Services

I know this topic has probably been covered a million times in the last couple years, but being a developer in a large corporate where "standards" are imposed to limit the craziness that would ensue without them. I have actually only now started to have a look at it. The reason for me looking at it now is the Spring 3 RESTful support, which I plan to play with in the next day or so.

Not really going to go into SOAP and XML-RPC, as I generally know that and they are the only type of webservices we implemented in my working environment. I know that SOAP and XML-RPC, uses an envelope, XML and is well defined / strongly typed... bla bla bla...

So from random sources this is what I found about REST:

REST stands for Representational State Transfer
Each unique URL is a representation of some object.
You can get the contents of that object using an HTTP GET.
You may use a POST, PUT, or DELETE to modify the object.
It is not necessary to use XML as a data format, so for small amounts of data it can be significantly lighter on bandwidth.
REST as a protocol does not define any form of message envelope.
REST takes advantage of HTTP caches. (A cache can't to do anything a POST; but they can cache GETs and expire those entries based on PUTs and DELETEs.).

So where to use RESTful ?
Well from what I just read, it is a smaller, quicker, HTTP interface which sounds perfect for replacing all "system internal" service calls happening between components where direct RPC is not ideal and it really isn't worth the time, effort and red tape to set up a WSDL / XSD.
(that was a really long sentence)

Ignore: just for proof of ownership: GUNCP4EUNAGF

Fluent Interfaces...Great.when.good && Fn.complicated.ifelse().true.

2010-06-21T22:59:00.000+02:00

While scanning the internet, looking for something to comment on and maybe learn, My wife, a .Net Developer, mentioned...

Sharp Test Ex... I went to check it out and was instantly intrigued...
With syntax shown below who can blame me, nice readable tests:

const string something = "something";
something.Should().Contain("some");
something.Should().Not.Contain("also");
something.ToUpperInvariant().Should().Not.Contain("some");

var ints = new[] { 1, 2, 3 };
ints.Should().Have.SameSequenceAs(new[] { 1, 2, 3 });
ints.Should().Not.Have.SameSequenceAs(new[] { 3, 2, 1 });
ints.Should().Not.Be.Null();
ints.Should().Not.Be.Empty();

I then remembered a couple years back I came across something Martin Fowler chatted about: Fluent Interface

I never paid too much attention to it then, but as the years add up and I get more lazy, this appeals to me more. So not to be out done by .Net I went looking for java equivalents. I found 2, FEST and OP4j.

FEST has a couple sub projects: Reflection, Swing, JavaFX, Mock, Assertions
To compare with the .Net example above I'll go to FEST Asserts.
FEST Asserts
"
Provides a fluent interface for writing assertions. Its main goal is to improve test code readability and make maintenance of tests easier.
"

With example Syntax below... nice ... simple ... readable... great, I'll be implementing that in the next project I have a little bit of spare time. It doesn't cater for everything but from the looks of it there will 0 ramp up time and should be usable / understandable right after adding it to the maven pom and downloading the libs.

int removed = employees.removeFired();
  assertThat(removed).isZero();

  List< Employee > newEmployees = employees.hired(TODAY);
  assertThat(newEmployees).hasSize(6)
                          .contains(frodo, sam);

  String[] newHires = employees.newHiresNames();
  assertThat(newHires).containsOnly("Gandalf", "Arwen", "Gimli"); 

  assertThat(yoda).isInstanceOf(Jedi.class)
                  .isEqualTo(foundJedi)
                  .isNotEqualTo(foundSith);

Now OP4J...
op4j

"
op4j is a powerful implementation of the Fluent Interface style of code. It allows you to create chained expressions which apply both predefined or user-defined functions to your objects in a fluid and readable way. This improves the way your code looks and greatly reduces the complexity of executing auxiliary low-level tasks in the highly bureaucratic, statically -and strongly- typed language that Java is.
"

"several hundred functions with more than one thousand different parameterizations"

Off the bat seems like a monster. They seem to have put a lot of time and effort into "bending the java spoon". From a quick look it seems they cater for a ton of common code occurences and allow you to condense quite a bit of code into 1 liners.
Although I do see value in this framework, I have some concerns on how much there is to learn, the ramp up time required, and how easy it would be for a junior developer to maintain.

Example syntax below:

// Without op4j
  Set< String > set = new LinkedHashSet< String >(list);      
  // With op4j
  Set< String > set = Op.on(list).toSet().get();


 Set < Calendar > set = Op.on(list).toSet().removeAllNull().forEach()
.exec(FnString.toCalendar("dd/MM/yyyy")).get();

 Function< List < String >,Set < Calendar > > dateProcessFunction = 
 Fn.onListOf(Types.STRING)
.toSet().map(FnString.toCalendar("dd/MM/yyyy"))
.removeAllNullOrTrue(FnCalendar.after(now)).get();

In conclusion, it is not entirely fair to compare OP4J with the much smaller "testing" fluent interfaces as they are targeted at different functionality. However, I think the idea of Fluent Interfaces is quite a nice one. On large systems with internal frameworks and structured code environments like large corporates and banks, defining and implementing a proprietary fluent interface type of framework could , if done correctly, be very very useful and simplify large amounts of commonly used code. There is however, a distinct danger that it may also lead to vastly harder to read code that is also harder to maintain and debug instead of the original concept of "The API is primarily designed to be readable and to flow"

What I do like of both "FEST Asserts" and the .Net "Sharp Test Ex" is that they are limited to testing and geared only towards quicker, more understandable test cases. Anything that makes testing quicker, easier or a little more fun is well worth it.

I also stumbled across the following,

Fluent Build eclipse builder.

How not yet had time to investigate it properly, but definitely worth keeping an eye on.

Spring 3: Scheduling... Take 2

2010-06-14T23:25:00.000+02:00

In the previous blog entry I used Spring annotation scheduling to fire an event, but there is also another way through the application context xml.

Annotation:
In the application context.

Then within the code annotate the method with @Scheduled(fixedDelay = 10000)

XML:
Now if you annotate your class with @Service or @Component (with component-scan), you can use the scheduled-tasks tag, and schedule a method on a service and specify a cron expression, which does give you all the control you'll need.

Since I personally can never remember the exact format for cron I am adding the next little bit for easy reference.

* * * * * *
| | | | | |
| | | | | +-- Year (range: 1900-3000)
| | | | +---- Day of the Week (range: 1-7, 1 standing for Monday)
| | | +------ Month of the Year (range: 1-12)
| | +-------- Day of the Month (range: 1-31)
| +---------- Hour (range: 0-23)
+------------ Minute (range: 0-59)

0 23 * * 1-5 * At 11:00PM on weekdays Mon-Fri
0 0 1 * * * At midnight, on the first day of each month
* * * * * ? Every Second
0 * * * * ? Every Minute

Spring 3: Scheduling, Components, PostBeanProcessors and bending the rules.

2010-06-13T13:55:00.000+02:00

This blog entry should come with one of those warnings "Do not attempt this at home, the following is done by trained professionals".. bla bla bla

To begin, I, being a developer always push the limits and bend the rules of what we should / or should not do. I set out to "bend" something in Spring for a change. I work in a 24/7/365 live type environment, where there are always people using the system and any downtime causes many reports and cascading pain flowing down the leadership hierarchy in a large corporate environment. Now sometimes in this environment even with tons of effort, signoffs, QA's, UAT's, Training something goes live, or happens in live that needs to be changed / switched off.

Now using Spring for injecting all that needs to be injected is awesome, but it does require that the application context be recreated / instantiated to pickup configuration changes. The following code changes that...

As with the method cache aspect, I will make the code available at my google code project:
Source Code

So we want to configure a field in a Spring managed bean to be available to be changed while the application is running.

I have specified an annotation @LiveConfig:

package javaitzen.spring.liveconfig;

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention(RetentionPolicy.RUNTIME) 
@Target({ElementType.FIELD })
@Documented
public @interface LiveConfig {
  String configLocation() default "liveconfig.properties";
}

To pick up all the @LiveConfig annotations, I created a BeanPostProcessor , that will be called when the ApplicationContext instantiates and keep track of all the @LiveConfig available to it.

package javaitzen.spring.liveconfig;

import java.lang.reflect.Field;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.config.BeanPostProcessor;
import org.springframework.stereotype.Component;

@Component
public class LiveConfigBeanProcessor implements BeanPostProcessor {
    private Log logger = LogFactory.getLog(this.getClass());

    @Autowired
    private ConfigLoader loader;
    public final Object postProcessBeforeInitialization(final Object bean, final String beanName) {

        Field[] fields = bean.getClass().getDeclaredFields();
        for (Field field : fields) {
            if (field.isAnnotationPresent(LiveConfig.class)) {

                try {
                    field.setAccessible(true);
                    LiveConfig ann = field.getAnnotation(LiveConfig.class);
                    LiveConfigData lcd = new LiveConfigData();
                    lcd.setFile(ann.configLocation());
                    lcd.setBean(bean);
                    lcd.setField(field);
                    loader.addConfig(field.getName(), lcd);
                    loader.load();
                } catch (Exception e) {
                    logger.warn(e);
                }
            }
        }
        return bean;
    }

    public Object postProcessAfterInitialization(final Object bean, final String beanName) {
        return bean;
    }

}

So now when the ApplicationContext loads, it will add LiveConfigData to the ConfigLoader. This config loader uses the new Spring 3 Scheduling annotations to fire every 30 seconds asynchronously.

package javaitzen.spring.liveconfig;

import java.io.FileInputStream;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
import java.util.Map.Entry;

import org.springframework.scheduling.annotation.Async;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;

/**
 * The Class ConfigLoaderImpl.
 */
@Component
public class ConfigLoaderImpl implements ConfigLoader {

    private Map< String, LiveConfigData > liveConfigs = new HashMap< String, LiveConfigData >();

    /**
     * Instantiates a new config loader impl.
     */
    public ConfigLoaderImpl() {

    }

    /**
     * Adds the config.
     * 
     * @param field
     *            the field
     * @param data
     *            the data
     */
    public void addConfig(final String field, final LiveConfigData data) {
        liveConfigs.put(field, data);
    }

    /**
     * Load.
     * 
     * @see javaitzen.spring.liveconfig.ConfigLoader#load()
     */
    @Override
    @Scheduled(fixedDelay = 30000)
    @Async
    public void load() {

        FileInputStream in = null;
        try {
            for (Entry< String, LiveConfigData > liveConfig : liveConfigs.entrySet()) {
                LiveConfigData lcd = liveConfig.getValue();
                Properties props = new Properties();
                in = new FileInputStream(lcd.getFile());
                props.load(in);
                for (Entry< Object, Object > entry : props.entrySet()) {
                    String key = (String) entry.getKey();
                    String value = (String) entry.getValue();
                    LiveConfigData data = liveConfigs.get(key);
                    data.getField().set(data.getBean(), value);
                }
            }
        } catch (IllegalArgumentException e) {
            e.printStackTrace();
        } catch (IllegalAccessException e) {
            e.printStackTrace();
        } catch (final IOException e) {
            e.printStackTrace();
        }
    }
}

The Application Context, is surprisingly simple. The context:component-scan picks up the classes annotated with @Component (ConfigLoaderImpl and the BeanPostProcessor). The task:annotation-driven lets Spring know to look for @Scheduled. The loader is @Autowired into the BeanPostProcessor.


 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:context="http://www.springframework.org/schema/context"
 xmlns:task="http://www.springframework.org/schema/task"
 xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
  http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task-3.0.xsd
  http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd">

For demostration purposes I just created a simple little WorkManager structure to show how to configure the fields and have a running application to test when changing the configuration changes. It also shows how to get config from a file that is not the default "liveconfig.properties"

package javaitzen.spring.liveconfig.service;

import java.util.Date;

import javaitzen.spring.liveconfig.LiveConfig;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;

@Service
public class TheWorkManager implements WorkManager {

    @LiveConfig
    private String filePickupURL;

    @LiveConfig
    private String enableDetail;

    @LiveConfig(configLocation = "duplicate.properties")
    private String allowDuplicates;

    @Autowired
    private Work worker;

    /**
     * process.
     * 
     * @see javaitzen.spring.liveconfig.service.WorkManager#process()
     */
    @Scheduled(fixedDelay = 10000)
    public void process() {
        System.out.println("processing: " + new Date());
        for (int i = 0; i < 3; i++) {
            worker.doWork("" + i, filePickupURL, enableDetail, allowDuplicates);
        }
    }

}

Neater Item Removal

2010-06-09T15:02:00.000+02:00

I stumbled apon a bit of code today, this removes items in a list and doesn't get a
java.util.ConcurrentModificationException, but it is pretty ugly.

List< Item > newItems = new LinkedList< Item >();

        for (Item item : items) {
            if (item.isAType() ) {
                newItems.add(item);
            }
        }
        items = new LinkedList< Item >();
        items.addAll(newItems);

As Part of reading that "5 Things you didn't know" series from IBM mentioned in a previous blog entry.

A much neater way is using the remove() method on an Iterator:

for (Iterator< Item > itemsIter = items.iterator(); itemsIter.hasNext();) {
     if (itemsIter.next().isAType()) {
  itemsIter.remove();
     }
 }

Spring, AspectJ, Ehcache Method Caching Aspect

2010-06-07T18:17:00.000+02:00

I know this is not a new concept, and I am pretty sure there a couple implementations of method caching out there, possibly some much more thorough, but I feel quite strongly about it and I think that it is still one of the most beneficial things ever to come out of the AOP design philosophy. It is also something that if implemented correctly can vastly improve the performance for a lot of systems out there.

A good couple years back, I stumbled across a blog post describing the concept of a method caching interceptor, that would allow you to cache method calls without changing code and thereby make some dramatic performance improvements on existing systems. I jumped on the concept, took some code from the blog, fixed an issue or 2, changed a couple things to my requirements, demo'ed it to the system architect at the time and implemented it for the next release cycle. That bit of code has been running for about 4 years now and had a huge impact on our performance and the pressures our legacy system placed on other internal teams.

Due to the joys that are large corporate organisations, the use of the latest version of Spring and especially AspectJ are not allowed by the standards and redtape, but that allows me to write a whole new neat version without infringing on too many IP issues :) and publish it here.

Full source code, maven pom and eclipse files are available for download:
Source Code
or through SVN:
MethodCacheAspect trunk

In the project there are 3 main java files of interest:
MethodCacheAspect.java
CacheKeyStrategy.java
DefaultCacheKeyStrategy.java

The CacheKeyStrategy interface came about when I noticed that even though objects were "equal" there were 2 separate items in ehcache. This was because of 2 issues actually, order of items and just using .toString() to build up a key for ehcache. Not wanting to enforce the order or method of generating a key, I created a the interface and a default implementation. If future users want to handle it any different all they need to do is implement it and specify the implementation in Spring. I have 2 custom implementations in my live project, but for this blog posting I have just created a simple default.

note:I removed all JavaDoc just to save space, the downloadable code has docs.

package javaitzen.spring.interceptors;

public interface CacheKeyStrategy {

    String generateKey() throws IllegalStateException;

    String classForStrategy();

    void setObject(final Object object);
}

Below is the simple default key generator, just for convenience, that does a Collections.sort and uses the objects' hashcode method.

package javaitzen.spring.interceptors;

import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.logging.Level;
import java.util.logging.Logger;

import org.aspectj.lang.ProceedingJoinPoint;

public class DefaultCacheKeyStrategy implements CacheKeyStrategy {

    private Logger logger = Logger.getLogger(DefaultCacheKeyStrategy.class.getName());
    private ProceedingJoinPoint pjp;

    public DefaultCacheKeyStrategy() {
    }

    public DefaultCacheKeyStrategy(final ProceedingJoinPoint pjp) {
        this.pjp = pjp;
    }

    public String generateKey() throws IllegalStateException {

        String targetName = pjp.getSignature().getDeclaringTypeName();
        String methodName = pjp.getSignature().getName();
        Object[] arguments = pjp.getArgs();

        return getCacheKey(targetName, methodName, arguments);
    }

    private String getCacheKey(final String targetName, final String methodName, final Object[] arguments) {
        StringBuilder sb = new StringBuilder();
        sb.append(targetName).append(".").append(methodName);

        for (Object arg : arguments) {
            if (arg instanceof Map< ?, ? >) {
                sb.append(expandMap((Map) arg));
            } else if (arg instanceof Collection< ? >) {
                sb.append(buildBuffer((Collection< ? >) arg));
            } else {
                sb.append(".").append(arg.hashCode());
            }
        }

        logger.log(Level.INFO, "Cache key is: [" + sb.toString() + "]");
        return sb.toString();
    }

    private StringBuilder buildBuffer(final Collection< ? > values) {
        StringBuilder bob = new StringBuilder();
        Collections.sort((List< ? >) values, new Comparator< Object >() {
            public int compare(final Object arg0, final Object arg1) {
                String thing1 = arg0.toString();
                String thing2 = arg1.toString();
                return thing1.compareTo(thing2);
            }
        });
        for(Object o : values){
            if (o != null) {
                bob.append(".").append(o.hashCode());
            }            
        }
        return bob;
    }

    private StringBuilder expandMap(final Map< ?, ? > args) {
        if (args == null) {
            return new StringBuilder();
        }

        List< Object > values = new ArrayList< Object >();
        for (Entry< ?, ? > entry : args.entrySet()) {
            values.add(entry.hashCode());
        }
        return buildBuffer(values);
    }

   public void setObject(final Object invocation) {
        this.pjp = (ProceedingJoinPoint) invocation;
    }

    public String classForStrategy() {
        return "*";
    }
}

Now for the juicy bit, the aspect checks if there are any cache key strategies compares types with information from the ProceedingJoinPoint, it extracts the relevant information and generates a key. It then checks if there is value for that key in the cache and returns that value if relevant, else it executes the method with "pjp.proceed()" and caches the result, so that for the next time the same method is called with the same parameters it can just be returned from the cache. I cant remember the exact numbers but with implementing this between our legacy system and another team we cut our daily calls to that teams web service from something like 65000 to around 6000.

package javaitzen.spring.interceptors;

import java.io.Serializable;
import java.util.LinkedList;
import java.util.List;
import java.util.logging.Level;
import java.util.logging.Logger;

import net.sf.ehcache.Cache;
import net.sf.ehcache.Element;

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Aspect;

@Aspect
public class MethodCacheAspect {

    private Logger logger = Logger.getLogger(MethodCacheAspect.class.getName());
    private Cache cache;
    private CacheKeyStrategy defaultKeyStrat;
    private List< CacheKeyStrategy > keyStrategies = new LinkedList< CacheKeyStrategy >();

    public void setCacheKeyStrategies(final List< CacheKeyStrategy > cacheKeys) {
        this.keyStrategies = cacheKeys;
    }

    public void addCacheKeyStrategy(final CacheKeyStrategy cacheKey) {
        this.keyStrategies.add(cacheKey);
    }

    public Object aroundAdvice(final ProceedingJoinPoint pjp) throws Throwable {

        Object[] arguments = pjp.getArgs();
        Object result;
        StringBuilder cacheKey = new StringBuilder();
        defaultKeyStrat = new DefaultCacheKeyStrategy(pjp);

         if (!keyStrategies.isEmpty()) {
            logger.log(Level.INFO, "Have a Key Strategy to use...");
            for (CacheKeyStrategy strat : keyStrategies) {
                if ((arguments != null) && (arguments.length != 0)) {
                    logger.log(Level.INFO, "Have Arguments...");
                    for (Object arg : arguments) {
                        if (Class.forName(strat.classForStrategy()).isInstance(arg)) {
                            strat.setObject(arg);
                            logger.log(Level.INFO, "Using Strategy...");
                            cacheKey.append(strat.generateKey());
                        }
                    }
                }
            }
        }

        if (cacheKey.length() == 0) {
            logger.log(Level.INFO, "Using Default...");
            cacheKey.append(defaultKeyStrat.generateKey());
        }
        Element element = cache.get(cacheKey.toString());

        // not in cache
        if (element == null) {
            result = pjp.proceed();
            if (result != null && !(result instanceof Serializable)) {
                throw new RuntimeException("[" + result.getClass().getName() + "] is not Serializable");
            }
            logger.log(Level.INFO, ">>> caching result - " + cacheKey);
            element = new Element(cacheKey.toString(), (Serializable) result);
            cache.put(element);
        } else {
            logger.log(Level.INFO, ">>> returning result from cache");
            return element.getValue();
        }

        return result;
    }

    public Cache getCache() {
        return cache;
    }

    public void setCache(final Cache cache) {
        this.cache = cache;
    }

}

In the application context below you will notice execution(* theBusinessMethod(..)). That only attaches the aspect for theBusinessMethod, in my test class in the downloadable source.

Just for quick reference here are a couple more examples from the Spring docs:
The execution of any public method:
execution(public * *(..))
The execution of any method with a name beginning with "set":
execution(* set*(..))
The execution of any method defined by the AccountService interface:
execution(* com.xyz.service.AccountService.*(..))
The execution of any method defined in the service package:
execution(* com.xyz.service.*.*(..))
The execution of any method defined in the service package or a sub-package:
execution(* com.xyz.service..*.*(..))

Application Context:


 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/aop
http://www.springframework.org/schema/aop/spring-aop-3.0.xsd">

 

 
  
   
  
 

 
  
   
   
  
 

 
   class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean">
  
   classpath:javaitzen/spring/interceptors/ehcache.xml
   
  
  
 

 
 
  
   
  
  
   testCache

And finally the ehcache.xml:


 
   timeToIdleSeconds="120" timeToLiveSeconds="120" overflowToDisk="true"
  diskPersistent="false" diskExpiryThreadIntervalSeconds="120">

   overflowToDisk="true" timeToIdleSeconds="300" timeToLiveSeconds="600">

Google, just doing it a little better than everyone else.

2010-06-04T07:38:00.000+02:00

I log onto my gmail this morning...

Get this nice red warning on top saying there is unusual activity on my account ... I open it..

Arrghhh!!!...
So... signout the other sessions... think of new ultra secure password... change World or Warcraft password... change Google account password...

Praise Google.

java.sql.Connection JDK 1.5 and 1.6 Compatibility Issue

2010-06-03T09:00:00.000+02:00

A certain former work colleague of mine found this just before skipping the country to work for some evil IT empire... not to mention any names... but it starts with an "O" and ends with a "racle".

Today I opened the project again and there is a little Red X in my IDE which gives me a strange tick and psychotic tendencies.

java.sql.Connection was implemented into a verbose connection to get more information for unit tests.

Problem is he was running 1.6 jdk ... (our environment is still on 1.5) our hudson build machine is running 1.6, building as 1.5, our maven pom is set to 1.5, and most the developers are using 1.5...

If you have 1.6 the compiler complains about:

The type VerboseConnection must implement the inherited abstract method Connection.createArrayOf(String, Object[])
The type VerboseConnection must implement the inherited abstract method Connection.createBlob()
The type VerboseConnection must implement the inherited abstract method Connection.createClob()
The type VerboseConnection must implement the inherited abstract method Connection.createNClob()
The type VerboseConnection must implement the inherited abstract method Connection.createSQLXML()
The type VerboseConnection must implement the inherited abstract method Connection.createStruct(String, Object[])
The type VerboseConnection must implement the inherited abstract method Connection.getClientInfo()
The type VerboseConnection must implement the inherited abstract method Connection.getClientInfo(String)
The type VerboseConnection must implement the inherited abstract method Connection.isValid(int)
The type VerboseConnection must implement the inherited abstract method Connection.setClientInfo(Properties)
The type VerboseConnection must implement the inherited abstract method Connection.setClientInfo(String, String)
The type VerboseConnection must implement the inherited abstract method Wrapper.isWrapperFor(Class)
The type VerboseConnection must implement the inherited abstract method Wrapper.unwrap(Class)

So you implement the above methods and there are no little red X's and all is good in the world... Then someone then opens it up in 1.5:

NClob cannot be resolved to a type
SQLClientInfoException cannot be resolved to a type
SQLClientInfoException cannot be resolved to a type
SQLXML cannot be resolved to a type
The import java.sql.NClob cannot be resolved
The import java.sql.SQLClientInfoException cannot be resolved
The import java.sql.SQLXML cannot be resolved

Ahhh convenience...

2010-06-02T17:05:00.000+02:00

Ok this may seem a bit mundane, but I have used Spring for a couple years now.. and only today by the chance discovered:
and thanks to MyEclipse 8.5 / Maven integration that automatically downloads Java docs and source code on Ctrl + click.

and yes it has been there since 2.0 ...

"This is a convenience method to load class path resources relative to a
given Class." - Spring Java Docs

public ClassPathXmlApplicationContext(String path, Class clazz) throws BeansException {
  this(new String[] {path}, clazz);
 }

I can't say how many times I have typed out something like:

new ClassPathXmlApplicationContext("com/some/really/long/config/package/config.xml");

but that now just becomes:

new ClassPathXmlApplicationContext("config.xml", TheClassIamIn.class);