netzooid

My friend Paul Brown and I just finished our presentation here at OSCON. It was a fun exploration of some of the non relational database technologies, a presentation of a key/value store based twitter clone that we built on Voldemort and some discussion of monitoring/deployment.

I started experimenting with some of the non relational databases a while back in an effort to learn a bit more. There are some very interesting problems involved in dealing with such systems. Somehow I managed to get Paul to get in on it as well and do this talk with me. He too must be a sick individual to play with this in his spare time.

We built out a functional twitter clone called BigBird based on voldemort (minus lots of things like direct messages, searching), and it’s now all open source and available on github. Our presentation is up as well (slideshare, pdf) and should give a good idea of how we architected things.

I hope that everyone finds our code useful and informative. If nothing else, it could server as a good basis for evaluating/testing Voldemort. We may improve it over time. The idea is that we will add different backends which interest us. HBase, for example, now seems much more worthy of attention than when I looked at it early this year. Cassandra seems interesting too now that it has a release out and I don’t have to deal with their crazy build process. Paul has also started on a Dynomite & Osmos implementation.  Although no word on when/if any of this will be finished.

I’ve been doing some POM editing to build distributions for the Mule RESTpack today. Normally this type of thing would’ve been a painful iteration back and forth between the command line and inspecting the assembly I built. But m2eclipse provides a great way to actually visualize your dependencies.

Screenshots say it all, so here’s a view of all my dependencies:

And here’s a view of me tracking down a specific dependency I want to remove:

Let the new appserver war begin.

Is there any standard way to package up applications for OSGi? Spring has PARs now. Equinox seems to have their way. There is also P2 for remote access.

I don’t confess to be an OSGi expert, but I’m wondering – does it make sense to find a way to package applications up for OSGi appservers in a standard fashion? Or is it really something that needs to be server specific?

I’ve recently started using the Maven Eclipse plugin that was started by Sonatype and I have to say -wow, this rocks.

I went to start a Maven plugin today. Normally I would’ve googled around until I found an example. Then I would copied that example and started tweaking it.

With m2eclipse it was dead easy to just do New->Maven Project, then select the Maven plugin archetype.

Next I needed to add a dependency. Previously I would’ve searched around the maven repo manually or use mvnrepository.com. Now I just do right click->Maven->Add Dependency. Then I can type in something, like say “groovy” and it will present me all the versions that match. Its great.

Next I needed to test my plugin. Normally I would have to intsall the plugin then test it from another project. The m2eclipse plugin allows me to test the plugin “in-situ”. I don’t have to go through the time consuming phase of installing the plugin first anymore!

There’s a whole host of other features too:

• I can run maven builds from the IDE with a simple keystroke/click of the mouse
• It synthesizes projects from your source control. No mvn eclipse:eclipse needed any more. It also works much much better than the command line eclipse plugin.
• Easily add plugins to POMs

This is really going to help bring Maven to the masses. My only gripe is their ugly red icon

Update: fixed the mvnrepository.com url.

Spring Security 2.0 final is out. It looks like a great step forward! I’m looking forward to upgrading Galaxy to it when I get chance.

Guess I’ll have to talk about other projects killing fairies now.

(Also, thanks to Ben Alex and the other Spring guys for having a good sense of humor about my original post. I didn’t expect the meme to live quite this long…)

This week I put the finishing touches on the first version of a Jersey connector for Mule. It makes it really easy to drop in JAX-RS services into a Mule application.

For the non-Mule users out there – you’re probably asking, why would I need Mule and JAX-RS?

Imagine this scenario: you receive messages over HTTP. You wish to process these messages asynchronously with your backend servers. With Mule its pretty easy to set up an HTTP listener and connect it to a JMS queue. You can then have pull messages off the queue and process them asynchronously within a transaction. JAX-RS makes it very easy to map your Java class to resources with it’s simple annotations.

While you could piece this together all yourself, Mule is like integration glue – it makes connecting all these things easier. </marketing>

Thanks to the great work of Dejan Bosanac, Jettison 1.0 final has been released! He popped on the mailing list the other day willing to fix things; before I knew it he had SVN access and was fixing everything he could get his hands on. This is looking a high quality release and should greatly reduce the number of problems people have with Jettison.

Thanks again Dejan, and enjoy the 1.0 release of Jettison everyone.

Now that Galaxy is announced, I can unleash my fury of blogging again. Or, as we all know will probably happen, I can blog just slightly less sporadically again.

One of the recent posts I’ve wanted to comment on is Atom is the New JCR (prompted by Sam Ruby). Adrian Sutton writes:

When the Java Content Repository (JCR) standard first came out it was supposed to bring in a new era of compatibility between content repositories and put an end to the content silo. There was, and still is, a lot of talk about it and just about everyone added JCR compliance to their marketing materials. Unfortunately, that’s mostly where things stopped – the implementation work that followed was generally done was buggy or incomplete and the only viable JCR implementations that I’ve seen have come out of Day Software, who lead the JCR spec effort.

We use JCR inside Galaxy for all our storage needs at the moment. Using it has been an interesting experience to say the least. One of my favorite “features” of JCR has been the fact that you can not have two seperate threads create a child node with the same name at the same time. Which means in our activity log, we can’t add two nodes called “/activity” to “/activities” at the same time.
The main reason we decided to go with JCR is that it supports both simple database type data and file type data via the same interface transparently. I’m not 100% sure it was the right choice though. I had to write an ORM like framework as the one in Jackrabbit seemed pretty immature and I didn’t have time to delve into the rabbit hole of the jcr-mapping module in Jackrabbit. Mine is limited but as least I could figure out how to use it. We’ll be reevaluating this for the future though now that Jackrabbit 1.4 is nearly out.

Adrian continues:

Then along came Atom which is all about remote access and manipulation of data and missing probably 90% of the functionality that JCR offers. It really isn’t a competitor to JCR at all and yet it’s doing more to break down content silos than JCR ever has. Atom support isn’t just being added to the marketing materials, it’s actually shipping and is usable in a lot of places – IBM’s Lotus Connections has Atom APIs to everything and, as best I can tell, only Atom APIs to it’s repository.

I completely agree that JCR isn’t very worthwhile as something that will break down content silos. It does have value as an API to work with data though. Atom is quite limited in the granularity it can work with data (which coincidentally is one of the reasons Web3S exists as well). And you still need to store your data somewhere.

Atom has not replaced JCR it has supplemented it. We use JCR as a content store for Galaxy. I’ve also written a generic JCR content store for AtomPub inside of Abdera. Sure, the vendor promises are probably wrong about the content-silos, but it sure simplifies some things when writing applications – which is what really counts!

Update: David Nuescheler has written a great follow up to this whole discussion.

Also, Jackrabbit 1.4 is out. 

I’m in Oslo this week for Javazone 2007. And while I missed the 8 AM heavy metal band suspending themselves unfortunately, it still has managed to be quite a good conference. Partly because I get to see lots of my European friends whom I may not see that often.

Oslo is a beautiful city. (With a disproportionate amount of beautiful women too!) Its very clean and friendly. Really the only bad thing I have to say about Oslo is that it is insanely expensive. Our cab ride last night which took us a .5 mile cost about $100 ($20 basically from the moment we stepped in the door)!!! And beers are about $10! Its nuts.

I gave an updated version of my Building Scalable, Reliable, and Secure RESTful services talk yesterday. I think it went pretty well – I got a fair amount of good feedback at least. It was the first time I thought I might have actually explained REST coherently to people.  (Have you ever tried explaining REST to someone who knows nothing about it?

I attended Ryan Heaton’s presentation on Enunciate today. He’s written some really cool stuff to package up and maintain web services. In particular I love the ability to document your services really easily and have it presented via a pretty UI to the user. You should check it out if you haven’t already.

For now, I’m headed off to the Spring/OSGi presentation shortly so I can pretend I understand OSGi when I talk to people.

Someone is blathering on again about how Spring sucks, and so startup performance came up in the conversation. CXF uses Spring under the covers by default (it’s optional, don’t worry). Initially there was some major slowness, but I’ve spent a lot of time profiling/improving CXF’s startup time. I thought I’d share what I learned so you can improve your startup time as well:

1. Use the latest version of Spring (2.0.5+). The latest version of Spring contains may performance improvements (including one I found while profiling CXF).
2. Reduce the number of configuration files. More configuration == more XML to parse.
3. Reduce the number of <bean>s in your configuration. Contrary to popular belief, not every friggin class needs to be a <bean>. Startup time is pretty proportional to the number of beans (if you follow the suggestion in #4).
4. Don’t use classpath*:*.xml type wildcards. These significantly slow down the starup process as Spring needs to search through all the jars. In some cases it needs to expand them (I think on certain appservers). Doing a classpath:/foo/*.xml search is much better as that can delegate to the JDK. Although further performance gains can be made by removing wildcards together I think.
5. Reduce your number of beans – just because you can make it a <bean> doesn’t mean you should.
6. lazy-init=”true” is your friend. Don’t load a <bean> until you need to.
7. Turn off schema validation (Alternately: anyone want to rewrite Xerces?) – this adds about 20% more time to startup in the CXF case.

If all goes well, you should be able to get the Spring startup time pretty low. At this point its probably other things, like your ORM layer (*cough* Hibernate *cough*), that are slowing things down.

Random tidbit: I was profiling the CXF startup the other day. About 33% of the time is in Introspector.getBeanInfo(), so its really not all Spring’s fault IMO . Maybe someone can hack the JDK to be faster there?

Have any more Spring performance tips? Add them to the comments!