OSCON: non relational DBs and a Twitter clone
Thursday, July 23rd, 2009My friend Paul Brown and I just finished our presentation here at OSCON. It was a fun exploration of some of the non relational database technologies, a presentation of a key/value store based twitter clone that we built on Voldemort and some discussion of monitoring/deployment.
I started experimenting with some of the non relational databases a while back in an effort to learn a bit more. There are some very interesting problems involved in dealing with such systems. Somehow I managed to get Paul to get in on it as well and do this talk with me. He too must be a sick individual to play with this in his spare time.
We built out a functional twitter clone called BigBird based on voldemort (minus lots of things like direct messages, searching), and it’s now all open source and available on github. Our presentation is up as well (slideshare, pdf) and should give a good idea of how we architected things.
I hope that everyone finds our code useful and informative. If nothing else, it could server as a good basis for evaluating/testing Voldemort. We may improve it over time. The idea is that we will add different backends which interest us. HBase, for example, now seems much more worthy of attention than when I looked at it early this year. Cassandra seems interesting too now that it has a release out and I don’t have to deal with their crazy build process. Paul has also started on a Dynomite & Osmos implementation. Although no word on when/if any of this will be finished. 