netzooid

Ross finally got a blog. Ross: You’re only a few years late, but glad to see it!

If you’re a Mule user, we’re putting together MuleCon 2008 right now. Come and discuss all things Mule, best practices, Mule product roadmaps, etc. I’ll be giving two talks, one on approaches to RESTful integration and another which will be an introduction to Galaxy & Governance.

If you mention this blog you’ll get an extra $50 discount. Just email mulecon2008@mulesource.com or call +1-415-229-2065!

I’m happy to announce the availability of the Galaxy 1.0-beta-2 release. Galaxy is our open source governance platform which is just a fancy of way of saying, its an open source registry/repository. There are some cool new features in this release….

Mule NetBoot

You can now boot a Mule instance directly from the repository. You simply upload a Mule distribution to the repository and then use Mule NetBoot to boot this instance over the network. This can greatly simplify deployments.

Maven Publish Plugin

The Maven Publish Plugin allows you to easily push resources and artifacts from your build to the repository as part of the release cycle. You can publish individual resources, your project’s jar, and/or your project’s dependencies. This works great with Mule NetBoot…

XML Schema Support

We now index XML schemas. Which allows you to do things like this.

Improved AtomPub API

The number of things you can do with the AtomPub API has been greatly expanded. It features a much improved URI space which allows you to do a number of things you couldn’t do before:

• List individual workspaces easily (No more GQL needed)
• List child workspaces
• Add/delete workspaces
• Edit artifact metadata
• Edit artifact lifecycle information

Feedback

Your comments, suggestions, criticisms, etc are all very welcome. Please sign up for the mailing list and let us know what you think!

Thanks to the great work of Dejan Bosanac, Jettison 1.0 final has been released! He popped on the mailing list the other day willing to fix things; before I knew it he had SVN access and was fixing everything he could get his hands on. This is looking a high quality release and should greatly reduce the number of problems people have with Jettison.

Thanks again Dejan, and enjoy the 1.0 release of Jettison everyone.

Tomorrow I move to San Francisco for a few months. I’m testing the waters to see if maybe I want to live out there. And my company is based out there, so it’ll be good to work in person with people more often.

If you’re based in SF and/or traveling through it, give me a ring! I’ll be ringing up those of you who I remember live around there, but I’m not sure I can remember everybody’s location.

Also, you can always find me on dopplr to track where I am.

I agree that stuffing a URI in an HTTP Content-Type header isn’t the best way to discover more metadata about a mime-type. And maybe decentralized mime types are a bad idea.

But, there are still times where you may encounter unknown elements and want to find a schema. I thought I’d describe a quick example of how you could use Galaxy to do this.

Searching for schemas by XML element name

1. Download Galaxy (this is a snapshot, but it’ll be released soon enough)

2. Run galaxy:

$ java -jar  galaxy-web-standalone-1.0-beta-2-SNAPSHOT.jar

3. Post your schema to Galaxy:

$ curl -v -d myschema.xsd -u admin:admin –header “Slug: myschema.xsd” –header “X-Artifact-Version: 1.0″ http://localhost:8080api/registry/Default%20Workspace

This adds your schema to the workspace “Default Workspace”. (Workspaces are just folders.)

4. Query Galaxy for a schema whenever you discover an XML element you don’t know [1]

$ curl -v -u admin:admin http://localhost:8080/api/registry?q=select%20artifact%20where%20xmlschema.targetNamespace%20=%20′http://www.example.org/test/’%20and%20xmlschema.element%20=%20′testElement’

5. Parse the result for /feed/entry/content/@src for links to any schemas which describe that XML element.

Searching by media type

Let’s pretend that you didn’t actually have to have your media type approved by IANA or just didn’t care. We can also lookup schemas by media types. We’ll intercept the above process at step #4.

4 (version 1). Log in to the web interface for Galaxy. Select the schema. Add a new property to the metadata where the id is “mediaType”, the description is “Media Type” and the value is “application/vnd.myContentType+xml”.

4 (version 2). Modify the metadata via Atom:

a) Get the atom representation of the schema:

$ curl -v -u admin:admin http://localhost:8080/api/registry/Default%20Workspace/myschema.xsd\;atom > entry.atom

b) Edit the entry.atom file and add the following to the entry/metadata element:

<property name="mediaType" value="application/vnd.myContentType+xml" />

c) PUT the new atom representation of the schema:

$ curl -v -u admin:admin -T entry.atom http://localhost:8080/api/registry/Default%20Workspace/myschema.xsd\;atom

5. Search for schemas associated with your media type:

$ curl -v -u admin;admin http://localhost:8080/api/registry?q=select%20artifact%20where%20mediaType%20=%20′application/vnd.myContentType%2Bxml’

6. Parse the result for /feed/entry/content/@src for links to any schemas which describe that media type

Conclusions

Pros: Easy to find and manage schemas.

Cons: Its not universal. You need to know about Galaxy. Decentralized media types are evidently bad (see Mark’s post).

Endnotes

1. Hooray for URL encoding. In case you’re wondering, the q query parameter is really:

select artifact where xmlschema.targetNamespace = ‘http://www.example.org/test/’ and xmlschema.element = ‘test’

More friendly URLs are on the way…

Several people objected to the proposal posted on Stefan’s blog from the WSO2 guys. Mark Baker and Bill de hÓra argue vehemently against it saying we should move toward a universal data model such as RDF/OWL. While I agree that the solution proposed is not necessarily full proof or what should ultimately be adopted, its a good that we’re discussing this and we need *something* like it.

This looks like another long, never ending, I-won’t-listen-to-you type of debate, but I’ll chime in a cent or two.

Universal All the Way to the Data Model?

Should we just use a data format like RDF/OWN which can be molded into just about anything? Paul Fremantle does a good job of answering this:

The problem of more general media types is that they just push the problem elsewhere. For example, we could do everything with RDF triples, but then you have just pushed the problem to be one of finding a suitable ontology.

One of the best things about the web is how minimal the number of truly centralized services are - and media types just doesn’t seem - to me - to be in the same class as DNS root servers and IP address allocation.

We need more protocols like AtomPub

AtomPub does a great job in generalizing things so you can achieve useful things without knowing as much about a service. This comes at a functionality tradeoff though. For those who say we need more stuff like AtomPub I would like to see proposals. Either you’re going to make it more generic, and reduce the amount of functionality you can access without knowing anything about the service. Or you’re going to make it more specific, reduce the number of use cases it applies to, and increase the functionality you can access without knowing anything about the service. Either way, not a solution to the problem at hand as you’re always leaving problems out of the solution.

Also, even in AtomPub you need to extend it to support extensions. We have OpenSearch extensions,  GData extensions, security extensions, etc etc. When I encounter one of these extensions how do I figure out what it is? Sure its easy if its some universal extension. But try inventing a real (semi-complex) service without your own extensions or your own Microformat. Nearly impossible.

I’m all for AtomPub and more things like it. But people should keep these two things in mind.

Get your head out of the Internet and into the intranet

Read Stu:

Media type proliferation is a governance problem. On the Internet, the IANA is the governing body. In an Intranet, …. it depends on your governance model. What’s clear is that having everyone’s IT department register their own vnd media type seems both silly and untenable because those media types will not likely be general. So they’ll have their own corporate & partners registry.

Sure you can say those media types should be general. See the above though.

Cultural Issues

There is this idea that we should stay far far away from anything that even remotely reminds anyone of WSDL (even if its not an IDL) because it can be misused. I can’t believe that no one isn’t throwing up their arms against this idea.  I suppose this will make me very non-cool, but just because a tool can be used in a bad way doesn’t means that you should never use it. I can use an ice pick to kill someone, but that doesn’t mean one shouldn’t have ice picks. (If you’re twisted you could even view this versatility as a strength of the ice pick ;-))

Update: Just to clarify, I agree with much of what Mark/Bill say. But, to say that the *idea* is completely misplaced seems wrong to me. I still think we need a way to:

a) Discover more about a media type which I create for my intranet/business partners/etc. All I need is a way to associate a media type with a link.

b) Discover more about XML extensions or microformats embedded in things like AtomPub.

Tatu on “So How Did It All Start?”:

As an author of a highly publicized software package, one often has to answer questions like above. That — in addition to the fame, money and chicks — is one of fringe benefits of being an uber-geek programmer doG.

But, ask you: how does that relate to me? Sadly, not in any way, shape or form. But I just thought I’ll start with such a claim to grab your attention.

The joys of being an open source developer! Read on for an amusing tail of how open source projects start.

I have a serious love-hate relationship with my iPhone. There is an ever growing list of things that really piss me off:

1. It takes a swipe, two button pushes, and anywhere between 1 to 5 taps to make a phone call
2. The contact directory sucks - I should be able to start typing and have it search the names like WinMo does.
3. Touchscreen keyboards don’t compare to chiclets
4. It only supports the bluetooth headset profile. It should at least support the audio and network profiles as well.
5. Its impossible to dial somebody while driving
6. It doesn’t sync more than one folder in your IMAP account.
7. There is no IM application (well if you’ve hacked your phone…)
8. It only syncs with iTunes. I want portable Rhapsody damn it!
9. Putting someone on speaker phone requires extra effort. With WinMo you just hold down the call button while talking. With the iPhone you have to do “Hide Keypad” then hit “Speaker”.
10. Its locked into one carrier.
11. The SDK requires you get certified and signed. This model sucks. WinMo does not require this. I can write an app which does whatever I want and deploy it to any phone.
12. Its slippery
13. The speaker jack is sharp
14. The speaker jack prevents nearly every non-Apple headset from working with it. I can think of no valid reason for this other than that Apple likes to lock you in to their gear.
15. When you add a Calendar event it doesn’t alert you by default.

The web browser and to some extent the mail client is really the reason I have one. It actually really sucks as a phone itself. Microsoft really got that part right in WinMo I think. Everytime I make a call I miss my Blackjack. I give it about a half star more than the Blackjack though because of the large screen and the web browser that is actually functional.

A little while back there was a lot of pontificating on about web service description languages in the blogosphere. I avoided weighing in partly because its a complex topic and partly because of lack of time. A recent thread on TSS and a commenter have prompted me to get back to actually writing this post though. So here we go…

RESTful services do not need an interface description language. They have a defined uniform interface.  This is fundamental.

There are some things which you do need to know to interact with a service though:

1. Documentation of the entry points into the hypertext engine.

This is to some extent solved. This is what OpenSearch is for.

2. What types of media types the resource supports.

On the retrieval side this can be done with a HEAD request and the Content-Type header. On the POST/PUT side, you can use a HEAD plus the Accept header.

3. What methods are available to use on a resource.

Use OPTIONS.

4. A description of the media type.

That is we need a description of the data, not the interface. This item is where I feel there is a large void.

Lets say I have an XML schema for a purchase order. And then I have my media type application/vnd.acme-purchase-order. I need a way to associate these two.

For those who tell me to look in the documentation: that isn’t a good enough answer. We need a machine to machine way to do this. Also, why are the first 3 items OK to do with machines and not this one? Seems rather silly to me.

I know of two possible options for solving this:

1. Define an HTTP header which points to a document which provides a media type to description mapping. Then when you HEAD a resource, you get a link to the description. Then you get a document which associates a media type with a link to a description of that resource.

2. Shove it in the content type header a la James’ proposal (see the comments). Ideally though I’d like it to mesh better with the accept header.

3. Create a document which creates a link between the media type and it’s description, then point people to it. The document basically just needs to have something like <link rel=”application/vnd.acme-foo” href=”foo.xsd”/>. (For bonus points it could have links to the OpenSearch documents too…)

The larger question

The larger question in my mind is if this is all a little.. obtuse. While its kind of cool architecturally, there is a lot of resource scanning going on here to use your service. There is a lot to be said for having a description of your service all in one place like WADL does. Users will be less confused (have you tried teaching what REST is to anyone lately?).  Who wants to go around and do a bunch of HEAD/OPTIONS requests while trying to figure out how to interact with a service? You can document your service in one place (which you need to do anyway). You can think more coherently about your service as its all defined in one place. You can give it to something like Jersey and get some service stubs generated (*gasp* who would want to do something this evil!?). And probably a bunch more stuff I’m not thinking of.

Conclusions

Does REST need an interface description language? No. That’s an oxymoron.

Do RESTful services need a way for you to determine how schemas and media types align? Yes.

Would having one document to describe a set of resources be really handy? Yes.

Is this document an IDL? No.

Should you forget about IDLs then and give resource description languages a second chance because anyone who doesn’t at least admit they are somewhat useful is clearly being one sided about this? YES.

In short, I think that having a resource description language is extremely useful from a practical standpoint and I generally support the effort to come up with one. It just simplifies my life too much for me to not want it.

Update: fixed a grammar problem or two and added a third way to discover media type descriptions.