Mixing JARs and OSGi Bundles with Maven
July 3rd, 2008Seeming that I’m on an OSGi roll here, I want to elaborate on why Maven and OSGi don’t mix well.
One of the things I like about Maven is that it has created a repository layout to store JARs/applications/etc for folks to use. There is a central repository, but you can also create your own repositories easily as well. Whether its log4j or Mule or Jetty or Spring, I can find it in the Maven repository.
The thing I want to discuss in this entry is how to create an application, with Maven, when dependencies are a mix of OSGi enabled bundles and just regular ol’ JARs. (Things such as how do I create an OSGi bundle from Maven or how to do I create a POM around an existing OSGi bundle are a separate discussion…)
Flawed Approaches
Spring Source went down the route of giving everything a new name in the Maven repository, or more specifically a new groupId which starts with com.springsource. This is a fundamentally flawed model as it breaks everything that Maven does. Now, I have some projects which depend on com.springsource.org.hibernate:hibernate and some which depend on org.hibernate:hibernate. It requires a complete clean room implementation of the Maven repository which is a time consuming and losing battle. Every time somebody adds a project, it has to be added to the SpringSource repository.
Also, what happens as projects add OSGi headers to their jars? Will everything in the repository be instantly switched back to the original groupId? If so its probably going to break all sorts of stuff in consumers’ POMs like dependency exclusions or usage of the assembly plugin. I could go on about how bad this approach is long term…
Next option: What if I used BND to modify JARs inside a repository and adding headers. Now you avoid all the issues around conflicting artifact/group ids, but you open up another can of worms. If the dependency is in your local repository, then you have the issue of having a different JAR from the regular Maven repository. Checksums and signatures: broken.
If you modify a JAR that sts in a public repository, then we have issues. First, who gives you the right to add the OSGi headers to somebody else’s JAR? Its their official distribution not yours. Then, how are you going to resign it? What are you going to do about the fact that some repositories might have the original JAR which you didn’t modify? You could end up with the non-OSGi enabled version of the JAR in your repository quite easily.
You could try to convince everybody to add OSGi headers upstream. I’m not waiting around for that to happen though.
A Way Forward?
I see a couple approaches which can be utilized.
First, you can use the mixed approach that David Savage advocates in a comment on my blog:
If you are building some new OSGi application that depends on classes from a plain old jar and those classes never need to be passed between OSGi code in different bundles. Then in this case you can embed the jar containing the classes directly into your bundle and add it to the local classpath of your bundle via the Bundle-Classpath header.
This gives you a deployable unit of code that automatically contains all code needed to function. Compared to a standard J2SE approach where you need to suppliment the local classpath with all the dependencies of a given jar.
The biggest problem with this is that you end up not gaining any of the strongly touted benefits of OSGi. If I’m going to do this, why even bother with OSGi?
The second approach that I can see is to use different version names. Instead of 2.0, you can use 2.0-osgi. This allows you to get around some of the disadvantages that I outlined above as its clear that this isn’t the original version of the JAR, its a new one. It also gets around the SpringSource problems as switching to an OSGi versio of the jar is as simple as adding the dependency (with the updated version) to your <dependencyManagement> section of the POM. Even better, as projects decide to add OSGi headers, you can remove these dependency listings from your <dependencyManagement> section, allowing your POM to shrink and become more manageable as time goes on.
(Along these lines, I wouldn’t mind seeing a Maven repository which could dynamically create bundles via a wiki like fashion. I as a user could submit an updated set of MANIFEST headers. Then the proxy would dynamically download the original dependency, add the headers, change the version and provide it to me as a user.)
Another approach that I wouldn’t mind seeing (and this is more long term) is to equip OSGi containers to be able to retrieve OSGi metadata to supplement JARs from a server. On a final note, I’l leave you with a tool that popped up on Steve’s blog to be able to do this - PAX URL wrap:
Pax URL Wrap is an OSGi URL handler that can process your legacy jar at runtime and transform it into an OSGi bundle.
Cool stuff. I have yet to try it out though.
July 3rd, 2008 at 3:07 pm
Dan, 2 comments:
1. Bundling dependencies via Bundle-Classpath in the same jar. Actually, in case of Mule this was the first thing that was tried long time ago. It doesn’t really work for a highly-modularized project like this (there were around 80 modules back then, now more). Just to mention that it was a 120MB bundle, monster… However, that’s the shortest path to get one started, and then work on pruning the deps and cleanup. And I don’t like it as a solution either, just replacing a problem with another type of. No, I don’t have a silver bullet solution either
2. Maven versions having ‘osgi’. I wonder why classifiers didn’t come to scene here, as they seem to be an ideal fit (osgi suffix is a classifier then, not part of a version). In fact, we did a very similar thing, when we had to retrotranslate jsr223 (scripting) jars to run on JDK 1.4.x. Just had a jdk14 classifier on the dependency. I know, there were weird issues with classifier handling in maven 2, but it’s beem a long time, and they had been resolved now (mostly?).
July 4th, 2008 at 4:36 am
“The biggest problem with this is that you end up not gaining any of the strongly touted benefits of OSGi. If I’m going to do this, why even bother with OSGi?”
Possibly I wasn’t clear, but the point I was trying to make was that in certain circumstances you can move forward with an OSGi architecture without having to rebuild every upstream dependency - which was the point of your first blog post.
You still get to build new code using OSGi patterns but can reuse “legacy” plain old jars without the upfront cost of converting the entire universe to the OSGi model.
One of the benefits of OSGi bundles is that they explicitly either contain or list the all the dependencies required to work at runtime. This means you can always administratively check prior to running whether you have all dependencies - compared to plain old jars which traditionally involved guessing games based on trapping NoClassDefFoundError at runtime.
Maven dependency resolution has certainly done some work to help get around this problem in the build process. But again it pretty much mandates that the entire world convert to Maven so is no less invasive.
OSGi’s focus on modularity via import-package/require-bundle is obviously very useful, but modularity for its own sake via an uber fine grained set of bundles is not that useful. A pragmatic approach is to define your modules such that they are self contained, reusable and modularised /as much as possible/.
As you point out the problem is that Maven and OSGi resolution processes are not compatible. But as Richard Hall commented in your previous post given OSGi has been around for a lot longer (1999) it is difficult to see how this is OSGi’s fault…Not casting stones - agree there’s never a silver bullet solution.
Finally I’d say OSGi is only the first step - you may want to look at the sort of things we can do with OSGi applications in http://newton.codecauldron.org
July 4th, 2008 at 12:15 pm
Few things:
1) Minor correction: SpringSource respects maven groupId’s in their SSEBR, it’s the artifactId’s that are prefixed.
2) I disagree that bundling dependencies using the Bundle-Classpath header makes using OSGI somewhat pointless. Their are clear cases both for hiding dependencies inside a bundle and for deploying dependencies as independent bundles. Some of the considerations to take into account when deciding this are:
- Is the dependency required by a single bundle or do multiple bundles require this dependency?
- Is there a requirement to allow users to choose/change the version of the dependency that is used?
- More bundles may mean a smaller distribution and allow the flexibility to switch out dependencies versions independent of application bundles but it can increase deployment complexity.
3) Having said that we still have the issue you put forward if we want to deploy a dependency as a bundle and it is not distributed with OSGI headers. Their are some good ideas here, we just need to make sure we don’t underestimate the importance of these OSGI header declarations as an integral part of a bundles API (as Felix says) which, in the case of jar’s without OSGI headers is missing.
Based on this, and taking into account that OSGI’ified artifacts and their OSGI headers would need to be frozen rather than living if people are to use/depend on them, I’m not very optimistic about a wiki-based approach. The need to depend on these bundles probably explains why we see most projects create their own bundles. Neither do I think that a loose “auto-magical bundleization” of jar’s is a good approach either, be it at compile-time or runtime.
So in conclusion, if we don’t want to wait around for authors to add headers upstream and we don’t want to have to create out own bundles I think the best approach is a multi project/vendor SSEBR style bundle repository with more generic artifact names using an maven identifier as Andrew suggested. Although this would be excellent in theory I’m not sure how it would work in practice. Who would have the definitive answer as to what packages a library should import and export or what versions of dependent packages are required if it’s not the author? Is there a single answer or does i depend partly on interpretations and needs? How would consensus be reached on this before the OSGI’ified jar is made available?
July 4th, 2008 at 1:57 pm
@Andrew: As I understand it, classifiers can’t be used as artifacts with the same groupId/artifactId but a different classifier are treated as different artifacts. So you would get both the non-osgi and osgi artifacts.
@David: Maven is less invasive. With maven the metadata and the actual artifact can be distributed separately. So I can download a JAR from somewhere, write my own metadata in a separate file and be good to go. OSGi is typically much more invasive as you have to modify everything and cannot distribute the metadata separately. This is not a requirement for OSGi as the PAX URL wrap utility shows, so why make it one?
@Daniel: Thanks for your insights and corrections. I agree that having a repository where anyone can modify the bundle data is somewhat difficult logistically. I think short term I’m probably more of a fan of just separating out the OSGi manifest information and the actual jar itself for non-OSGi enabled artifacts.
July 5th, 2008 at 11:53 am
I’m sorry I have to disagree, maven is just as invasive - the problem with your argument is that you are comparing different ends of the graph resolution dependency problem (a bit like comparing the head of one arrow to the flight of another and then announcing that the first arrow is much sharper).
In maven the resolution dependencies are handled by the pom and the result is a set of artifacts. In OSGi the resolution dependencies are manifest headers and the result is a set of jars.
In both cases the results are simple entities that are portable between environments in that you can use an OSGi bundle or an artifact as a plain old jar if you have to.
But at the other end of the problem the dependencies must be specified up front by the person authoring the component. Just as it is very difficult to see how you can autogenerate the pom dependencies from a set of class files it is just as difficult to generate the manifest headers for an existing jar. Both require human interaction to make complex choices that machines are just not that good at making.
Good conversation anyway
Regards,
Dave
July 5th, 2008 at 12:25 pm
Clarification - to avoid confusion or circular arguments (serves me right for posting on a Sunday night).
Just to confirm the invasiveness I’m referring to is in the /use/ of the dependency resolution information, you are talking about invasiveness in terms of modifying the jar so we’re possibly talking about apples and oranges.
If you were building a maven build out of a set of non maven jars you can obviously generate a pom for a particular jar by inspecting the code and via a series of trial and error steps to prune out bad versions etc.
In the OSGi case you can package a set of non OSGi jars into a parent bundle and again resolve external dependencies through a similar pattern.
In both cases the result is fragile as if the module author updates the code then you as a user have to update your dependency information. But at least it gets you moving.
This is where it becomes much more robust for the author to supply the resolution information as part of their module - which is where the upstream pressure of OSGi and Maven info comes in as in both cases module authors can add a lot of benefit to clients if they just declare their dependencies up front.
Regards,
Dave
July 6th, 2008 at 1:49 pm
I think everyone has good points in this discussion. I agree that we can’t expect to reap significant benefits from OSGi without actually designing and packaging libs according to the new paradigm, just as we couldn’t reap significant benefits from Maven’s handling of transitive dependencies until most libs started to have correct POMs. At the same time I am sympathetic to Dan’s frustration with having to “OSGify” the known universe of dependencies, as it really is a PITA and reminiscent of what we went through not so long ago with “Mavenification” of the known universe.
Perhaps one thing that might help is for the OSGi Frameworks themselves to provide some default settings or even heuristics for being able to load a non-OSGified jar (e.g., make all packages private, all public, guess the package id of the jar’s author and only expose those, etc.). Maybe only marginally useful, but perhaps it would be better done by the Framework itself than by each and every vendor/project trying to introduce OSGi to their software.
July 6th, 2008 at 8:31 pm
@Travis: to me there is an important difference between Maven & OSGi though. With Maven I create the POM myself and have it separate from the actual build artifact. With OSGi I can’t.
July 8th, 2008 at 12:20 am
Pax (URL) Wrap is good.
I’ve been using it all of the time, and in some cases also persuading upstream to OSGify their libs.
Pax Construct, Pax Swissbox, Pax Logging, Maven Bnd Plugin, Pax Web Service, … (Pax Script!) .. they’re so useful…
July 8th, 2008 at 12:22 am
@Travis
“Perhaps one thing that might help is for the OSGi Frameworks themselves to provide some default settings or even heuristics for being able to load a non-OSGified jar (e.g., make all packages private, all public, guess the package id of the jar’s author and only expose those, etc.).”
This is recently known as what Dan just covered, SpringSource Application Platform
September 26th, 2008 at 8:43 am
order phentermine line…
yeasts swallowtail bivalve!…