Murphy’s law

If all goes well next week you won’t even know the difference, which actually is a bit sad. 😉

But since most of us work in IT, we might need to turn it off and on again.

Early next week the domain “packagedrone.org” will be transferred to the Eclipse Foundation. The blog will still run on the same server, for now. But this is one of the steps in becoming an Eclipse project.

So if you experience some problems connecting to this site, it is just a hick up caused by the domain transfer.

0.9.4, 0.10.0-m1 and more

In the last few days we got a few maintenance released for the 0.9.x release branch and some more progress on the upcoming 0.10.0 version.

Package Drone 0.9.4

Due do the fact that the system grows (which is a good thing) it also gets more complex. This caused a few issues in the 0.9.x release which indicates that it is time to re-think the release policy (a thing which is not written down). In the past there was rarely a milestone release and never actually a release branch. This worked quite well for some time, but it turns out that we passed the point of being able to make such quick releases any longer. The beginning with the 0.9.x release we now have a maintenance branch for 0.9.x releases and will provide a set of milestone releases for the next major release 0.10.0, before doing the final 0.10.0 release. This also means that releases will take longer, but possibly can have more changes in each release.

Milestone release 0.10.0-m1

This brings us to the first milestone release, 0.10.0-m1. Beside the usual minor fixes and cleanups it contains the new RPM and YUM aspects and adapters and contains a re-written locking mechnism in order to finally fix issue #97. It also tries to give MySQL a second chance. Due to the fact that MySQL actually does quite a few table locks, it is impossible to use MySQL for concurrently accessing different channels. The new locking manager will take care of this and lock the complete storage service for write operations. Of course this is not an optimal solution, therefore it is recommended to use PostgreSQL instead, which allows to access different channels in parallel.

Apache Felix Configuration Admin

Also will we try to finally fix issue #103 by switching the Configuration Admin implementation from Eclipse Equinox to Apache Felix. Oh how wonderful OSGi can be 😉 Yes, we are switching a system service from one OSGi implementation to another one.

Beside fixing this one issue which causes troubles when upgrading, the Apache Felix implementation also stores the configuration in plain property files, which can be read and edited manually. Beside that it is possible to use a storage directory outside the data storage area of OSGi, so that it can be placed anywhere in the file system. This also allows pre-provisioning all configuration outside the OSGi container.

OSGi HTTP service

Package Drone still uses its own Jetty based OSGi setup. Jetty itself brings some very extensible OSGi functionality, but does not provide an OSGi HTTP service. Therefore it is not possible right now to add servlets using the HttpService or the HTTP whiteboard pattern, a thing which is required it we want to add a real REST interface.

The problem is that, although Eclipse Equinox provides an HttpService based on a recent version of Jetty (9.2.x), it currently contains a blocking issue, #467859. There is a fix in Eclipse Gerrit, but unfortunately this seems to break some functionality in Liferay, so the fix was rolled back. So maybe it is necessary to fork this specific module of Equinox in order to use the HttpService based approach.

Screenshot

See the new YUM help screen in action:


Package Drone 0.9.0

Whhappp … Package Drone 0.9.0 is out!

Be sure to check out the most recent release on github.

The most interesting new feature is the basic POM file generation from OSGi meta data (pull requests: 81, 86 and 89, thanks @maggu2810). This aspect can create a basic POM file from the extracted OSGi metadata. Hopefully, in the future, this aspect will also create dependencies in the POM files as well.

In addition to that a few bugs got fixed (thanks @afischer211 for reporting and helping). The most interesting bug was caused by the cleanup process cleaning up an artifact which was being uploaded in the same time. But now Package Drone can handle this case as well 😉

Thanks to all contributors!

Package Drone 0.8.0

Rooooar … Package Drone 0.8.0 is out!

Be sure to check out the most recent release on github.

The main two features in this release are validation and the upload API.

Validation allows each aspect to provide information if artifacts or channels are valid (WARNING, ERROR) and provide a message. Two use cases are OSGi bundles with invalid names, which would cause a P2 repository error later on. Second the case when two P2 installable units have the same ID but different MD5 checksums. Again P2 would complain and Package Drone can fix and warn about this now.

The Upload “API” (actually just a URL, see the Upload API in the wiki) allows to create a new artifact using a simple PUT/POST request using the deploy key system. So it is possible to use curl or any other command line tool to upload artifacts.

Using this API Package Drone also now has a Jenkins Plugin, which can be used to deploy directly from Jenkins to Package Drone.

Thanks to the contributors!

Package Drone @ Eclipse DemoCamp Mars 2015 in Munich

Package Drone will be presented at the Eclipse DemoCamp Mars 2015 in Munich [1]. Although the even is free (as in beer), you still need to register [2].

We will present the current state of Package Drone and what the future might bring, since we only have small time slot. But there will be beer, food and the opportunity to have a longer talk at the end of the official schedule.

So, let’s meet!

PS: If the event is already booked out, you can still register and put yourself on the wait list.

[1] https://wiki.eclipse.org/Eclipse_DemoCamps_Mars_2015/Munich
[2] https://www.eventbrite.de/e/eclipse-democamp-mars-2015-tickets-16747508248

Package Drone 0.7.0

Flappppp … Package Drone 0.7.0 is out!

Be sure to check out the most recent release on github.

This release of Package Drone brings in ideas, help and code from @cornzy, @CptMauli and @maggu2810. Thank you for contributing!

Beside a few nice little enhancements like channel help pages for maven and p2, a freetext HTML documentation page for each channel and a few cleanup there are two nice little features which are worth mentioning. Of course the full release notes are available on GitHub.

First of all there is Nexus scraping. Yes, Package Drone can provide a full remote index for the Nexus repository. Well, I you like to do that of course 😉 Package Drone provides the same meta data index files as Nexus does and so the full remote index information can be used by Nexus.

Second there was a discussion in the GitHub issue tracker about the idea to make it easier to consume artifacts using maven. Before 0.7.0 it was necessary to upload artifacts with “mvn deploy” in order to be able to download them. Now when a JAR file gets uploaded to Package Drone, the “Maven Extractor” process can unpack the embedded POM file from a JAR file and extract the Maven meta data information (group, artifact, version) from the POM file. The rest stays the same, the Maven Repository index will pick up this data and create a Maven repository structure for this. Another nice little feature.

Thanks to the contributors!

Short April update

So what happened in April? Not much from a release perspective 😉

However a few things have happened behind the scenes. So first we hopefully have two additional contributors, they provided a few minor fixed and a lot of great ideas! Hopefully there will be more in the future.

In addition to a few bug fixes there is issue #71 in GitHub. Based on the idea that each JAR file generated by Maven contains a POM file, it would be possible to a) extract this POM file and b) create necessary metadata out of it in order to provide the necessary maven repository information.

In short this means that the next version of Package Drone will allow the upload of a simple, Maven generated, JAR and can provide it using the Maven repository adapter.

So hopefully there will be some time in the next weeks to make another release of Package Drone 🙂

There also is a Google Group now for Package Drone. Of course everything which is a bug, feature request or similar should go the the GitHub issue tracker. However questions and discussions now can be handled better at the new Package Drone Google Group.

Package Drone 0.6.0

Wuuuushhh … Package Drone 0.6.0 is out!

Be sure to check out the most recent release on github.

The big feature in this release is the file system BLOB store. So due to the issues with MySQL and PostgreSQL when it comes to big BLOBs (see previous posts) the (some would say, long overdue) file system BLOB store was added. It acts as an overlay to the database which can be activated at any time and after this store the BLOBs in the file system. BLOBs which had been stored in the database before are still loaded from the database.

Now although this was possible before on machines with more memory, it is now possible to host a full build of Eclipse Orbit in the virtual machine running thedrone.packagedrone.org, our demo system. The channel consists of an imported ZIP archive, containing the actual build output of Eclipse Orbit. The artifacts are extracted as virtual artifacts and metadata is being generated on those. Adding a second build of Orbit would merge those two results, but still each build would be an atomic unit, since they got uploaded as ZIPs.

The documentation also got a bit of attention, so there is a little bit more on the file system blob store, system properties and backup & restore.

Package Drone 0.5.0

Sssgduschh … Package Drone 0.5.0 is out!

Be sure to check out the most recent release on github.

The most notable feature in this release is the support for PostgreSQL. So during the setup of Package Drone there is now a real choice of databases, MySQL or PostgreSQL.

Also have a lots of bugs been fixed. The channel aspects are now versioned and request an update of metadata when required. And finally there is way to backup and restore a single or all channels.

Since both JDBC drivers, MySQL and PostgreSQL, have issues with streaming BLOBs, the next version of Package Drone will introduce a file system layer for storing artifacts. A transparent transition mechanism will ensure that it won’t cause any troubles migrating to the next version.

Package Drone 0.5.0-rc1

Chakka! Package Drone 0.5.0-rc1 is out!

Be sure to check out the most recent release on github.

There were a lot of changes after the 0.4.0 release that we thought it would be best to start with at least one RC before releasing the 0.5.0.

The most notable changes are that Package Drone now can run on PostgreSQL! Which also performs better than MySQL for big artifacts. PostgreSQL does fully support streaming. So starting with 0.5.0, PostgreSQL will be the recommended database for Package Drone. Thanks to @cptmauli and PostgreSQL JDBC 9.4, which is OSGi compliant out of the box!

Beside that there are a lot of enhancements and improvements. Aspects are now versioned and there is process to upgrade them.

Screenshots will be present in the final release announcement. Check the release note for the RC1 for more information.