Build Scala projects with Eclipse Buckminster

In one of my previous posts, I talked about how to build Scala projects with PDE Build. However, PDE Build is rather old and has long been superseded by Buckminster. The great advantage of Buckminster is that it is actually a lightweight Eclipse SDK with all you need to build an RCP application: a workspace of projects, a target platform, and so on. To build Scala projects in Eclipse, you will most likely install the Scala IDE. Unfortunately, this plug-in cannot be installed in Buckminster right away. But there is a workaround…

Why Buckminster?

Let’s first talk about why I prefer Buckminster over other solutions. Compared to other build tools, Buckminster has one great advantage: it provides an environment very similar to the one you’re developing in—the Eclipse SDK. You can see Buckminster as a lightweight Eclipse SDK without all the distracting UI stuff, but with a command line interface instead. However, Buckminster still has a workspace where you can import your projects into, and of course a target platform which defines the environment for your RCP application. Running a Buckminster-driven build with continuous integration systems such as Jenkins is also very easy, since all you have to do is to create some configuration files—a resource map and a CQuery—and then run a simple command on the shell: buckminster build. There is also a plug-in available for Jenkins, which automatically downloads and installs Buckminster if necessary.

Buckminster is also one of the few tools that are able to build a full-featured Eclipse RCP application. There are extensions for other build tools such as Ant or—since we’re talking about building Scala projects here—sbt. For example, Bnd is one of them. It allows building OSGi bundles without the need for creating a MANIFEST.MF file manually. Besides, you can always package your OSGi bundles yourself using the jar tool, but that is really not enough for serious RCP development.

If you want to build your RCP application headlessly, you have to use a tool that is really aware of what it is building. It has to understand the complex dependencies between your projects and the OSGi bundles from the target platform you use. Apart from that, it has to be able to parse files like MANIFEST.MF, build.properties, plugin.xml as well as configurations stored in .product or feature.xml files.

As far as I know, there are only three products available which support all these features (or at least most of them): PDE Build, Eclipse Tycho and, of course, Buckminster. I mention PDE Build here since it was formerly the recommended way to build RCP applications but has now been superseded by Buckminster. PDE Build is also only a set of Ant scripts and does not support all of the aforementioned features. Tycho, on the other hand, has recently become a mature solution. It integrates very tightly into the well-known Maven build tool. This makes it very easy for people who are using Tycho for the first time, but already know Maven. Apart from that, Tycho contains some cool features like archetypes which automatically create pom.xml files for the bundles to build.

Nevertheless, I found Tycho always being slower than Buckminster—at least for very large projects with a lot of bundles. I don’t know if this has something to do with the support for incremental builds in multi-module projects (Is this still a problem with Maven 3 anyhow?) or with the way Tycho is resolving the target platform and how it calculates bundle dependencies. Apart from that, Tycho builds projects in a slightly different environment than they are developed in. This eliminates all of the aforementioned advantages of having a build environment that is very similar to the Eclipse SDK. As I said, this is in fact what you have when you use Buckminster. These drawbacks make it of course not impossible to build RCP applications, but they are the reason why I would prefer Buckminster over Tycho. Of course, this may change eventually, since Tycho is continuously developed.

How to install Buckminster and the Scala IDE normally

There is a Buckminster plug-in for the Eclipse SDK that adds a context menu with build tasks to all workspace projects. You simply install it using the p2 update manager. The current update site can be found at https://www.eclipse.org/buckminster/downloads.html.

Installing this plug-in and then installing the Scala IDE is very easy and lets you build Scala-featured OSGi bundles in a few minutes. Nevertheless, setting up Buckminster with the Scala IDE for headless operation is a little bit harder. If you follow the documentation, very strictly, you will do the following things:

First, you have to download the latest version of the p2 director application: director_latest.zip. You can then install the Buckminster RCP application with the following command:

$ director -r [URL TO HEADLESS REPO] -d [BUCKMINSTER INSTALL FOLDER] \
      -p Buckminster -i org.eclipse.buckminster.cmdline.product

This will download Buckminster from the supplied update site—which can be found on this site—and install it into the given folder. After that, you will most likely want to install some additional features in order to enable headless builds. Change into the directory where you installed Buckminster and run the following commands, for example:

$ buckminster install [HEADLESS REPO] org.eclipse.buckminster.core.headless.feature
$ buckminster install [HEADLESS REPO] org.eclipse.buckminster.emma.headless.feature
$ buckminster install [HEADLESS REPO] org.eclipse.buckminster.git.headless.feature
$ buckminster install [HEADLESS REPO] org.eclipse.buckminster.pde.headless.feature

Additionally, you have to install the Scala IDE into Buckminster. Go to http://download.scala-ide.org and select the update site that fits your needs. Then run the following command:

$ buckminster install [SCALA IDE REPO] org.scala-ide.sdt.feature

This will print:

Cannot complete the install because one or more required items could not be found.
[0]Software being installed: Scala IDE for Eclipse 2.0.0
     (org.scala-ide.sdt.feature.feature.group 2.0.0)
[0]Missing requirement: Scala IDE for Eclipse 2.0.0
     (org.scala-ide.sdt.feature.feature.group 2.0.0)
   requires 'org.eclipse.ui.navigator 0.0.0' but it could not be found

Oops! What happened? It seems like the Scala IDE requires user interface bundles, but Buckminster runs on the command line!

Workaround

In order to work around this problem, you have to install Buckminster a little bit differently. First, you need the p2 director application again, but this time you will install the Eclipse SDK:

$ director -r [ECLIPSE SDK REPO URL] -d [ECLIPSE INSTALL FOLDER] \
      -profileProperties org.eclipse.update.install.features=true \
      -p SDKProfile -i org.eclipse.sdk.ide

After that, change into the directory where you just installed Eclipse and run the following commands to install Buckminster:

$ eclipse -nosplash -application org.eclipse.equinox.p2.director \
      -r [URL TO BUCKMINSTER HEADLESS REPO] \
      -i org.eclipse.buckminster.core.headless.feature.feature.group \
      -i org.eclipse.buckminster.emma.headless.feature.feature.group \
      -i org.eclipse.buckminster.git.headless.feature.feature.group \
      -i org.eclipse.buckminster.pde.headless.feature.feature.group

Note the additional .feature.group at the end of each feature to install.

Finally, you can install the Scala IDE:

$ eclipse -nosplash -application org.eclipse.equinox.p2.director
      -r [SCALA IDE REPO]
      -i org.scala-ide.sdt.feature.feature.group
      -i org.scala-ide.sdt.weaving.feature.feature.group

This should succeed without errors.

From now on you can launch Buckminster with the following command:

$ eclipse -nosplash -application org.eclipse.buckminster.cmdline.headless

You may get the following exception when the Scala IDE bundles are started:

java.lang.IllegalStateException: Workbench has not been created yet

This is due to the fact that you don’t have a UI and hence no workbench. Fortunately, there’s an (undocumented?) system property you can set to avoid this exception. Simply add the following line at the end of your eclipse.ini:

-Dsdtcore.headless

This tells the Scala IDE that you’re running it in headless mode and that it should avoid accessing the workbench.

Summary

In this article, I described how to headlessly build Scala projects with Buckminster. I talked about why I prefer Buckminster over other build tools and presented a workaround for installing Buckminster and the Scala IDE with the command line.

After installing Buckminster this way, you will have a full fledged tool to build Eclipse RCP applications including Scala plug-ins. However, installing a complete SDK would not be necessary if the Scala IDE bundles did not depend on Eclipse’s UI. It’s probably possible to create a feature containing only the core plugins like it has been done for the Groovy plug-in, but that’s something the Scala IDE team has to figure out in the future.

Nevertheless, a complete Eclipse SDK has also an advantage worth mentioning: everything that works on your local machine will also work on your headless build server. Actually, you may even install your local Eclipse the same way as described above. Then you can absolutely be sure that your application is built in the same environment as it is developed in. This can make debugging a lot easier.

Posted by Michel Krämer
on 26 June 2011