Wednesday, July 8, 2009

Grails Read-Only Plugin Repository

One of my favorite enhancements to Grails 1.1 is the automatic transitive plugin resolution. This feature is great for team development. When a new plug-in is introduced to a project only one developer needs to install it. When they do, the application.properties is updated with the plugin name and version numbers like Listing 1 which shows an application.properties containing the hibernate, image-tools and jsecurity plugins.

Listing 1 - application.properties file with plugin references.

app.version=0.1
app.servlet.version=2.4
app.grails.version=1.1
plugins.hibernate=1.1.1
plugins.image-tools=1.0.4
app.name=pfj
plugins.jsecurity=0.3

When other members of the team get the updated application.properties file they just run grails run-app and the plugins will automatically be downloaded and installed on their machine. Life couldn’t be easier. Well, maybe it could be. See for this to work, the plugin must be installed in the central grails plugin repository found at http://plugins.grails.org/ or you can use another one of my favorite new features of Grails 1.1, multiple repositories. To configure multiple repositories just create a BuildConfig.groovy in your application’s grails-app/conf directory and add a new entry for your additional repository like Listing 2.

Listing 2 - Additional plugin repository entry in BuildConfig.groovy.

grails.plugin.repos.discovery.myRepository="http://company.com/grails"

The only problem with this is it assumes the repository is hosted in Subversion (SVN). This is not always the case. With the popularity of git many of the plug-in developers have gone to using github for hosting their plugins. One example is the popular image-tools plugin.

Another important use case takes advantage of my second favorite feature of Grails 1.1, the Apache Ant and Apache Ivy integration which enables you to do a continuous integration builds in something like Hudson by executing the Ant build.xml without even having to have Grails installed on the build machine. Everything required is installed as a bootstrap. Now that is awesome. Of course this assumes the plugins are in a configured repository that is backed by SVN.

To simplify developer setup and continuous integration, I looked for documentation to setup a custom Grails plugin repository for plugins I did not maintain but I still want to access using the new Grails 1.1 features. All I found were instructions on how to do it if you are the plugin provider or using the Grails central repository. If that is your goal it can not be easier. For deploying to the central repository you first need to request permission to the Grails plugin SVN repository. Then to deploy the plugin just type grails release-plugin. For a custom plugin repository, you will need an additional configuration in your BuildConfig.groovy file to point to your SVN repository like the one found in Listing 3 and type grails release-plugin -repository=myRepository where the repository parameter is the last part of the property name. That’s it.

Listing 3 - BuildConfig.groovy with an entry to a repository for distribution.

grails.plugin.repos.distribution.myRepository="https://company.com/grails/"

I could not find any documentation about creating a custom read-only repository for third party plugins you don’t want to maintain or if you just don’t want to have to maintain a SVN repository just for those third-party plugins. So after a little bit of exploring the central repository and some trial and error, this is how I created my own read-only repository saving time and simplifying my team setup.

Note: for a read-only repository you don’t need SVN all you need is an HTTP based server that can serve files.

1. Create a BuildConfig.groovy file in your application’s grails-app/config directory containing grails.plugin.repos.discovery.myRepository="http://company.com/grails/" entry where the URL is addressable via HTTP and is the root of the plugin repository.

2. Create a plugins-list.xml file in a .plugin-meta data directory under the repository root. This file needs to contain your plugin(s) meta data like the following for image-tools:




<plugins revision="2">
<plugin release="1.0.4" name="image-tools">
<release tag="RELEASE_1_0_4" type="svn" version="1.0.4">

<title>Image Tools</title>
<author>Arquetipos Software</author>
<authoremail>projects@arquetipos.co.cr</authoremail>

<description>Images Tools</description>
<documentation>http://www.grails.org/ImageTools+plugin</documentation>

<file>http://www.arquetipos.co.cr/blog/files/grails-image-tools-1.0.4.zip</file>

 </release>
</plugin>

</plugins>


3. Create a grails-/trunk directory under the repository root and copy the zipped plugin package there with the name grails--.zip.

Implementing these 3 steps made my third-party plugins available to the list-plugin and install-plugin commands but more importantly to the automatic transitive plugin resolutions for developer and continuous integration servers.

10 comments:

Josh Brown said...

This is really cool - thanks for sharing!

Willy said...

Thanks. However, this doesn't seem to work right of the box. When trying to install a plugin the new local repository, I get a 'Latest release information is not available for plugin 'plugin name', specify concrete release to install' error message.

Christopher M Judd said...

Willy,

I don't understand your question. If are trying to do a "grails release", it won't work. This is a read-only solution. To make the plugin available you will have to copy it to a location where it can be accessed it via Http. If you want a version that you can release to you will need to use SVN.

Willy said...

Copying the .zip files generated via the grails package-plugin command to a location accessible via http is exactly what I did.
I nonetheless get a 'Latest release information is not available for plugin 'plugin name', specify concrete release to install' error message when trying to install the plugin from my local read-only repository.
I think, there must be some meta information missing.

Christopher M Judd said...

If that is what you did, you are probably right, it is a meta data problem, can you share the meta data and Http based paths?

TH said...

This was really helpful. I think I almost have it working. My only problem is that I'm trying to use our subversion repository as the HTTP server and it requires authentication. When I run 'grails list-plugins', I'm prompted for SVN authentication information and it works. But our CI server isn't quite so smart. It skips past the prompts and leaves the plugin out of the generated war file.

Any thoughts about how to solve this? If I could just use a local path or somehow pass in the SVN authentication info, it would be ideal, but if I have to, I can identify another http server to use as my repository.

Christopher M Judd said...

TH, I have never done it but it would be HTTP basic authorization not SVN authentication, you would be using. Ivy seems to support HTTP Basic Auth in a couple of ways. Once reference I have found was at http://ant.apache.org/ivy/history/latest-milestone/use/settings.html.

TH said...

Thanks for the follow up, Christopher. I ended up deciding to take a different approach. My goal was to have the plugin in SVN and I realized I could just leave it expanded in my plugins folder and check that into SVN (while leaving the folders for downloaded plugins svn:ignore'd). It wouldn't help if I was trying to share the plugin across multiple projects, but it meets my current needs.

Al Rodgers said...

This is great, thanks.

I hit an issue using a local repository because I have a proxy configured in Grails ('grails set-proxy') so I can see the public repositories through our corporate firewall. This got in the way when it came to resolving my internally-hosted HTTP server. I got round it by configuring the 'http.nonProxyHosts' Java networking setting.

This involved editing {GRAILS_HOME}/scripts/_GrailsProxy.groovy to set the 'nonproxyhosts' attribute on the 'ant.setproxy' command and also add the 'http.nonProxyHosts' to the system properties map.

Finally, I also modified {GRAILS_HOME}/scripts/SetProxy_.groovy so that it captures the value from user input when calling 'grails set-proxy' and adds it to the map in ProxyConfig.groovy.

Kimble said...

Great blog post!

Does anyone know whether it's possible to get basic http authentication without an svn server?

Configuring the repository in Grails with http://username:password@... initiates a svn request.

AWS EC2 Hibernate Java SDK v2 Example

I recently wanted to automate the creation of developer VMs in AWS using EC2 instances. To improve the developer experience (DX), I didn'...