Using Amazon S3 as a Maven repository
October 15th, 2009I’ve been creating several small projects that I’ve shared with friends and collegues. An often recurring problem is that as soon as I share a project I receive the same complaint: “it doesn’t build. Maven complains about an artifact that it cannot find”. It works great on my local development laptop since all my Maven dependencies are safely located in my local repository. The solution is to provide my artifacts through a publically available Maven repository.
Among the many Maven Repository Managers, Nexus is a popular choice. I’m quite familiar with it, but in this case I’d rather just drop off my artifacts on a remote server. Reading this blog, I decided to deploy all my artifacts to a bucket on my Amazon S3 account. These are the steps required to make it all happen.
Wagon
First of all I had to checkout and build Wagon-s3 from source. Wagon-S3 is a protocol provider to the Wagon Maven extension. Wagon is used by, for example, the maven-deploy-plugin to upload artifacts to a Maven repository. As the same suggeste, Wagon-S3 enables Wagon to transfer artifacts to an Amazon S3 bucket.
First I needed to checkout and build the main Wagon project:
<code> svn checkout http://svn.apache.org/repos/asf/maven/wagon/tags/wagon-1.0-beta-6 wagon cd wagon/ mvn install </code>
Then I need to checkout the Wagon-S3 project:
<code> svn checkout http://svn.apache.org/repos/asf/maven/sandbox/trunk/wagon/wagon-s3 wagon-s3 </code>
Make sure the parent POM of the Wagon-S3 project points to the same versjon as the main Wagon project you just build:
<code lang="XML">
<parent>
<artifactId>wagon-providers</artifactId>
<groupId>org.apache.maven.wagon</groupId>
<version><strong>1.0-beta-6</strong></version>
</parent>
</code>
Now you should have a brand new build of Wagon S3 safely in your local Maven repository. Now it’s time to make it possible to deploy your project artifact to S3 using this extension.
Maven Dependency Management
The maven-deploy-plugin makes use of the information provided in your POMs dependencyManagement section:
<code lang="XML">
<distributionManagement>
<repository>
<id>s3-release</id>
<name>jForce S3 Release</name>
<url><b>s3rest://release.jforce.no/</b></url>
</repository>
<snapshotRepository>
<id>s3-snapshot</id>
<name>jForce S3 Snapshot</name>
<url><b>s3rest://snapshot.jforce.no/</b></url>
</snapshotRepository>
</distributionManagement>
</code>
Trying to deploy your project now will result in nothing but a stacktrace on your console. The next step is to create an Amazon S3 account.
Amazon S3
Howto create an S3 account is covered by several blog articles and tutorials on the Internet, thus I’ll skip this part here. In my case I installed the Firefox S3 plugin S3Fox. Using this tool I created two buckets, namely snapshot.jforce.no and release.jforce.no and edited their ACL to allow read access to everyone.
Uploading artifacts to the buckets requires authorization by Maven. This is done by adding the following snippet to your settings.xml file:
<code lang="XML">
<servers>
<server>
<id><strong>s3-release</strong></id>
<privateKey>[YOUR AWS ACCESS KEY]</privateKey>
<passphrase>[YOUR AWS SECRET KEY]</passphrase>
</server>
<server>
<id><strong>s3-snapshot</strong></id>
<privateKey>[YOUR AWS ACCESS KEY]</privateKey>
<passphrase>[YOUR AWS SECRET KEY]</passphrase>
</server>
</servers>
</code>
Notice how the server id is equal to the repository id in the distribution-management section. Your Amazon Web Services (AWS) credentials are available from the Your Account menu on the Amazon S3 webpage.
Deploy
Now you are ready to deploy your artifacts. This is done using the following Maven command:
<code> mvn clean deploy </code>
When done you can browse your bucket with S3Fox and find all your project artifacts in the familiar Maven directory structure.
Downloading from S3
Now that deployment to S3 has been taken care of, other projects that have dependencies to your projects can simply add the following to their POM (or to their repository manager):
<code lang="XML">
<repositories>
<repository>
<id>s3-snapshot</id>
<name>jForce S3 Snapshot</name>
<url><strong>http://snapshot.jforce.no.s3.amazonaws.com/</strong></url>
</repository>
<repository>
<id>s3-release</id>
<name>jForce S3 Release</name>
<url><strong>http://release.jforce.no.s3.amazonaws.com/</strong></url>
</repository>
</repositories>
</code>
This should complete the circle. Now you are able to deploy to S3 and your friends and collegues can resolve/download dependencies from S3. Enjoy!
