From Subversion to Mercurial. Part 3, Implementation Day and Beyond

If you followed the steps described in the first and second parts of this series, you should have a Mercurial (Hg) repository ready to replace your project’s Subversion (SVN) repository.  In this third and last part we’ll go over Implementation Day, with particular detail on how to implement this migration on the SourceForge infrastructure.

Test

Can’t test enough.  Your script produces an Hg repository that looks OK on superficial investigation with tools like hg log and hg view.  But does the code build?  Hugin’s build system had a couple of dependencies on SVN and they needed to be updated.  Thomas Modes and Kornel Benko stepped up to the task.  Can developers and builders use this repository?

Educate

On Implementation Day the project will transition from SVN to Hg.  While all relevant contributors are proficient in SVN, Hg was new territory for many.  While progressing on the conversion I kept the community informed and took every opportunity to encourage learning of the new tools, including public tutoring that continues after the transition.  You want to encourage people to share their experiences and learn from each other.  Conceptually the biggest difference between SVN and Hg is that with Hg the repository sits on your local client.  A check in to SVN is the equivalent of a check in and a push to Hg.  Offline operation is not possible with SVN but it is with Hg.  However both are revision control system (RCS) and very similar to use.

Implementation Day Overview

Warn everybody one last time.  Create a new repository on SourceForge for each migrating code line.  Lock down SVN by revoking write access to everybody but a few maintainers who will clean up after the transition.  Run once again the whole migration on a green field, from scratch, to be sure that everything to the very last SVN commit is included.  Test one last time this new local repository (compare it to previous results); create the new repository on SourceForge and push your local repository to it. Last but not least, configure the repository on SourceForge and announce the transition to the world.  Sounds easy.  The devil is in the detail.

Mercurial on SourceForge

SourceForge has been very generous with the projects it is hosting: we can have unlimited Hg repositories.  Unfortunately there are rough edges.

To activate Mercurial for your project:

• Login via the web as a project administrator and go to the “Develop” page for your project.
• Select the Project Admin menu, and click on “Feature Settings”.
• Select “Available Features”.
• Select the checkbox to the left of the “Mercurial” heading. Your repository will be instantly enabled.

This first repository will be fine but if you want to activate more than one repository you will have to manually set them to be group writable.  To activate additional repositories:

• Log on to SourceForge’s shell service (assuming you have set up your SSH key) with `ssh -t USER,PROJECTUNIXNAME@shell.sourceforge.net create`
• Navigate to your project’s Mercurial space with `cd /home/scm_hg/P/PR/PROJECTUNIXNAME`, e.g. for Hugin this would be `cd /home/scm_hg/h/hu/hugin`
• Create a new directory with the name you want for the repository.  E.g. for Hugin’s website this was `mkdir hugin-web`
• Execute `hg init DIRNAME` (where DIRNAME is the directory you just created, e.g. `hg init hugin-web`). This will initialize the new repository.
• Inside the new repository, edit the configuration file .hg/hgrc (see configuration section below)
• SourceForge rough edge: group write access must be given manually `chmod -R g+w /home/scm_hg/P/PR/PROJECTUNIXNAME/DIRNAME`

Configuration of the Mercurial Repository on SourceForge

SVN support on SourceForge is mature and projects are used to amenities such as email commit notifications.  Hg support is better than what the scant documentation suggests.  Most standard functionality, including email notification, works, even if it is officially unsupported.  One only has to find out how to configure it.  I played around with some trial and error already when optimizing the Enblend repository last year.  This is the hgrc file template that works for us:

[hooks]
changegroup.notify = python:hgext.notify.hook

[email]
from = NOTIFICATION_ADDRESS@lists.sourceforge.net

[smtp]
host = localhost

[web]
baseurl = http://PROJECT.hg.sourceforge.net/hgweb/PROJECT/DIRNAME

[notify]
sources = serve push pull bundle
test = False
config =
template = \ndetails:   {baseurl}{webroot}/rev/{node|short}\nchangeset: {rev}:{node|short}\nuser:      {author}\ndate:       {date|date}\ndescription:\n{desc}\n
maxdiff = -1

[usersubs]
NOTIFICATION_ADDRESS@lists.sourceforge.net = **[trusted]

users = *

You’ll have to replace your own project unix name PROJECT; your own Hg repository top directory DIRNAME; and your own NOTIFICATION_ADDRESS mailing list.  The configuration options are documented.

Committer Write Access

With a dRCS like Mercurial write access has a completely different meaning.  Everybody can `hg clone` an existing repository and once cloned has full write access and can publish their own repository.  The d in dRCS stands for distributed.  Technically there are no more hierarchies and no more central control.  All clones are equal.  Whoever owns a clone can decide to publish it on the web, e.g. with `hg serve`, and give write access to whomever they want.  Granting SourceForge write access only means that the committer can push to the repository hosted on SourceForge.  What makes a repository authoritative is user’s trust, and this is given implicitly by pulling from it.

SourceForge Rough Edges, Again

I wish there was a way to group-manage access rights on SourceForge.  I have not found it.  I needed to revoke SVN access to most developers, and grant them Hg access.  I had to click through each and every contributor registered with the project and single handedly managed their access rights.  To make things worse the pseudo-ajax web interface of SourceForge is everything but asynchronous: it reload the page after each change.  Ajax-cosmetics with underlying old technology from the last century.

One point projects on SourceForge will need to pay attention to are default access rights.  I did not find a place to change those, so every new project member gets by default SVN access right, unless you explicitly remove them.  It seems to me that the defaults on SourceForge are based on the principle of random uncoordinated historical growth.  Have they ever heard of the generally accepted principle of least privilege?  And the default file access for newly created extra Hg repositories is less than reasonable least privilege (see above).

<rant>And don’t tell me about SourceForge’s IdeaTorrent and ways to request and enhancement.  In my experience it does not work and some things on that site have been broken for years when the fix is simple, easy, and does not take much time.  Have you tried to use a SourceForge mailing list archive?</rant>

Push

Now that everything is set, you can simply `hg push` from your local repository to the SourceForge one.  Or if you’re really confident, you can rsync the .hg directory (but don’t forget to edit the .hg/hgrc configuration file on the SourceForge end).

CMake Build System

Our CMake build system depended on SVN and after the push it was broken.  Kornel Benko and Thomas Modes fixed it.  Bruno Postle added a break in the CMake build system in the SVN repository, to warn users of that repository that newer versions are in Hg.  Harry van der Wolf updated the OSX build system.

Conclusion

The disruption was short.  A few hours after going live, developers started committing again, using Hg.  Builders started building and distributing again, using Hg.  The Google Summer of Code students cloned away their own copies of the source code and started working on the next major developments for Hugin.  After taking on the most complex of the code lines in the SVN first,  I migrated the remaining ones over a few hours Sunday night.  Hugin and most of its related projects live now happily in Hg and can easily be converted to other formats, including Bazaar, git, and even SVN.  Initially I thought to mirror the default code branch from Hg to SVN, but our project does not really need that.  Subversion has been made completely redundant by a newer, better, superior tool. Mercurial and its likes would not exist without Subversion, and should be seen as a continuum in the lineage rather than a break from the past.  With Mercurial, Hugin is freer than ever, and you are free to take it further on a journey to the future.  For now Hugin still lives on SourceForge, where the next critical bit of infrastructure is the bug tracker.  But with Mercurial the dependency on SourceForge; and the dependency on any single central service or person; has been further reduced.  Long and Free live Hugin.