jms1.net - Setting up Subversion and Trac

Subversion is a source code revision control system, similar to RCS or CVS. Their web site has a full list of reasons why Subversion (or "svn") is better than cvs. The idea is that you have a self-organizing "repository" of code... whenever you create a file, you "add" it to the repository, and when you update it, you update the repo. If you make a mistake and need to "back out" a change, the previous versions of each file are stored in the repo as well. The repo can be used by multiple people, and as long as nobody is working on the same exact file at the same time, everything stays organized correctly by itself.

There is definitely more to be said about the benefits of using a source conotrol system. I've been aware of the idea for many years, but I was never a fan- I always saw it as excess overhead, one more thing to keep track of. However, now that I've worked on a project with a client who was already using it, and seen the benefits first-hand, I find myself wishing I had started using it a lot sooner.

Trac is an open-source web-based project management tool. It provides a wiki, a web interface to a Subversion repo (which allows you to not only look at any given version of a file, but to see a colour-coded "diff" of any two versions), and a bug reporting and ticket tracking system, all in one package. This client was also using Trac, and the combination of Subversion and Trac made it possible for a team of eight people, in different parts of the world, to all know what each other were working on, and to share their work with each other.

Background

I have very little experience in "team programming". Normally I work on projects by myself, and what little "source control" I've done in the past has always consisted of copying the latest working version to another disk, as a backup.

Earlier this year I suffered a hard drive crash, and lost what turned out to be the only copy of the source code for a project I had written for a client last year. The client was running a Subversion repository for their own code, but I had never asked about adding my source to it- and when I was done, both of us neglected to get a copy of my source to them. (Live and learn.) When the client called a few months ago and wanted some updates, I had to go back and re-create as much as I could using my own memory and the output from what I wrote, and managed to get it working (and improved) but wasted a lot of time in the process. This time the client created a directory within their Subversion repo for me to use, and after ten minutes of reading the documentation about the svn client, I started keeping each version of my work in their repo.

After using svn with that one project, I found myself wishing I had an svn server of my own. A few weeks ago, I installed the subversion package on my server (running CentOS 5) and discovered that it comes with the executables to set up a repository. However, a few hours' worth of reading through the documentation left me a little bit confused- there are several different ways to set up a Subversion server, and the docs cover every one of them, in excruciating detail. I finally managed to get a subversion repo running under daemontools, but I wasn't really happy with it, since it involved sending everything across the net in plain text, and it couldn't easily be tied into a Trac system (which to me is one of the coolest things about Subversion.)

A few days ago, I decided to have another go at it, and found the book Version Control with Subversion. The book itself is published by O'Reilly Media, and is available under a license which allows it to be legally downloaded for free- how cool is that? (Cool enough that even though I've downloaded it, I've also purchased a printed copy. Just another reason to like O'Reilly.)

This time, after reading through this book (which for some reason seems to be easier to understand, even though it says the same things which are in the documentation which came with the software) I got a repository running under Apache, which means it uses SSL. It also makes it possible to run Trac with the repo (both the subversion server and the trac software need write access to the repo files, and with both running under Apache, the files can be owned by the "apache" user, and everybody is happy.)

I kept notes as I went along. The information below is a cleaned-up version of those notes. It's mostly here for my own use, but if others are able to use it, so much better.

These instructions are for running both Subversion and Trac under Apache. If you don't plan to use Trac and just want a subversion repository, and you don't need SSL, you can run subversion under daemontools (djb's service control framework, as commonly used with qmail and djbdns) as well.

These instructions assume that you already have Apache installed, and if you plan to use SSL, that you have mod_ssl or something similar installed as well.

Installing the software

Obviously, we need the "subversion" and "trac" packages. However, there are a few pre-requisites for each one.

A Subversion repository running under Apache actually uses the WebDAV protocol, with some subversion-specific extensions. This means you will need to install or enable the "mod_dav" and "mod_dav_svn" modules within Apache.

The trac program itself is written in Python, and uses a Python library called clearsilver, which means you will need the "python" and "python-clearsilver" packages installed. In addition, trac will run MUCH more quickly if you use "mod_python" to embed a Python interpreter into Apache. This is not strictly required, it is possible to run trac as a CGI script, but it runs very slowly. Trust me on this, you WANT to use mod_python.

My own server is running CentOS 5. The normal CentOS repositories contain a subversion package, however it's slightly out of date. The RPMforge repository has a current subversion package, as well as a trac package (which the CentOS repositories don't have.)

If you're using CentOS, or some other RedHat-flavoured version of Linux, start by adding the rpmforge repository to your system, and then you can install all of the necessary packages, with their pre-requisites, using a single command like this:

If you are running some other OS, there should be a way to use your system's package-management commands to install the packages. And of course you can always compile everything from source.

Setting up the first subversion repo

The directions below are written using the actual directory names I use on my own server. I use "/www/sitename" as the top directory for each web site, with "/www/sitename/docs" as the DocumentRoot, and "/www/sitename/error.log" as the error log file for each web site. This allows my clients to see any error messages pertaining to their own site, plus it gives me a place to set up directories for things which are related to web sites, but not part of the actual content- like a web-based subversion repository.

Whatever directory you choose, it cannot be within the DocumentRoot of the site which will host it. This is mentioned in the Subversion FAQ but sometimes people miss it. You don't want the repo to be directly visible- the file structures within the repo are not directly usable without using a client like svn or Trac.

This example creates the "/www/secure.jms1.net/svn" directory, which serves as a top-level container for the actual subversion repositories on the "secure.jms1.net" web site. (The rest of this page uses this directory name as well.)

After creating the top-level container, the next step is to create the first repository itself. For this example, the name of the repository will be "dummy". After creating it, we will change its ownership so the apache anonymous user owns it (and therefore has full access to it.)

# cd /www/secure.jms1.net/svn
# svnadmin create dummy
# chown -R apache:apache dummy

The next step is to set up a file containing the userids and hashed passwords for the repository. This file needs to be readable by the apache user, but should not be writeable, because any other CGI script running on the system will be able to read the file and we don't want them to be able to accidentally (or purposefully) change it.

Technically, the file can be anywhere and have any name. On my server the file is in the "/www/secure.jms1.net/svn" directory (since it's part of the subversion "system") and has the name ".htusers. If you have multiple repositories, you could set up different password files for each one- however it's easier to have one file which is used by all of the repositories.

# cd /www/secure.jms1.net/svn
# htpasswd -cm .htusers admin
New password: You will not see the password as you type it
Re-type new password:
Adding password for user admin
# chown root:apache .htusers
# chmod 640 .htusers

The file now exists and has the proper permissions. To add more users to an existing file, do not use the htpasswd command's "-c" option.

# htpasswd -m .htusers jms1
New password:
Re-type new password:
Adding password for user jms1

Next we need to create a text file which defines what access each user has to each repository. Each user can be given "read/write", "read-only", or "no" access to a repository, or if necessary, to a specific directory within a repository. The book shows exactly how to do this; this example shows a very simple version.

Again, the file can be anywhere and have any name; I keep it in the same directory, with the name ".authz". (I'm using filenames which begin with "." because I don't know what kind of repository names I'll need in the future.) The same caveats apply to this file's ownership and permissions as were on the .htusers file.

# cd /www/secure.jms1.net/svn
# nano .authz Obviously you can use whatever text editor you like. The file's contents are:

[groups]
admin = admin,jms1

[/]
@admin = rw
* =

[dummy:/]
@admin = rw
* = r

# chown root:apache .authz
# chmod 640 .authz

After setting up the access control files, we need to tell Apache about the repositories. This is done with directives in the httpd.conf file (or in other files which are Include'd from the main httpd.conf file.)

This means that any /etc/httpd/conf.d/*.conf files will be read at that point, and treated as if their contents had been contained within the main file.

The mod_dav_svn package installed a /etc/httpd/conf.d/subversion.conf file, which loads the two apache modules needed in order to "speak" the subversion-specific extended version of WebDAV, and to understand the authorization file we created above. If your system didn't include a separate file like this, you may need to add (or un-comment) these two lines:

LoadModule dav_svn_module modules/mod_dav_svn.so LoadModule authz_svn_module modules/mod_authz_svn.so

The other thing which needs to be added is a "<Location>" block which defines the container for the svn repositories. On my server, this block looks like:

<Location "/svn"> Dav svn SVNParentPath /www/secure.jms1.net/svn AuthzSVNAccessFile /www/secure.jms1.net/svn/.authz Satisfy Any Require valid-user AuthType Basic AuthName "secure.jms1.net subversion repository" AuthUserFile /www/secure.jms1.net/svn/.htusers </Location>

You can then test the repository. This can be done from any machine which has the subversion client "svn" installed, and which is able to reach the web server (so if you set it up on a machine which is behind a firewall and didn't allow the appropriate port through, you probably won't be able to reach it.)

On a different machine...
$ svn ls https://secure.jms1.net/svn/dummy
If you are using SSL and have a self-signed certificate, you may see a warning at this point. If you know how to verify the fingerprint, do so, and then decide if you want to accept the certificate permanently or not.

If you configured the dummy repository to give read-write or read-only access to anonymous users, you probably won't see anything else. Otherwise you should be prompted for a userid and password:
Authentication realm: <https://secure.jms1.net:443> secure.jms1.net subversion repository
Password for 'testuser':
Authentication realm: <https://secure.jms1.net:443> secure.jms1.net subversion repository
Username: jms1
Password for 'jms1':

As long as you don't see any error messages involving "access denied" or "timeout", the repository is working and you can start creating directories and committing changes.

Setting up additional subversion repos

If you wish to create additional repositories (for different clients, different projects, or however you want to keep things organized) you only need to repeat the following steps. Apache itself should not need to be re-configured; the block above will make it treat ALL directories below the /www/secure.jms1.net/svn directory as subversion repositories.

# cd /www/secure.jms1.net/svn
# svnadmin create name
# chown -R apache:apache name

If you are creating a new userid for this repository...
# htpasswd -m .htusers username

To set the access rules for this repository...
# nano .authz

Note that mod_dav_svn also provides a VERY simple interface to the repositories. The same URL you would use to access the repo using the svn command-line client, can be entered into a browser, and you will be shown the most recent revision of the code in the repo. This interface has no way to show you older versions, let alone show you what changed from one version to the next.

Setting up the first trac

Just as subversion has individual repositories, trac has individual "environments", each of which corresponds to a specific subversion repo (but which doesn't have to be connected to a repo- trac is usable as a general wiki and ticket tracking system without tying it into a subversion repository.) Just as the first step for installing subversion was to create a directory to contain all of the repositories, the same is true of trac- we need to create a directory to contain all of the trac environments. And since trac is "part of" a web site, and the files need to be writeable to the apache anonymous user, we will create it "next to" the subversion and docs directories.

# mkdir -m 710 /www/secure.jms1.net/trac
# chown root:apache /www/secure.jms1.net/trac

Setting up a trac environment is similar to setting up a subversion repo. We need to run a command to set up the files, set the ownership of those files, set the permissions for the users, and configure Apache to use the trac software to serve the pages.

For the sake of consistency (and to avoid confusing your users) we will be configuring trac to use the same user/password file that subversion uses.

Creating a trac environment for our "dummy" project (from the subversion examples above) looks like this:

# cd /www/secure.jms1.net/trac
# trac-admin dummy initenv
The trac-admin program generates a lot of output to explain each of the questions. I am skipping the output and just showing the questions and how I'm answering them.
Project Name [My Project]> Dummy Project
...
Database connection string [sqlite:db/trac.db]>
...
Repository type [svn]>
...
Path to repository [/path/to/repos]> /www/secure.jms1.net/svn/dummy
...
Templates directory [/usr/share/trac/templates]>
...
# chown -R apache:apache dummy

Trac will use the userids and passwords from the .htusers file in the subversion directory (unless we point it to a different file) but it can't use the .authz file because trac supports a lot more options for each user. These permissions are stored within the trac environment, and administered using the trac-admin command.

The default permissions given to a newly created trac environment will allow anybody (i.e. "anonymous", or non-authenticated users) to view the entire contents of the trac environment (wiki, code repository, tickets) as well as enter and change trouble tickets. In order to be able to use the system, you will probably want to give more permissions to one or more users, or adjust the privileges available to anonymous users.

This command will give full administrator access (or "TRAC_ADMIN" rights) for the "dummy" trac environment, to the userid "jms1".)

# cd /www/secure.jms1.net/trac
# trac-admin dummy permission add jms1 TRAC_ADMIN

This command will set the permissions so that non-authenticated users (or "anonymous" users) have NO permissions to look at the content of the trac environment at all.

# cd /www/secure.jms1.net/trac
# trac-admin dummy permission remove anonymous '*'

If you want to make the trac environment fully read-only, so that guests can look but not touch, you may want to use a command like this:

# cd /www/secure.jms1.net/trac
# trac-admin dummy permission remove anonymous TICKET_CREATE TICKET_MODIFY WIKI_CREATE WIKI_MODIFY

Once the permissions are set, add this block to your httpd.conf file in order to tell Apache to use the trac software to handle any URLs which start with "/trac/___" on the web site:

<LocationMatch "/trac/.+/"> SetHandler mod_python PythonInterpreter main_interpreter PythonHandler trac.web.modpython_frontend PythonOption TracEnvParentDir /www/secure.jms1.net/trac PythonOption TracUriRoot /trac </LocationMatch> <LocationMatch "/trac/[^/]+/login"> AuthType Basic AuthName "Dummy Project trac" AuthUserFile /www/secure.jms1.net/svn/.htusers Require valid-user </LocationMatch>

After adding this block, restart Apache and you should be able to visit your new trac environment using a URL like "https://secure.jms1.net/trac/dummy/".

Note that the URL for the parent, in this case "https://secure.jms1.net/trac/", will still function as a normal directory. If you create a directory with this name under your normal DocumentRoot area, whatever index that directory would generate (by virtue of an "index.html" file, for example) will work as if the trac system were not installed. This allows you to set up your own index of trac environments, which doesn't necessarily have to be complete- if you have trac environments for projects you don't want people snooping into, simply not listing them in this index.html will prevent people from finding them. However, if they find the name through some other means, they will still be able to access them- so make sure to set the trac permissions, using the "trac-admin ____ permission" command.

And if you're curious, yes, https://secure.jms1.net/trac/ is my real trac parent directory, set up with a custom index file, exactly as described on this web page. And there IS a "dummy" project there, which is read-only to the entire world.

Creating additional trac environments

Just as you can create addition subversion repositories without having to touch Apache, you can also create additional trac environments without touching Apache. For example...

# cd /www/secure.jms1.net/trac
# trac-admin name initenv
...
Project Name [My Project]> Project Name
...
Database connection string [sqlite:db/trac.db]>
...
Repository type [svn]>
...
Path to repository [/path/to/repos]> /www/secure.jms1.net/svn/name
...
Templates directory [/usr/share/trac/templates]>
...
# chown -R apache:apache name
# trac-admin name permission add jms1 TRAC_ADMIN
# trac-admin name permission remove anonymous '*'