Accessing the FieldTrip source code through SVN

The FieldTrip code base is managed using the Subversion source code version control system, also known as SVN. Below you will find details on how to access the source code repository. The documentation below only applies to developers, regular users should use the copy on /home/common/matlab/fieldtrip (inside the DCCN) or the copy on the ftp server (outside the DCCN, see download).

A read-only version of the SVN repository is available on http://code.google.com/p/fieldtrip. The Google code version of FieldTrip is synchronized frequently and automatically with the in-house development version. You can use it to keep up to date with the latest updates of the source code, to revert to older versions, and to track all individual changes to the files. The link above allows you to browse through the code with your webbrowser, your SVN client (e.g. TortoiseSVN) should be pointing to the actual repository at http://fieldtrip.googlecode.com/svn/trunk.

To keep an eye on the changes to the code, you can follow FieldTrip on Twitter: all SVN changes appear on http://twitter.com/fieldtriptoolbx.

Getting a SVN client on your desktop or workstation

On Linux and Mac OSX machines the SVN command line utility is usually installed by default. The recommended SVN client for Windows is http://tortoisesvn.tigris.org. Please scroll down for detailed information on how to use SVN on Windows.

A word of caution: it is not recommended to use both Linux and Windows clients to access the same working copy of an SVN repository. This can lead to unpredictable results, or, in the worst case, a corrupt source base. Just pick either Linux or Windows SVN, and stick with it. Or, if you want, use two separate working copies, one for Linux and one for Windows.

Logging in on the SVN server

The source code is managed using SVN/Subversion and the repository is hosted on svn.fcdonders.nl. The following information is aimed at those people that have been granted write access to the SVN repository.

The server that is hosting the FieldTrip source code repository is only accessible through ssh. That means that you can only access it if you have an ssh account. The default account names and passwords for people working at the DCCN are the same as for the SSH tunnel server. External contributors can apply for an account by sending Robert an email.

To test the access, you should try the following

ssh yourname@svn.fcdonders.nl

After logging in on the svn server using ssh, you can also change your password with the “passwd” command. Once you have changed your password, please log out again.

To prevent that you have to type your password every time you do “svn update” or “svn commit”, you can exchange your ssh keys between the mentats and the svn server. That is explained in more detail on the DCCN intranet.

You should in general not use the “svn” command on the svn server directly. The only reason to log in on the svn server is to test the connection and to change the password. Please do not checkout the source code in your homedir on the svn server, as space is limited.

Getting started with SVN

Checking out a local copy -> your working version

To check out your new working copy of fieldtrip, you can type

svn checkout svn+ssh://yourname@svn.fcdonders.nl/home/svnroot/fieldtrip/trunk

or if you are using the read-only version from google

svn checkout http://fieldtrip.googlecode.com/svn/trunk

This gives you the trunk, i.e. the main development and release branch. It being called the “trunk” is the SVN convention and not my choice. For your convenience, I strongly recommend to do

mv trunk fieldtrip

or

mv trunk fieldtrip-dev

to rename the directory to something more usefull. There is no harm in that. If you go into that directory, you'll see that it contains the complete fieldtrip code base (at leat the parts that were part of the normal release version). This is where we should continue working on.

Commit your changes to existing code to the repository

If you have modified a function, you should send the modifications to the repository like this

svn commit existingfunction.m

The svn command will then ask you to write a log message. Please see the code guidelines for more information on how to write a log message. The default editor for writing the log message is vi, which is a very powerful, but also slightly user-unfriendly editor. Once in vi, you should press the “i” button, type the text you want, press escape, type ”:wq” and the log message will be recorded. See http://www.linuxjournal.com/article/6542 for a “getting started with vi” document.

If you haven't set a standard text editor yet, you may receive an error when first trying to commit code with svn. The editor is used to specify the comments that are logged. There are different ways to solve this, see http://svnbook.redbean.com/en/1.5/svn.advanced.externaleditors.html. What you can do is to add a line 'editor-cmd = vi' under the heading [helpers] in the ~/.subversion/config file. Another option would be to set an environment variable in your .bashrc script such that it will be created every time you login. Yet another option is to use

svn commit existingfunction.m -m 'my log message'

in order to specify a log message while committing.

In general it is wise to carefully check the changes prior to committing. You can check the difference between your present working version and the version in the repository like this:

svn diff existingfunction.m

So in general the safe way of committing your changes to the function is by typing

svn diff existingfunction.m

svn commit existingfunction.m

Add a new file to the repository

If you create a new function, you have to add it to the repository like this:

svn add    newfunction.m
svn commit newfunction.m

SVN on Windows

TortoiseSVN (http://tortoisesvn.tigris.org) is a fairly comfortable SVN client for the Windows platform. It integrates into the Windows Explorer and shell context menus and provides GUI controls for most (if not all) SVN tasks. Since the FieldTrip repository needs to be accessed via SSH, it is suggested to also install Putty and use a private/public key pair to avoid typing passwords repeatedly.

Create environment variable SVN_SSH

In order to make SSH work smoothly with SVN, you have to create an environment variable named SVN_SSH on your computer (see picture below). Depending on your Windows version, go to “System Properties” by right clicking on “Computer” and click on “Properties”. Then look for the “Advanced”-Tab (Windows XP and below) or “Advanced system settings” (Windows Vista and Windows 7). Click on the button “Environment Variables…” and create the SVN_SSH user variable. It should point to either “C:/Program Files/TortoiseSVN/bin/TortoisePlink.exe” or “C:/Program Files/Putty/bin/plink.exe” (without ”). Be sure to use use the forward slash / instead of a backslash \.

Using PuTTY / Pageant with TortoiseSVN

Using public/private key authentication is completely optional, but really convenient in the long term. With one exception, you will need to do the following steps only once.

If you don't already have it, get the PuTTY software package from http://www.putty.org/. After installation, start the application PuTTYgen to create an SSH key: Select SSH-2 RSA and 1024 bits, then press the Generate button and wiggle the mouse for a few seconds until your key is generated. You should now see a public key, a fingerprint, a key comment, and two input fields for a Key passphrase. This is something like a password that protects your key, and it is strongly recommended to use one, because effectively the passphrase will play a similar role to the password on the SSH server. Save your private key to some location that only you can access, and add your public key (the contents of the top-most textbox) to the authorized_keys file on the SVN server.

If you didn't understand the last part, these are the steps: With your PuTTYgen still open, start PuTTY and type svn.fcdonders.nl into the Host name field. Click Openand login with your username and password. Type

ls -la

and make sure that you have a directory called .ssh. If this does not exist, type

mkdir .ssh

You should also make sure that this directory can only be accessed by you, so if you just created it, please also type

chmod 700 .ssh

Next, change into the directory and edit the authorized_keys file using

cd .ssh
vi authorized_keys

Now comes a tricky part: You need to paste the public key from the PuTTYgen window into this file. First, mark the contents of that text field and press CTRL-C, then change back to the PuTTY window and press i for insert. Right-click on your mouse and the key should appear. Press Return, Esc, and then type :wq and again press Return. The text editor will now save the file and exit. You can check if things went right using

cat authorized_keys

which should print out your public key. Finally, please make sure that only you can read this file and close the SSH session:

chmod 600 authorized_keys
exit

That should be it.

Once you have the public key in place, you will need to start Pageant (from the PuTTY collection) on your local computer. You will get a new symbol (computer with a hat) in your system tray. Double-click this icon, select “Add Key”, point to your private key file, and finally enter the passphrase to activate the SSH key. From now on, you should be able to login without typing the password, which you might want to try using PuTTY itself.

Your passphrase and key are only stored until you log off from your Windows box, so you will need to start Pageant and re-enter your passphrase everytime you come back.

Initial checkout of your FieldTrip copy

You should create an empty directory (here we use D:\src\fieldtrip) and open this directory in an Explorer window. With TortoiseSVN installed, you should get a context menu like the following:

After selecting “SVN Checkout”, enter the following into the dialog window, but replace LOGIN by the username given to you:

Click OK to start downloading the files. If you don't use Pageant, you will need your SSH password.

Usual workflow

TortoiseSVN comes with in-depth documentation which you can browse here: http://tortoisesvn.net/docs/release/TortoiseSVN_en/tsvn-dug.html For convenience, we'll just cover the three most important use cases.

Updating your local copy

In order to merge changes from other people into your local copy, navigate to the directory you want to update, right-click and choose “SVN Update”.

In case you and other people have worked on the same parts of the same files, these “conflicts” will show up in the message window of TortoiseSVN, and also the icon of those files will look different.

Viewing changes

To compare your local version to what is currently in the central repository, right-click a file and select TortoiseSVN - Diff. A new window will display both versions of that file side-by-side and also highlight the differences.

Committing changes

Once you are sure about your modifications and want to merge your work back into the repository, right-click a file or a folder and select “SVN Commit”. If you get an error message, you probably first need to update your local copy and possibly resolve conflicts.

How to maintain two copies of the same file

Sometimes it is needed to have one MATLAB function at two locations, especially it it is a private function (i.e. located in a private directory) that is used by multiple modules. FieldTrip uses an “svn hook” to ensure that changes to the function at one location propagate to all other locations.

Adding a function to two locations is done by

svn add private/mynewfunction.m
svn commit -m "Added my new function to fieldtrip/private"
svn copy private/mynewfunction.m module/private/mynewfunction.m
svn commit -m "Also added my new function to module/private"

Subsequently you should set the (FieldTrip specific) svn property “autosync” to “true” on both occurences of the files.

svn propset autosync true private/mynewfunction.m
svn propset autosync true module/private/mynewfunction.m
svn commit -m "set autosync=true on both copies of my new function"

If you now make a change to one of the copies and “svn commit”, it will automatically updated to all other copies of the function. After the svn commit you should do “svn update” to get all your local copies updated.

One can easily check the functions that have autosync set to true:

svn propset autosync private/*.m 

Once the property “autosync” is set to “true” on all occurrences of a file, you can svn commit the changes to one of the occurrences just like you would do for a normal file. Invisible to you the svn server will automatically copy your modified file over all other occurrences and commit once more. After you have done the commit (and the svn server has taken care of the replication), you should do an svn update to get the other occurrences in your FieldTrip working copy updated.

Keeping the SVN repository tidy

We try to keep a consistent representation of the files in the repository. Below are a few pieces of linux command-line code to clean it up.

vi ~/.subversion/config
 
[miscellany]
enable-auto-props = yes

[auto-props]
*.sh        = svn:eol-style=native;svn:executable='*';svn:keywords=Id Rev
*.exe       = svn:executable='*'
*.dll       = svn:executable='*'
*.mexa64    = svn:executable='*'
*.mexglx    = svn:executable='*'
*.mexmac    = svn:executable='*'
*.mexmaci   = svn:executable='*'
*.mexmaci64 = svn:executable='*'
*.mexw32    = svn:executable='*'
*.mexw64    = svn:executable='*'
*.tex       = svn:eol-style=native;svn:keywords=Id Rev
*.txt       = svn:eol-style=native;svn:keywords=Id Rev
*.xml       = svn:eol-style=native;svn:keywords=Id Rev
Makefile    = svn:eol-style=native;svn:keywords=Id Rev
*.c         = svn:eol-style=native;svn:keywords=Id Rev
*.cpp       = svn:eol-style=native;svn:keywords=Id Rev
*.h         = svn:eol-style=native;svn:keywords=Id Rev
*.m         = svn:eol-style=native;svn:keywords=Id Rev
*.man       = svn:eol-style=native;svn:keywords=Id Rev

Remove all file-externals from your working version

curdir=`pwd`
for dir in `find . -type d ! -path '*/.svn*'` ; do
cd $dir
svn propget svn:externals | tr -s ' '  | cut -f 2 -d ' ' | awk  '/./ {print "rm " $1}' | sh 
cd $curdir
done

Remove the executable flag from the *.m files and fix some other settings

for file in `find . -name \*.m -or -name \*.c -or -name \*.h -or -name \*.txt -or -name README -or -name COPYING` ; do 
svn propdel svn:executable $file
svn propset svn:eol-style native $file
svn propset svn:keywords 'Rev Id' $file
done

for file in `find . -name \*.exe -or -name \*.glnx86 -or -name \*.glnxa64 -or -name \*.mac -or -name \*.maci -or -name \*.maci64` ; do 
svn propset svn:executable '*' $file
svn propdel svn:eol-style  $file
svn propdel svn:keywords   $file
done

for file in `find . -name \*.dll -or -name \*.mexw64 -or -name \*.mexw32 -or -name \*.mexglx -or -name \*.mexa64 -or -name \*.mexmac -or -name \*.mexmaci -or -name \*.mexmaci64` ; do 
svn propset svn:executable '*' $file
svn propdel svn:eol-style  $file
svn propdel svn:keywords   $file
done

Desired is that all mex-files, for all platforms, in all subfolders, have the svn:executable property set to '*' (the only allowed value). This will solve the 'Access is denied' issue with the mex-files on Windows, when accessing them through a Samba server.

How to find files or directories, excluding the .svn ones

 find . -type f ! -path '*/.svn*'
 find . -type d ! -path '*/.svn*'

How to work with an svn branch

Before you consider making an svn branch, please ask Robert or another svn expert whether a branch is really the solution to your problem.

You should carefully read the SVN documentation on branching and merging before you start using this.

You start by creating a copy of the trunk. It should be a completely clean copy and it is done on the svn server (the following should all be on a single line in a linux terminal):

svn copy svn+ssh://roboos@svn.fcdonders.nl/home/svnroot/fieldtrip/trunk svn+ssh://roboos@svn.fcdonders.nl/home/svnroot/fieldtrip/branches/username-purpose -m "Creating a branch of the fieldtrip trunk for the purpose of ..."

Now that you've created a branch of fieldtrip, you can check out a new working copy to start using it:

svn checkout svn+ssh://roboos@svn.fcdonders.nl/home/svnroot/fieldtrip/branches/username-purpose

You have to keep your branch in sync with all changes that happen in the trunk, e.g. by your coworkers. That is done by regularly using the svn merge command. Note that you need to commit the merge (of course after checking whether all went well). More information can be found here. First, you have to identify the range of revisions that you need to merge into your branch. This typically runs from the revision number at which the branch was created (or the revision number until which the last merge was executed), until the current revision number.

svn merge -r number1:number2 svn+ssh://roboos@svn.fcdonders.nl/home/svnroot/fieldtrip/trunk .

After merging the latest changes from the trunk into your branch, you have to check and test that they did not cause conflicts with your modifications. Subsequently, you need to commit these changes to your branch. It is good practice then to log which revisions have been merged:

svn commit -m "Merged changes from trunk to BRANCHNAME: svn merge -r number1:number2 svn+ssh://roboos@svn.fcdonders.nl/home/svnroot/fieldtrip/trunk ."

Once you are done with all changes in your branch, you can reintegrate the branch back into the trunk. You must first ensure that there are no pending changes in the trunk which you have not yet merged into your branch.

In your branch version

svn update
svn merge ^/calc/trunk
# test whether everything is fine in your branch

In your copy of the trunk

 svn merge --reintegrate ^/branches/username-purpose
 # test whether everything is fine in the trunk

svn commit -m "Merge the username-purpose back into trunk!"

More information

If you need more information on svn, this can be found in the faq http://subversion.tigris.org/faq.html and in the online documentation http://svnbook.red-bean.com/.

development/svn.txt · Last modified: 2014/04/17 21:30 by robert

You are here: startdevelopmentsvn
This DokuWiki features an Anymorphic Webdesign theme, customised by Eelke Spaak and Stephen Whitmarsh.
www.chimeric.de Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0