Gitolite is a git repository manager. Here's how to add git-annex support to gitolite, so you can git annex copy files to a gitolite repository, and git annex get files from it.

A nice feature of using gitolite with git-annex is that users can be given read-only access to a repository, and this allows them to git annex get file contents, but not change anything.

First, you need new enough versions:

  • gitolite 2.2 is needed -- this version contains a git-annex-shell ADC and supports "ua" ADCs.
  • git-annex 3.20111016 or newer needs to be installed on the gitolite server. Don't install an older version, it wouldn't be secure!

And here's how to set it up. The examples are for gitolite as installed on Debian with apt-get, but the changes described can be made to any gitolite installation, just with different paths.

Set $GL_ADC_PATH in .gitolite.rc, if you have not already done so.

echo '$GL_ADC_PATH = "/usr/local/lib/gitolite/adc/;"' >>~gitolite/.gitolite.rc

Make the ADC directory, and a "ua" subdirectory.

   
mkdir -p /usr/local/lib/gitolite/adc/ua

Install the git-annex-shell ADC into the "ua" subdirectory and make it executable.

   
cd /usr/local/lib/gitolite/adc/ua/
wget https://raw.github.com/sitaramc/gitolite/pu/contrib/adc/git-annex-shell
chmod +x git-annex-shell

Now all gitolite repositories can be used with git-annex just as any ssh remote normally would be used. For example:

# git clone gitolite@localhost:testing
Cloning into testing...
Receiving objects: 100% (18/18), done.
# cd testing
# git annex init
init  ok
# cp /etc/passwd my-cool-big-file
# git annex add my-cool-big-file
add my-cool-big-file ok
(Recording state in git...)
# git commit -m added
[master d36c8b4] added
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 120000 my-cool-big-file
# git push --all
Counting objects: 17, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (12/12), done.
Writing objects: 100% (14/14), 1.39 KiB, done.
Total 14 (delta 0), reused 1 (delta 0)
To gitolite@localhost:testing
   c552a38..db4653e  git-annex -> git-annex
   29cd204..d36c8b4  master -> master
# git annex copy --to origin
copy my-cool-big-file (checking origin...) (to origin...) 
WORM-s2502-m1318875140--my-cool-big-file
        2502 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/1)

sent 2606 bytes  received 31 bytes  1758.00 bytes/sec
total size is 2502  speedup is 0.95
ok

Troubleshooting

I got an error like this when setting up gitolite after setting up a local git repo and git annex:

git-annex-shell: First run: git-annex init
Command ssh ["git@git.example.com","git-annex-shell 'configlist' '/~/myrepo.git'"] failed; exit code 1

because I forgot to "git push --all" after adding the new gitolite remote.

Looks like you are missing a closing double quote on the line:

echo '$GL_ADC_PATH = "/usr/local/lib/gitolite/adc/;' >>~gitolite/.gitolite.rc

right after /;

I got this working by the way - great stuff.

Comment by http://www.openid.albertlash.com/openid/ Sat Dec 24 06:08:45 2011
I've fixed the typo (anyone can edit pages in this wiki FWIW.)
Comment by http://joey.kitenet.net/ Sat Dec 24 16:54:31 2011
I'm confused by the fact that the git-annex-shell adc rejects any repo names that don't start with /~/ since none of my repos start that way. It seems work ok if I just delete /\~ from the front of the regex, but I feel like I must be missing something.
Comment by bremner Fri Dec 30 21:41:13 2011

Well a repo url like gitolite@localhost:testing puts it in the gitolite user's /~/testing

This worked when I added the gitolite stuff, anyway.. Let's see if it still does:

joey@gnu:~/tmp>mkdir g
joey@gnu:~/tmp>cd g
joey@gnu:~/tmp/g>git init
Initialized empty Git repository in /home/joey/tmp/g/.git/
joey@gnu:~/tmp/g>git annex init
init  ok
joey@gnu:~/tmp/g>git remote add test 'gitolite@localhost:testing'
joey@gnu:~/tmp/g>touch foo
joey@gnu:~/tmp/g>git annex add foo
add foo (checksum...) ok
(Recording state in git...)
joey@gnu:~/tmp/g>git annex copy foo --to test --debug
git ["--git-dir=/home/joey/tmp/g/.git","--work-tree=/home/joey/tmp/g","ls-files","--cached","-z","--","foo"]
git ["--git-dir=/home/joey/tmp/g/.git","--work-tree=/home/joey/tmp/g","check-attr","annex.numcopies","-z","--stdin"]
git ["--git-dir=/home/joey/tmp/g/.git","--work-tree=/home/joey/tmp/g","show-ref","--hash","refs/heads/git-annex"]
git ["--git-dir=/home/joey/tmp/g/.git","--work-tree=/home/joey/tmp/g","show-ref","git-annex"]
git ["--git-dir=/home/joey/tmp/g/.git","--work-tree=/home/joey/tmp/g","cat-file","--batch"]
Running: ssh ["-4","gitolite@localhost","git-annex-shell 'configlist' '/~/testing'"]

Still seems right, the ADC's regexp will match this the git-annex shell command.

Comment by http://joey.kitenet.net/ Sat Dec 31 00:29:45 2011

I guess there is some path rewriting going in in gitolite proper because if try a url of the form ssh://git@localhost/testing, then it still works with gitolite, but fails with the ADC because the repo is passed as /testing:

Running: ssh ["git@host","git-annex-shell 'configlist' '/recommend'"]
Running: ssh ["git@host","git-annex-shell 'configlist' '/recommend'"]

What I have to ask Sitaram and or find in the docs is if this is a bug or a feature in gitolite. I can see how the leading slash would get swallowed up by this line

$repo = "'$REPO_BASE/$repo.git'"

in gl-auth-command, but I guess that isn't the whole story.

Comment by bremner Sat Dec 31 01:50:49 2011

I confirmed with Sitaram that this is intentional, if probably under-documented. Since the ADC strips the leading /~/ in assigning $start anyway, I guess something like the following will work


diff --git a/contrib/adc/git-annex-shell b/contrib/adc/git-annex-shell
index 7f9f5b8..523dfed 100755
--- a/contrib/adc/git-annex-shell
+++ b/contrib/adc/git-annex-shell
@@ -28,7 +28,7 @@ my $cmd=$ENV{SSH_ORIGINAL_COMMAND};
 # the second parameter.
 # Further parameters are not validated here (see below).
 die "bad git-annex-shell command: $cmd"
-    unless $cmd =~ m#^(git-annex-shell '\w+' ')/\~/([0-9a-zA-Z][0-9a-zA-Z._\@/+-
+    unless $cmd =~ m#^(git-annex-shell '\w+' ')/(?:\~\/)?([0-9a-zA-Z][0-9a-zA-Z.
 my $start = $1;
 my $repo = $2;
 my $end = $3;
Comment by bremner Sat Dec 31 03:34:17 2011

That patch seems ok, it doesn't seem to allow through any repo locations that were blocked before.

So, it has my blessing.. but the ADC is in gitolite and will need to be patched there.

Comment by http://joey.kitenet.net/ Sat Dec 31 18:32:28 2011

After some debugging printing, here is my current understanding.

  • urls of the form git@host:~repo or ssh://git@host

    • git sends commands like "git-receive-pack '~/repo'
    • gitolite converts these to $REPO_BASE/~/repo which fails. ~/repo would also fail fwiw.
    • git-annex sends seems /~/repo, which works
  • urls of the form git@host:/repo or ssh://git@host/repo

    • git sends "git-receive-pack '/db/cs3383'"
    • gitolite converts this to $REPO_BASE/repo which works
    • git annex sends "git-annex-shell 'inannex' '/repo' ..." which works, but only with the patch above.
  • urls of the form git@host:repo

    • git sends "git-receive-pack 'repo'
    • gitolite converts this to $REPO_BASE/repo, which works
    • git-annex sends "git-annex-shell 'inannex' '/~/db/cs3383'...", which also works for git-annex-shell.

So the weird case is the last one where git and git-annex are sending different things over the wire. I don't know if you have other motivations for doing the url normalization on the client side, but it isn't needed for gitolite, and in some sense complicates things a little. On the other hand, now that I see what is going on, it isn't a big deal to just strip the leading /~ off in the adc. It does lead to the odd situation of some URLs working for git-annex but not git.

Comment by bremner Sat Dec 31 22:29:38 2011
Ah right. git-annex normalizes all git ssh style user@host:dir to valid uris, which is where the /~/ comes from. I don't anticipate this changing on the git-annex side.
Comment by http://joey.kitenet.net/ Mon Jan 2 16:27:55 2012
Comments on this page are closed.