Today I worked on the race conditions, and fixed two of them. Both were fixed by avoiding using git add, which looks at the files currently on disk. Instead, git annex watch injects symlinks directly into git's index, using git update-index.

There is one bad race condition remaining. If multiple processes have a file open for write, one can close it, and it will be added to the annex. But then the other can still write to it.


Getting away from race conditions for a while, I made git annex watch not annex .gitignore and .gitattributes files.

And, I made it handle running out of inotify descriptors. By default, /proc/sys/fs/inotify/max_user_watches is 8192, and that's how many directories inotify can watch. Now when it needs more, it will print a nice message showing how to increase it with sysctl.

FWIW, DropBox also uses inotify and has the same limit. It seems to not tell the user how to fix it when it goes over. Here's what git annex watch will say:

Too many directories to watch! (Not watching ./dir4299)
Increase the limit by running:
  echo fs.inotify.max_user_watches=81920 | sudo tee -a /etc/sysctl.conf; sudo sysctl -p
Actually, Dropbox giver you a warning via libnotify inotify. It tends to go away too quickly to properly read though, much less actually copy down the command...
When I work on the webapp, I'm planning to make it display this warning, and any other similar warning messages that might come up.
Comment by http://joeyh.name/ Wed Jun 6 23:25:57 2012

For the unfamiliar, it's hard to tell if a command like that would persist. I'd suggest being as clear as possible, e.g.:

Increase the limit for now by running:
  sudo sysctl fs.inotify.max_user_watches=81920
Increase the limit now and automatically at every boot by running:
  echo fs.inotify.max_user_watches=81920 | sudo tee -a /etc/sysctl.conf; sudo sysctl -p
Good thought Jim. I've done something like that.
Comment by http://joeyh.name/ Thu Jun 7 04:48:15 2012
Comments on this page are closed.