Automatically schedule repair process (and backups?)

[mmore]

1.have Tahoe an automatic repair, i.e. , the introducer monitor the replicas numbers, for example if we have configuration of 10 /3 , so introducer trigger repair process when the replicas became only 4 to keep the file available?

2. what are the threshold that triggers the repair process?

[terrell]

As I understand it, the repairer will try to keep 10 copies (in your configuration) on the grid at all times. If the checker determines that the current number of good shares is less than 10, the repairer will create/repair enough to get back up to 10.

Is this an enhancement request for a separate configurable repair threshold (some number < N) ?

[mmore]

Replying to terrell:

As I understand it, the repairer will try to keep 10 copies (in your configuration) on the grid at all times. If the checker determines that the current number of good shares is less than 10, the repairer will create/repair enough to get back up to 10.

Is this an enhancement request for a separate configurable repair threshold (some number < N) ?

indeed , my explanation was: currently has Tahoe automatic repair ?, i.e the lost share can be rebuild in order to keep the file available.

[zooko]

It is currently not "automatic" in the sense that there is an established process which you can tell it: "Please repair the following set of files on the following repeating schedule.". Such a process could be built, for example with a bash script or a short Python program, using the wapi or the cli to trigger the repair processes.

Brian Warner has been writing code like that, to trigger repair processes at certain times. I think it isn't quite documented yet but already runs. Also, you could write your own, which might actually be better as the exact set of file to repair and the exact schedule of when to repair them might fit your use better.

[zooko]

This introductory article in arstechnica.com reminds me that I've had many conversations with people introducing them to Tahoe-LAFS in which I have to explain that the repair process is not automatic. Anytime I have to repeatedly explain something to newcomers, this makes me think it ought to be changed so that the newcomers's assumptions are true. In this case, that means making a scheduler which automatically kicks off a repair process and runs it in the background.

(The comment by thraxil after the arstechnica.com article incorrectly states that this is currently done automatically. In fact, the user has to learn a special command-line tool and run it themselves or configure it to run as a cron job or whatever.)

[zooko]

​http://allmydata.org/pipermail/tahoe-dev/2009-August/002509.html

[zooko]

Kevin Reid posted a cron job to do this ​http://allmydata.org/pipermail/tahoe-dev/2009-October/003012.html and I linked to it from the RelatedProjects page.

[zooko]

So, even though Kevin has demonstrated how to do it with a cron job, this ticket still remains open to make it an integrated and easy-to-configure feature of Tahoe-LAFS itself.

[davidsarah]

To schedule tasks from Python under both Unix and Windows:

​http://pypi.python.org/pypi/python-crontab/0.7 ​http://tarekziade.wordpress.com/2007/11/01/scheduling-tasks-in-windows-with-pywin32/

[davidsarah]

Better link: ​http://pypi.python.org/pypi/python-crontab/

[davidsarah]

An earlier ticket discussing a repair service (now partly overtaken by other changes) was #483, now marked as a duplicate of #543.

[davidsarah]

Replying to davidsarah:

To schedule tasks from Python under [...] Windows:

​http://tarekziade.wordpress.com/2007/11/01/scheduling-tasks-in-windows-with-pywin32/

However, see #1274 (eliminate pywin32 dependency). The code on that page could be translated to use ctypes, although it's slightly more complicated than usual because it uses COM.

[davidsarah]

amiller had some difficulty configuring tahoe backup to run from cron. If we were to add automatic scheduling of repair, we could make it schedule tahoe backup as well, with little extra work.

[zhan0903]

Is there some implementation about automatical repair? I found it is not convenient to repair only one file or directory every time you run the repair tool

sometimes, you even donnot know which file should be repaired, so I think it will be good to make this tool run as a service and check files automatically based on some principles or algorithm

actually, I would like to do something about that