[tahoe-dev] zfec right for me?

[Jack Lloyd]

On Fri, Jul 16, 2010 at 09:18:46AM +0200, Markus Beyer wrote:
> Hi,
> 
> I am trying to build a data store for my files.
> I would like to shield the file in it against corruption, eg bit errors on the disk and the like.
> Can I achive this with zfec? If so, what m and k would you advice I use?

I think (depending on your disk failure model) this could work well,
though I don't have enough insight to offer particular suggestions on
what parameters you would want.

> Also, does zfec have the ability to detect if one of the pieces is
> corrupted so it won't be used for reconstruction of the original
> (leading to corrupt original data) ?

No it does not - you'll want to use an external checksum for this
purpose.

(A while back I put together a draft of a new file format for zfec
that will be more useful in the general case, for instance with easily
identifiable magic numbers, per-share checksums, etc - right now zfec
only implements the bare error correction code. However I haven't had
time to flesh it out and actually implement it, so for the time being
with zfec you're more or less on your own with the error checking and
so forth).

-Jack