[tahoe-dev] blocks instead of files?

[Jody Harris]

On Wed, Mar 10, 2010 at 4:27 PM, David-Sarah Hopwood <
david-sarah at jacaranda.org> wrote:

> Jody Harris wrote:
> > Has the Tahoe Dev ever discussed the possibility of encrypting and
> storing
> > blocks instead of files?
> >
> > I don't know:
> > 1) if this has already been discussed, or
> > 2) if it would actually solve the problem it seems to me it would solve.
> >
> > If the encrypt/store algorithms encrypt in (for example) 128 KB blocks,
> > then
> >  - you would never hit the large memory overhead problems seen with 100
> MB
> > and larger files.
> >  - processing and storage would just take the file one chunk at a time
> >  - retrieval and decryption (seems) that it would stream nicely since
> > allocation of the next block(s) could take place while the current block
> is
> > streaming out.
>
> This should all already be the case for immutable files.
>
> That it isn't true for mutable files is, as far as I know, just a temporary
> limitation of the current mutable download/upload code (which does work in
> terms of segments, but hasn't been optimized to reduce its memory usage by
> doing so).
>
> I'm glad I asked. I've learned a lot today.

thanks!

jody
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://allmydata.org/pipermail/tahoe-dev/attachments/20100310/70ddf6fb/attachment.htm