[tahoe-dev] 'Client' caching?

[Brian Warner]

On Nov 25, 2009, at 11:45 PM, Nathan Eisenberg  
<nathan at atlasnetworks.us> wrote:
> I apologize if this is not the correct place to ask questions.
>

Welcome! Yes, this is the perfect place for questions like this.

>  since it appears that the ‘client’ portion of tahoe does not  
> maintain cached files in memory for any period of time.
>
That's correct. At some point, to support random-access reads more  
efficiently, we'll probably add a minimal cache for individual  
segments of a single file, but so far whole-file caching has not been  
on the requirement list.

It's useful to note the distinction between mutable and immutable  
files here. Caching immutable files is perfectly safe: it's a simple  
tradeoff between local storage consumption and performance, assuming a  
given locality-of-reference / reader behavior. Caching *mutable* files  
(or directories), on the other hand, is not safe: the tradeoff  
includes a correctness aspect, since someone else might change the  
contents and you might use your (now stale) cached copy. In general,  
we try to avoid putting any sorts of heuristics about correctness into  
tahoe itself, so any caching layer that requires a decision on a  
correctness-vs-performance policy would need to be placed above the  
tahoe node.
>  My first instinct was to turn to Squid.
>

Hm, I'm surprised that didn't work. Do you know what caused Squid to  
believe the file was changing each time? Maybe a quick peek at the  
returned headers would be informative.

Basically, any URL that starts with /uri/URI:CHK: should be immutable  
and an ideal choice for caching. We'd planned (although I can't check  
right now to see whether we got around to implementing it or not) to  
add an ETag: header with the file's UEB header, which is basically a  
hash of the contents, and thus an ideal etag. And I know we aren't  
intentionally adding any Cache-Control or date headers that might make  
the file look uncacheable.

(for mutable files, there is a similar value called the "roothash"  
which covers the file contents, and allows If-ETag-Differs: -type  
queries to do the right thing, but I don't know if we've
actually implemented that either).

hope that helps,
  -Brian

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://allmydata.org/pipermail/tahoe-dev/attachments/20091126/e58c1778/attachment.htm