[tahoe-dev] weekly Tahoe dev call report: 2012-07-10

[Terrell Russell]

For the cloud version of leasedb, if EBS has problems, which it is
known to have had, and a leasedb is corrupted/locked/lost - is that
okay?

Terrell



On Wed, Jul 11, 2012 at 9:09 AM, Zooko Wilcox-O'Hearn <zooko at zooko.com> wrote:
> • 1.9.2 release! Yay David-Sarah, Release Manager. Thanks to everyone
> who contributed bug reports, patches, testing, packaging, etc.
>
> • Add-only sets: Can servers exercise "editorial power" over add-only
> sets, remixing different legitimate adder-signed sets to form new
> sets? Zooko thinks this could be a problem, and that add-only sets
> should be designed against it, but he can't remember why he thinks
> that. Brian thinks that it is hardly a problem because the presence of
> other servers giving answers renders any one server's ability to
> select among legitimate answers almost moot. Andrew and David-Sarah
> both think that the notion of a *set* as opposed to a fully serialized
> sequence must surely require that readers accept unions. We agreed to
> drop the subject for now and move on to lease database.
>
> • lease database
>
>    • keep information about leases in some separate location instead
> of bundled with each share
>
>    • let's use a sqlite db through the pysqlite API, like we do with backupdb
>
>    • for the cloud backend (that Least Authority Enterprises is
> building), the leasedb will be stored on persistent storage e.g Amazon
> Elastic Block Store (EBS), while the shares are stored on cloud
> storage, e.g. Amazon Simple Storage Service (S3).
>
>    • people can manually add shares, such as by just dropping a share
> file into a disk backend filesystem, or uploading a share object to
> Amazon S3, and the lease system will eventually discover them and
> maintain them as long as they are leased, and then delete them when
> they are no longer leased.
>
>    • people can manually delete shares, such as by just rm'ing a share
> file from a disk backend filesystem, or deleting a share object to
> Amazon S3, and the lease system will not break when it discovers that
> it is gone.
>
>    • There can be race conditions between such external actions and
> the progress of the crawler which is inspecting leases. A state
> machine must be carefully analyzed to see that in handles all such
> possible sequences of events. See
> https://tahoe-lafs.org/trac/tahoe-lafs/wiki/Summit2Day4#leasedbcrawler
> for initial notes about that. A state transition diagram would be a
> good way to analyze and communicate that.
>
>    • Brian was about to write a new lease database as the next step in
> his Accounting work, and Least Authority Enterprises is about to write
> a new lease database as the next step in our DARPA research grant
> contract, with a deadline of July 26. So, let's cooperate. We need to
> agree on separation of responsibilities.
>
>    • The crawler can be a "background task" that doesn't take up
> resources (CPU) for too long at a time, so there's a configurable knob
> for "how many seconds in a row do I run" and "how many seconds do I
> idle in between runs", and another knob for "how soon should I start a
> new pass after I've already finished the last pass".
>
>    • Should "account ids" or "lease-owner ids" be public keys or
> things derived from symmetric secrets? Brian wisely suggests
> decoupling that question from the rest of the lease db design. But,
> then Brian and David-Sarah agreed that the lease-owner ids should be
> small integers. Zooko disagrees, but nobody asked him.
>
>    • Least Authority Enterprises might someday want to store their
> lease databases in funky cloud databases things like Amazon's Cloud
> SQL DB or Microsoft's Cloud SQL DB. But for now we're just going to
> use pysqlite and local storage such as Amazon EBS.
>
> Brian will review his branch and write up some stuff about how the
> accounting branch ought to go. Brian and David-Sarah will
> synchronously work on it Tuesday and Thursday.
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at tahoe-lafs.org
> https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev