[tahoe-dev] compatibility break, test grid flushed

[Brian Warner]

It's that time of the week again. In the closing hours before our big 0.8.0
release, we pushed a number of large changes into the Tahoe codebase under
the "better now than later" banner. Once we committed to making at least one
compatibility-breaking change, we figured it was an opportunity to push
through a number of other ones.

In the last few days, we've changed the following items:

 * we changed the storage server's share-naming algorithm to allow servers to
   store more shares before running into the ext3 32000-entry-per-directory
   limit. More changes might be necessary in the future, as storage servers
   grow, but these changes will not need to be visible outside that server.

 * we switched all binary-to-ascii representations from using z-base32 to
   base62 to regular base32. This invalidates all URIs and the
   'private/secret' file that is used by the client to compute lease
   renewal/cancellation secrets

 * we switched to using SHA-256d (as defined by Ferguson and Schneier in
   _Practical Cryptography_) for all hashing purposes, which is not
   vulnerable to the length-extension attack that normal SHA algorithms are.
   This invalidates all URIs too.

 * hash tags have been updated and cleaned up to more accurately reflect
   their meanings. This invalidates all URIs too.

What does this mean for you?

 * When you upgrade a storage server, the old shares will no longer be useful
   to the new version of the server. Zooko is planning to write a conversion
   tool to migrate these shares, but that probably won't help you, since:

 * All URIs in all extant dirnodes and bookmarks are now invalid. All data is
   effectively flushed, so storage server operators might as well delete
   their storage/shares/ directory when upgrading their nodes.

 * The 'private/secret' file in client directories needs to be deleted before
   you restart your client with the new code.


We hope for this to be the last time we make non-backwards-compatible changes.

I've updated the TestGrid wiki page with the new public shared directory cap:
URI:DIR2:rymosvvxauojktsga6ukfbffsu:d4b2rqamue744nejtu4h7akxicllb7ta7iybckfywx7ba3lcqj7a
, and populated it with some random documents.


Stay tuned for a shiny new 0.8.0 release tomorrow!

cheers,
 -Brian