added secret to protect convergent encryption from confirmation-of-a-file and learn-partial-information attacks

[zooko]

Per ​the discussion on tahoe-dev, p2p-hackers, cryptography, we're going to make a chk_secret file that lives in $BASEDIR/private, and mix the contents of that file into the symmetric encryption key when doing content-hash keying.

If you don't want to use a chk secret at all, thus doing traditional convergent encryption, then you accomplished this by writing the empty string into the chk_secret file (making it a zero-length file). If there is no chk_secret file at all, it will treat it as if it had the empty string.

​http://allmydata.com plans to make their customers' chk_secrets be generated from a tagged hash of the customers' root directory capability, thus giving convergence across all files upload by the same customer.

Brian and I already wrote code for this and unit tests, but it hasn't been double-checked and committed. Also the version we wrote had an unnecessary complication of treating the absent chk_secret file differently than the zero-length chk_secret file, so I want Brian's opinion about the design proposed in this note before I commit it.

[zooko]

I'm sorry -- what we plan is not to interpret an empty chk_secret file as being the same as a zero-length chk added secret, but rather, if there is no chk_secret file, to create a new random secret and put it in the file.

This is the way to get a random secret for use as a chk added secret.

It means that rm'ing the private/chk_secret file will cause the automatic generation of a new one the next time you start your node, but zeroing it out (i.e., rm'ing it and then touching it) will cause the use of the zero-length chk added secret.

Make sense? Any objections?

[warner]

Let's discuss this more.. the last plan that I'd put any thought into was to have the allmydata.com "install wizard" write a secret into that file, rather than having the node create one by itself. It's a question of defaults.

We have to decide if we want to turn off convergence for all users, or turn it off for allmydata.com users and give pure-tahoe users the information to make that decision for themselves.

[warner]

My friend Drew Hess came up with another concern with publishing hashes of your files: if you've installed some network-accessible service (say, phpbb, or a database of some sort), and the default configuration uses some well-known password (like "admin"), then publishing a hash of the unmodified config file is equivalent to telling the world that you are running this vulnerable configuration.

These files might be of moderate size, so any heuristic which tries to decide whether convergence is safe or not needs to take these files into account. In addition, if the user edits the config file to replace the default password with some low-entropy string, then the guess-partial-information attack applies, as the attacker hashes a dictionary of passwords into that portion of the config file and looks for a match.

[zooko]

Fixed by 29b844efaf7f2eee and 3b7b1932a9869cac. However, the way to control your node's convergent-encryption behavior (see description in comments above) hasn't yet been written into docs/configuration.txt.

[zooko]

documentation added by 4b46f1cd5368820f

[zooko]

The patches that fixed this are now listed in the trac changeset timeline as fc3bd0c9871c7674 and 29b844efaf7f2eee, and the documentation was added in 7f14b933411ff43b.