[tahoe-dev] introducing BLAKE2: an alternative to SHA-3, SHA-2, SHA-1, and MD5

[Zooko O'Whielacronx]

Folks:

I just posted this letter to the cryptography mailing list about a new
secure hash function that I helped (a little bit) to define: BLAKE2
(https://blake2.net).

http://lists.randombit.net/pipermail/cryptography/2012-December/003554.html

Tahoe-LAFS is mentioned in the documentation of BLAKE2, as an example
of a system that needs data integrity, and that deals with large
enough chunks of data that the performance of the hash function
matters. See Eirik Haver and Pål Ruud's experiment on an earlier
version of Tahoe-LAFS that showed there was a measurable performance
difference from different hash functions:

https://tahoe-lafs.org/trac/tahoe-lafs/wiki/Bibliography#HashFunctions

I think the biggest difference that they saw between Tahoe-LAFS using
fast or slow hash functions was about 10%.

Other experiments
(https://tahoe-lafs.org/trac/tahoe-lafs/wiki/Performance) have shown
that there are several major performance issues in Tahoe-LAFS
currently, mostly due to unnecessary round trips or pipeline stalls in
the networking protocol, and a few due to unnecessarily complex
algorithms in the client. I believe that the fact that hash function
cost is measurable *at all* in Eirik and Pål's experiment suggests
that the hash function cost would be major issue if the huge
performance problems in the network protocol and the client-side code
were fixed. ☺

Anyway, as you probably know by now the Tahoe-LAFS project is very
conservative about these sorts of things — security, compatibility,
and so forth — so you don't have to worry about Tahoe-LAFS switching
from SHA-256d to BLAKE2 anytime soon. The process that we have starts
with me spending a few years convincing Brian that it is a good idea,
followed by a long, slow process of developing "forward-compatibility"
features to make sure that when we actually do switch over, users
won't experience a disruption of service because of it.

The process also involves us measuring performance, fixing the biggest
performance issues, and measuring again until eventually the biggest
remaining performance issues have to do with the speed of the secure
hash function.

The process will also hopefully involve cryptographers studying BLAKE2
and reporting on whether they find any flaw in it, and whether they
are convinced by the argument that the previous cryptanalysis of
BLAKE1 lends confidence in the strength of BLAKE2.

Note, I feel a lot more urgent about the addition of an extra
XOR'ed-in stream cipher, XSalsa20 — ticket #1164 — and I hope to land
the added XSalsa20 in Tahoe-LAFS v1.11 in early 2013. That's because
attackers in the future may take advantage of ciphertext and other
information (including timing information) which was previously
produced by users. If such attackers can violate confidentiality, then
the users will, at that future time, have the confidentiality of their
old data breached, even if by then they have upgraded their encryption
scheme. On the other hand, attackers from the future can't use a break
of SHA-256 to violate the integrity of old files once users have
upgraded their cryptographic hash function. See the difference?
Upgrading your cryptographic hash function instantly ends the
attacker's ability to hurt you by cracking your old one. Upgrading
your encryption function doesn't. So we should hurry and upgrade our
encryption function. That was the biggest lesson that I learned from
the One Hundred Year Cryptography thought-experiment:

https://tahoe-lafs.org/trac/tahoe-lafs/wiki/OneHundredYearCryptography

Regards,

Zooko

https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1164# use
XSalsa20+AES-128 encryption