Segmentation Fault in function X86_SHA256_HashBlocks

[francois]

Sometimes I get a segfault when I launch Tahoe's 'make test' on my Ubuntu karmic 64 bit workstation. Thanks to 'ulimit -c unlimited' a coredump was generated.

Here's what gdb has to tell.

Core was generated by `python setup.py test -s allmydata'.
Program terminated with signal 11, Segmentation fault.
#0  0x00002b1fdf884efc in X86_SHA256_HashBlocks (this=<value optimized out>, input=0x30456e4, length=47412143980619) at cryptopp/sha.cpp:435
435	cryptopp/sha.cpp: No such file or directory.
	in cryptopp/sha.cpp

Here is the interesting part of the stacktrace:

#0  0x00002b1fdf884efc in X86_SHA256_HashBlocks (this=<value optimized out>, input=0x30456e4, length=47412143980619) at cryptopp/sha.cpp:435
#1  CryptoPP::SHA256::HashMultipleBlocks (this=<value optimized out>, input=0x30456e4, length=47412143980619) at cryptopp/sha.cpp:453
#2  0x00002b1fdf88311e in CryptoPP::IteratedHashBase<unsigned int, CryptoPP::HashTransformation>::Update(unsigned char const*, unsigned long) ()
   from /usr/local/lib/python2.6/dist-packages/pycryptopp-0.5.15-py2.6-linux-x86_64.egg/pycryptopp/_pycryptopp.so
#3  0x00002b1fdf826004 in CryptoPP::PK_MessageAccumulatorBase::Update (this=0x3cc6940, input=0x30456e4 "", length=47412143980619) at cryptopp/pubkey.h:292
#4  0x00002b1fdf872419 in CryptoPP::PK_Verifier::VerifyMessage (this=0x67eb720, message=0x30456e4 "", messageLen=47412143980619, signature=0x38e1c24 "\001s\001\205", 
    signatureLength=66) at cryptopp/cryptlib.cpp:781
#5  0x00002b1fdf8dcef9 in VerifyingKey_verify (self=0x5bab600, args=<value optimized out>, kwdict=<value optimized out>) at pycryptopp/publickey/rsamodule.cpp:77
#6  0x00000000004a290d in PyEval_EvalFrameEx ()
#7  0x00000000004a2e47 in PyEval_EvalFrameEx ()
#8  0x00000000004a40e0 in PyEval_EvalCodeEx ()

As you can see, this is pycryptopp-0.5.15 running, I will try to upgrade to see if the same bug also appears with 0.5.17.

[zooko]

It looks like the Crypto++ hackers may have fixed this. But how can we confirm?

​http://tahoe-lafs.org/pipermail/tahoe-dev/2010-July/004604.html

[weidai]

That doesn't look like one of the bugs that were fixed in Crypto++ recently. All of those had to do with incorrect computation of the hash, and shouldn't have caused a segfault. Looking at the stack, it shows messageLen=47412143980619 at the first call into Crypto++, CryptoPP::PK_Verifier::VerifyMessage?. If that is correct and not just the stacktrace showing garbage due to optimizations etc., it seems to imply that the problem occurred prior to the call into Crypto++.

[warner]

I observed something similar on a karmic system (in this case the linode.com slice to which we're planning to migrate tahoe-lafs.org, the host of this here Trac instance). The symptom was a hang in the pycryptopp unit tests (and also a hang in the first Tahoe unit test that touched pycryptopp). I didn't get a crash, but 'strace' did mention a SIGILL of some sort. Running it under GDB didn't seem to reveal anything useful: the process hung instead of getting a signal, and interrupting GDB with control-C told me that the stack was somewhere in this same X86_SHA256_HashBlocks function, but it also gave a lot of "Cannot access memory at address .." errors and couldn't show me anything further. The last few lines of the Tahoe unit test run's "strace" capture were:

rt_sigaction(SIGILL, {0xb73f5470, [ILL], SA_RESTART}, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGILL, {SIG_DFL, [ILL], SA_RESTART}, {0xb73f5470, [ILL], SA_RESTART}, 8) = 0
rt_sigaction(SIGILL, {0xb73f5470, [ILL], SA_RESTART}, {SIG_DFL, [ILL], SA_RESTART}, 8) = 0
rt_sigaction(SIGILL, {SIG_DFL, [ILL], SA_RESTART}, {0xb73f5470, [ILL], SA_RESTART}, 8) = 0
rt_sigaction(SIGILL, {0xb73f5440, [ILL], SA_RESTART}, {SIG_DFL, [ILL], SA_RESTART}, 8) = 0
rt_sigaction(SIGILL, {SIG_DFL, [ILL], SA_RESTART}, {0xb73f5440, [ILL], SA_RESTART}, 8) = 0
--- SIGQUIT (Quit) @ 0 (0) ---
+++ killed by SIGQUIT +++

When I rebuilt pycryptopp with "--disable-embedded-cryptopp", the unit tests worked perfectly.

This was using pycryptopp-0.5.19 . The system's version of libcrypto++8 was 5.6.0-3 . I don't know what version of Crypto++ is embedded in pycryptopp-0.5.19, but the embedded cryptopp/Readme.txt file claims "Version 5.6.0 (3/15/2009)".

My first instinct is to think that something has clobbered the stack, and some sort of SIGILL handler has gotten stuck in an infinite loop (maybe a C++ exception handler or something?).

[zooko]

I tried ​Crypto++ r492 on Brian's linode and it didn't change the behavior.

[zooko]

Oh! Wei Dai pointed out that this might be ​https://bugs.launchpad.net/fedora/+source/binutils/+bug/461303, and indeed it is:

zooko@localhost:~/playground/pycryptopp$ dpkg --status binutils
Package: binutils
Status: install ok installed
Priority: optional
Section: devel
Installed-Size: 9508
Maintainer: Ubuntu Core developers <ubuntu-devel-discuss@lists.ubuntu.com>
Architecture: i386
Version: 2.20-0ubuntu1
Provides: elf-binutils
Depends: libc6 (>= 2.8), zlib1g (>= 1:1.1.4)
Suggests: binutils-doc (>= 2.20-0ubuntu1)
Conflicts: elf-binutils, gas, modutils (<< 2.4.19-1)
Description: The GNU assembler, linker and binary utilities
 The programs in this package are used to assemble, link and manipulate
 binary and object files.  They may be used in conjunction with a compiler
 and various libraries to build programs.
Original-Maintainer: Matthias Klose <doko@debian.org>

The fixed Ubuntu package of binutils for Karmic is 2.20-0ubuntu2:
​https://bugs.launchpad.net/fedora/+source/binutils/+bug/461303/comments/22

[zooko]

Brian: please upgrade the binutils on the linode and click force-rebuild to rebuild the same version of the pycryptopp source code with the new binutils.

[zooko]

confirmed: ​http://tahoe-lafs.org/buildbot-pycryptopp/builders/Brian%20Ubuntu-amd64%20linode/builds/3

[zooko]

Brian suggested adding doc to the Debian/Ubuntu? pages of Tahoe-LAFS that you have to add a special repo -- "lucid-updates" I think -- to get the fixed binutils. I'm not going to add that doc, I'm just mentioning it here in case someone wants to. ​http://tahoe-lafs.org/trac/tahoe-lafs/wiki/DownloadDebianPackages

[zooko]

duplicate of #31