#804 closed defect (fixed)

provide binary .egg's for pycryptopp for all supported platforms and versions of Python

Reported by: arch_o_median Owned by: zooko
Priority: minor Milestone: 1.8.1
Component: packaging Version: 1.5.0
Keywords: binaries install dependency pycryptopp linux x86-64 Cc:
Launchpad Bug:

Description (last modified by tilllt)

Some System Configuration Information

arc@showertimedepot:~/tahoe$ darcs --version
2.0.2 (release)
arc@showertimedepot:~/tahoe$ uname -a
Linux showertimedepot 2.6.18.8-x86_64-linode7 #1 SMP Fri Aug 14 17:14:08 UTC 2009 x86_64 GNU/Linux
arc@showertimedepot:~/tahoe$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 8.10
Release:        8.10
Codename:       intrepid

Downloads

Per these instructions I ran:

darcs get --lazy http://allmydata.org/source/tahoe/trunk-hashedformat tahoe

I've deviated from the prescribed protocol since the default install docs do not recommend acquiring the source from the darcs repo.
Perhaps differences between the tarball and the repo versions generate this bug.
To check this I also attempt to build from the tarball in a separate location.

I.E. I ran:

 wget http://allmydata.org/source/tahoe/releases/allmydata-tahoe-1.5.0.zip

Build From Repo

Per these instructions I ran:

~/tahoe$ python ./setup.py build &> from_repo_build_log.txt

in the "from-repo" directory.

Upon inspection of the output (attached) I see several error-types.

I guess these two to be not-fatal:

  1. "darcs failed: Not a repository: ..." and
  2. "cc1plus: warning:..." lines are not fatal.

Following those two error-types the next error is:

pycryptopp/_pycryptoppmodule.cpp:2:20: error: Python.h: No such file or directory

Difference Between First and Second Build Attempts in From-Tarball Version

In the "from-tar" directory I ran:

python ./setup.py build &> from_tar_build_log_initial.txt

(Actually I changed the log names to be more descriptive, after the fact.)

I do not notice obvious differences between repo and tar.

I do notice differences between initial and subsequent build-attempts.

(Perhaps I should be writing "setup-" vice "build-" attempts.)

arc@showertimedepot:~/tahoe/allmydata-tahoe-1.5.0$ grep zope from_tar_build_log_initial.txt | grep Python.h
src/zope/interface/_zope_interface_coptimizations.c:15:20: error: Python.h: No such file or directory
arc@showertimedepot:~/tahoe/allmydata-tahoe-1.5.0$ grep zope from_tar_build_log_second.txt | grep Python.h

(Log names changed after the fact for clarity.)

Intriguingly "zope" appears to be missing "Python.h" on the first build attempt, this is not reported on subsequent build attempts.
The missing Python.h in pycryptopp is reported in initial and subsequent build attempts:

arc@showertimedepot:~/tahoe/allmydata-tahoe-1.5.0$ grep pycryp from_tar_build_log_initial.txt | grep Python.h
pycryptopp/_pycryptoppmodule.cpp:2:20: error: Python.h: No such file or directory
arc@showertimedepot:~/tahoe/allmydata-tahoe-1.5.0$ grep pycryp from_tar_build_log_second.txt | grep Python.h
pycryptopp/_pycryptoppmodule.cpp:2:20: error: Python.h: No such file or directory

First Versus Second From Repo Setup

Is the difference between initial and subsequent setup bugs also
seen in the from repo version? To find out I tested a fresh from-repo setup attempt.

arc@showertimedepot:~/tahoe-darcs$ darcs get --lazy http://allmydata.org/source/tahoe/trunk-hashedformat tahoe
Finished getting.                           
arc@showertimedepot:~/tahoe-darcs/tahoe$ grep -iEe"zope|pycryp" | grep Python.h from_repo_*
from_repo_initial_setup.txt:src/zope/interface/_zope_interface_coptimizations.c:15:20: error: Python.h: No such file or directory
from_repo_initial_setup.txt:pycryptopp/_pycryptoppmodule.cpp:2:20: error: Python.h: No such file or directory
from_repo_second_setup.txt:pycryptopp/_pycryptoppmodule.cpp:2:20: error: Python.h: No such file or directory

So the difference between zope and pycryptopp initial vs.
second installs generalizes to installs from tar and from repo.

Bug Interpretation

It looks like Python.h of pycryptopp is not found so I guess that I lack some necessary dependency.

Calling All Dependencies

(Since my bug does not seem to vary between from-repo and from-tar, I stick to from repo from here on.)

Where're these elusive Python.h's? Sound like C headers to me....

Naive:

arc@showertimedepot:~/tahoe-darcs/tahoe$ find . -iname "*Python.h*"
arc@showertimedepot:~/tahoe-darcs/tahoe$ 

OK, not surprising since docs say all repo is python, and what's missing is C. So, my belief that I have a dependency problem grows stronger.

Following More Details, I learn that I can view "the canonical list of dependencies" in _auto_deps.py.

Of course, I cannot run "tahoe --version" yet so I skip that advice...

Aha! A "complete list of all dependencies..."

PYTHONPATH=${PYTHONPATH}:./support/lib/python2.5/site-packages/ python -c "import pkg_resources;print ', '.join([d.project_name+': '+d.version for d in set(pkg_resources.require('allmydata-tahoe'))])"
arc@showertimedepot:~/tahoe-darcs/tahoe$ PYTHONPATH=${PYTHONPATH}:./support/lib/python2.5/site-packages/ python -c "import pkg_resources;print ', '.join([d.project_name+': '+d.version for d in set(pkg_resources.require('allmydata-tahoe'))])"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/arc/tahoe-darcs/tahoe/support/lib/python2.5/site-packages/setuptools-0.6c12dev.egg/pkg_resources.py", line 626, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/arc/tahoe-darcs/tahoe/support/lib/python2.5/site-packages/setuptools-0.6c12dev.egg/pkg_resources.py", line 524, in resolve
    raise DistributionNotFound(req)  # XXX put more info here
pkg_resources.DistributionNotFound: pycryptopp>=0.5.15
arc@showertimedepot:~/tahoe-darcs/tahoe$ 

Hmmm.... next step... submit and wait... if no help then dissect above expression...

OK, I did try to dissect a tiny bit:

In ipython interpreter:

In [1]: import sys

In [2]: sys.path
Out[2]: 
['',
 '/usr/bin',
 '/usr/lib/python2.5',
 '/usr/lib/python2.5/plat-linux2',
 '/usr/lib/python2.5/lib-tk',
 '/usr/lib/python2.5/lib-dynload',
 '/usr/local/lib/python2.5/site-packages',
 '/usr/lib/python2.5/site-packages',
 '/var/lib/python-support/python2.5',
 '/var/lib/python-support/python2.5/IPython/Extensions',
 '/home/arc/.ipython']

In [3]: sys.path.append('./support/lib/python2.5/site-packages/')

In [4]: import pkg_resources
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)

/home/arc/tahoe-darcs/tahoe/<ipython console> in <module>()

ImportError: No module named pkg_resources

(And just to be sure I tried above with explicit path name from root to relevant site-packages.)

Which is:

arc@showertimedepot:~/tahoe-darcs/tahoe$ ls -la ./support/lib/python2.5/site-packages/
total 32
drwxr-xr-x 3 arc arc 4096 2009-09-14 22:32 .
drwxr-xr-x 3 arc arc 4096 2009-09-14 22:30 ..
-rw-r--r-- 1 arc arc   35 2009-09-14 22:32 allmydata-tahoe.egg-link
-rw-r--r-- 1 arc arc  297 2009-09-14 22:30 easy-install.pth
drwxr-xr-x 4 arc arc 4096 2009-09-14 22:32 setuptools-0.6c12dev.egg
-rw-r--r-- 1 arc arc   27 2009-09-14 22:32 setuptools.pth
-rw-r--r-- 1 arc arc 2327 2009-09-14 22:30 site.py
-rw-r--r-- 1 arc arc 1833 2009-09-14 22:30 site.pyc

Attachments (5)

from_repo_initial_setup.txt (34.7 KB) - added by arch_o_median at 2009-09-14T22:59:13Z.
from_repo_second_setup.txt (8.5 KB) - added by arch_o_median at 2009-09-14T22:59:21Z.
from_tar_build_log_initial.txt (35.1 KB) - added by arch_o_median at 2009-09-14T22:59:31Z.
from_tar_build_log_second.txt (9.0 KB) - added by arch_o_median at 2009-09-14T22:59:39Z.
pycryptopp-0.5.25_r757-py2.6-netbsd-5.0.2-i386.egg (1.8 MB) - added by midnightmagic at 2010-11-22T00:23:27Z.
here's a binary egg for netbsd-5.0.2 i386, from the non-syslib buildslave.

Download all attachments as: .zip

Change History (55)

Changed at 2009-09-14T22:59:13Z by arch_o_median

Changed at 2009-09-14T22:59:21Z by arch_o_median

Changed at 2009-09-14T22:59:31Z by arch_o_median

Changed at 2009-09-14T22:59:39Z by arch_o_median

comment:1 Changed at 2009-09-15T04:03:42Z by arch_o_median

  • Resolution set to fixed
  • Status changed from new to closed
  • Summary changed from Cannot Install, No Python.h (zope, pycryptopp), Cannot Obtain Dependency List to Cannot Install,

OK, solved.

For my system:

sudo apt-get install python-dev python-openssl
wget http://pypi.python.org/packages/source/p/pyutil/pyutil-1.4.1.tar.gz
...
sudo python setup.py install (in pyutil directory)

Unfortunately when I ran setup.py install without sudo, I received a complaint about test_suite needing to be a list...

Anywho, it looks like basic install's OK now. Maybe I should write up an amateur install doc, for ubuntu people?

comment:2 Changed at 2009-09-15T16:09:18Z by zooko

  • Resolution fixed deleted
  • Status changed from closed to reopened

Thank you for the bug report. This isn't fixed yet though, until either the process in the current install.html works, or the install.html is changed to describe a process that works.

I'm not sure exactly how to accomplish that, yet...

comment:3 Changed at 2009-09-15T17:41:29Z by arch_o_median

I see two distinct "bugs".

  1. We need a doc that catalogs how to get dependencies (in this case for old Ubuntu).
  2. We need to know why sudo is necessary for installing pyutil, and/or change the docs to reflect this.

comment:4 Changed at 2009-09-19T20:37:49Z by zooko

sudo? You had to use sudo? I'm sorry, I haven't read your bug report thoroughly enough yet. You definitely don't need to use sudo.

comment:5 Changed at 2009-09-20T02:50:36Z by zooko

As to your issue "i.", the goal is to make it so that the list of dependencies in http://allmydata.org/source/tahoe/trunk/docs/install.html is really sufficient:

  • Python >= 2.4.2 and < 3.0
  • if on Windows, pywin32

(Indeed, the goal would be to automate the installation of pywin32 too, so that the list of dependencies would be just Python. See http://twistedmatrix.com/trac/ticket/3238 (patch: declare that twisted requires pywin32 if it is to offer process management or iocp reactor on Windows), #756 (if pywin32 has been manually installed, setuptools still doesn't detect it), #142 (pywin32 can't be installed with setuptools), http://bugs.python.org/setuptools/issue18 (can't install pywin32), http://sourceforge.net/tracker/?func=detail&atid=551954&aid=1799934&group_id=78018 (easy_install silently fails).)

Anyway, my point is that the problem you encountered is that pycryptopp couldn't be automatically installed, and there are two ways to improve the situation: either (a) make pycryptopp be automatically installed on your platform and/or (b) document pycryptopp as something that has to be manually installed on your platform.

The former would be accomplished by building a binary egg of pycryptopp for your platform. I intend to configure the pycryptopp buildslaves to build eggs and upload them to http://allmydata.org in the same way that they currently build .deb's and upload those. One problem with this is http://bugs.python.org/setuptools/issue78 (egg platform names don't reflect unicode variant (UCS2, UCS4)).

The latter would be accomplished by editing the InstallDetails wiki page to tell Ubuntu users to do what you did. (sudo apt-get install python-dev)

comment:6 Changed at 2009-09-27T19:38:25Z by zooko

  • Owner changed from nobody to arch_o_median
  • Status changed from reopened to new

Assigning to arc.

arc: what shall we do? I think we should narrow this ticket to be about the lack of a binary egg for pycryptopp on Linux. What do you think? If you agree, please change the title of this ticket and set it to be assigned to "nobody" instead of yourself. Perhaps we could *also* open another ticket saying that the InstallDetails page should include information about binary eggs of pycryptopp. Or instead of spending time opening that ticket, just spend the same time editing the wiki.

comment:7 Changed at 2009-10-05T01:16:08Z by warner

  • Component changed from unknown to packaging

comment:8 Changed at 2009-12-11T05:09:50Z by imhavoc

This bug has surfaced again.

System info:

tahoe@cat:~/allmydata-tahoe-1.5.0$ uname -a Linux cat 2.6.24-24-generic #1 SMP Fri Sep 18 16:49:39 UTC 2009 i686 GNU/Linux

tahoe@cat:~/allmydata-tahoe-1.5.0$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 8.04.3 LTS Release: 8.04 Codename: hardy

tahoe@cat:~/allmydata-tahoe-1.5.0$ cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 15 model : 79 model name : AMD Athlon(tm) 64 Processor 3500+ stepping : 2 cpu MHz : 2210.208 cache size : 512 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 1 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow up pni cx16 lahf_lm svm extapic cr8_legacy ts fid vid ttp tm stc bogomips : 4420.41 clflush size : 64

I tried using the solutions listed in this ticket. No joy.

comment:9 Changed at 2009-12-11T05:12:08Z by imhavoc

This bug has surfaced again.

System info:

tahoe@cat:~/allmydata-tahoe-1.5.0$ uname -a
Linux cat 2.6.24-24-generic #1 SMP Fri Sep 18 16:49:39 UTC 2009 i686 GNU/Linux
tahoe@cat:~/allmydata-tahoe-1.5.0$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 8.04.3 LTS
Release:        8.04
Codename:       hardy
tahoe@cat:~/allmydata-tahoe-1.5.0$ cat /proc/cpuinfo
processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 79
model name      : AMD Athlon(tm) 64 Processor 3500+
stepping        : 2
cpu MHz         : 2210.208
cache size      : 512 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt rdtscp lm 3dnowext 3dnow up pni cx16 lahf_lm svm extapic cr8_legacy ts fid vid ttp tm stc
bogomips        : 4420.41
clflush size    : 64

I tried using the solutions listed in this ticket. Build still failed.

comment:10 follow-up: Changed at 2009-12-11T05:58:32Z by zooko

  • Owner changed from arch_o_median to zooko
  • Status changed from new to assigned

Okay I'm changing this ticket to say that we should take strategy (a) from comment:5 -- make binary .egg's of pycryptopp be installed for supported systems. arc's and imhavoc's systems are both amd64 with UCS4, and we have a pycryptopp buildbot that fits that description: http://allmydata.org/buildbot-pycryptopp/builders/linux-amd64-ubuntu-karmic-yukyuk . So to close this ticket, configure that buildbot to generate binary eggs and upload them to the tahoe-deps repository. (Then check if the builds start working for arc and imhavoc.) Oh, but arc was using Python 2.5 and imhavoc was using Python 2.6. Actually I think arc has subsequently upgraded his Ubuntu to Karmic so he has 2.6 as well. At least for now I'm defining this ticket as "provide binary .egg's for pycryptopp for linux-amd64-py2.6-ucs4".

comment:11 Changed at 2009-12-11T06:12:33Z by zooko

  • Summary changed from Cannot Install, to provide binary .egg's for pycryptopp for linux-amd64-py2.6-ucs4

comment:12 in reply to: ↑ 10 Changed at 2009-12-11T15:10:21Z by imhavoc

Replying to zooko:

[snip] Oh, but arc was using Python 2.5 and imhavoc was using Python 2.6. Actually I think arc has subsequently upgraded his Ubuntu to Karmic so he has 2.6 as well....

Actually, on this system, Python is at version 2.5.2. Sorry.

comment:13 Changed at 2009-12-11T16:10:26Z by zooko

  • Summary changed from provide binary .egg's for pycryptopp for linux-amd64-py2.6-ucs4 to provide binary .egg's for pycryptopp for linux-amd64-py2.[56]-ucs4

Okay, to close this ticket we need to generate binary .egg's for both Python 2.5 and Python 2.6 on Linux-amd64-ucs4.

comment:14 Changed at 2009-12-11T22:08:48Z by imhavoc

It turns out that my /home is a separate partition.

That partition was being mounted as 'noexec.'

When I edited fstab to make the partition mount as 'exec' and rebooted, tahoe compiled fine on the system in question.

comment:15 Changed at 2009-12-11T23:27:11Z by zooko

Okay, so basically it turn out that you did have a C compiler, once you allowed it to be executable. Wait, was the cc1plus executable on your home partition?

In any case this ticket is about making precompiled pycryptopp binaries available for people who don't have a compiler.

comment:16 Changed at 2009-12-11T23:36:14Z by imhavoc

I may have needed to install build-essential also.... I think I did. So, yes, I did have a C compiler available. So my problem may have been miss-files in this ticket.

comment:17 Changed at 2009-12-11T23:47:58Z by zooko

So, the problem wasn't that your home directory was mounted no-exec, it was that you hadn't installed build-essential? That would explain the error message saying that cc1plus didn't work. Hey waitasecond, where is the error message? I thought I saw a paste from you showing cc1plus failing, but scanning back up through this ticket I don't see it.

What was the exact error message that you got?

comment:18 Changed at 2009-12-12T00:02:26Z by imhavoc

I emailed it to you. I think that I never pasted it here.

comment:19 Changed at 2009-12-12T00:36:06Z by zooko

Oh, okay here was the original error message that imhavoc mailed to me:

gcc: error trying to exec 'cc1plus': execvp: No such file or directory
error: Setup script exited with error: command 'gcc' failed with exit status 1

Yep, that looks like build-essential wasn't installed. Now, once this ticket is fixed, then it will just get the binary .egg of pycryptopp and it won't even try to execute gcc.

comment:20 Changed at 2009-12-12T02:18:28Z by davidsarah

  • Keywords pycryptopp added

comment:21 Changed at 2009-12-12T02:18:59Z by davidsarah

  • Keywords linux x86-64 added

comment:22 Changed at 2009-12-14T04:39:41Z by imhavoc

One of my grid participants has run into this bug again.

Ubuntu 8.10

AMD (32-bit CPU)

Looks like pycrypto is not available. There is also no python-pycryptopp available in apt.

We're trying to get a grid online, but installing tahoe-lafs seems to be, by far, the hardest part of getting a grid online.

comment:23 follow-up: Changed at 2009-12-14T04:49:13Z by zooko

The workaround for Ubuntu is to install build-essential so that a pycryptopp will be built automatically from source.

comment:24 Changed at 2009-12-14T04:52:58Z by imhavoc

python -V

 2.5.2

gcc -v

Using built-in specs.
Target: i486-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.3.2-1ubuntu12' --with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.3 --program-suffix=-4.3 --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --enable-mpfr --enable-targets=all --enable-checking=release --build=i486-linux-gnu --host=i486-linux-gnu --target=i486-linux-gnu
Thread model: posix
gcc version 4.3.2 (Ubuntu 4.3.2-1ubuntu12) 

cat /proc/cpuinfo

processor	: 0
vendor_id	: AuthenticAMD
cpu family	: 15
model		: 44
model name	: Mobile AMD Sempron(tm) Processor 3000+
stepping	: 2
cpu MHz		: 800.000
cache size	: 128 KB
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt 3dnowext 3dnow up pni lahf_lm
bogomips	: 1600.04
clflush size	: 64
power management: ts fid vid ttp tm stc

build fail output:

someUser@someUser-ubuntu:~/Desktop/allmydata-tahoe-1.5.0$ python setup.py build
running darcsver
setup.py darcsver: Failure from attempt to find version tags with 'darcs changes', and src/allmydata/_version.py already exists, so leaving it alone.
running develop
Not found: tahoe-deps
Not found: ../tahoe-deps
running egg_info
writing requirements to src/allmydata_tahoe.egg-info/requires.txt
writing src/allmydata_tahoe.egg-info/PKG-INFO
writing top-level names to src/allmydata_tahoe.egg-info/top_level.txt
writing dependency_links to src/allmydata_tahoe.egg-info/dependency_links.txt
writing entry points to src/allmydata_tahoe.egg-info/entry_points.txt
reading manifest file 'src/allmydata_tahoe.egg-info/SOURCES.txt'
writing manifest file 'src/allmydata_tahoe.egg-info/SOURCES.txt'
running build_ext
Processing setuptools-0.6c12dev.egg
creating /home/someUser/.local/share/Trash/files/allmydata-tahoe-1.5.0/support/lib/python2.5/site-packages/setuptools-0.6c12dev.egg
Extracting setuptools-0.6c12dev.egg to /home/someUser/.local/share/Trash/files/allmydata-tahoe-1.5.0/support/lib/python2.5/site-packages
Adding setuptools 0.6c12dev to easy-install.pth file
Installing easy_install script to support/bin
Installing easy_install-2.5 script to support/bin

Installed /home/someUser/.local/share/Trash/files/allmydata-tahoe-1.5.0/support/lib/python2.5/site-packages/setuptools-0.6c12dev.egg
Creating /home/someUser/.local/share/Trash/files/allmydata-tahoe-1.5.0/support/lib/python2.5/site-packages/allmydata-tahoe.egg-link (link to src)
allmydata-tahoe 1.5.0 is already the active version in easy-install.pth
Installing tahoe script to support/bin

Installed /home/someUser/.local/share/Trash/files/allmydata-tahoe-1.5.0/src
Processing dependencies for allmydata-tahoe==1.5.0
Searching for pycryptopp>=0.5.15
Reading http://allmydata.org/source/tahoe/deps/tahoe-dep-sdists/
Reading http://allmydata.org/source/tahoe/deps/tahoe-dep-eggs/
Best match: pycryptopp 0.5.17
Downloading http://allmydata.org/source/tahoe/deps/tahoe-dep-sdists/pycryptopp-0.5.17.tar.gz
Processing pycryptopp-0.5.17.tar.gz
Running pycryptopp-0.5.17/setup.py -q bdist_egg --dist-dir /tmp/easy_install-pzTdsC/pycryptopp-0.5.17/egg-dist-tmp-7fK0dS
setup.py darcsver: Failure from attempt to find version tags with 'darcs changes', and pycryptopp/_version.py already exists, so leaving it alone.
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
pycryptopp/_pycryptoppmodule.cpp:2:20: error: Python.h: No such file or directory
In file included from pycryptopp/_pycryptoppmodule.cpp:5:
pycryptopp/publickey/rsamodule.hpp:5: error: variable or field ‘init_rsa’ declared void
pycryptopp/publickey/rsamodule.hpp:5: error: ‘PyObject’ was not declared in this scope
pycryptopp/publickey/rsamodule.hpp:5: error: ‘module’ was not declared in this scope
pycryptopp/publickey/rsamodule.hpp:7: error: ‘PyMethodDef’ does not name a type
pycryptopp/publickey/rsamodule.hpp:9: error: expected initializer before ‘*’ token
pycryptopp/publickey/rsamodule.hpp:13: error: expected initializer before ‘*’ token
pycryptopp/publickey/rsamodule.hpp:17: error: expected initializer before ‘*’ token
In file included from pycryptopp/_pycryptoppmodule.cpp:6:
pycryptopp/hash/sha256module.hpp:5: error: variable or field ‘init_sha256’ declared void
pycryptopp/hash/sha256module.hpp:5: error: ‘PyObject’ was not declared in this scope
pycryptopp/hash/sha256module.hpp:5: error: ‘module’ was not declared in this scope
In file included from pycryptopp/_pycryptoppmodule.cpp:7:
pycryptopp/cipher/aesmodule.hpp:5: error: variable or field ‘init_aes’ declared void
pycryptopp/cipher/aesmodule.hpp:5: error: ‘PyObject’ was not declared in this scope
pycryptopp/cipher/aesmodule.hpp:5: error: ‘module’ was not declared in this scope
pycryptopp/_pycryptoppmodule.cpp:9: error: expected constructor, destructor, or type conversion before ‘(’ token
pycryptopp/_pycryptoppmodule.cpp:20: error: ‘PyMethodDef’ does not name a type
pycryptopp/_pycryptoppmodule.cpp: In function ‘void init_pycryptopp()’:
pycryptopp/_pycryptoppmodule.cpp:32: error: ‘PyObject’ was not declared in this scope
pycryptopp/_pycryptoppmodule.cpp:32: error: ‘module’ was not declared in this scope
pycryptopp/_pycryptoppmodule.cpp:34: error: ‘_pycryptopp_functions’ was not declared in this scope
pycryptopp/_pycryptoppmodule.cpp:34: error: ‘_pycryptopp__doc__’ was not declared in this scope
pycryptopp/_pycryptoppmodule.cpp:34: error: ‘Py_InitModule3’ was not declared in this scope
pycryptopp/_pycryptoppmodule.cpp:39: error: ‘init_rsa’ was not declared in this scope
pycryptopp/_pycryptoppmodule.cpp:40: error: ‘init_sha256’ was not declared in this scope
pycryptopp/_pycryptoppmodule.cpp:41: error: ‘init_aes’ was not declared in this scope
error: Setup script exited with error: command 'gcc' failed with exit status 1
someUser@someUser-ubuntu:~/Desktop/allmydata-tahoe-1.5.0$ 

comment:25 in reply to: ↑ 23 Changed at 2009-12-14T04:53:31Z by imhavoc

Replying to zooko:

The workaround for Ubuntu is to install build-essential so that a pycryptopp will be built automatically from source.

We tried installing build-essential, and it continued to fail.

comment:26 follow-up: Changed at 2009-12-14T05:32:27Z by zooko

The informative line in that voluminous error output is:

 pycryptopp/_pycryptoppmodule.cpp:2:20: error: Python.h: No such file or
 directory

So the work-around is to install build-essential and python-dev. :-)

comment:27 in reply to: ↑ 26 Changed at 2009-12-14T05:39:17Z by imhavoc

Replying to zooko:

The informative line in that voluminous error output is:

 pycryptopp/_pycryptoppmodule.cpp:2:20: error: Python.h: No such file or
 directory

So the work-around is to install build-essential and python-dev. :-)

It would be immensely helpful if this information was included in the download/install documentation. Python is not throwing useful errors for people who are not familiar with python builds.

So, let me start a list of system requirements that should be in place if you expect to see a successful build: build-essential, python-dev...

I see that I also installed python-openssl on my 8.04 system before getting a successful build.

Am I missing anything?

comment:28 Changed at 2009-12-14T05:57:09Z by zooko

There are some notes about such things on http://allmydata.org/trac/tahoe/wiki/InstallDetails . Those notes aren't on http://allmydata.org/source/tahoe/trunk/docs/install.html because those steps aren't supposed to be necessary. The fact that they are necessary (on some platforms) is a bug, and is the subject of this ticket. Improving the documentation for those platforms where binaries of pycryptopp and the other dependencies are not available would be the subject of... let's see... #282 (more detailed and targeted docs about installing from source). Hm, see also #701 (The instructions in "Installing Tahoe" do not mention C compiler requirement.) which is pretty much the same issue as you are having but opened by my brother Nathan.

comment:29 Changed at 2009-12-14T06:09:20Z by imhavoc

I stumbled across this page from the InstallDetails page: http://allmydata.org/trac/tahoe/wiki/DownloadDebianPackages

That's useful information. It should be linked from the install.html page.

comment:30 Changed at 2009-12-14T06:12:33Z by zooko

It is!

comment:31 Changed at 2009-12-14T06:36:33Z by zooko

  • Summary changed from provide binary .egg's for pycryptopp for linux-amd64-py2.[56]-ucs4 to provide binary .egg's for pycryptopp for linux-{i386,amd64}-py2.{5,6}-ucs4

Err, sorry, I meant that the InstallDetails page is linked from the install.html page. The DownloadDebianPackages page is too platform-specific to be linked from the install.html page.

Since imhavoc's friend is using 32-bit, I hereby enlarge this ticket to also provide 32-bit binaries of pycryptopp.

comment:32 Changed at 2010-10-31T06:53:52Z by zooko

  • Keywords binaries added

Okay, we've now fixed #1094 which is to automatically build bdist_eggs of pycryptopp on all supported platforms. Now the question is whether the supported platforms (see the pycryptopp buildbot and the tahoe-dep-eggs directory on tahoe-lafs.org) include the platforms that the users on this ticket needed...

comment:33 Changed at 2010-10-31T18:35:06Z by zooko

  • Summary changed from provide binary .egg's for pycryptopp for linux-{i386,amd64}-py2.{5,6}-ucs4 to provide binary .egg's for pycryptopp for all supported platforms and versions of Python

comment:34 Changed at 2010-11-14T09:00:03Z by zooko

  • Milestone changed from undecided to 1.8.1

comment:35 Changed at 2010-11-20T07:59:32Z by zooko

Here is a nice table which shows which binary packages we are currently hosting for which platforms.

comment:36 Changed at 2010-11-20T08:00:45Z by zooko

Any platform in http://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-dep-eggs/README.html which has a row that goes all the way across with every cell filled in is a platform where Tahoe-LAFS can be easily installed by someone who doesn't have a compiler or development libraries.

Changed at 2010-11-22T00:23:27Z by midnightmagic

here's a binary egg for netbsd-5.0.2 i386, from the non-syslib buildslave.

comment:37 Changed at 2011-01-06T21:21:06Z by warner

  • Milestone changed from 1.8.1 to soon (release n/a)

comment:38 Changed at 2011-01-06T21:59:35Z by zooko

  • Milestone changed from soon (release n/a) to 1.8.1
  • Resolution set to fixed
  • Status changed from assigned to closed

I'm going to call this good enough. We have binary pycryptopp eggs for every platform on our Supported Builder list (go to http://tahoe-lafs.org/buildbot/ and click on "Supported Builders") except for Solaris. I guess I'll open a new ticket asking for binary eggs for all of the needed dependencies for Solaris (those are: Twisted, pyOpenSSL, pycrypto, pycryptopp, pywin32, zfec, zope.interface as shown on http://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-dep-eggs/README.html .) There, the Solaris row of the matrix is now #1294.

comment:39 Changed at 2013-05-24T15:53:44Z by tilllt

  • Description modified (diff)

The bug still occurred for me on my raspberry pi running Linux 3.6.11+ armv6l and Python 2.7.3 where there is no pycryptoapp binary package. installing the suggested dependencies (python-dev python-openssl) solved this for me.

comment:40 Changed at 2013-05-24T20:15:46Z by zooko

Thanks for the bug report, tilllt. Could you please paste in the output from python misc/build_helpers/show-tool-versions.py? That will tell us what kind of binary someone would need to provide to make it install on your platform without needing the dev packages.

comment:41 Changed at 2013-05-24T20:35:50Z by tilllt

well, i was too quick in assuming the build had worked. actually the python setup.py build command finishes without error, but then tahoe won't run, complaining it cannot find it's dependency pycryptopp >=0.6. which is kind of weird. as even the first half hour of python setup.py trial did not have complains until i stopped it.

this is the output http://pastebin.com/eASzrxzk you asked for. so apparently somehow an outdated pycryptopp was installed along the way.

comment:42 Changed at 2013-05-24T20:41:58Z by tilllt

now i did easy_install for pycryptopp from this page:

https://tahoe-lafs.org/trac/pycryptopp

and now tahoe fails with

Traceback (most recent call last):
  File "/usr/local/bin/tahoe", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2707, in <module>
    working_set.require(__requires__)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 686, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 584, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: Twisted>=11.0.0

so i guess it is a different bug than the others in here had.

Last edited at 2013-05-24T21:08:19Z by zooko (previous) (diff)

comment:43 follow-up: Changed at 2013-05-24T20:50:10Z by tilllt

ok, final update, after running:

sudo easy_install Twisted>=11.0.0

the build helpers look like this (and tahoe runs):

locale.locale: (None, None)

buildbot: no such file or directory
buildslave: no such file or directory
gcc: gcc (Debian 4.6.3-14+rpi1) 4.6.3
g++: g++ (Debian 4.6.3-14+rpi1) 4.6.3
cryptest: no such file or directory
git: git version 1.7.10.4 
openssl: OpenSSL 1.0.1c 10 May 2012 (Library: OpenSSL 1.0.1e 11 Feb 2013) 
darcs: no such file or directory
darcs-exact-version: no such file or directory
flappclient: Foolscap version: 0.6.4 Twisted version: 13.0.0 
valgrind: no such file or directory
lzip: no such file or directory
as: GNU assembler (GNU Binutils for Debian) 2.22 Copyright 2011 Free Software Foundation, Inc. This program is free software; you may redistribute it under the terms of the GNU General Public License version 3 or later. This program has absolutely no warranty. This assembler was configured for a target of `arm-linux-gnueabihf'.  
setuptools: [distribute 0.6.24dev-r0 (/usr/lib/python2.7/dist-packages)]

coverage: DistributionNotFound

trialcoverage: DistributionNotFound

pyflakes: DistributionNotFound

zope.interface: [zope.interface 3.6.1 (/usr/lib/python2.7/dist-packages)]
zope.interface module: <module 'zope.interface' from '/usr/lib/python2.7/dist-packages/zope/interface/__init__.pyc'>

setuptools_darcs: DistributionNotFound

darcsver: DistributionNotFound

Twisted: [Twisted 13.0.0 (/usr/local/lib/python2.7/dist-packages/Twisted-13.0.0-py2.7-linux-armv6l.egg), zope.interface 3.6.1 (/usr/lib/python2.7/dist-packages)]
Twisted module: <module 'twisted' from '/usr/local/lib/python2.7/dist-packages/Twisted-13.0.0-py2.7-linux-armv6l.egg/twisted/__init__.pyc'>
Twisted __version__: 13.0.0

TwistedCore: DistributionNotFound
TwistedCore module: <module 'twisted.python' from '/usr/local/lib/python2.7/dist-packages/Twisted-13.0.0-py2.7-linux-armv6l.egg/twisted/python/__init__.pyc'>

TwistedWeb: DistributionNotFound
TwistedWeb module: <module 'twisted.web' from '/usr/local/lib/python2.7/dist-packages/Twisted-13.0.0-py2.7-linux-armv6l.egg/twisted/web/__init__.pyc'>
TwistedWeb __version__: 13.0.0

TwistedConch: DistributionNotFound
TwistedConch module: <module 'twisted.conch' from '/usr/local/lib/python2.7/dist-packages/Twisted-13.0.0-py2.7-linux-armv6l.egg/twisted/conch/__init__.pyc'>
TwistedConch __version__: 13.0.0

pycryptopp: [pycryptopp 0.6.0.1206569328141510525648634803928199668821045408958 (/usr/local/lib/python2.7/dist-packages/pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-armv6l.egg), distribute 0.6.24dev-r0 (/usr/lib/python2.7/dist-packages)]
pycryptopp module: <module 'pycryptopp' from '/usr/local/lib/python2.7/dist-packages/pycryptopp-0.6.0.1206569328141510525648634803928199668821045408958-py2.7-linux-armv6l.egg/pycryptopp/__init__.pyc'>
pycryptopp __version__: 0.6.0.1206569328141510525648634803928199668821045408958
Last edited at 2013-05-24T21:07:27Z by zooko (previous) (diff)

comment:44 in reply to: ↑ 43 Changed at 2013-05-24T21:09:42Z by zooko

Replying to tilllt:

ok, final update, after running:

sudo easy_install Twisted>=11.0.0

the build helpers look like this (and tahoe runs):

locale.locale: (None, None)

buildbot: no such file or directory

Hey wait a minute. The part of the output that I wanted is what comes before locale.locale and buildbot. The first line should be platform and the second machine. Here's what it looks like on my system:

$ python misc/build_helpers/show-tool-versions.py  |  head
platform: Linux-3.9.3cz9-x86_64-with-LinuxMint-14-nadia
machine:  x86_64
linux_distribution: ('LinuxMint', '14', 'nadia')

python: 2.7.3 (default, Sep 26 2012, 21:51:14)  [GCC 4.7.2]
maxunicode: 1114111

locale: LANG=en_US.UTF-8 LANGUAGE= LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL= 
filesystem.encoding: UTF-8
locale.getpreferredencoding: UTF-8

comment:45 Changed at 2013-05-24T21:22:17Z by tilllt

ah sorry, i am running the raspberry pi headless and had "screen" running in the terminal which cut off the beginning of the output:

platform: Linux-3.6.11+-armv6l-with-debian-7.0
machine:  armv6l
linux_distribution: ('debian', '7.0', '')

python: 2.7.3 (default, Jan 13 2013, 11:20:46)  [GCC 4.6.3]
maxunicode: 1114111

locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
locale: LANG=en_GB.UTF-8 LANGUAGE= LC_CTYPE=UTF-8 LC_NUMERIC="en_GB.UTF-8" LC_TIME="en_GB.UTF-8" LC_COLLATE="en_GB.UTF-8" LC_MONETARY="en_GB.UTF-8" LC_MESSAGES="en_GB.UTF-8" LC_PAPER="en_GB.UTF-8" LC_NAME="en_GB.UTF-8" LC_ADDRESS="en_GB.UTF-8" LC_TELEPHONE="en_GB.UTF-8" LC_MEASUREMENT="en_GB.UTF-8" LC_IDENTIFICATION="en_GB.UTF-8" LC_ALL= 
filesystem.encoding: ANSI_X3.4-1968
locale.getpreferredencoding: ANSI_X3.4-1968
got exception from locale.getdefaultlocale():  unknown locale: UTF-8
locale.locale: (None, None)
Last edited at 2013-05-24T21:30:05Z by zooko (previous) (diff)

comment:46 Changed at 2013-05-24T21:31:39Z by zooko

Cool, thanks! Okay, I think you will be able to install Tahoe-LAFS on Raspberry Pi without compiling anything from source as soon as there is a new row on https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/README.html in the "Python 2.7" table which row is labeled "linux-armv6l" and which has an egg in each of its six columns.

Okay, so which Raspberry Pi owner wants to build these six eggs for us?

comment:47 follow-ups: Changed at 2013-05-25T01:52:19Z by daira

Perhaps we should have a ticket for each platform that needs eggs building for it?

Note that some eggs need newer versions because of these dependency increases for 1.10:

  • Twisted >= 11.0.0
  • pycryptopp >= 0.6.0
  • zope.interface == 3.6.0, == 3.6.1, == 3.6.2, >= 3.6.5

comment:48 in reply to: ↑ 47 Changed at 2013-05-25T02:11:35Z by zooko

Replying to daira:

Perhaps we should have a ticket for each platform that needs eggs building for it?

Good idea. I created #1983.

comment:49 in reply to: ↑ 47 Changed at 2013-05-25T02:24:59Z by daira

Replying to daira:

Perhaps we should have a ticket for each platform that needs eggs building for it?

Filed as #1978 to #1985.

Version 0, edited at 2013-05-25T02:24:59Z by daira (next)

comment:50 Changed at 2013-05-25T02:48:24Z by daira

Now we need to recruit some egg-building volunteers to assign the tickets to.

Note: See TracTickets for help on using tickets.