tahoe 'make' fails at twisted, on leopard (macos 10.5)

[robk]

brian thought that this was related to #229

a quick scan of that seems to suggest it's something slightly different.

this is after upgrading my mac to 10.5, and removing the old built-for-10.4 python from my path, and trying to 'make' build tahoe.

Installed /Users/robk/trees/tahoe/support/lib/python2.5/site-packages/setuptools-0.6c8.egg
Searching for Twisted==2.5.0
Reading http://allmydata.org/trac/tahoe/wiki/Dependencies
Reading http://pypi.python.org/simple/Twisted/
Reading http://twistedmatrix.com/
Reading http://www.twistedmatrix.com
Reading http://twistedmatrix.com/products/download
Reading http://twistedmatrix.com/projects/core/
No local packages or download links found for Twisted==2.5.0
error: Could not find suitable distribution for Requirement.parse('Twisted==2.5.0')
make[1]: *** [build-once] Error 1
make: *** [build] Error 2

[robk]

I seem to remember there being something weird about the twisted that ships with 10.5 (which itself ships with python 2.5)

e.g. as far as python is concerned 'twisted' is already installed as v 2.5.0

#[Thu Jun 19 17:15:49] otto:~$ echo $PATH
~/Library/bin:/usr/local/bin:/usr/local/bin:/bin:/sbin:/usr/bin:/usr/sbin
#[Thu Jun 19 17:15:52] otto:~$ python
Python 2.5.1 (r251:54863, Apr 15 2008, 22:57:26) 
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import twisted
>>> from twisted.copyright import version
>>> version
'2.5.0'
>>> ^D

otoh, the notes in #229 suggest asking pkg_resources, which reports 2.4.0

#[Thu Jun 19 17:16:34] otto:~$ python -c 'import pkg_resources;print pkg_resources.require("twisted")'
[Twisted 2.4.0 (/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python)]

neither of which goes any obvious way to explaining why setuptools requirement on 'twisted>=2.4.0' caused it to try and find 2.5.0 to install, given that 2.5.0 is not available in any of the places it looks.

I'm going to try installing the 8.0.1 .pkg installer from the twisted web site.

[zooko]

Could you try this again with the current trunk, please?

Thanks.

[zooko]

The original bug report was against v1.1.0 and the build system has changed a lot since then. Hm, in fact the presence of Twisted 2.5.0 in the bug report and the fact that it is on Mac reminds me that there was a badly packaged copy of Twisted 2.5.0 that came with Mac OS X for a while. I'm going to presumptively close this ticket as "invalid". (Even though I don't think there was anything invalid about the original bug report.)

[zooko]

#1025 is a new report of this problem.

[zooko]

Okay part of this looks a lot like ​http://bugs.python.org/setuptools/issue17 (easy_install will install a package that is already there) because it is detecting the existence of Twisted 2.5.0 and then bogusly attempting to download Twisted == 2.5.0 instead of using the one that it just detected.

Another part of this is probably that Apple shipped a Twisted 2.5.0 with some breakage in it. I searched for a ticket about that but couldn't find it.

A-ha, here is the previous ticket where we worked-around this problem in Tahoe-LAFS: #229. You can follow that ticket to see that the work-around was 5ba922555931d268. We then bundled a toothpick of setuptools in 5c0d937eececedbc which was believed to fix ​http://bugs.python.org/setuptools/issue17 and then removed the workaround in 91a28da3aa2e98c9.

So if josipl could please try that workaround on his instance of the problem and also report the output of:

python -c 'import pkg_resources;print pkg_resources.require("setuptools")'
python -c 'import pkg_resources;print pkg_resources.require("distribute")'
python -c 'import pkg_resources;print pkg_resources.require("Twisted")'

Thanks!

[zooko]

Oh and tahoe --version-and-path once you get Tahoe-LAFS running.

[josipl]

Replying to zooko:

A-ha, here is the previous ticket where we worked-around this problem in Tahoe-LAFS: #229. You can follow that ticket to see that the work-around was 5ba922555931d268. We then bundled a toothpick of setuptools in 5c0d937eececedbc which was believed to fix ​http://bugs.python.org/setuptools/issue17 and then removed the workaround in 91a28da3aa2e98c9.

It looks like adding site-dirs to setup.cfg as in 5ba922555931d268 fixed the problem:

$ python -c 'import pkg_resources;print pkg_resources.require("setuptools")'
[distribute 0.6.8 (/Library/Python/2.5/site-packages/distribute-0.6.8-py2.5.egg)]
$ python -c 'import pkg_resources;print pkg_resources.require("distribute")'
[distribute 0.6.8 (/Library/Python/2.5/site-packages/distribute-0.6.8-py2.5.egg)]
$ python -c 'import pkg_resources;print pkg_resources.require("Twisted")'
[Twisted 2.4.0 (/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python)]

And here's the ​output of tahoe --version-and-path.

[zooko]

Here I'm copying in the output of tahoe --version-and-path from ​http://pastie.org/923921:

allmydata-tahoe: 1.6.1-r4268 (/Library/Python/2.5/site-packages/allmydata_tahoe-1.6.1_r4268-py2.5.egg), foolscap: 0.4.2 (/Library/Python/2.5/site-packages/foolscap-0.4.2-py2.5.egg), pycryptopp: 0.5.17-r683 (/Library/Python/2.5/site-packages/pycryptopp-0.5.17_r683-py2.5-macosx-10.5-i386.egg), zfec: 1.4.5 (/Library/Python/2.5/site-packages/zfec-1.4.5-py2.5-macosx-10.5-i386.egg), Twisted: 2.4.0 (/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python), Nevow: 0.9.33-rUnknown (/Library/Python/2.5/site-packages/Nevow-0.9.33_rUnknown-py2.5.egg), zope.interface: 3.3.0 (/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python), python: 2.5.1 (/System/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python), platform: Darwin-9.8.0-i386-32bit (None), windmill: 1.3 (/Library/Python/2.5/site-packages/windmill-1.3-py2.5.egg), sqlite: 3.4.0 (unknown), simplejson: 2.0.9 (/Library/Python/2.5/site-packages/simplejson-2.0.9-py2.5-macosx-10.5-i386.egg), distribute: 0.6.8 (/Library/Python/2.5/site-packages/distribute-0.6.8-py2.5.egg), argparse: 0.9.1 (/Library/Python/2.5/site-packages/argparse-0.9.1-py2.5.egg), pyOpenSSL: 0.6 (/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python), pyutil: 1.3.34 (/Library/Python/2.5/site-packages/pyutil-1.3.34-py2.5.egg), zbase32: 1.1.1 (/Library/Python/2.5/site-packages/zbase32-1.1.1-py2.5.egg), setuptools: 0.6 (/Library/Python/2.5/site-packages/distribute-0.6.8-py2.5.egg/setuptools), pysqlite: 2.3.2 (/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/sqlite3)

[zooko]

Okay I'll look at this issue. I'll probably need to ask Josip some more questions or ask for some more experiments when I do.

[zooko]

I believe this was due to ​http://bugs.python.org/setuptools/issue17 , which was fixed in setuptools, but then regressed in distribute: ​https://bitbucket.org/tarek/distribute/issue/142/easy_install-will-install-a-package-that-is-already , but then was fixed in distribute. Therefore, we can close this as fixed.