host tahoe dependencies

[zooko]

Let's upload 3rd party libraries which are needed to build tahoe to the Tahoe Test Grid, along with an HTML page which easy_install can parse in order to know which files are which. :-)

Things we especially need because the current releases are not yet hosted on pypi:

• pyOpenSSL

Assigning to Chris. :-)

[zooko]

This ticket is linked from ​https://sourceforge.net/tracker2/?func=detail&aid=1799934&group_id=78018&atid=551954 (easy_install silently fails)

[zooko]

Brian said he was uncomfortable with the likelihood of the test grid failing and thus impeding someone from building Tahoe. More discussion is probably warranted. We can always specify more than one source and setuptools will try the next one if one fails. We could also use the allmydata.com production grid. Personally, I'm still happy using the Tahoe test grid -- it seems to be way more available than pypi.python.org...

[zooko]

Here is a test grid directory to which I am uploading binary distributions (eggs produced with ${PYTHON} setup.py bdist_egg) for various platforms:

​http://testgrid.allmydata.org:3567/uri/URI%3ADIR2-RO%3Asnrfwfxatrci35zdgjnzxxx2ke%3Aunarxv347edtku3xzmefy4mcdmfngxzeb72iyqcadbjzjpczjx5a/index.html

[zooko]

Now I just need to upload more of these so that the supported platforms which don't have a C compiler have binary packages. I guess that mostly means Windows.

ndurner on IRC reported a problem using my binary distribution of pycryptopp for python 2.6 for Windows.

[warner]

It sounds like this is not a dependency for 1.3.0, so we can push it out.

[zooko]

Someone (I'm sorry, I forgot who) reported several problems trying to install Tahoe-LAFS on Windows, many of which would have been fixed if he could have gotten the binaries which are hosted on the test grid.

Since there are outstanding issues with the test grid having very slow response times (a failure of availability), what we should do is copy all the files currently stored in ​http://testgrid.allmydata.org:3567/uri/URI%3ADIR2-RO%3Asnrfwfxatrci35zdgjnzxxx2ke%3Aunarxv347edtku3xzmefy4mcdmfngxzeb72iyqcadbjzjpczjx5a/index.html into ​http://allmydata.org/source/tahoe/deps/tahoe-deps .

I hate relying on a boring old apache running on ubuntu instead of an awesome distributed, secure web hosting system, but I guess we need to release v1.5.0 so people can use it, without worrying about the test grid availability issues.

I really hope we fix these outstanding issues of performance/availability soon so that we can start relying on tahoe grids with intermittently connected servers for web hosting. According to setup.cfg​, these issues might include #193, #253, #287, #651, #653.

Okay, so what we should do right now is copy all those files to the apache server and then bump this ticket out of the 1.5.0 Milestone.

[zooko]

I'm changing the name of this ticket to "host tahoe dependencies". The next step is to copy everything from the test grid dir to ​http://allmydata.org . Once that is done, please change the name back to "host tahoe dependencies on a tahoe grid" and bump this ticket out of the 1.5 Milestone!

(Actually I expect to be the person to do this, so the imperative voice there is directed at myself.)

[zooko]

Okay I've downloaded the packages from the Tahoe test grid to ​http://allmydata.org/source/tahoe/deps/tahoe-dep-eggs and 760bab7d2c57cde7 adds that directory to the find_links in setup.cfg​.

Bumping this ticket out of v1.5 Milestone. To close this ticket, find a TahoeLAFS grid that is reliable and fast enough to be a component that builds of TahoeLAFS can rely upon, and upload all those packages to that grid, and change setup.cfg to point to it.

[zooko]

NODA, Kai reported #1012 (Downloading pysqlite takes quite a long time), which would be solved by hosting copies of pysqlite on Tahoe-LAFS and telling our installer not to look at the original project's web site -- look only at our Tahoe-LAFS grid.

[zooko]

This is done, without the "host Tahoe dependencies on LAFS itself" awesomeness. Here's the boring old nginx-on-ubuntu version: ​https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/README.html

[daira]

Replying to zooko:

This is done, without the "host Tahoe dependencies on LAFS itself" awesomeness.

BTW, I never agreed that was awesome, because it is a cyclic dependency.