cloud backend: merge to trunk

[davidsarah]

Rebase the cloud backend branch for landing on trunk, after the v1.10 release.

Depends on #1818.

[davidsarah]

Starting the merge now. It will be done on my github branch at ​https://github.com/davidsarah/tahoe-lafs/commits/1819-cloud-merge.

[Zancas]

Replying to davidsarah:

If I Understand Correctly the proper URL for _cloning_ this repository is now named: ​https://github.com/davidsarah/tahoe-lafs.git

[davidsarah]

Replying to Zancas:

If I Understand Correctly the proper URL for _cloning_ this repository is now named: ​https://github.com/davidsarah/tahoe-lafs.git

That's correct.

[daira]

The branch to look at is now ​https://github.com/LeastAuthority/tahoe-lafs/commits/1819-cloud-merge (at ​https://github.com/LeastAuthority/tahoe-lafs.git ).

[daira]

The following tickets block merging the cloud backend to trunk: blocks-cloud-merge keyword query

[daira]

Preserving these github comments relating to ​b0fc0987 to stop them getting clobbered by a force-push:

zooko: How is this different than setting expire.enabled = false (as described in docs/garbage-collection.rst​)?

daira: expire.enabled controls whether leases expire (that is, whether leases past their expiry timestamp are removed). This setting, which is just intended for debugging, controls whether shares with no leases are removed. Setting expire.enabled to false would not have had any effect on the kind of bug I believed was present in the accounting crawler.

zooko: I see. Thanks. I'd rather we not commit code to trunk which is, like this, solely for debugging.

daira: Hmm. We have lots of code that is just for debugging.

zooko: Well, this patch, I don't feel particularly good about. It makes it harder for someone reading the code, and for little gain. We could use other ways to disable share deletion for debugging, next time we want to do that.

daira: I'm sorry you don't feel good about this patch. I'll probably omit it when rebasing the branch to trunk, but I'd like to keep it on this branch until I've debugged #1921 and confirmed that #1987 is invalid.

zooko: Okay!

[zooko]

I made a branch called ​1819-cloud-merge-opensource which has the same licensing declarations as trunk and which omits the four cloud storage connectors (googlestorage, msazure, openstack, s3), and I updated ​1819-cloud-merge to state its own (non-Free/Open-Source) licensing statement.

I posted a lengthy description of the existence of this code and the licensing situation and LeastAuthority.com's business plans.

[daira]

This branch has been moved to ​https://github.com/tahoe-lafs/tahoe-lafs/commits/1819-cloud-merge. It is now fully open-source.

See the ​blocks-cloud-merge keyword for things remaining to be done.

[daira]

This needs to be rebased because some of the patches on it (for dbutil and deferredutil) were cherry-picked into the #2406 branches for Magic Folder, and later onto master.

[daira]

Rebased to ​https://github.com/tahoe-lafs/tahoe-lafs/commits/1819.cloud-merge.2. The rebase was complicated (it was quite a deep fork) and could have has introduced bugs.

This branch is currently failing with:

allmydata.PackagingError: 
PackagingError: no version info for txAWS == 0.2.1.post5
PackagingError: no version info for oauth2client == 1.1.0

which prevents running the test suite.

[daira]

The dependency problem is fixed, now the test suite fails with some real bugs:

FAILED (skips=13, expectedFailures=4, failures=12, errors=18, successes=1222)

[daira]

I'm so glad we have a decent test suite. Merges like this would be just impossible otherwise.

I'm going to leave it there for now since working on Magic Folder is a higher priority.

[warner]

Milestone renamed

[warner]

renaming milestone

[exarkun]

So far as I can 1819.cloud-merge.2 no longer reflects the current state of this effort. There is an 1819.cloud-merge.3 branch but even that is not current. Instead, ticket:2237 is where all of the action is happening these days.

It seems like ticket:1818 and ticket:1819 were efforts at avoiding an unmergeable mega-branch and that strikes me as a good idea. However, everything seems to have been balled up for ticket:2237 at this point (perhaps due to communication failures, perhaps for other reasons, I don't really know).

There is a plan on ticket:2237 to get this work merged. I think this ticket probably no longer serves much purpose. We will need tickets for the pieces of ticket:2237 which are broken off for merge into master but the shape of those tickets will be dictated entirely by the shape of the pieces of code that can easily be broken off of the mega-branch.

I suggest this be closed.

[exarkun]

Closing. See ticket:2237.