#1987 closed defect (invalid)
cloud backend: for a very large upload, the accounting crawler deletes shares before they are leased
| Reported by: | daira | Owned by: | daira |
|---|---|---|---|
| Priority: | major | Milestone: | soon |
| Component: | code-storage | Version: | cloud-branch |
| Keywords: | leasedb accounting-crawler | Cc: | |
| Launchpad Bug: |
Description
I tested uploading a 10 GB (sic) file to the cloud backend on Azure. Before the upload had finished, the accounting crawler ran and started deleting the uploaded chunks. I thought this had been fixed, but it clearly hasn't. (A share is supposed to be considered leased while it is being uploaded, i.e. while it is in STATE_COMING.)
I took a copy of leasedb.sqlite while it was doing this so that I can examine the share state.
Change History (8)
comment:1 Changed at 2013-05-25T22:32:09Z by daira
comment:2 Changed at 2013-05-27T20:27:22Z by zooko
comment:3 Changed at 2013-05-27T21:08:18Z by daira
I would like to understand why the current code is failing, anyway. It may be a symptom of shares being in the wrong state when they are being written, or something similar.
comment:4 Changed at 2013-05-28T01:07:35Z by daira
After examining the logs more closely, I think I misinterpreted the problem. The upload failed because four consecutive HTTP PUT requests to Azure failed (with ConnectionLost TimeoutError exceptions). Then the share chunks were deleted because that is the behaviour coded in BucketWriter._abort.
comment:5 Changed at 2013-05-28T01:09:13Z by daira
I'm retrying the 10 GB upload; if it succeeds this time then I'll reenable share deletion.
comment:6 Changed at 2013-05-30T17:25:54Z by daira
- Resolution set to invalid
- Status changed from new to closed
The 10 GB upload failed but for an unrelated reason (#1991), and uploads up to 2 GB succeeded concurrently with an accounting crawler run.
#1921 may be the same bug as this. I'm not marking them as duplicates because I'm not sure of that yet.