[tahoe-dev] [tahoe-lafs] #1157: new downloader should reuse shares with only partial corruption

[tahoe-lafs]

#1157: new downloader should reuse shares with only partial corruption
---------------------------+------------------------------------------------
 Reporter:  warner         |           Owner:           
     Type:  enhancement    |          Status:  new      
 Priority:  minor          |       Milestone:  undecided
Component:  code-encoding  |         Version:  1.8β     
 Keywords:                 |   Launchpad Bug:           
---------------------------+------------------------------------------------
 during the work on #1154, I was reminded that I made an
 expedient/conservative choice during my recent new-downloader work: once
 we see any corruption in a share, we completely give up on it. It would be
 nice if we could get value out of partially-corrupted shares.
 Specifically, when the block data for e.g. segment0 is corrupted, we
 should still be able to get block data for segment1.

 This change will show up in
 source:src/allmydata/immutable/downloader/fetcher.py#L222 , in
 {{{SegmentFetcher._block_request_activity}}}, in the handling of
 {{{state=CORRUPT}}} (that state will probably go away, or become per-
 segment).

 I plan to defer this work until we get a "sort shares by quality" scheme
 in place, where the general idea is to put the "best" shares (fastest,
 most-data-already-downloaded) at the top of the list, occasionally try out
 new shares for serendipity, and put slow/tardy shares at the bottom. In
 this system, corruption would move a share to the bottom of the list, but
 would not discard it completely.

 There is a test (test_download.py,
 {{{DownloadTest.test_simultaneous_onefails}}}) which will need to be
 updated when this is fixed, since it asserts that two simultaneous
 segment reads (one for a segment that we've intentionally corrupted, the
 other for an uncorrupted segment) results in two failures, whereas once
 we've fixed this it will result in one failure and one success.

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1157>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage