[tahoe-dev] [tahoe-lafs] #1201: 'rm' via sftp+sshfs

[tahoe-lafs]

#1201: 'rm' via sftp+sshfs
-------------------------+--------------------------------------------------
     Reporter:  bj0      |       Owner:  davidsarah                 
         Type:  defect   |      Status:  assigned                   
     Priority:  major    |   Milestone:  undecided                  
    Component:  unknown  |     Version:  1.8β                       
   Resolution:           |    Keywords:  hang sftp sshfs reliability
Launchpad Bug:           |  
-------------------------+--------------------------------------------------
Changes (by davidsarah):

  * owner:  nobody => davidsarah
  * status:  new => assigned


Comment:

 The most likely explanation is that the .sync on {{{<GeneralSFTPFile
 #7>(/Bonnie.3520)}}} is hanging because some previous operation on that
 file object failed to complete. By design, operations that would be
 synchronous in a POSIX filesystem (like unlink/delete) wait for previous
 operations on the same file, which is necessary to avoid race conditions.
 (Otherwise, create-followed-by-delete could be reordered to delete-
 followed-by-create, which would leave the file in place.)

 When an SFTP client connection is closed, we still may have operations
 that were performed on that connection that need to complete, so we don't
 just drop open handles to a file at that point. However, there's no
 timeout for how long an operation takes, so operations that never complete
 can result in a file becoming "stuck" and undeletable, until the gateway
 is restarted. This seems to be much more likely to happen if the sshfs
 process is killed (even though the server ''should'' be able to tolerate
 that). I'll have to have a good think about how to resolve this; it isn't
 obvious how to do it without risking integrity.

 (The calls to .abandon mark the file handle as being immediately
 closeable, but in this case it isn't closed before the .sync, so that
 doesn't help.)

 It's a bit strange that there are so many copies of the same file object
 in the heisenfile dicts (shown after '{{{files =}}}' in line 6 of the
 log). There should normally be no more than two per open handle to a file.

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1201#comment:1>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage