I want an indicator of progress on a backup

[zooko]

When running "tahoe backup" on a directory, I'd like an indicator about approximately how much of the total job is has done and approximately how much longer it will probably take.

[dsoprea]

What's actually blocking this ticket?

I was a little surprised that there wasn't already a mechanism to do this from the beginning, but I'm even more surprised that this [major] ticket has been open for six years and nary a comment. Thoughts?

[exarkun]

One minor improvement that could be made relatively easily here is just to report progress by file count. This isn't the most accurate kind of progress report. However, it would be an incremental step towards something based on actual size of data transfer required. The current implementation doesn't know how many files it is working on, it discovers them as it goes. Reporting on file-count-based progress would require looking at all files in advance - which will also be required for a data-size-based progress report.

I suggest that, to replace the current behavior of no-output, we have something like:

$ tahoe backup src dst
 Found 123 files to back up.
 Backing up 1/123...  0:00:0 elapsed...
 Backing up 2/123...  0:00:7 elapsed...
 .
 .
 .
 123 files uploaded (0 reused), 0 files skipped, 2 directories created (0 reused), 0 
directories skipped
 backup done, elapsed time: 0:00:41    

This serves three purposes. First, it gives the user some feedback some _very_ rough feedback on the size of the job they've just initiated. Second, it gives them some _very_ rough indication of what kind of progress has been made through the job (and that _some_ progress is still being made). Third, it gives them a running report of how much time has been spent on this job up to the current point.

A variation on this could be that the Backing up ... line is updated rather than being re-written for each line. I'm not sure what assumptions the Tahoe-LAFS CLI is happy making about its output destination though. Are basic control bytes are acceptable? If so, an updating status line with \b is easy enough.

I imagine in the future this could be improved to something like:

$ tahoe backup src dst
 Found 456 bytes in 123 files to back up.
 Backing up file 1/123, 20/456 bytes...  0:00:0 elapsed... estimated completion 1:23:45
 .
 .
 .
 123 files uploaded (0 reused), 0 files skipped, 2 directories created (0 reused), 0 
directories skipped
 backup done, elapsed time: 0:00:41    

Perhaps also including a b/s transfer rate. And the "Backing up ..." line could again either be updating or repeating (if updating, some basic tabular formatting might be nice).

Thoughts?

[GitHub <noreply@…>]

In 6ec5005/trunk:

Merge pull request #474 from exarkun/1587.basic-progress-report

Basic progress reporting for tahoe backup

Fixes ticket:1587