Some potential issues with GET /v1/immutable/:storage_index

[itamarst]

Downloading multiple shares is specified via the query string.

The current spec gives an example:

GET /v1/immutable/:storage_index?share=:s0&share=:sN&offset=o1&size=z0&offset=oN&size=zN

First, that example is clearly wrong (should be "offset=o0", not "offset=o1".).

Second, it's not made explicit how to match shares, offsets, and sizes. Implicitly it seems like the pattern is:

1. First all the shares.
2. Then, pairs of offset and size, in same order as shares.

This pattern should be explicitly documented.

Third, as part of documenting, it's worth thinking about the pattern in context of HTTP server implementations. HTTP server frameworks often don't preserve order _between_ parameters, only preserving order for multiple values of a single parameter. So e.g. ?share=1&share=2&offset=0&offset=100 will look the same as ?share=1&offset=0&share=2&offset=100 if you're using Twisted Web (or Flask), because you get back a mapping of argument key to list of values.

Fourth, can I request multiple chunks from same share?

Fifth, taking a step back and looking at the big picture, I am not certain that supporting reading from multiple shares (or more broadly, reading multiple chunks) in a single query is actually useful.

1. It's not clear to me it simplifies the client implementation. Clients can just send multiple HTTP requests in parallel.
2. There are Python API design difficulties of receive arbitrarily sized multiple chunks in a single stream.
3. Using single CBOR result streaming isn't really easy to support. Alternative is to concatenate multiple separately CBOR-encoded values (i.e. writing multiple results of cbor2.dumpb or whatever), which CBOR libraries can handle better.
4. Given support for parallel requests, it doesn't improve latency (though it does have minor reduction in bandwidth usage).

[itamarst]

All the above applies to mutables too, I guess.

[exarkun]

​https://github.com/tahoe-lafs/tahoe-lafs/pull/1119

[GitHub <noreply@…>]

In bec813f/trunk:

Merge pull request #1119 from LeastAuthority?/3777.gbs-immutable-download-spec-improvements

GBS immutable download spec improvements

Fixes: ticket:3777