Opened at 2021-08-23T14:11:04Z
Last modified at 2021-09-07T21:52:11Z
#3777 closed task
Some potential issues with GET /v1/immutable/:storage_index — at Version 3
Reported by: | itamarst | Owned by: | exarkun |
---|---|---|---|
Priority: | normal | Milestone: | HTTP Storage Protocol |
Component: | unknown | Version: | n/a |
Keywords: | Cc: | ||
Launchpad Bug: |
Description (last modified by itamarst)
Downloading multiple shares is specified via the query string.
The current spec gives an example:
GET /v1/immutable/:storage_index?share=:s0&share=:sN&offset=o1&size=z0&offset=oN&size=zN
First, that example is clearly wrong (should be "offset=o0", not "offset=o1".).
Second, it's not made explicit how to match shares, offsets, and sizes. Implicitly it seems like the pattern is:
- First all the shares.
- Then, pairs of offset and size, in same order as shares.
This pattern should be explicitly documented.
Third, as part of documenting, it's worth thinking about the pattern in context of HTTP server implementations. HTTP server frameworks often don't preserve order _between_ parameters, only preserving order for multiple values of a single parameter. So e.g. ?share=1&share=2&offset=0&offset=100 will look the same as ?share=1&offset=0&share=2&offset=100 if you're using Twisted Web (or Flask), because you get back a mapping of argument key to list of values.
Fourth, can I request multiple chunks from same share?
Fifth, taking a step back and looking at the big picture, I am not certain that supporting reading from multiple shares (or more broadly, reading multiple chunks) in a single query is actually useful.
- It's not clear to me it simplifies the client implementation. Clients can just send multiple HTTP requests in parallel.
- There are Python API design difficulties of receive arbitrarily sized multiple chunks in a single stream.
- Using single CBOR result streaming isn't really easy to support. Alternative is to concatenate multiple separately CBOR-encoded values (i.e. writing multiple results of cbor2.dumpb or whatever), which CBOR libraries can handle better.
- Given support for parallel requests, it doesn't improve latency (though it does have minor reduction in bandwidth usage).
Change History (3)
comment:1 Changed at 2021-08-23T14:22:42Z by itamarst
- Description modified (diff)
- Milestone changed from undecided to HTTP Storage Protocol
- Owner set to exarkun
- Type changed from defect to task
comment:2 Changed at 2021-08-23T15:27:12Z by itamarst
comment:3 Changed at 2021-08-23T15:27:41Z by itamarst
- Description modified (diff)
All the above applies to mutables too, I guess.