Opened at 2022-12-12T16:25:44Z
Closed at 2023-02-23T15:30:54Z
#3952 closed enhancement (fixed)
Benchmarks for basic immutable Tahoe operations (upload and download)
Reported by: | itamarst | Owned by: | |
---|---|---|---|
Priority: | normal | Milestone: | HTTP Storage Protocol |
Component: | unknown | Version: | n/a |
Keywords: | Cc: | ||
Launchpad Bug: |
Description
Motivation:
- We don't want HTTP to be slower than Foolscap.
- It's not clear how fast these operations are in general.
Thus: there should be a way to measure how long these operations take.
First pass will run over loopback, but latency can be added-on on Linux (e.g. https://bencane.com/2012/07/16/tc-adding-simulated-network-latency-to-your-linux-server/).
Change History (6)
comment:1 Changed at 2022-12-12T16:26:00Z by itamarst
- Summary changed from Benchmarks for basic immutable Tahoe operations to Benchmarks for basic immutable Tahoe operations (upload and download)
comment:2 Changed at 2022-12-12T20:01:04Z by itamarst
comment:3 Changed at 2022-12-13T18:19:57Z by itamarst
comment:4 Changed at 2022-12-13T18:42:49Z by itamarst
Changing default segment size to be bigger helps a lot (mostly a workaround).
comment:5 Changed at 2023-01-23T16:51:42Z by itamarst
The problems with increasing default segment size:
- Some code actually relies on it (e.g. https://github.com/tahoe-lafs/tahoe-lafs/blob/master/src/allmydata/immutable/downloader/node.py#L74) though _hopefully_ that's just an optimization. Would need testing of all relevant places that rely on it, at least.
- It doesn't help with existing downloads.
comment:6 Changed at 2023-02-23T15:30:54Z by itamarst
- Resolution set to fixed
- Status changed from new to closed
Note: See
TracTickets for help on using
tickets.
For a small 2000 byte file with a single share uploaded to a single node, an upload is 3 sequential HTTP requests, and a download is 4 sequential HTTP requests. So with high latency this gets quite slow.
Next, I tested HTTP over loopback, but with RTT latency of 60ms (30ms one way) artificially added. Bandwidth is quite high, so not likely to be a bottleneck.
This is all sharing a single CPU core for both client and server, it's one big process.
So at first glance:
I then tested Foolscap with 20MB file. Upload 3,200ms clock time, 500ms CPU time, so HTTP upload is worse but not too bad in comparison. Foolscap download was 20,800ms clock time and 2,100 CPU time. So HTTP download is much worse than Foolscap, but Foolscap downloads are still quite bad.