Opened at 2022-12-14T16:59:02Z
Last modified at 2023-01-23T16:44:54Z
#3957 new enhancement
Make HTTP no slower than Foolscap for mutable uploads and downloads
Reported by: | itamarst | Owned by: | |
---|---|---|---|
Priority: | normal | Milestone: | HTTP Storage Protocol v2 |
Component: | unknown | Version: | n/a |
Keywords: | Cc: | ||
Launchpad Bug: |
Change History (4)
comment:1 Changed at 2023-01-06T16:18:43Z by itamarst
comment:2 Changed at 2023-01-20T16:44:16Z by itamarst
Looking at profiling results. 10-15% of CPU time is at least CDDL validation and CBOR decoding. And there's some HTTP parsing. So that's probably a big part of it.
comment:3 Changed at 2023-01-23T13:37:59Z by itamarst
My first attempt to speed up CDDL validation was a failure. As a short-term solution (and probably a good idea anyway) I'm going to move CDDL validation and CBOR parsing into a thread, so at least it doesn't block the event loop (or block it as much). CDDL validation is the bulk of the time at the moment (which is embarassing, but OK) and it will release the GIL. This won't improve latency, but it will help server parallelism: #3968
comment:4 Changed at 2023-01-23T16:44:54Z by itamarst
- Milestone changed from HTTP Storage Protocol to HTTP Storage Protocol v2
For a tens-of-MB mutable SDMF upload, with no network latency, HTTP upload is about twice as slow as Foolscap. Downloads are slower, but not as much. The issue seems to mostly be CPU.
With network latency of 60ms, the difference between HTTP and Foolscap becomes less marked. The latency penalty is not terrible.
Conclusion: As a first pass, for SDMF figure out why HTTP is using more CPU.