Changes between Version 3 and Version 4 of Performance/Sep2011


Ignore:
Timestamp:
2011-09-27T03:20:30Z (13 years ago)
Author:
warner
Comment:

fix MDMF tests to include proper 'k' graph

Legend:

Unmodified
Added
Removed
Modified

  • Performance/Sep2011

    v3 v4  
    2424
    2525MDMF is fast! Trunk downloads 1MB/10MB/100MB MDMF files at around 4MBps.
    26 ~~The download speed seems unaffected by k (from 1 to 60)~~. Partial
    27 reads take the expected amount of time: O(data_read), slightly quantized
    28 near the 128KiB segment size.
     26Download speed drops linearly with k, k=60 takes roughly 2x more time
     27than k=30. Partial reads take the expected amount of time: O(data_read),
     28slightly quantized near the 128KiB segment size.
    2929
    30 (update 25-Sep: the test harness appears to have been flawed, and all
    31 MDMF files were actually k=3. A new test is in the works to get proper
    32 data on how MDMF read speed varies with k).
    33 
    34 * MDMF read versus k, 100MB attachment:MDMF-100MB-vs-k.png (timing5.out)
    3530* MDMF partial reads, 100MB attachment:MDMF-100MB-partial.png (timing6.out)
    3631* MDMF partial reads, 1MB attachment:MDMF-1MB-partial.png (timing6.out)
    … …  
    4439WAN links, and it's only the fast LAN connections of the atlasperf1 grid that
    4540exposes the delay). In addition, both old and new downloaders suffer from a
    46 linear slowdown as k increases. On the new-downloader, k=60 takes roughly 3x
    47 more time than k=15. Trunk contains a fix for #1268 that might improve speeds
     41linear slowdown as k increases. On the new-downloader, k=60 takes roughly 2x
     42more time than k=30. Trunk contains a fix for #1268 that might improve speeds
    4843by 5% compared to 1.8.2. Partial reads take the expected amount of time,
    4944although the segsize-quantization was nowhere nearly as clear as with MDMF.
    … …  
    5146
    5247* CHK (1.7.1/1.8.2/trunk) read versus k, 1MB attachment:CHK-1MB-vs-k.png (t4/t/t3)
    53 * CHK (1.7.1/1.8.2/trunk) read versus k, 100MB attachment:CHK-100MB-vs-k.png (t4/t/t3)
     48* CHK (1.7.1/1.8.2/trunk) and MDMF (trunk) read versus k, 100MB attachment:CHKMDMF-100MB-vs-k.png (t4/t/t3/t8)
    5449
    5550* CHK (1.8.2) read versus segsize, 1MB attachment:CHK-1MB-vs-segsize.png (t2)
    … …  
    7469  the memory footprint goals (requires O(numsegments) memory), but probably
    7570  tolerable unless the filesize is really large.
     71 * note that 1.7.1 "cheats" with both the crypttext_hash_tree and the
     72   block_hash_tree. MDMF in trunk cheats with the block_hash_tree and
     73   does not have a crypttext_hash_tree
    7674* encourage use of larger segsize for large files (at the expense of
    7775  alacrity)
    7876* use unencrypted HTTP for share reads
    7977
    80 readv() is the least-work/most-promising, ~~since MDMF has readv() and
    81 achieves high speeds. First step is to do whatever MDMF is doing~~
     78readv() is the least-work/most-promising, since MDMF has readv() and
     79is faster than trunk, although not as fast as 1.7.1.
    8280
    8381== Future Tests ==
    8482
    8583* measure alacrity: ask for random single byte, measure elapsed time
    86 * measure partial-read speeds for CHK
    8784* measure SDMF/MDMF modification times
    8885* measure upload times