1 | Sun May 30 22:44:07 Paris, Madrid (heure d'été) 2010 freestorm77@gmail.com |
---|
2 | * munin_stats-gatherer.darcs.txt |
---|
3 | |
---|
4 | - Move stats-gatherer plugins to munin/stats-gatherer_plugins |
---|
5 | - Create munin/stats-gatherer_plugins/README file |
---|
6 | - Remove munin/tahoe-stats.plugin-conf |
---|
7 | - Create tahoe-stats_conf/tahoe-stats_conf |
---|
8 | |
---|
9 | |
---|
10 | New patches: |
---|
11 | |
---|
12 | [munin_stats-gatherer.darcs.txt |
---|
13 | freestorm77@gmail.com**20100530204407 |
---|
14 | Ignore-this: f65230719c8a0146d4c9de83870485b4 |
---|
15 | |
---|
16 | - Move stats-gatherer plugins to munin/stats-gatherer_plugins |
---|
17 | - Create munin/stats-gatherer_plugins/README file |
---|
18 | - Remove munin/tahoe-stats.plugin-conf |
---|
19 | - Create tahoe-stats_conf/tahoe-stats_conf |
---|
20 | |
---|
21 | ] { |
---|
22 | adddir ./misc/munin/stats-gatherer_plugins |
---|
23 | move ./misc/munin/tahoe_stats ./misc/munin/stats-gatherer_plugins/tahoe_stats |
---|
24 | addfile ./misc/munin/stats-gatherer_plugins/REAME |
---|
25 | hunk ./misc/munin/stats-gatherer_plugins/REAME 1 |
---|
26 | - |
---|
27 | += Tahoe-LAFS Stats-Gatherer plugins = |
---|
28 | + |
---|
29 | +== Helper plugins == |
---|
30 | + |
---|
31 | +These plugins need a Tahoe-LAFS Helper |
---|
32 | + |
---|
33 | +chk_upload_helper.encoded_bytes: Shows the number of bytes encoded by the helper |
---|
34 | +chk_upload_helper.upload_need_upload: Shows the number of uploads whose files are not already present in the grid |
---|
35 | +tahoe_helper_active_uploads: Shows the number of files actively being processed by the helper |
---|
36 | +tahoe_helper_encoded_bytes: Shows the number of bytes encoded by the helper |
---|
37 | +tahoe_helper_encoding_files: Shows the number of encoding files |
---|
38 | +tahoe_helper_encoding_filesize: Shows total size of encoding files |
---|
39 | +tahoe_helper_encoding_files_old: Shows total size of old encoding files |
---|
40 | +tahoe_helper_fetched_bytes: Shows the number of bytes fetched by the helper |
---|
41 | +tahoe_helper_incoming_files: Shows the number of incoming files |
---|
42 | +tahoe_helper_incoming_filesize: Shows total size of incoming files |
---|
43 | +tahoe_helper_incoming_files_old: Shows total size of old incoming files |
---|
44 | +tahoe_helper_upload_already_present: Shows the number of uploads whose files are already present in the grid |
---|
45 | +tahoe_helper_upload_need_upload: Shows the number of uploads whose files are not already present in the grid |
---|
46 | +tahoe_helper_upload_requests: Shows the number of upload requests arriving at the helper |
---|
47 | + |
---|
48 | +== Traffic plugins == |
---|
49 | + |
---|
50 | +tahoe_mutable_files_published: Shows the number of mutable files published |
---|
51 | +tahoe_mutable_files_retrieved: Shows the number of files retrieved |
---|
52 | +tahoe_uploader_bytes_uploaded: Shows the number of bytes uploaded |
---|
53 | +tahoe_uploader_files_uploaded: Shows the number of files uploaded |
---|
54 | + |
---|
55 | +== Storage plugins == |
---|
56 | + |
---|
57 | +tahoe_storage_allocated: Shows space allocated |
---|
58 | +tahoe_storage_bytes_added: Shows cummulative bytes added |
---|
59 | +tahoe_storage_bytes_freed: Shows cummulative bytes removed |
---|
60 | +tahoe_storage_consumed: Shows space consumed |
---|
61 | +tahoe_storage_operations_allocate: Shows how many allocate_buckets operations occured per second. |
---|
62 | + Each immutable file upload causes one such operation per server |
---|
63 | + |
---|
64 | +tahoe_storage_operations_get: This graph shows how many get_bucket operations occured per second. |
---|
65 | + Each immutable file download/check causes one such operation per server |
---|
66 | + |
---|
67 | +tahoe_storage_operations_readv: Shows how many readv operations occured per second. |
---|
68 | + Each dirnode read causes one such operation per server. |
---|
69 | + |
---|
70 | +tahoe_storage_operations_writev: Shows how many writev operations occured per second. |
---|
71 | + Each mutable file/dirnode write causes one such operation per server |
---|
72 | + |
---|
73 | +== Load plugins == |
---|
74 | + |
---|
75 | +tahoe_runtime_load_peak: Shows peak reactor delay |
---|
76 | +tahoe_runtime_load_avg: Shows average reactor delay |
---|
77 | + |
---|
78 | += Installation = |
---|
79 | + |
---|
80 | +This file need to be renamed/copied/linked to the plugin name you want. |
---|
81 | +you need to do this for each graph you want. |
---|
82 | + |
---|
83 | +For example: |
---|
84 | +If you want to show the "Tahoe-LAFS Runtime Load Average Graph": |
---|
85 | +Just copy tahoe_stats to '/etc/munin/plugins/tahoe_stats_runtime_load_avg' |
---|
86 | + |
---|
87 | +If you want all graph you can run theses command: |
---|
88 | + cp tahoe_stats /etc/munin/plugins/tahoe_stats.bak |
---|
89 | + # the .bak extension is needed, otherwise Munin-Node will try to run it. |
---|
90 | + cd /etc/munin/plugins/ |
---|
91 | + chmod +x tahoe_stats.bak |
---|
92 | + for FILE in `grep "'tahoe_" tahoe_stats.bak | cut -d\' -f2`; do ln -s tahoe_stats.bak $FILE;done; |
---|
93 | + |
---|
94 | + After edit the configuration file "tahoe-stats_conf" and copy it to /etc/munin/plugin-conf.d/ |
---|
95 | + Reload the Munin-Node process: /etc/init.d/munin-node reload |
---|
96 | + |
---|
97 | + |
---|
98 | +Note: |
---|
99 | +You need to wait 5min before Munin-Node perform graphs |
---|
100 | addfile ./misc/munin/stats-gatherer_plugins/tahoe-stats_conf |
---|
101 | hunk ./misc/munin/stats-gatherer_plugins/tahoe-stats_conf 1 |
---|
102 | +# |
---|
103 | +# This file contains configuration options for all Tahoe Stats-Gatherer plugins. |
---|
104 | +# |
---|
105 | +# You need to copy this file in your /etc/munin/plugin-conf.d/ directory |
---|
106 | +# |
---|
107 | +# Three options are understood by munin-node itself: |
---|
108 | +# |
---|
109 | +# user <user> # Set the user to run the plugin as. (not mandatory) |
---|
110 | +# group <group> # Set the group to run the plugin as. (not mandatory) |
---|
111 | +# env.statsfile # Set your Tahoe Stats-Gatherer pickle file path |
---|
112 | +# |
---|
113 | + |
---|
114 | +[tahoe_stats*] |
---|
115 | +user root |
---|
116 | +group root |
---|
117 | +env.statsfile /home/tahoe-stats-gatherer/stats.pickle |
---|
118 | + |
---|
119 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 13 |
---|
120 | |
---|
121 | PLUGINS = { |
---|
122 | # LOAD AVERAGE |
---|
123 | - 'tahoe_runtime_load_avg': |
---|
124 | + 'tahoe_stats_runtime_load_avg': |
---|
125 | { 'statid': 'load_monitor.avg_load', |
---|
126 | 'category': 'stats', |
---|
127 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 16 |
---|
128 | - 'configheader': '\n'.join(['graph_title Tahoe Runtime Load Average', |
---|
129 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Runtime Load Average', |
---|
130 | 'graph_vlabel load', |
---|
131 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 18 |
---|
132 | - 'graph_category tahoe', |
---|
133 | + 'graph_category Tahoe-LAFS Stats-Gatherer', |
---|
134 | 'graph_info This graph shows average reactor delay', |
---|
135 | ]), |
---|
136 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
137 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 28 |
---|
138 | ]), |
---|
139 | }, |
---|
140 | |
---|
141 | - 'tahoe_runtime_load_peak': |
---|
142 | + 'tahoe_stats_runtime_load_peak': |
---|
143 | { 'statid': 'load_monitor.max_load', |
---|
144 | 'category': 'stats', |
---|
145 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 31 |
---|
146 | - 'configheader': '\n'.join(['graph_title Tahoe Runtime Load Peak', |
---|
147 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Runtime Load Peak', |
---|
148 | 'graph_vlabel load', |
---|
149 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 33 |
---|
150 | - 'graph_category tahoe', |
---|
151 | + 'graph_category Tahoe-LAFS Stats-Gatherer', |
---|
152 | 'graph_info This graph shows peak reactor delay', |
---|
153 | ]), |
---|
154 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
155 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 44 |
---|
156 | }, |
---|
157 | |
---|
158 | # STORAGE ALLOCATION (BYTES) |
---|
159 | - 'tahoe_storage_consumed': |
---|
160 | + 'tahoe_stats_storage_consumed': |
---|
161 | { 'statid': 'storage_server.consumed', |
---|
162 | 'category': 'stats', |
---|
163 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 47 |
---|
164 | - 'configheader': '\n'.join(['graph_title Tahoe Storage Server Space Consumed', |
---|
165 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Storage Server Space Consumed', |
---|
166 | 'graph_vlabel bytes', |
---|
167 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 49 |
---|
168 | - 'graph_category tahoe_storage_server', |
---|
169 | + 'graph_category Tahoe-LAFS Stats-Gatherer_storage_server', |
---|
170 | 'graph_info This graph shows space consumed', |
---|
171 | 'graph_args --base 1024', |
---|
172 | ]), |
---|
173 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 60 |
---|
174 | ]), |
---|
175 | }, |
---|
176 | |
---|
177 | - 'tahoe_storage_allocated': |
---|
178 | + 'tahoe_stats_storage_allocated': |
---|
179 | { 'statid': 'storage_server.allocated', |
---|
180 | 'category': 'stats', |
---|
181 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 63 |
---|
182 | - 'configheader': '\n'.join(['graph_title Tahoe Storage Server Space Allocated', |
---|
183 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Storage Server Space Allocated', |
---|
184 | 'graph_vlabel bytes', |
---|
185 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 65 |
---|
186 | - 'graph_category tahoe_storage_server', |
---|
187 | + 'graph_category Tahoe-LAFS Stats-Gatherer_storage_server', |
---|
188 | 'graph_info This graph shows space allocated', |
---|
189 | 'graph_args --base 1024', |
---|
190 | ]), |
---|
191 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 76 |
---|
192 | ]), |
---|
193 | }, |
---|
194 | |
---|
195 | - 'tahoe_storage_bytes_added': |
---|
196 | + 'tahoe_stats_storage_bytes_added': |
---|
197 | { 'statid': 'storage_server.bytes_added', |
---|
198 | 'category': 'counters', |
---|
199 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 79 |
---|
200 | - 'configheader': '\n'.join(['graph_title Tahoe Storage Server Bytes Added', |
---|
201 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Storage Server Bytes Added', |
---|
202 | 'graph_vlabel bytes', |
---|
203 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 81 |
---|
204 | - 'graph_category tahoe_storage_server', |
---|
205 | + 'graph_category Tahoe-LAFS Stats-Gatherer_storage_server', |
---|
206 | 'graph_info This graph shows cummulative bytes added', |
---|
207 | ]), |
---|
208 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
209 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 91 |
---|
210 | ]), |
---|
211 | }, |
---|
212 | |
---|
213 | - 'tahoe_storage_bytes_freed': |
---|
214 | + 'tahoe_stats_storage_bytes_freed': |
---|
215 | { 'statid': 'storage_server.bytes_freed', |
---|
216 | 'category': 'counters', |
---|
217 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 94 |
---|
218 | - 'configheader': '\n'.join(['graph_title Tahoe Storage Server Bytes Removed', |
---|
219 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Storage Server Bytes Removed', |
---|
220 | 'graph_vlabel bytes', |
---|
221 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 96 |
---|
222 | - 'graph_category tahoe_storage_server', |
---|
223 | + 'graph_category Tahoe-LAFS Stats-Gatherer_storage_server', |
---|
224 | 'graph_info This graph shows cummulative bytes removed', |
---|
225 | ]), |
---|
226 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
227 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 106 |
---|
228 | ]), |
---|
229 | }, |
---|
230 | |
---|
231 | - 'tahoe_storage_operations_allocate': |
---|
232 | + 'tahoe_stats_storage_operations_allocate': |
---|
233 | { 'statid': 'storage_server.allocate', |
---|
234 | 'category': 'counters', |
---|
235 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 109 |
---|
236 | - 'configheader': '\n'.join(['graph_title Tahoe Storage Server Allocate_Bucket Operations', |
---|
237 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Storage Server Allocate_Bucket Operations', |
---|
238 | 'graph_vlabel operations per second', |
---|
239 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 111 |
---|
240 | - 'graph_category tahoe_storage_server', |
---|
241 | + 'graph_category Tahoe-LAFS Stats-Gatherer_storage_server', |
---|
242 | 'graph_info This graph shows how many allocate_buckets operations occured per second. Each immutable file upload causes one such operation per server.', |
---|
243 | ]), |
---|
244 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
245 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 123 |
---|
246 | ]), |
---|
247 | }, |
---|
248 | |
---|
249 | - 'tahoe_storage_operations_get': |
---|
250 | + 'tahoe_stats_storage_operations_get': |
---|
251 | { 'statid': 'storage_server.get', |
---|
252 | 'category': 'counters', |
---|
253 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 126 |
---|
254 | - 'configheader': '\n'.join(['graph_title Tahoe Storage Server get_bucket Operations', |
---|
255 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Storage Server get_bucket Operations', |
---|
256 | 'graph_vlabel operations per second', |
---|
257 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 128 |
---|
258 | - 'graph_category tahoe_storage_server', |
---|
259 | + 'graph_category Tahoe-LAFS Stats-Gatherer_storage_server', |
---|
260 | 'graph_info This graph shows how many get_bucket operations occured per second. Each immutable file download/check causes one such operation per server.', |
---|
261 | ]), |
---|
262 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
263 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 140 |
---|
264 | ]), |
---|
265 | }, |
---|
266 | |
---|
267 | - 'tahoe_storage_operations_writev': |
---|
268 | + 'tahoe_stats_storage_operations_writev': |
---|
269 | { 'statid': 'storage_server.writev', |
---|
270 | 'category': 'counters', |
---|
271 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 143 |
---|
272 | - 'configheader': '\n'.join(['graph_title Tahoe Storage Server writev Operations', |
---|
273 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Storage Server writev Operations', |
---|
274 | 'graph_vlabel operations per second', |
---|
275 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 145 |
---|
276 | - 'graph_category tahoe_storage_server', |
---|
277 | - 'graph_info This graph shows how many writev operations occured per second. Each mutable file / dirnode write causes one such operation per server.', |
---|
278 | + 'graph_category Tahoe-LAFS Stats-Gatherer_storage_server', |
---|
279 | + 'graph_info This graph shows how many writev operations occured per second. Each mutable file/dirnode write causes one such operation per server.', |
---|
280 | ]), |
---|
281 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
282 | '%(name)s.type DERIVE', |
---|
283 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 157 |
---|
284 | ]), |
---|
285 | }, |
---|
286 | |
---|
287 | - 'tahoe_storage_operations_readv': |
---|
288 | + 'tahoe_stats_storage_operations_readv': |
---|
289 | { 'statid': 'storage_server.readv', |
---|
290 | 'category': 'counters', |
---|
291 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 160 |
---|
292 | - 'configheader': '\n'.join(['graph_title Tahoe Storage Server readv Operations', |
---|
293 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Storage Server readv Operations', |
---|
294 | 'graph_vlabel operations per second', |
---|
295 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 162 |
---|
296 | - 'graph_category tahoe_storage_server', |
---|
297 | + 'graph_category Tahoe-LAFS Stats-Gatherer_storage_server', |
---|
298 | 'graph_info This graph shows how many readv operations occured per second. Each dirnode read causes one such operation per server.', |
---|
299 | ]), |
---|
300 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
301 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 176 |
---|
302 | |
---|
303 | |
---|
304 | # HELPER |
---|
305 | - 'tahoe_helper_incoming_files': |
---|
306 | + 'tahoe_stats_helper_incoming_files': |
---|
307 | { 'statid': 'chk_upload_helper.incoming_count', |
---|
308 | 'category': 'stats', |
---|
309 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 179 |
---|
310 | - 'configheader': '\n'.join(['graph_title Tahoe Upload Helper Incoming File Count', |
---|
311 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Upload Helper Incoming File Count', |
---|
312 | 'graph_vlabel n files', |
---|
313 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 181 |
---|
314 | - 'graph_category tahoe_helper', |
---|
315 | - 'graph_info This graph shows number of incoming files', |
---|
316 | + 'graph_category Tahoe-LAFS Stats-Gatherer_helper', |
---|
317 | + 'graph_info This graph shows the number of incoming files', |
---|
318 | ]), |
---|
319 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
320 | '%(name)s.draw LINE1', |
---|
321 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 190 |
---|
322 | 'graph_render': '\n'.join(['%(name)s.value %(value)s', |
---|
323 | ]), |
---|
324 | }, |
---|
325 | - 'tahoe_helper_incoming_filesize': |
---|
326 | + 'tahoe_stats_helper_incoming_filesize': |
---|
327 | { 'statid': 'chk_upload_helper.incoming_size', |
---|
328 | 'category': 'stats', |
---|
329 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 193 |
---|
330 | - 'configheader': '\n'.join(['graph_title Tahoe Upload Helper Incoming File Size', |
---|
331 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Upload Helper Incoming File Size', |
---|
332 | 'graph_vlabel bytes', |
---|
333 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 195 |
---|
334 | - 'graph_category tahoe_helper', |
---|
335 | + 'graph_category Tahoe-LAFS Stats-Gatherer_helper', |
---|
336 | 'graph_info This graph shows total size of incoming files', |
---|
337 | ]), |
---|
338 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
339 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 204 |
---|
340 | 'graph_render': '\n'.join(['%(name)s.value %(value)s', |
---|
341 | ]), |
---|
342 | }, |
---|
343 | - 'tahoe_helper_incoming_files_old': |
---|
344 | + 'tahoe_stats_helper_incoming_files_old': |
---|
345 | { 'statid': 'chk_upload_helper.incoming_size_old', |
---|
346 | 'category': 'stats', |
---|
347 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 207 |
---|
348 | - 'configheader': '\n'.join(['graph_title Tahoe Upload Helper Incoming Old Files', |
---|
349 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Upload Helper Incoming Old Files', |
---|
350 | 'graph_vlabel bytes', |
---|
351 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 209 |
---|
352 | - 'graph_category tahoe_helper', |
---|
353 | + 'graph_category Tahoe-LAFS Stats-Gatherer_helper', |
---|
354 | 'graph_info This graph shows total size of old incoming files', |
---|
355 | ]), |
---|
356 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
357 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 219 |
---|
358 | ]), |
---|
359 | }, |
---|
360 | |
---|
361 | - 'tahoe_helper_encoding_files': |
---|
362 | + 'tahoe_stats_helper_encoding_files': |
---|
363 | { 'statid': 'chk_upload_helper.encoding_count', |
---|
364 | 'category': 'stats', |
---|
365 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 222 |
---|
366 | - 'configheader': '\n'.join(['graph_title Tahoe Upload Helper Encoding File Count', |
---|
367 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Upload Helper Encoding File Count', |
---|
368 | 'graph_vlabel n files', |
---|
369 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 224 |
---|
370 | - 'graph_category tahoe_helper', |
---|
371 | - 'graph_info This graph shows number of encoding files', |
---|
372 | + 'graph_category Tahoe-LAFS Stats-Gatherer_helper', |
---|
373 | + 'graph_info This graph shows the number of encoding files', |
---|
374 | ]), |
---|
375 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
376 | '%(name)s.draw LINE1', |
---|
377 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 233 |
---|
378 | 'graph_render': '\n'.join(['%(name)s.value %(value)s', |
---|
379 | ]), |
---|
380 | }, |
---|
381 | - 'tahoe_helper_encoding_filesize': |
---|
382 | + 'tahoe_stats_helper_encoding_filesize': |
---|
383 | { 'statid': 'chk_upload_helper.encoding_size', |
---|
384 | 'category': 'stats', |
---|
385 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 236 |
---|
386 | - 'configheader': '\n'.join(['graph_title Tahoe Upload Helper Encoding File Size', |
---|
387 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Upload Helper Encoding File Size', |
---|
388 | 'graph_vlabel bytes', |
---|
389 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 238 |
---|
390 | - 'graph_category tahoe_helper', |
---|
391 | + 'graph_category Tahoe-LAFS Stats-Gatherer_helper', |
---|
392 | 'graph_info This graph shows total size of encoding files', |
---|
393 | ]), |
---|
394 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
395 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 247 |
---|
396 | 'graph_render': '\n'.join(['%(name)s.value %(value)s', |
---|
397 | ]), |
---|
398 | }, |
---|
399 | - 'tahoe_helper_encoding_files_old': |
---|
400 | + 'tahoe_stats_helper_encoding_files_old': |
---|
401 | { 'statid': 'chk_upload_helper.encoding_size_old', |
---|
402 | 'category': 'stats', |
---|
403 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 250 |
---|
404 | - 'configheader': '\n'.join(['graph_title Tahoe Upload Helper Encoding Old Files', |
---|
405 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Upload Helper Encoding Old Files', |
---|
406 | 'graph_vlabel bytes', |
---|
407 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 252 |
---|
408 | - 'graph_category tahoe_helper', |
---|
409 | + 'graph_category Tahoe-LAFS Stats-Gatherer_helper', |
---|
410 | 'graph_info This graph shows total size of old encoding files', |
---|
411 | ]), |
---|
412 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
413 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 262 |
---|
414 | ]), |
---|
415 | }, |
---|
416 | |
---|
417 | - 'tahoe_helper_active_uploads': |
---|
418 | + 'tahoe_stats_helper_active_uploads': |
---|
419 | { 'statid': 'chk_upload_helper.active_uploads', |
---|
420 | 'category': 'stats', |
---|
421 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 265 |
---|
422 | - 'configheader': '\n'.join(['graph_title Tahoe Upload Helper Active Files', |
---|
423 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Upload Helper Active Files', |
---|
424 | 'graph_vlabel n files', |
---|
425 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 267 |
---|
426 | - 'graph_category tahoe_helper', |
---|
427 | - 'graph_info This graph shows number of files actively being processed by the helper', |
---|
428 | + 'graph_category Tahoe-LAFS Stats-Gatherer_helper', |
---|
429 | + 'graph_info This graph shows the number of files actively being processed by the helper', |
---|
430 | ]), |
---|
431 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
432 | '%(name)s.draw LINE1', |
---|
433 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 277 |
---|
434 | ]), |
---|
435 | }, |
---|
436 | |
---|
437 | - 'tahoe_helper_upload_requests': |
---|
438 | + 'tahoe_stats_helper_upload_requests': |
---|
439 | { 'statid': 'chk_upload_helper.upload_requests', |
---|
440 | 'category': 'counters', |
---|
441 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 280 |
---|
442 | - 'configheader': '\n'.join(['graph_title Tahoe Upload Helper Upload Requests', |
---|
443 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Upload Helper Upload Requests', |
---|
444 | 'graph_vlabel requests', |
---|
445 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 282 |
---|
446 | - 'graph_category tahoe_helper', |
---|
447 | + 'graph_category Tahoe-LAFS Stats-Gatherer_helper', |
---|
448 | 'graph_info This graph shows the number of upload requests arriving at the helper', |
---|
449 | ]), |
---|
450 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
451 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 293 |
---|
452 | 'graph_render': '\n'.join(['%(name)s.value %(value)s', |
---|
453 | ]), |
---|
454 | }, |
---|
455 | - 'tahoe_helper_upload_already_present': |
---|
456 | + 'tahoe_stats_helper_upload_already_present': |
---|
457 | { 'statid': 'chk_upload_helper.upload_already_present', |
---|
458 | 'category': 'counters', |
---|
459 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 296 |
---|
460 | - 'configheader': '\n'.join(['graph_title Tahoe Upload Helper Uploads Already Present', |
---|
461 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Upload Helper Uploads Already Present', |
---|
462 | 'graph_vlabel requests', |
---|
463 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 298 |
---|
464 | - 'graph_category tahoe_helper', |
---|
465 | + 'graph_category Tahoe-LAFS Stats-Gatherer_helper', |
---|
466 | 'graph_info This graph shows the number of uploads whose files are already present in the grid', |
---|
467 | ]), |
---|
468 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
469 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 309 |
---|
470 | 'graph_render': '\n'.join(['%(name)s.value %(value)s', |
---|
471 | ]), |
---|
472 | }, |
---|
473 | - 'tahoe_helper_upload_need_upload': |
---|
474 | + 'tahoe_stats_helper_upload_need_upload': |
---|
475 | { 'statid': 'chk_upload_helper.upload_need_upload', |
---|
476 | 'category': 'counters', |
---|
477 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 312 |
---|
478 | - 'configheader': '\n'.join(['graph_title Tahoe Upload Helper Uploads Needing Upload', |
---|
479 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Upload Helper Uploads Needing Upload', |
---|
480 | 'graph_vlabel requests', |
---|
481 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 314 |
---|
482 | - 'graph_category tahoe_helper', |
---|
483 | + 'graph_category Tahoe-LAFS Stats-Gatherer_helper', |
---|
484 | 'graph_info This graph shows the number of uploads whose files are not already present in the grid', |
---|
485 | ]), |
---|
486 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
487 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 325 |
---|
488 | 'graph_render': '\n'.join(['%(name)s.value %(value)s', |
---|
489 | ]), |
---|
490 | }, |
---|
491 | - 'tahoe_helper_encoded_bytes': |
---|
492 | + 'tahoe_stats_helper_encoded_bytes': |
---|
493 | { 'statid': 'chk_upload_helper.encoded_bytes', |
---|
494 | 'category': 'counters', |
---|
495 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 328 |
---|
496 | - 'configheader': '\n'.join(['graph_title Tahoe Upload Helper Encoded Bytes', |
---|
497 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Upload Helper Encoded Bytes', |
---|
498 | 'graph_vlabel bytes', |
---|
499 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 330 |
---|
500 | - 'graph_category tahoe_helper', |
---|
501 | + 'graph_category Tahoe-LAFS Stats-Gatherer_helper', |
---|
502 | 'graph_info This graph shows the number of bytes encoded by the helper', |
---|
503 | ]), |
---|
504 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
505 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 341 |
---|
506 | 'graph_render': '\n'.join(['%(name)s.value %(value)s', |
---|
507 | ]), |
---|
508 | }, |
---|
509 | - 'tahoe_helper_fetched_bytes': |
---|
510 | + 'tahoe_stats_helper_fetched_bytes': |
---|
511 | { 'statid': 'chk_upload_helper.fetched_bytes', |
---|
512 | 'category': 'counters', |
---|
513 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 344 |
---|
514 | - 'configheader': '\n'.join(['graph_title Tahoe Upload Helper Fetched Bytes', |
---|
515 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Upload Helper Fetched Bytes', |
---|
516 | 'graph_vlabel bytes', |
---|
517 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 346 |
---|
518 | - 'graph_category tahoe_helper', |
---|
519 | + 'graph_category Tahoe-LAFS Stats-Gatherer_helper', |
---|
520 | 'graph_info This graph shows the number of bytes fetched by the helper', |
---|
521 | ]), |
---|
522 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
523 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 359 |
---|
524 | }, |
---|
525 | |
---|
526 | # WEBAPI |
---|
527 | - 'tahoe_uploader_bytes_uploaded': |
---|
528 | + 'tahoe_stats_uploader_bytes_uploaded': |
---|
529 | { 'statid': 'uploader.bytes_uploaded', |
---|
530 | 'category': 'counters', |
---|
531 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 362 |
---|
532 | - 'configheader': '\n'.join(['graph_title Tahoe Uploader Bytes Uploaded', |
---|
533 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Uploader Bytes Uploaded', |
---|
534 | 'graph_vlabel bytes', |
---|
535 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 364 |
---|
536 | - 'graph_category tahoe_traffic', |
---|
537 | + 'graph_category Tahoe-LAFS Stats-Gatherer_traffic', |
---|
538 | 'graph_info This graph shows the number of bytes uploaded', |
---|
539 | ]), |
---|
540 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
541 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 375 |
---|
542 | 'graph_render': '\n'.join(['%(name)s.value %(value)s', |
---|
543 | ]), |
---|
544 | }, |
---|
545 | - 'tahoe_uploader_files_uploaded': |
---|
546 | + 'tahoe_stats_uploader_files_uploaded': |
---|
547 | { 'statid': 'uploader.files_uploaded', |
---|
548 | 'category': 'counters', |
---|
549 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 378 |
---|
550 | - 'configheader': '\n'.join(['graph_title Tahoe Uploader Bytes Uploaded', |
---|
551 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Uploader Files Uploaded', |
---|
552 | 'graph_vlabel files', |
---|
553 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 380 |
---|
554 | - 'graph_category tahoe_traffic', |
---|
555 | + 'graph_category Tahoe-LAFS Stats-Gatherer_traffic', |
---|
556 | 'graph_info This graph shows the number of files uploaded', |
---|
557 | ]), |
---|
558 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
559 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 391 |
---|
560 | 'graph_render': '\n'.join(['%(name)s.value %(value)s', |
---|
561 | ]), |
---|
562 | }, |
---|
563 | - 'tahoe_mutable_files_published': |
---|
564 | + 'tahoe_stats_mutable_files_published': |
---|
565 | { 'statid': 'mutable.files_published', |
---|
566 | 'category': 'counters', |
---|
567 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 394 |
---|
568 | - 'configheader': '\n'.join(['graph_title Tahoe Mutable Files Published', |
---|
569 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Mutable Files Published', |
---|
570 | 'graph_vlabel files', |
---|
571 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 396 |
---|
572 | - 'graph_category tahoe_traffic', |
---|
573 | + 'graph_category Tahoe-LAFS Stats-Gatherer_traffic', |
---|
574 | 'graph_info This graph shows the number of mutable files published', |
---|
575 | ]), |
---|
576 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
577 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 407 |
---|
578 | 'graph_render': '\n'.join(['%(name)s.value %(value)s', |
---|
579 | ]), |
---|
580 | }, |
---|
581 | - 'tahoe_mutable_files_retrieved': |
---|
582 | + 'tahoe_stats_mutable_files_retrieved': |
---|
583 | { 'statid': 'mutable.files_retrieved', |
---|
584 | 'category': 'counters', |
---|
585 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 410 |
---|
586 | - 'configheader': '\n'.join(['graph_title Tahoe Mutable Files Retrieved', |
---|
587 | + 'configheader': '\n'.join(['graph_title Tahoe-LAFS Mutable Files Retrieved', |
---|
588 | 'graph_vlabel files', |
---|
589 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 412 |
---|
590 | - 'graph_category tahoe_traffic', |
---|
591 | + 'graph_category Tahoe-LAFS Stats-Gatherer_traffic', |
---|
592 | 'graph_info This graph shows the number of files retrieved', |
---|
593 | ]), |
---|
594 | 'graph_config': '\n'.join(['%(name)s.label %(name)s', |
---|
595 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 456 |
---|
596 | for tubid, nodestats in stats.items(): |
---|
597 | if check_time and (now - nodestats.get('timestamp', 0)) > STAT_VALIDITY: |
---|
598 | continue |
---|
599 | - name = smash_name("%s_%s" % (nodestats['nickname'], tubid[:4])) |
---|
600 | + #name = smash_name("%s_%s" % (nodestats['nickname'], tubid[:4])) |
---|
601 | + name = smash_name(nodestats['nickname']) |
---|
602 | #value = nodestats['stats'][plugin_conf['category']].get(plugin_conf['statid']) |
---|
603 | category = plugin_conf['category'] |
---|
604 | statid = plugin_conf['statid'] |
---|
605 | hunk ./misc/munin/stats-gatherer_plugins/tahoe_stats 466 |
---|
606 | args = { 'name': name, 'value': value } |
---|
607 | print plugin_conf[output_section] % args |
---|
608 | |
---|
609 | + |
---|
610 | + |
---|
611 | if len(argv) > 1: |
---|
612 | if sys.argv[1] == 'config': |
---|
613 | print plugin_conf['configheader'] |
---|
614 | hunk ./misc/munin/tahoe-stats.plugin-conf 1 |
---|
615 | -[tahoe_storage_allocated] |
---|
616 | -env.statsfile /home/robk/trees/tahoe/stats_gatherer/stats.pickle |
---|
617 | -[tahoe_storage_consumed] |
---|
618 | -env.statsfile /home/robk/trees/tahoe/stats_gatherer/stats.pickle |
---|
619 | -[tahoe_runtime_load_avg] |
---|
620 | -env.statsfile /home/robk/trees/tahoe/stats_gatherer/stats.pickle |
---|
621 | -[tahoe_runtime_load_peak] |
---|
622 | -env.statsfile /home/robk/trees/tahoe/stats_gatherer/stats.pickle |
---|
623 | -[tahoe_storage_bytes_added] |
---|
624 | -env.statsfile /home/robk/trees/tahoe/stats_gatherer/stats.pickle |
---|
625 | -[tahoe_storage_bytes_freed] |
---|
626 | -env.statsfile /home/robk/trees/tahoe/stats_gatherer/stats.pickle |
---|
627 | rmfile ./misc/munin/tahoe-stats.plugin-conf |
---|
628 | } |
---|
629 | |
---|
630 | Context: |
---|
631 | |
---|
632 | [Document reason for the trialcoverage version requirement being 0.3.3. |
---|
633 | david-sarah@jacaranda.org**20100525004444 |
---|
634 | Ignore-this: 2f9f1df6882838b000c063068f258aec |
---|
635 | ] |
---|
636 | [Downgrade version requirement for trialcoverage to 0.3.3 (from 0.3.10), to avoid needing to compile coveragepy on Windows. |
---|
637 | david-sarah@jacaranda.org**20100524233707 |
---|
638 | Ignore-this: 9c397a374c8b8017e2244b8a686432a8 |
---|
639 | ] |
---|
640 | [Suppress deprecation warning for twisted.web.error.NoResource when using Twisted >= 9.0.0. |
---|
641 | david-sarah@jacaranda.org**20100516205625 |
---|
642 | Ignore-this: 2361a3023cd3db86bde5e1af759ed01 |
---|
643 | ] |
---|
644 | [docs: CREDITS for Jeremy Visser |
---|
645 | zooko@zooko.com**20100524081829 |
---|
646 | Ignore-this: d7c1465fd8d4e25b8d46d38a1793465b |
---|
647 | ] |
---|
648 | [test: show stdout and stderr in case of non-zero exit code from "tahoe" command |
---|
649 | zooko@zooko.com**20100524073348 |
---|
650 | Ignore-this: 695e81cd6683f4520229d108846cd551 |
---|
651 | ] |
---|
652 | [setup: upgrade bundled zetuptoolz to zetuptoolz-0.6c15dev and make it unpacked and directly loaded by setup.py |
---|
653 | zooko@zooko.com**20100523205228 |
---|
654 | Ignore-this: 24fb32aaee3904115a93d1762f132c7 |
---|
655 | Also fix the relevant "make clean" target behavior. |
---|
656 | ] |
---|
657 | [setup: remove bundled zipfile egg of setuptools |
---|
658 | zooko@zooko.com**20100523205120 |
---|
659 | Ignore-this: c68b5f2635bb93d1c1fa7b613a026f9e |
---|
660 | We're about to replace it with bundled unpacked source code of setuptools, which is much nicer for debugging and evolving under revision control. |
---|
661 | ] |
---|
662 | [setup: remove bundled copy of setuptools_trial-0.5.2.tar |
---|
663 | zooko@zooko.com**20100522221539 |
---|
664 | Ignore-this: 140f90eb8fb751a509029c4b24afe647 |
---|
665 | Hopefully it will get installed automatically as needed and we won't bundle it anymore. |
---|
666 | ] |
---|
667 | [setup: remove bundled setuptools_darcs-1.2.8.tar |
---|
668 | zooko@zooko.com**20100522015333 |
---|
669 | Ignore-this: 378b1964b513ae7fe22bae2d3478285d |
---|
670 | This version of setuptools_darcs had a bug when used on Windows which has been fixed in setuptools_darcs-1.2.9. Hopefully we will not need to bundle a copy of setuptools_darcs-1.2.9 in with Tahoe-LAFS and can instead rely on it to be downloaded from PyPI or bundled in the "tahoe deps" separate tarball. |
---|
671 | ] |
---|
672 | [tests: fix pyflakes warnings in bench_dirnode.py |
---|
673 | zooko@zooko.com**20100521202511 |
---|
674 | Ignore-this: f23d55b4ed05e52865032c65a15753c4 |
---|
675 | ] |
---|
676 | [setup: if the string '--reporter=bwverbose-coverage' appears on sys.argv then you need trialcoverage |
---|
677 | zooko@zooko.com**20100521122226 |
---|
678 | Ignore-this: e760c45dcfb5a43c1dc1e8a27346bdc2 |
---|
679 | ] |
---|
680 | [tests: don't let bench_dirnode.py do stuff and have side-effects at import time (unless __name__ == '__main__') |
---|
681 | zooko@zooko.com**20100521122052 |
---|
682 | Ignore-this: 96144a412250d9bbb5fccbf83b8753b8 |
---|
683 | ] |
---|
684 | [tests: increase timeout to give François's ARM buildslave a chance to complete the tests |
---|
685 | zooko@zooko.com**20100520134526 |
---|
686 | Ignore-this: 3dd399fdc8b91149c82b52f955b50833 |
---|
687 | ] |
---|
688 | [run_trial.darcspath |
---|
689 | freestorm77@gmail.com**20100510232829 |
---|
690 | Ignore-this: 5ebb4df74e9ea8a4bdb22b65373d1ff2 |
---|
691 | ] |
---|
692 | [docs: line-wrap README.txt |
---|
693 | zooko@zooko.com**20100518174240 |
---|
694 | Ignore-this: 670a02d360df7de51ebdcf4fae752577 |
---|
695 | ] |
---|
696 | [Hush pyflakes warnings |
---|
697 | Kevan Carstensen <kevan@isnotajoke.com>**20100515184344 |
---|
698 | Ignore-this: fd602c3bba115057770715c36a87b400 |
---|
699 | ] |
---|
700 | [setup: new improved misc/show-tool-versions.py |
---|
701 | zooko@zooko.com**20100516050122 |
---|
702 | Ignore-this: ce9b1de1b35b07d733e6cf823b66335a |
---|
703 | ] |
---|
704 | [Improve code coverage of the Tahoe2PeerSelector tests. |
---|
705 | Kevan Carstensen <kevan@isnotajoke.com>**20100515032913 |
---|
706 | Ignore-this: 793151b63ffa65fdae6915db22d9924a |
---|
707 | ] |
---|
708 | [Remove a comment that no longer makes sense. |
---|
709 | Kevan Carstensen <kevan@isnotajoke.com>**20100514203516 |
---|
710 | Ignore-this: 956983c7e7c7e4477215494dfce8f058 |
---|
711 | ] |
---|
712 | [docs: update docs/architecture.txt to more fully and correctly explain the upload procedure |
---|
713 | zooko@zooko.com**20100514043458 |
---|
714 | Ignore-this: 538b6ea256a49fed837500342092efa3 |
---|
715 | ] |
---|
716 | [Fix up the behavior of #778, per reviewers' comments |
---|
717 | Kevan Carstensen <kevan@isnotajoke.com>**20100514004917 |
---|
718 | Ignore-this: 9c20b60716125278b5456e8feb396bff |
---|
719 | |
---|
720 | - Make some important utility functions clearer and more thoroughly |
---|
721 | documented. |
---|
722 | - Assert in upload.servers_of_happiness that the buckets attributes |
---|
723 | of PeerTrackers passed to it are mutually disjoint. |
---|
724 | - Get rid of some silly non-Pythonisms that I didn't see when I first |
---|
725 | wrote these patches. |
---|
726 | - Make sure that should_add_server returns true when queried about a |
---|
727 | shnum that it doesn't know about yet. |
---|
728 | - Change Tahoe2PeerSelector.preexisting_shares to map a shareid to a set |
---|
729 | of peerids, alter dependencies to deal with that. |
---|
730 | - Remove upload.should_add_servers, because it is no longer necessary |
---|
731 | - Move upload.shares_of_happiness and upload.shares_by_server to a utility |
---|
732 | file. |
---|
733 | - Change some points in Tahoe2PeerSelector. |
---|
734 | - Compute servers_of_happiness using a bipartite matching algorithm that |
---|
735 | we know is optimal instead of an ad-hoc greedy algorithm that isn't. |
---|
736 | - Change servers_of_happiness to just take a sharemap as an argument, |
---|
737 | change its callers to merge existing_shares and used_peers before |
---|
738 | calling it. |
---|
739 | - Change an error message in the encoder to be more appropriate for |
---|
740 | servers of happiness. |
---|
741 | - Clarify the wording of an error message in immutable/upload.py |
---|
742 | - Refactor a happiness failure message to happinessutil.py, and make |
---|
743 | immutable/upload.py and immutable/encode.py use it. |
---|
744 | - Move the word "only" as far to the right as possible in failure |
---|
745 | messages. |
---|
746 | - Use a better definition of progress during peer selection. |
---|
747 | - Do read-only peer share detection queries in parallel, not sequentially. |
---|
748 | - Clean up logging semantics; print the query statistics whenever an |
---|
749 | upload is unsuccessful, not just in one case. |
---|
750 | |
---|
751 | ] |
---|
752 | [Alter the error message when an upload fails, per some comments in #778. |
---|
753 | Kevan Carstensen <kevan@isnotajoke.com>**20091230210344 |
---|
754 | Ignore-this: ba97422b2f9737c46abeb828727beb1 |
---|
755 | |
---|
756 | When I first implemented #778, I just altered the error messages to refer to |
---|
757 | servers where they referred to shares. The resulting error messages weren't |
---|
758 | very good. These are a bit better. |
---|
759 | ] |
---|
760 | [Change "UploadHappinessError" to "UploadUnhappinessError" |
---|
761 | Kevan Carstensen <kevan@isnotajoke.com>**20091205043037 |
---|
762 | Ignore-this: 236b64ab19836854af4993bb5c1b221a |
---|
763 | ] |
---|
764 | [Alter the error message returned when peer selection fails |
---|
765 | Kevan Carstensen <kevan@isnotajoke.com>**20091123002405 |
---|
766 | Ignore-this: b2a7dc163edcab8d9613bfd6907e5166 |
---|
767 | |
---|
768 | The Tahoe2PeerSelector returned either NoSharesError or NotEnoughSharesError |
---|
769 | for a variety of error conditions that weren't informatively described by them. |
---|
770 | This patch creates a new error, UploadHappinessError, replaces uses of |
---|
771 | NoSharesError and NotEnoughSharesError with it, and alters the error message |
---|
772 | raised with the errors to be more in line with the new servers_of_happiness |
---|
773 | behavior. See ticket #834 for more information. |
---|
774 | ] |
---|
775 | [Eliminate overcounting iof servers_of_happiness in Tahoe2PeerSelector; also reorganize some things. |
---|
776 | Kevan Carstensen <kevan@isnotajoke.com>**20091118014542 |
---|
777 | Ignore-this: a6cb032cbff74f4f9d4238faebd99868 |
---|
778 | ] |
---|
779 | [Change stray "shares_of_happiness" to "servers_of_happiness" |
---|
780 | Kevan Carstensen <kevan@isnotajoke.com>**20091116212459 |
---|
781 | Ignore-this: 1c971ba8c3c4d2e7ba9f020577b28b73 |
---|
782 | ] |
---|
783 | [Alter Tahoe2PeerSelector to make sure that it recognizes existing shares on readonly servers, fixing an issue in #778 |
---|
784 | Kevan Carstensen <kevan@isnotajoke.com>**20091116192805 |
---|
785 | Ignore-this: 15289f4d709e03851ed0587b286fd955 |
---|
786 | ] |
---|
787 | [Alter 'immutable/encode.py' and 'immutable/upload.py' to use servers_of_happiness instead of shares_of_happiness. |
---|
788 | Kevan Carstensen <kevan@isnotajoke.com>**20091104111222 |
---|
789 | Ignore-this: abb3283314820a8bbf9b5d0cbfbb57c8 |
---|
790 | ] |
---|
791 | [Alter the signature of set_shareholders in IEncoder to add a 'servermap' parameter, which gives IEncoders enough information to perform a sane check for servers_of_happiness. |
---|
792 | Kevan Carstensen <kevan@isnotajoke.com>**20091104033241 |
---|
793 | Ignore-this: b3a6649a8ac66431beca1026a31fed94 |
---|
794 | ] |
---|
795 | [Alter CiphertextDownloader to work with servers_of_happiness |
---|
796 | Kevan Carstensen <kevan@isnotajoke.com>**20090924041932 |
---|
797 | Ignore-this: e81edccf0308c2d3bedbc4cf217da197 |
---|
798 | ] |
---|
799 | [Revisions of the #778 tests, per reviewers' comments |
---|
800 | Kevan Carstensen <kevan@isnotajoke.com>**20100514012542 |
---|
801 | Ignore-this: 735bbc7f663dce633caeb3b66a53cf6e |
---|
802 | |
---|
803 | - Fix comments and confusing naming. |
---|
804 | - Add tests for the new error messages suggested by David-Sarah |
---|
805 | and Zooko. |
---|
806 | - Alter existing tests for new error messages. |
---|
807 | - Make sure that the tests continue to work with the trunk. |
---|
808 | - Add a test for a mutual disjointedness assertion that I added to |
---|
809 | upload.servers_of_happiness. |
---|
810 | - Fix the comments to correctly reflect read-onlyness |
---|
811 | - Add a test for an edge case in should_add_server |
---|
812 | - Add an assertion to make sure that share redistribution works as it |
---|
813 | should |
---|
814 | - Alter tests to work with revised servers_of_happiness semantics |
---|
815 | - Remove tests for should_add_server, since that function no longer exists. |
---|
816 | - Alter tests to know about merge_peers, and to use it before calling |
---|
817 | servers_of_happiness. |
---|
818 | - Add tests for merge_peers. |
---|
819 | - Add Zooko's puzzles to the tests. |
---|
820 | - Edit encoding tests to expect the new kind of failure message. |
---|
821 | - Edit tests to expect error messages with the word "only" moved as far |
---|
822 | to the right as possible. |
---|
823 | - Extended and cleaned up some helper functions. |
---|
824 | - Changed some tests to call more appropriate helper functions. |
---|
825 | - Added a test for the failing redistribution algorithm |
---|
826 | - Added a test for the progress message |
---|
827 | - Added a test for the upper bound on readonly peer share discovery. |
---|
828 | |
---|
829 | ] |
---|
830 | [Alter various unit tests to work with the new happy behavior |
---|
831 | Kevan Carstensen <kevan@isnotajoke.com>**20100107181325 |
---|
832 | Ignore-this: 132032bbf865e63a079f869b663be34a |
---|
833 | ] |
---|
834 | [Replace "UploadHappinessError" with "UploadUnhappinessError" in tests. |
---|
835 | Kevan Carstensen <kevan@isnotajoke.com>**20091205043453 |
---|
836 | Ignore-this: 83f4bc50c697d21b5f4e2a4cd91862ca |
---|
837 | ] |
---|
838 | [Add tests for the behavior described in #834. |
---|
839 | Kevan Carstensen <kevan@isnotajoke.com>**20091123012008 |
---|
840 | Ignore-this: d8e0aa0f3f7965ce9b5cea843c6d6f9f |
---|
841 | ] |
---|
842 | [Re-work 'test_upload.py' to be more readable; add more tests for #778 |
---|
843 | Kevan Carstensen <kevan@isnotajoke.com>**20091116192334 |
---|
844 | Ignore-this: 7e8565f92fe51dece5ae28daf442d659 |
---|
845 | ] |
---|
846 | [Test Tahoe2PeerSelector to make sure that it recognizeses existing shares on readonly servers |
---|
847 | Kevan Carstensen <kevan@isnotajoke.com>**20091109003735 |
---|
848 | Ignore-this: 12f9b4cff5752fca7ed32a6ebcff6446 |
---|
849 | ] |
---|
850 | [Add more tests for comment:53 in ticket #778 |
---|
851 | Kevan Carstensen <kevan@isnotajoke.com>**20091104112849 |
---|
852 | Ignore-this: 3bb2edd299a944cc9586e14d5d83ec8c |
---|
853 | ] |
---|
854 | [Add a test for upload.shares_by_server |
---|
855 | Kevan Carstensen <kevan@isnotajoke.com>**20091104111324 |
---|
856 | Ignore-this: f9802e82d6982a93e00f92e0b276f018 |
---|
857 | ] |
---|
858 | [Minor tweak to an existing test -- make the first server read-write, instead of read-only |
---|
859 | Kevan Carstensen <kevan@isnotajoke.com>**20091104034232 |
---|
860 | Ignore-this: a951a46c93f7f58dd44d93d8623b2aee |
---|
861 | ] |
---|
862 | [Alter tests to use the new form of set_shareholders |
---|
863 | Kevan Carstensen <kevan@isnotajoke.com>**20091104033602 |
---|
864 | Ignore-this: 3deac11fc831618d11441317463ef830 |
---|
865 | ] |
---|
866 | [Refactor some behavior into a mixin, and add tests for the behavior described in #778 |
---|
867 | "Kevan Carstensen" <kevan@isnotajoke.com>**20091030091908 |
---|
868 | Ignore-this: a6f9797057ca135579b249af3b2b66ac |
---|
869 | ] |
---|
870 | [Alter NoNetworkGrid to allow the creation of readonly servers for testing purposes. |
---|
871 | Kevan Carstensen <kevan@isnotajoke.com>**20091018013013 |
---|
872 | Ignore-this: e12cd7c4ddeb65305c5a7e08df57c754 |
---|
873 | ] |
---|
874 | [Update 'docs/architecture.txt' to reflect readonly share discovery |
---|
875 | kevan@isnotajoke.com**20100514003852 |
---|
876 | Ignore-this: 7ead71b34df3b1ecfdcfd3cb2882e4f9 |
---|
877 | ] |
---|
878 | [Alter the wording in docs/architecture.txt to more accurately describe the servers_of_happiness behavior. |
---|
879 | Kevan Carstensen <kevan@isnotajoke.com>**20100428002455 |
---|
880 | Ignore-this: 6eff7fa756858a1c6f73728d989544cc |
---|
881 | ] |
---|
882 | [Alter wording in 'interfaces.py' to be correct wrt #778 |
---|
883 | "Kevan Carstensen" <kevan@isnotajoke.com>**20091205034005 |
---|
884 | Ignore-this: c9913c700ac14e7a63569458b06980e0 |
---|
885 | ] |
---|
886 | [Update 'docs/configuration.txt' to reflect the servers_of_happiness behavior. |
---|
887 | Kevan Carstensen <kevan@isnotajoke.com>**20091205033813 |
---|
888 | Ignore-this: 5e1cb171f8239bfb5b565d73c75ac2b8 |
---|
889 | ] |
---|
890 | [Clarify quickstart instructions for installing pywin32 |
---|
891 | david-sarah@jacaranda.org**20100511180300 |
---|
892 | Ignore-this: d4668359673600d2acbc7cd8dd44b93c |
---|
893 | ] |
---|
894 | [web: add a simple test that you can load directory.xhtml |
---|
895 | zooko@zooko.com**20100510063729 |
---|
896 | Ignore-this: e49b25fa3c67b3c7a56c8b1ae01bb463 |
---|
897 | ] |
---|
898 | [setup: fix typos in misc/show-tool-versions.py |
---|
899 | zooko@zooko.com**20100510063615 |
---|
900 | Ignore-this: 2181b1303a0e288e7a9ebd4c4855628 |
---|
901 | ] |
---|
902 | [setup: show code-coverage tool versions in show-tools-versions.py |
---|
903 | zooko@zooko.com**20100510062955 |
---|
904 | Ignore-this: 4b4c68eb3780b762c8dbbd22b39df7cf |
---|
905 | ] |
---|
906 | [docs: update README, mv it to README.txt, update setup.py |
---|
907 | zooko@zooko.com**20100504094340 |
---|
908 | Ignore-this: 40e28ca36c299ea1fd12d3b91e5b421c |
---|
909 | ] |
---|
910 | [Dependency on Windmill test framework is not needed yet. |
---|
911 | david-sarah@jacaranda.org**20100504161043 |
---|
912 | Ignore-this: be088712bec650d4ef24766c0026ebc8 |
---|
913 | ] |
---|
914 | [tests: pass z to tar so that BSD tar will know to ungzip |
---|
915 | zooko@zooko.com**20100504090628 |
---|
916 | Ignore-this: 1339e493f255e8fc0b01b70478f23a09 |
---|
917 | ] |
---|
918 | [setup: update comments and URLs in setup.cfg |
---|
919 | zooko@zooko.com**20100504061653 |
---|
920 | Ignore-this: f97692807c74bcab56d33100c899f829 |
---|
921 | ] |
---|
922 | [setup: reorder and extend the show-tool-versions script, the better to glean information about our new buildslaves |
---|
923 | zooko@zooko.com**20100504045643 |
---|
924 | Ignore-this: 836084b56b8d4ee8f1de1f4efb706d36 |
---|
925 | ] |
---|
926 | [CLI: Support for https url in option --node-url |
---|
927 | Francois Deppierraz <francois@ctrlaltdel.ch>**20100430185609 |
---|
928 | Ignore-this: 1717176b4d27c877e6bc67a944d9bf34 |
---|
929 | |
---|
930 | This patch modifies the regular expression used for verifying of '--node-url' |
---|
931 | parameter. Support for accessing a Tahoe gateway over HTTPS was already |
---|
932 | present, thanks to Python's urllib. |
---|
933 | |
---|
934 | ] |
---|
935 | [backupdb.did_create_directory: use REPLACE INTO, not INSERT INTO + ignore error |
---|
936 | Brian Warner <warner@lothar.com>**20100428050803 |
---|
937 | Ignore-this: 1fca7b8f364a21ae413be8767161e32f |
---|
938 | |
---|
939 | This handles the case where we upload a new tahoe directory for a |
---|
940 | previously-processed local directory, possibly creating a new dircap (if the |
---|
941 | metadata had changed). Now we replace the old dirhash->dircap record. The |
---|
942 | previous behavior left the old record in place (with the old dircap and |
---|
943 | timestamps), so we'd never stop creating new directories and never converge |
---|
944 | on a null backup. |
---|
945 | ] |
---|
946 | ["tahoe webopen": add --info flag, to get ?t=info |
---|
947 | Brian Warner <warner@lothar.com>**20100424233003 |
---|
948 | Ignore-this: 126b0bb6db340fabacb623d295eb45fa |
---|
949 | |
---|
950 | Also fix some trailing whitespace. |
---|
951 | ] |
---|
952 | [docs: install.html http-equiv refresh to quickstart.html |
---|
953 | zooko@zooko.com**20100421165708 |
---|
954 | Ignore-this: 52b4b619f9dde5886ae2cd7f1f3b734b |
---|
955 | ] |
---|
956 | [docs: install.html -> quickstart.html |
---|
957 | zooko@zooko.com**20100421155757 |
---|
958 | Ignore-this: 6084e203909306bed93efb09d0e6181d |
---|
959 | It is not called "installing" because that implies that it is going to change the configuration of your operating system. It is not called "building" because that implies that you need developer tools like a compiler. Also I added a stern warning against looking at the "InstallDetails" wiki page, which I have renamed to "AdvancedInstall". |
---|
960 | ] |
---|
961 | [Fix another typo in tahoe_storagespace munin plugin |
---|
962 | david-sarah@jacaranda.org**20100416220935 |
---|
963 | Ignore-this: ad1f7aa66b554174f91dfb2b7a3ea5f3 |
---|
964 | ] |
---|
965 | [Add dependency on windmill >= 1.3 |
---|
966 | david-sarah@jacaranda.org**20100416190404 |
---|
967 | Ignore-this: 4437a7a464e92d6c9012926b18676211 |
---|
968 | ] |
---|
969 | [licensing: phrase the OpenSSL-exemption in the vocabulary of copyright instead of computer technology, and replicate the exemption from the GPL to the TGPPL |
---|
970 | zooko@zooko.com**20100414232521 |
---|
971 | Ignore-this: a5494b2f582a295544c6cad3f245e91 |
---|
972 | ] |
---|
973 | [munin-tahoe_storagespace |
---|
974 | freestorm77@gmail.com**20100221203626 |
---|
975 | Ignore-this: 14d6d6a587afe1f8883152bf2e46b4aa |
---|
976 | |
---|
977 | Plugin configuration rename |
---|
978 | |
---|
979 | ] |
---|
980 | [setup: add licensing declaration for setuptools (noticed by the FSF compliance folks) |
---|
981 | zooko@zooko.com**20100309184415 |
---|
982 | Ignore-this: 2dfa7d812d65fec7c72ddbf0de609ccb |
---|
983 | ] |
---|
984 | [setup: fix error in licensing declaration from Shawn Willden, as noted by the FSF compliance division |
---|
985 | zooko@zooko.com**20100309163736 |
---|
986 | Ignore-this: c0623d27e469799d86cabf67921a13f8 |
---|
987 | ] |
---|
988 | [CREDITS to Jacob Appelbaum |
---|
989 | zooko@zooko.com**20100304015616 |
---|
990 | Ignore-this: 70db493abbc23968fcc8db93f386ea54 |
---|
991 | ] |
---|
992 | [desert-island-build-with-proper-versions |
---|
993 | jacob@appelbaum.net**20100304013858] |
---|
994 | [docs: a few small edits to try to guide newcomers through the docs |
---|
995 | zooko@zooko.com**20100303231902 |
---|
996 | Ignore-this: a6aab44f5bf5ad97ea73e6976bc4042d |
---|
997 | These edits were suggested by my watching over Jake Appelbaum's shoulder as he completely ignored/skipped/missed install.html and also as he decided that debian.txt wouldn't help him with basic installation. Then I threw in a few docs edits that have been sitting around in my sandbox asking to be committed for months. |
---|
998 | ] |
---|
999 | [TAG allmydata-tahoe-1.6.1 |
---|
1000 | david-sarah@jacaranda.org**20100228062314 |
---|
1001 | Ignore-this: eb5f03ada8ea953ee7780e7fe068539 |
---|
1002 | ] |
---|
1003 | Patch bundle hash: |
---|
1004 | 6c2f9b7e34529b50050a7262b41e014cf34dbde6 |
---|