1 | This is a proposal for handing accounts and quotas in Tahoe. Nothing is final |
---|
2 | yet.. we are still evaluating the options. |
---|
3 | |
---|
4 | |
---|
5 | = Accounts = |
---|
6 | |
---|
7 | The basic Tahoe account is defined by a DSA key pair. The holder of the |
---|
8 | private key has the ability to consume storage in conjunction with a specific |
---|
9 | account number. |
---|
10 | |
---|
11 | The Account Server has a long-term keypair. Valid accounts are marked as such |
---|
12 | by the Account Server's signature on a "membership card", which binds a |
---|
13 | specific pubkey to an account number and declares that this pair is a valid |
---|
14 | account. |
---|
15 | |
---|
16 | Each Storage Server which participates in the AS's domain will have the AS's |
---|
17 | pubkey in its list of valid AS keys, and will thus accept membership cards |
---|
18 | that were signed by that AS. If the SS accepts multiple ASs, then it will |
---|
19 | give each a distinct number, and leases will be labled with an (AS#,Account#) |
---|
20 | pair. If there is only one AS, then leases will be labeled with just the |
---|
21 | Account#. |
---|
22 | |
---|
23 | Each client node is given the FURL of their personal Account object. The |
---|
24 | Account will accept a DSA public key and return a signed membership card that |
---|
25 | authorizes the corresponding private key to consume storage on behalf of the |
---|
26 | account. The client will create its own DSA keypair the first time it |
---|
27 | connects to the Account, and will then use the resulting membership card for |
---|
28 | all subsequent storage operations. |
---|
29 | |
---|
30 | == Storage Server Goals == |
---|
31 | |
---|
32 | The Storage Server cares about two things: |
---|
33 | |
---|
34 | 1: maintaining an accurate refcount on each bucket, so it can delete the |
---|
35 | bucket when the refcount goes to zero |
---|
36 | 2: being able to answer questions about aggregate usage per account |
---|
37 | |
---|
38 | The SS conceptually maintains a big matrix of lease information: one column |
---|
39 | per account, one row per storage index. The cells contain a boolean |
---|
40 | (has-lease or no-lease). If the grid uses per-lease timers, then each |
---|
41 | has-lease cell also contains a lease timer. |
---|
42 | |
---|
43 | This matrix may be stored in a variety of ways: entries in each share file, |
---|
44 | or items in a SQL database, according to the desired tradeoff between |
---|
45 | complexity, robustness, read speed, and write speed. |
---|
46 | |
---|
47 | Each client (by virtue of their knowledge of an authorized private key) gets |
---|
48 | to manipulate their column of this matrix in any way they like: add lease, |
---|
49 | renew lease, delete lease. (TODO: for reconcilliation purposes, the should |
---|
50 | also be able to enumerate leases). |
---|
51 | |
---|
52 | == Storage Operations == |
---|
53 | |
---|
54 | Side-effect-causing storage operations come in three forms: |
---|
55 | |
---|
56 | 1: allocate bucket / add lease to existing bucket |
---|
57 | arguments: storage_index=, storage_server=, ueb_hash=, account= |
---|
58 | 2: renew lease |
---|
59 | arguments: storage_index=, storage_server=, account= |
---|
60 | 3: cancel lease |
---|
61 | arguments: storage_index=, storage_server=, account= |
---|
62 | |
---|
63 | (where lease renewal is only relevant for grids which use per-lease timers). |
---|
64 | Clients do add-lease when they upload a file, and cancel-lease when they |
---|
65 | remove their last reference to it. |
---|
66 | |
---|
67 | Storage Servers publish a "public storage port" through the introducer, which |
---|
68 | does not actually enable storage operations, but is instead used in a |
---|
69 | rights-amplification pattern to grant authorized parties access to a |
---|
70 | "personal storage server facet". This personal facet is the one that |
---|
71 | implements allocate_bucket. All clients get access to the same public storage |
---|
72 | port, which means that we can improve the introduction mechanism later (to |
---|
73 | use a gossip-based protocol) without affecting the authority-granting |
---|
74 | protocols. |
---|
75 | |
---|
76 | The public storage port accepts signed messages asking for storage authority. |
---|
77 | It responds by creating a personal facet and making it available to the |
---|
78 | requester. The account number is curried into the facet, so that all |
---|
79 | lease-creating operations will record this account number into the lease. By |
---|
80 | restricting the nature of the personal facets that a client can access, we |
---|
81 | restrict them to using their designated account number. |
---|
82 | |
---|
83 | |
---|
84 | ======================================== |
---|
85 | |
---|
86 | There are two kinds of signed messages: use (other names: connection, |
---|
87 | FURLification, activation, reification, grounding, specific-making, ?), and |
---|
88 | delegation. The FURLification message results in a FURL that points to an |
---|
89 | object which can actually accept RIStorageServer methods. The delegation |
---|
90 | message results in a new signed message. |
---|
91 | |
---|
92 | The furlification message looks like: |
---|
93 | |
---|
94 | (pubkey, signed(serialized({limitations}, beneficiary_furl))) |
---|
95 | |
---|
96 | The delegation message looks like: |
---|
97 | |
---|
98 | (pubkey, signed(serialized({limitations}, delegate_pubkey))) |
---|
99 | |
---|
100 | The limitations dict indicates what the resulting connection or delegation |
---|
101 | can be used for. All limitations for the cert chain are applied, and the |
---|
102 | result must be restricted to their overall minimum. |
---|
103 | |
---|
104 | The following limitation keys are defined: |
---|
105 | |
---|
106 | 'account': a number. All resulting leases must be tagged with this account |
---|
107 | number. A chain with multiple distinct 'account' limitations is |
---|
108 | an error (the result will not permit leases) |
---|
109 | 'SI': a storage index (binary string). Leases may only be created for this |
---|
110 | specific storage index, no other. |
---|
111 | 'serverid': a peerid (binary string). Leases may only be created on the |
---|
112 | storage server identified by this serverid. |
---|
113 | 'UEB_hash': (binary string): Leases may only be created for shares which |
---|
114 | contain a matching UEB_hash. Note: this limitation is a nuisance |
---|
115 | to implement correctly: it requires that the storage server |
---|
116 | parse the share and verify all hashes. |
---|
117 | 'before': a timestamp (seconds since epoch). All leases must be made before |
---|
118 | this time. In addition, all liverefs and FURLs must expire and |
---|
119 | cease working at this time. |
---|
120 | 'server_size': a number, measuring share size (in bytes). A storage server |
---|
121 | which sees this message should keep track of how much storage |
---|
122 | space has been consumed using this liveref/FURL, and throw |
---|
123 | an exception when receiving a lease request that would bring |
---|
124 | this total above 'server_size'. Note: this limitation is |
---|
125 | a nuisance to implement (it works best if 'before' is used |
---|
126 | and provides a short lifetime). |
---|
127 | |
---|
128 | Actually, let's merge the two, and put the type in the limitations dict. |
---|
129 | 'furl_to' and 'delegate_key' are mutually exclusive. |
---|
130 | |
---|
131 | 'furl_to': (string): Used only on furlification messages. This requests the |
---|
132 | recipient to create an object which implements the given access, |
---|
133 | then send a FURL which references this object to an |
---|
134 | RIFURLReceiver.furl() call at the given 'furl_to' FURL. |
---|
135 | |
---|
136 | To reduce the number of extra roundtrips, both foolscap calls |
---|
137 | include an extra (ignored) argument that will carry the object |
---|
138 | being referenced by the FURL, used to pre-load the recipient's |
---|
139 | foolscap table. In addition, the response message will contain a |
---|
140 | nonce, to allow the same beneficiary to be used for multiple |
---|
141 | messages: |
---|
142 | |
---|
143 | def process(limitations, nonce, ignored): |
---|
144 | facet = create_storage_facet(limitations) |
---|
145 | facet_furl = tub.registerReference(facet) |
---|
146 | d = tub.getReference(limitations['furl_to']) |
---|
147 | d.addCallback(lambda rref: rref.furl(facet_furl, nonce, facet)) |
---|
148 | |
---|
149 | The server must always send the facet/facet_furl to the furl_to |
---|
150 | beneficiary, and never to the 'ignored' argument (even though for |
---|
151 | well-behaved clients these will both refer to the same target). |
---|
152 | This is to prevent a rogue server from echoing a client's signed |
---|
153 | message to some other server, to try to steal the client's |
---|
154 | authority. |
---|
155 | |
---|
156 | The facet_furl should be persistent, so to reduce storage space, |
---|
157 | facet_furl should contain an HMAC'ed list of all limitations, and |
---|
158 | create_storage_facet() should be deferred until the client |
---|
159 | actually tries to use the furl. This leads to 150-200 byte base32 |
---|
160 | swissnums. |
---|
161 | |
---|
162 | 'delegate_key': (binary string, a DSA pubkey). Used only on delegation |
---|
163 | messages. This requests all observers to accept messages |
---|
164 | signed by the given public key and to apply the associated |
---|
165 | limitations. |
---|
166 | |
---|
167 | I also want to keep the message size small, so I'm going to define a custom |
---|
168 | netstring-based encoding format for it (JSON expands binary data by about |
---|
169 | 3.5x). Each dict entry will be encoded as netstring(key)+netstring(value). |
---|
170 | The container is responsible for providing the size of this serialized |
---|
171 | structure. |
---|
172 | |
---|
173 | The actual message will then look like: |
---|
174 | |
---|
175 | def make_message(privkey, limitations): |
---|
176 | message_to_sign = "".join([ netstring(k) + netstring(v) |
---|
177 | for k,v in limitations ]) |
---|
178 | signature = privkey.sign(message_to_sign) |
---|
179 | pubkey = privkey.get_public_key() |
---|
180 | msg = netstring(message_to_sign) + netstring(signature) + netstring(pubkey) |
---|
181 | return msg |
---|
182 | |
---|
183 | The deserialization code MUST throw an exception if the same limitations key |
---|
184 | appears twice, to ensure that everybody interprets the dict the same way. |
---|
185 | |
---|
186 | These messages are passed over foolscap connections as a single string. They |
---|
187 | are also saved to disk in this format. Code should only store them in a |
---|
188 | deserialized form if the signature has been verified, the cert chain |
---|
189 | verified, and the limitations accumulated. |
---|
190 | |
---|
191 | |
---|
192 | The membership card is just the following: |
---|
193 | |
---|
194 | membership_card = make_message(account_server_privkey, |
---|
195 | {'account': account_number, |
---|
196 | 'before': time.time() + 1*MONTH, |
---|
197 | 'delegate_key': client_pubkey}) |
---|
198 | |
---|
199 | This card is provided on demand by the given user's Account facet, for |
---|
200 | whatever pubkey they submit. |
---|
201 | |
---|
202 | When a client learns about a new storage server, they create a new receiver |
---|
203 | object (and stash the peerid in it), and submit the following message to the |
---|
204 | RIStorageServerWelcome.get_personal_facet() method: |
---|
205 | |
---|
206 | class Receiver(foolscap.Referenceable): |
---|
207 | def remote_furl(self, facet_furl, nonce, ignored_facet): |
---|
208 | self.stash = facet_furl |
---|
209 | receiver = Receiver() |
---|
210 | nonce = make_nonce() |
---|
211 | mymsg = make_message(client_privkey, {'furl_to': receiver_furl}) |
---|
212 | send([membership_card, mymsg], nonce, receiver) |
---|
213 | |
---|
214 | Note that the receiver_furl will probably not have a routeable address, but |
---|
215 | this won't matter because the client is already attached, so foolscap can use |
---|
216 | the existing connection. The receiver should use facet_furl in preference to |
---|
217 | ignored_facet for consistency, but (unlike the server's use of receiver_furl) |
---|
218 | there is no security risk in using ignored_facet (since both are coming from |
---|
219 | the same source). |
---|
220 | |
---|
221 | The server will validate the cert chain (see below) and wind up with a |
---|
222 | complete list of limitations that are to be applied to the facet it will |
---|
223 | provide to the caller. This list must combine limitations from the entire |
---|
224 | chain: in particular it must enforce the account= limitation from the |
---|
225 | membership card. |
---|
226 | |
---|
227 | The server will then serialize this limitation dict into a string, compute a |
---|
228 | fixed-size HMAC code using a server-private secret, then base32 encode the |
---|
229 | (hmac+limitstring) value (and prepend a "0-" version indicator). The |
---|
230 | resulting string is used as the swissnum portion of the FURL that is sent to |
---|
231 | the furl_to target. |
---|
232 | |
---|
233 | Later, when the client tries to dereference this FURL, a |
---|
234 | Tub.registerNameLookupHandler hook will notice the attempt, claim the "0-" |
---|
235 | namespace, base32decode the string, check the HMAC, decode the limitation |
---|
236 | dict, then create and return an RIStorageServer facet with these limitations. |
---|
237 | |
---|
238 | The client should cache the (peerid, FURL) mapping in persistent storage. |
---|
239 | Later, when it learns about this storage server again, it will use the cached |
---|
240 | FURL instead of signing another message. If the getReference or the storage |
---|
241 | operation fails with StorageAuthorityExpiredError, the cache entry should be |
---|
242 | removed and the client should sign a new message to obtain a new one. |
---|
243 | |
---|
244 | (security note: an evil storage server can take 'mymsg' and present it to |
---|
245 | someone else, but other servers will only send the resulting authority to |
---|
246 | the client's receiver_furl, so the evil server cannot benefit from this. The |
---|
247 | receiver object has the serverid curried into it, so the evil server can |
---|
248 | only affect the client's mapping for this one serverid, not anything else, |
---|
249 | so the server cannot hurt the client in any way other than denying service |
---|
250 | to itself. It might be a good idea to include serverid= in the message, but |
---|
251 | it isn't clear that it really helps anything). |
---|
252 | |
---|
253 | When the client wants to use a Helper, it needs to delegate some amount of |
---|
254 | storage authority to the helper. The first phase has the client send the |
---|
255 | storage index to the helper, so it can query servers and decide whether the |
---|
256 | file needs to be uploaded or not. If it decides yes, the Helper creates a new |
---|
257 | Uploader object and a receiver object, and sends the Uploader liveref and the |
---|
258 | receiver FURL to the client. |
---|
259 | |
---|
260 | The client then creates a message for the helper to use: |
---|
261 | |
---|
262 | helper_msg = make_message(client_privkey, {'furl_to': helper_rx_furl, |
---|
263 | 'SI': storage_index, |
---|
264 | 'before': time.time() + 1*DAY, #? |
---|
265 | 'server_size': filesize/k+overhead, |
---|
266 | }) |
---|
267 | |
---|
268 | The client then sends (membership_card, helper_msg) to the helper. The Helper |
---|
269 | sends (membership_card, helper_msg) to each storage server that it needs to |
---|
270 | use for the upload. This gives the Helper access to a limited facet on each |
---|
271 | storage server. This facet gives the helper the authority to upload data for |
---|
272 | a specific storage index, for a limited time, using leases that are tagged by |
---|
273 | the user's account number. The helper cannot use the client's storage |
---|
274 | authority for any other file. The size limit prevents the helper from storing |
---|
275 | some other (larger) file of its own using this authority. The time |
---|
276 | restriction allows the storage servers to expire their 'server_size' table |
---|
277 | entry quickly, and prevents the helper from hanging on to the storage |
---|
278 | authority indefinitely. |
---|
279 | |
---|
280 | The Helper only gets one furl_to target, which must be used for multiple SS |
---|
281 | peerids. The helper's receiver must parse the FURL that gets returned to |
---|
282 | determine which server is which. [problems: an evil server could deliver a |
---|
283 | bogus FURL which points to a different server. The Helper might reject the |
---|
284 | real server's good FURL as a duplicate. This allows an evil server to block |
---|
285 | access to a good server. Queries could be sent sequentially, which would |
---|
286 | partially mitigate this problem (an evil server could send multiple |
---|
287 | requests). Better: if the cert-chain send message could include a nonce, |
---|
288 | which is supposed to be returned with the FURL, then the helper could use |
---|
289 | this to correlate sends and receives.] |
---|
290 | |
---|
291 | === repair caps === |
---|
292 | |
---|
293 | There are three basic approaches to provide a Repairer with the storage |
---|
294 | authority that it needs. The first is to give the Repairer complete |
---|
295 | authority: allow it to place leases for whatever account number it wishes. |
---|
296 | This is simple and requires the least overhead, but of course it give the |
---|
297 | Repairer the ability to abuse everyone's quota. The second is to give the |
---|
298 | Repairer no user authority: instead, give the repairer its own account, and |
---|
299 | build it to keep track of which leases it is holding on behalf of one of its |
---|
300 | customers. This repairer will slowly accumulate quota space over time, as it |
---|
301 | creates new shares to replace ones that have decayed. Eventually, when the |
---|
302 | client comes back online, the client should establish its own leases on these |
---|
303 | new shares and allow the repairer to cancel its temporary ones. |
---|
304 | |
---|
305 | The third approach is in between the other two: give the repairer some |
---|
306 | limited authority over the customer's account, but not enough to let it |
---|
307 | consume the user's whole quota. |
---|
308 | |
---|
309 | To create the storage-authority portion of a (one-month) repair-cap, the |
---|
310 | client creates a new DSA keypair (repair_privkey, repair_pubkey), and then |
---|
311 | creates a signed message and bundles it into the repaircap: |
---|
312 | |
---|
313 | repair_msg = make_message(client_privkey, {'delegate_key': repair_pubkey, |
---|
314 | 'SI': storage_index, |
---|
315 | 'UEB_hash': file_ueb_hash}) |
---|
316 | repair_cap = (verify_cap, repair_privkey, (membership_card, repair_msg)) |
---|
317 | |
---|
318 | This gives the holder of the repair cap a time-limited authority to upload |
---|
319 | shares for the given storage index which contain the given data. This |
---|
320 | prohibits the repair-cap from being used to upload or repair any other file. |
---|
321 | |
---|
322 | When the repairer needs to upload a new share, it will use the delegated key |
---|
323 | to create its own signed message: |
---|
324 | |
---|
325 | upload_msg = make_message(repair_privkey, {'furl_to': repairer_rx_furl}) |
---|
326 | send(membership_card, repair_msg, upload_msg) |
---|
327 | |
---|
328 | The biggest problem with the low-authority approaches is the expiration time |
---|
329 | of the membership card, which limits the duration for which the repair-cap |
---|
330 | authority is valid. It would be nice if repair-caps could last a long time, |
---|
331 | years perhaps, so that clients can be offline for a similar period of time. |
---|
332 | However to retain a reasonable revocation interval for users, the membership |
---|
333 | card's before= timeout needs to be closer to a month. [it might be reasonable |
---|
334 | to use some sort of rights-amplification: the repairer has a special cert |
---|
335 | which allows it to remove the before= value from a chain]. |
---|
336 | |
---|
337 | |
---|
338 | === chain verification === |
---|
339 | |
---|
340 | The server will create a chain that starts with the AS's certificate: an |
---|
341 | unsigned message which derives its authority from being manually placed in |
---|
342 | the SS's configdir. The only limitation in the AS certificate will be on some |
---|
343 | kind of meta-account, in case we want to use multiple account servers and |
---|
344 | allow their account numbers to live in distinct number spaces (think |
---|
345 | sub-accounts or business partners to buy storage in bulk and resell it to |
---|
346 | users). The rest of the chain comes directly from what the client sent. |
---|
347 | |
---|
348 | The server walks the chain, keeping an accumulated limitations dictionary |
---|
349 | along the way. At each step it knows the pubkey that was delegated by the |
---|
350 | previous step. |
---|
351 | |
---|
352 | == client config == |
---|
353 | |
---|
354 | Clients are configured with an Account FURL that points to a private facet on |
---|
355 | the Account Server. The client generates a private key at startup. It sends |
---|
356 | the pubkey to the AS facet, which will return a signed delegate_key message |
---|
357 | (the "membership card") that grants the client's privkey any storage |
---|
358 | authority it wishes (as long as the account number is set to a specific |
---|
359 | value). |
---|
360 | |
---|
361 | The client stores this membership card in private/membership.cert . |
---|
362 | |
---|
363 | |
---|
364 | RIStorageServer messages will accept an optional account= argument. If left |
---|
365 | unspecified, the value is taken from the limitations that were curried into |
---|
366 | the SS facet. In all cases, the value used must meet those limitations. The |
---|
367 | value must not be None: Helpers/Repairers or other super-powered storage |
---|
368 | clients are obligated to specify an account number. |
---|
369 | |
---|
370 | == server config == |
---|
371 | |
---|
372 | Storage servers are configured with an unsigned root authority message. This |
---|
373 | is like the output of make_message(account_server_privkey, {}) but has empty |
---|
374 | 'signature' and 'pubkey' strings. This root goes into |
---|
375 | NODEDIR/storage_authority_root.cert . It is prepended to all chains that |
---|
376 | arrive. |
---|
377 | |
---|
378 | [if/when we accept multiple authorities, storage_authority_root.cert will |
---|
379 | turn into a storage_authority_root/ directory with *.cert files, and each |
---|
380 | arriving chain will cause a search through these root certs for a matching |
---|
381 | pubkey. The empty limitations will be replaced by {domain=X}, which is used |
---|
382 | as a sort of meta-account.. the details depend upon whether we express |
---|
383 | account numbers as an int (with various ranges) or as a tuple] |
---|
384 | |
---|
385 | The root authority message is published by the Account Server through its web |
---|
386 | interface, and also into a local file: NODEDIR/storage_authority_root.cert . |
---|
387 | The admin of the storage server is responsible for copying this file into |
---|
388 | place, thus enabling clients to use storage services. |
---|
389 | |
---|
390 | |
---|
391 | ---------------------------------------- |
---|
392 | |
---|
393 | -- Text beyond this point is out-of-date, and exists purely for background -- |
---|
394 | |
---|
395 | Each storage server offers a "public storage port", which only accepts signed |
---|
396 | messages. The Introducer mechanism exists to give clients a reference to a |
---|
397 | set of these public storage ports. All clients get access to the same ports. |
---|
398 | If clients did all their work themselves, these public storage ports would be |
---|
399 | enough, and no further code would be necessary (all storage requests would we |
---|
400 | signed the same way). |
---|
401 | |
---|
402 | Fundamentally, each storage request must be signed by the account's private |
---|
403 | key, giving the SS an authenticated Account Number to go with the request. |
---|
404 | This is used to index the correct cell in the lease matrix. The holder of the |
---|
405 | account privkey is allowed to manipulate their column of the matrix in any |
---|
406 | way they like: add leases, renew leases, delete leases. (TODO: for |
---|
407 | reconcilliation purposes, they should also be able to enumerate leases). The |
---|
408 | storage request is sent in the form of a signed request message, accompanied |
---|
409 | by the membership card. For example: |
---|
410 | |
---|
411 | req = SIGN("allocate SI=123 SSID=abc", accountprivkey) , membership_card |
---|
412 | -> RemoteBucketWriter reference |
---|
413 | |
---|
414 | Upon receipt of this request, the storage server will return a reference to a |
---|
415 | RemoteBucketWriter object, which the client can use to fill and close the |
---|
416 | bucket. The SS must perform two DSA signature verifications before accepting |
---|
417 | this request. The first is to validate the membership card: the Account |
---|
418 | Server's pubkey is used to verify the membership card's signature, from which |
---|
419 | an account pubkey and account# is extracted. The second is to validate the |
---|
420 | request: the account pubkey is used to verify the request signature. If both |
---|
421 | are valid, the full request (with account# and storage index) is delivered to |
---|
422 | the internal StorageServer object. |
---|
423 | |
---|
424 | Note that the signed request message includes the Storage Server's node ID, |
---|
425 | to prevent this storage server from taking the signed message and echoing to |
---|
426 | other storage servers. Each SS will ignore any request that is not addressed |
---|
427 | to the right SSID. Also note that the SI= and SSID= fields may contain |
---|
428 | wildcards, if the signing client so chooses. |
---|
429 | |
---|
430 | == Caching Signature Verification == |
---|
431 | |
---|
432 | We add some complexity to this simple model to achieve two goals: to enable |
---|
433 | fine-grained delegation of storage capabilities (specifically for renewers |
---|
434 | and repairers), and to reduce the number of public-key crypto operations that |
---|
435 | must be performed. |
---|
436 | |
---|
437 | The first enhancement is to allow the SS to cache the results of the |
---|
438 | verification step. To do this, the client creates a signed message which asks |
---|
439 | the SS to return a FURL of an object which can be used to execute further |
---|
440 | operations *without* a DSA signature. The FURL is expected to contain a |
---|
441 | MAC'ed string that contains the account# and the argument restrictions, |
---|
442 | effectively currying a subset of arguments into the RemoteReference. Clients |
---|
443 | which do all their operations themselves would use this to obtain a private |
---|
444 | storage port for each public storage port, stashing the FURLs in a local |
---|
445 | table, and then later storage operations would be done to those FURLs instead |
---|
446 | of creating signed requests. For example: |
---|
447 | |
---|
448 | req = SIGN("FURL(allocate SI=* SSID=abc)", accountprivkey), membership_card |
---|
449 | -> FURL |
---|
450 | Tub.getReference(FURL).allocate(SI=123) -> RemoteBucketWriter reference |
---|
451 | |
---|
452 | == Renewers and Repairers |
---|
453 | |
---|
454 | A brief digression is in order, to motivate the other enhancement. The |
---|
455 | "manifest" is a list of caps, one for each node that is reachable from the |
---|
456 | user's root directory/directories. The client is expected to generate the |
---|
457 | manifest on a periodic basis (perhaps once a day), and to keep track of which |
---|
458 | files/dirnodes have been added and removed. Items which have been removed |
---|
459 | must be explicitly dereferenced to reclaim their storage space. For grids |
---|
460 | which use per-file lease timers, the manifest is used to drive the Renewer: a |
---|
461 | process which renews the lease timers on a periodic basis (perhaps once a |
---|
462 | week). The manifest can also be used to drive a Checker, which in turn feeds |
---|
463 | work into the Repairer. |
---|
464 | |
---|
465 | The manifest should contain the minimum necessary authority to do its job, |
---|
466 | which generally means it contains the "verify cap" for each node. For |
---|
467 | immutable files, the verify cap contains the storage index and the UEB hash: |
---|
468 | enough information to retrieve and validate the ciphertext but not enough to |
---|
469 | decrypt it. For mutable files, the verify cap contains the storage index and |
---|
470 | the pubkey hash, which also serves to retrieve and validate ciphertext but |
---|
471 | not decrypt it. |
---|
472 | |
---|
473 | If the client does its own Renewing and Repairing, then a verifycap-based |
---|
474 | manifest is sufficient. However, if the user wants to be able to turn their |
---|
475 | computer off for a few months and still keep their files around, they need to |
---|
476 | delegate this job off to some other willing node. In a commercial network, |
---|
477 | there will be centralized (and perhaps trusted) Renewer/Repairer nodes, but |
---|
478 | in a friendnet these may not be available, and the user will depend upon one |
---|
479 | of their friends being willing to run this service for them while they are |
---|
480 | away. In either of these cases, the verifycaps are not enough: the Renewer |
---|
481 | will need additional authority to renew the client's leases, and the Repairer |
---|
482 | will need the authority to create new shares (in the client's name) when |
---|
483 | necessary. |
---|
484 | |
---|
485 | A trusted central service could be given all-account superpowers, allowing it |
---|
486 | to exercise storage authority on behalf of all users as it pleases. If this |
---|
487 | is the case, the verifycaps are sufficient. But if we desire to grant less |
---|
488 | authority to the Renewer/Repairer, then we need a mechanism to attenuate this |
---|
489 | authority. |
---|
490 | |
---|
491 | The usual objcap approach is to create a proxy: an intermediate object which |
---|
492 | itself is given full authority, but which is unwilling to exercise more than |
---|
493 | a portion of that authority in response to incoming requests. The |
---|
494 | not-fully-trusted service is then only given access to the proxy, not the |
---|
495 | final authority. For example: |
---|
496 | |
---|
497 | class Proxy(RemoteReference): |
---|
498 | def __init__(self, original, storage_index): |
---|
499 | self.original = original |
---|
500 | self.storage_index = storage_index |
---|
501 | def remote_renew_leases(self): |
---|
502 | return self.original.renew_leases(self.storage_index) |
---|
503 | renewer.grant(Proxy(target, "abcd")) |
---|
504 | |
---|
505 | But this approach interposes the proxy in the calling chain, requiring the |
---|
506 | machine which hosts the proxy to be available and on-line at all times, which |
---|
507 | runs opposite to our use case (turning the client off for a month). |
---|
508 | |
---|
509 | == Creating Attenuated Authorities == |
---|
510 | |
---|
511 | The other enhancement is to use more public-key operations to allow the |
---|
512 | delegation of reduced authority to external helper services. Specifically, we |
---|
513 | want to give then Renewer the ability to renew leases for a specific file, |
---|
514 | rather than giving it lease-renewal power for all files. Likewise, the |
---|
515 | Repairer should have the ability to create new shares, but only for the file |
---|
516 | that is being repaired, not for unrelated files. |
---|
517 | |
---|
518 | If we do not mind giving the storage servers the ability to replay their |
---|
519 | inbound message to other storage servers, then the client can simply generate |
---|
520 | a signed message with a wildcard SSID= argument and leave it in the care of |
---|
521 | the Renewer or Repairer. For example, the Renewer would get: |
---|
522 | |
---|
523 | SIGN("renew-lease SI=123 SSID=*", accountprivkey), membership_card |
---|
524 | |
---|
525 | Then, when the Renewer needed to renew a lease, it would deliver this signed |
---|
526 | request message to the storage server. The SS would verify the signatures |
---|
527 | just as if the message came from the original client, find them good, and |
---|
528 | perform the desired operation. With this approach, the manifest that is |
---|
529 | delivered to the remote Renewer process needs to include a signed |
---|
530 | lease-renewal request for each file: we use the term "renew-cap" for this |
---|
531 | combined (verifycap + signed lease-renewal request) message. Likewise the |
---|
532 | "repair-cap" would be the verifycap plus a signed allocate-bucket message. A |
---|
533 | renew-cap manifest would be enough for a remote Renewer to do its job, a |
---|
534 | repair-cap manifest would provide a remote Repairer with enough authority, |
---|
535 | and a cancel-cap manifest would be used for a remote Canceller (used, e.g., |
---|
536 | to make sure that file has been dereferenced even if the client does not |
---|
537 | stick around long enough to track down and inform all of the storage servers |
---|
538 | involved). |
---|
539 | |
---|
540 | The only concern is that the SS could also take this exact same renew-lease |
---|
541 | message and deliver it to other storage servers. This wouldn't cause a |
---|
542 | concern for mere lease renewal, but the allocate-share message might be a bit |
---|
543 | less comfortable (you might not want to grant the first storage server the |
---|
544 | ability to claim space in your name on all other storage servers). |
---|
545 | |
---|
546 | Ideally we'd like to send a different message to each storage server, each |
---|
547 | narrowed in scope to a single SSID, since then none of these messages would |
---|
548 | be useful on any other SS. If the client knew the identities of all the |
---|
549 | storage servers in the system ahead of time, it might create a whole slew of |
---|
550 | signed messages, but a) this is a lot of signatures, only a fraction of which |
---|
551 | will ever actually be used, and b) new servers might be introduced after the |
---|
552 | manifest is created, particularly if we're talking about repair-caps instead |
---|
553 | of renewal-caps. The Renewer can't generate these one-per-SSID messages from |
---|
554 | the SSID=* message, because it doesn't have a privkey to make the correct |
---|
555 | signatures. So without some other mechanism, we're stuck with these |
---|
556 | relatively coarse authorities. |
---|
557 | |
---|
558 | If we want to limit this sort of authority, then we need to introduce a new |
---|
559 | method. The client begins by generating a new DSA keypair. Then it signs a |
---|
560 | message that declares the new pubkey to be valid for a specific subset of |
---|
561 | storage operations (such as "renew-lease SI=123 SSID=*"). Then it delivers |
---|
562 | the new privkey, the declaration message, and the membership card to the |
---|
563 | Renewer. The renewer uses the new privkey to sign its own one-per-SSID |
---|
564 | request message for each server, then sends the (signed request, declaration, |
---|
565 | membership card) triple to the server. The server needs to perform three |
---|
566 | verification checks per message: first the membership card, then the |
---|
567 | declaration message, then the actual request message. |
---|
568 | |
---|
569 | == Other Enhancements == |
---|
570 | |
---|
571 | If a given authority is likely to be used multiple times, the same |
---|
572 | give-me-a-FURL trick can be used to cut down on the number of public key |
---|
573 | operations that must be performed. This is trickier with the per-SI messages. |
---|
574 | |
---|
575 | When storing the manifest, things like the membership card should be |
---|
576 | amortized across a set of common entries. An isolated renew-cap needs to |
---|
577 | contain the verifycap, the signed renewal request, and the membership card. |
---|
578 | But a manifest with a thousand entries should only include one copy of the |
---|
579 | membership card. |
---|
580 | |
---|
581 | It might be sensible to define a signed renewal request that grants authority |
---|
582 | for a set of storage indicies, so that the signature can be shared among |
---|
583 | several entries (to save space and perhaps processing time). The request |
---|
584 | could include a Bloom filter of authorized SI values: when the request is |
---|
585 | actually sent to the server, the renewer would add a list of actual SI values |
---|
586 | to renew, and the server would accept all that are contained in the filter. |
---|
587 | |
---|
588 | == Revocation == |
---|
589 | |
---|
590 | The lifetime of the storage authority included in the manifest's renew-caps |
---|
591 | or repair-caps will determine the lifetime of those caps. In particular, if |
---|
592 | we implement account revocation by using time-limited membership cards |
---|
593 | (requiring the client to get a new card once a month), then the repair-caps |
---|
594 | won't work for more than a month, which kind of defeats the purpose. |
---|
595 | |
---|
596 | A related issue is the FURL-shortcut: the MAC'ed message needs to include a |
---|
597 | validity period of some sort, and if the client tries to use a old FURL they |
---|
598 | should get an error message that will prompt them to try and acquire a newer |
---|
599 | one. |
---|
600 | |
---|
601 | ------------------------------ |
---|
602 | |
---|
603 | The client can produce a repair-cap manifest for a specific Repairer's |
---|
604 | pubkey, so it can produce a signed message that includes the pubkey (instead |
---|
605 | of needing to generate a new privkey just for this purpose). The result is |
---|
606 | not a capability, since it can only be used by the holder of the |
---|
607 | corresponding privkey. |
---|
608 | |
---|
609 | So the generic form of the storage operation message is the request (which |
---|
610 | has all the argument values filled in), followed by a chain of |
---|
611 | authorizations. The first authorization must be signed by the Account |
---|
612 | Server's key. Each authorization must be signed by the key mentioned in the |
---|
613 | previous one. Each one adds a new limitation on the power of the following |
---|
614 | ones. The actual request is bounded by all the limitations of the chain. |
---|
615 | |
---|
616 | The membership card is an authorization that simply limits the account number |
---|
617 | that can be used: "op=* SI=* SSID=* account=4 signed-by=CLIENT-PUBKEY". |
---|
618 | |
---|
619 | So a repair manifest created for a Repairer with pubkey ABCD could consist of |
---|
620 | a list of verifycaps plus a single authorization (using a Bloom filter to |
---|
621 | identify the SIs that were allowed): |
---|
622 | |
---|
623 | SIGN("allocate SI=[bloom] SSID=* signed-by=ABCD") |
---|
624 | |
---|
625 | If/when the Repairer needed to allocate a share, it would use its own privkey |
---|
626 | to sign an additional message and send the whole list to the SS: |
---|
627 | |
---|
628 | request=allocate SI=1234 SSID=EEFS account=4 shnum=2 |
---|
629 | SIGN("allocate SI=1234 SSID=EEFS", ABCD) |
---|
630 | SIGN("allocate SI=[bloom] SSID=* signed-by=ABCD", clientkey) |
---|
631 | membership: SIGN("op=* SI=* SSID=* account=4 signed-by=clientkey", ASkey) |
---|
632 | [implicit]: ASkey |
---|
633 | |
---|
634 | ---------------------------------------- |
---|
635 | |
---|
636 | Things would be a lot simpler if the Repairer (actually the Re-Leaser) had |
---|
637 | everybody's account authority. |
---|
638 | |
---|
639 | One simplifying approach: the Repairer/Re-Leaser has its own account, and the |
---|
640 | shares it creates are leased under that account number. The R/R keeps track |
---|
641 | of which leases it has created for whom. When the client eventually comes |
---|
642 | back online, it is told to perform a re-leasing run, and after that occurs |
---|
643 | the R/R can cancel its own temporary leases. |
---|
644 | |
---|
645 | This would effectively transfer storage quota from the original client to the |
---|
646 | R/R over time (as shares are regenerated by the R/R while the client remains |
---|
647 | offline). If the R/R is centrally managed, the quota mechanism can sum the |
---|
648 | R/R's numbers with the SS's numbers when determining how much storage is |
---|
649 | consumed by any given account. Not quite as clean as storing the exact |
---|
650 | information in the SS's lease tables directly, but: |
---|
651 | |
---|
652 | * the R/R no longer needs any special account authority (it merely needs an |
---|
653 | accurate account number, which can be supplied by giving the client a |
---|
654 | specific facet that is bound to that account number) |
---|
655 | * the verify-cap manifest is sufficient to perform repair |
---|
656 | * no extra DSA keys are necessary |
---|
657 | * account authority could be implemented with either DSA keys or personal SS |
---|
658 | facets: i.e. we don't need the delegability aspects of DSA keys for use by |
---|
659 | the repair mechanism (we might still want them to simplify introduction). |
---|
660 | |
---|
661 | I *think* this would eliminate all that complexity of chained authorization |
---|
662 | messages. |
---|