Ticket #833: all-833-darcspatch.txt

File all-833-darcspatch.txt, 276.2 KB (added by davidsarah, at 2010-01-27T23:23:25Z)

Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements (supercedes previous patches)

Line 
1Wed Jan 27 06:44:30 GMT Standard Time 2010  david-sarah@jacaranda.org
2  * Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
3
4Wed Jan 27 07:03:09 GMT Standard Time 2010  david-sarah@jacaranda.org
5  * Miscellaneous documentation, test, and code formatting tweaks.
6
7Wed Jan 27 23:06:42 GMT Standard Time 2010  david-sarah@jacaranda.org
8  * Address comments by Kevan on 833 and add test for stripping spaces
9
10New patches:
11
12[Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
13david-sarah@jacaranda.org**20100127064430
14 Ignore-this: 5ef6a3554cf6bef0bf0712cc7d6c0252
15] {
16hunk ./contrib/fuse/impl_c/blackmatch.py 4
17 #!/usr/bin/env python
18 
19 #-----------------------------------------------------------------------------------------------
20-from allmydata.uri import CHKFileURI, DirectoryURI, LiteralFileURI
21+from allmydata.uri import CHKFileURI, DirectoryURI, LiteralFileURI, is_literal_file_uri
22 from allmydata.scripts.common_http import do_http as do_http_req
23 from allmydata.util.hashutil import tagged_hash
24 from allmydata.util.assertutil import precondition
25hunk ./contrib/fuse/impl_c/blackmatch.py 338
26                 self.fname = self.tfs.cache.tmp_file(os.urandom(20))
27                 if self.fnode is None:
28                     log('TFF: [%s] open() for write: no file node, creating new File %s' % (self.name, self.fname, ))
29-                    self.fnode = File(0, 'URI:LIT:')
30+                    self.fnode = File(0, LiteralFileURI.BASE_STRING)
31                     self.fnode.tmp_fname = self.fname # XXX kill this
32                     self.parent.add_child(self.name, self.fnode, {})
33                 elif hasattr(self.fnode, 'tmp_fname'):
34hunk ./contrib/fuse/impl_c/blackmatch.py 365
35                     self.fname = self.fnode.tmp_fname
36                     log('TFF: reopening(%s) for reading' % self.fname)
37                 else:
38-                    if uri.startswith("URI:LIT") or not self.tfs.async:
39+                    if is_literal_file_uri(uri) or not self.tfs.async:
40                         log('TFF: synchronously fetching file from cache for reading')
41                         self.fname = self.tfs.cache.get_file(uri)
42                     else:
43hunk ./contrib/fuse/impl_c/blackmatch.py 1240
44 
45     def get_file(self, uri):
46         self.log('get_file(%s)' % (uri,))
47-        if uri.startswith("URI:LIT"):
48+        if is_literal_file_uri(uri):
49             return self.get_literal(uri)
50         else:
51             return self.get_chk(uri, async=False)
52hunk ./docs/frontends/webapi.txt 153
53 
54 === Child Lookup ===
55 
56-Tahoe directories contain named children, just like directories in a regular
57-local filesystem. These children can be either files or subdirectories.
58+Tahoe directories contain named child entries, just like directories in a regular
59+local filesystem. These child entries, called "dirnodes", consist of a name,
60+metadata, a write slot, and a read slot. The write and read slots normally contain
61+a write-cap and read-cap referring to the same object, which can be either a file
62+or a subdirectory. The write slot may be empty (actually, both may be empty,
63+but that is unusual).
64 
65 If you have a Tahoe URL that refers to a directory, and want to reference a
66 named child inside it, just append the child name to the URL. For example, if
67hunk ./docs/frontends/webapi.txt 397
68           } } } ]
69  }
70 
71+ For forward-compatibility, a mutable directory can also contain caps in
72+ a format that is unknown to the webapi server. When such caps are retrieved
73+ from a mutable directory in a "ro_uri" field, they will be prefixed with
74+ the string "ro.", indicating that they must not be decoded without
75+ checking that they are read-only. The "ro." prefix must not be stripped
76+ off without performing this check. (Future versions of the webapi server
77+ will perform it where necessary.)
78+
79+ If both the "rw_uri" and "ro_uri" fields are present in a given PROPDICT,
80+ and the webapi server recognizes the rw_uri as a write cap, then it will
81+ reset the ro_uri to the corresponding read cap and discard the original
82+ contents of ro_uri (in order to ensure that the two caps correspond to the
83+ same object and that the ro_uri is in fact read-only). However this may not
84+ happen for caps in a format unknown to the webapi server. Therefore, when
85+ writing a directory the webapi client should ensure that the contents
86+ of "rw_uri" and "ro_uri" for a given PROPDICT are a consistent
87+ (write cap, read cap) pair if possible. If the webapi client only has
88+ one cap and does not know whether it is a write cap or read cap, then
89+ it is acceptable to set "rw_uri" to that cap and omit "ro_uri". The
90+ client must not put a write cap into a "ro_uri" field.
91+
92  Note that the webapi-using client application must not provide the
93  "Content-Type: multipart/form-data" header that usually accompanies HTML
94  form submissions, since the body is not formatted this way. Doing so will
95hunk ./docs/frontends/webapi.txt 432
96 
97  Like t=mkdir-with-children above, but the new directory will be
98  deep-immutable. This means that the directory itself is immutable, and that
99- it can only contain deep-immutable objects, like immutable files, literal
100- files, and deep-immutable directories. A non-empty request body is
101- mandatory, since after the directory is created, it will not be possible to
102- add more children to it.
103+ it can only contain objects that are treated as being deep-immutable, like
104+ immutable files, literal files, and deep-immutable directories.
105+
106+ For forward-compatibility, a deep-immutable directory can also contain caps
107+ in a format that is unknown to the webapi server. When such caps are retrieved
108+ from a deep-immutable directory in a "ro_uri" field, they will be prefixed
109+ with the string "imm.", indicating that they must not be decoded without
110+ checking that they are immutable. The "imm." prefix must not be stripped
111+ off without performing this check. (Future versions of the webapi server
112+ will perform it where necessary.)
113+
114+ The cap for each child may be given either in the "rw_uri" or "ro_uri"
115+ field of the PROPDICT (not both). If a cap is given in the "rw_uri" field,
116+ then the webapi server will check that it is an immutable read-cap of a
117+ *known* format, and give an error if it is not. If a cap is given in the
118+ "ro_uri" field, then the webapi server will still check whether known
119+ caps are immutable, but for unknown caps it will simply assume that the
120+ cap can be stored, as described above. Note that an attacker would be
121+ able to store any cap in an immutable directory, so this check when
122+ creating the directory is only to help non-malicious clients to avoid
123+ accidentally giving away more authority than intended.
124+
125+ A non-empty request body is mandatory, since after the directory is created,
126+ it will not be possible to add more children to it.
127 
128 POST /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir
129 PUT /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir
130hunk ./docs/frontends/webapi.txt 462
131 
132  Create new directories as necessary to make sure that the named target
133  ($DIRCAP/SUBDIRS../SUBDIR) is a directory. This will create additional
134- intermediate directories as necessary. If the named target directory already
135- exists, this will make no changes to it.
136+ intermediate mutable directories as necessary. If the named target directory
137+ already exists, this will make no changes to it.
138 
139  If the final directory is created, it will be empty.
140 
141hunk ./docs/frontends/webapi.txt 467
142- This will return an error if a blocking file is present at any of the parent
143- names, preventing the server from creating the necessary parent directory.
144+ This operation will return an error if a blocking file is present at any of
145+ the parent names, preventing the server from creating the necessary parent
146+ directory; or if it would require changing an immutable directory.
147 
148  The write-cap of the new directory will be returned as the HTTP response
149  body.
150hunk ./docs/frontends/webapi.txt 476
151 
152 POST /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir-with-children
153 
154- Like above, but if the final directory is created, it will be populated with
155- initial children from the POST request body, as described above in the
156- /uri?t=mkdir-with-children operation.
157+ Like /uri?t=mkdir-with-children, but the final directory is created as a
158+ child of an existing mutable directory. This will create additional
159+ intermediate mutable directories as necessary. If the final directory is
160+ created, it will be populated with initial children from the POST request
161+ body, as described above.
162+
163+ This operation will return an error if a blocking file is present at any of
164+ the parent names, preventing the server from creating the necessary parent
165+ directory; or if it would require changing an immutable directory; or if
166+ the immediate parent directory already has a a child named SUBDIR.
167 
168 POST /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir-immutable
169 
170hunk ./docs/frontends/webapi.txt 489
171- Like above, but the final directory will be deep-immutable, with the
172- children specified as a JSON dictionary in the POST request body.
173+ Like /uri?t=mkdir-immutable, but the final directory is created as a child
174+ of an existing mutable directory. The final directory will be deep-immutable,
175+ and will be populated with the children specified as a JSON dictionary in
176+ the POST request body.
177+
178+ In Tahoe 1.6 this operation creates intermediate mutable directories if
179+ necessary, but that behaviour should not be relied on; see ticket #920.
180+
181+ This operation will return an error if the parent directory is immutable,
182+ or already has a child named SUBDIR.
183 
184 POST /uri/$DIRCAP/[SUBDIRS../]?t=mkdir&name=NAME
185 
186hunk ./docs/frontends/webapi.txt 502
187- Create a new empty directory and attach it to the given existing directory.
188- This will create additional intermediate directories as necessary.
189+ Create a new empty mutable directory and attach it to the given existing
190+ directory. This will create additional intermediate directories as necessary.
191 
192hunk ./docs/frontends/webapi.txt 505
193- The URL of this form points to the parent of the bottom-most new directory,
194- whereas the previous form has a URL that points directly to the bottom-most
195- new directory.
196+ This operation will return an error if a blocking file is present at any of
197+ the parent names, preventing the server from creating the necessary parent
198+ directory, or if it would require changing any immutable directory.
199+
200+ The URL of this operation points to the parent of the bottommost new directory,
201+ whereas the /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir operation above has a URL
202+ that points directly to the bottommost new directory.
203 
204 POST /uri/$DIRCAP/[SUBDIRS../]?t=mkdir-with-children&name=NAME
205 
206hunk ./docs/frontends/webapi.txt 515
207- As above, but the new directory will be populated with initial children via
208- the POST request body, as described in /uri?t=mkdir-with-children above.
209+ Like /uri/$DIRCAP/[SUBDIRS../]?t=mkdir&name=NAME, but the new directory will
210+ be populated with initial children via the POST request body. This command
211+ will create additional intermediate mutable directories as necessary.
212+
213+ This operation will return an error if a blocking file is present at any of
214+ the parent names, preventing the server from creating the necessary parent
215+ directory; or if it would require changing an immutable directory; or if
216+ the immediate parent directory already has a a child named NAME.
217+
218  Note that the name= argument must be passed as a queryarg, because the POST
219  request body is used for the initial children JSON.
220 
221hunk ./docs/frontends/webapi.txt 529
222 POST /uri/$DIRCAP/[SUBDIRS../]?t=mkdir-immutable&name=NAME
223 
224- As above, but the new directory will be deep-immutable, with the children
225- specified as a JSON dictionary in the POST request body. Again, the name=
226- argument must be passed as a queryarg.
227+ Like /uri/$DIRCAP/[SUBDIRS../]?t=mkdir-with-children&name=NAME, but the
228+ final directory will be deep-immutable. The children are specified as a
229+ JSON dictionary in the POST request body. Again, the name= argument must be
230+ passed as a queryarg.
231+
232+ In Tahoe 1.6 this operation creates intermediate mutable directories if
233+ necessary, but that behaviour should not be relied on; see ticket #920.
234+
235+ This operation will return an error if the parent directory is immutable,
236+ or already has a child named NAME.
237 
238 === Get Information About A File Or Directory (as JSON) ===
239 
240hunk ./docs/frontends/webapi.txt 761
241   "childinfo" is a dictionary that contains "rw_uri", "ro_uri", and
242   "metadata" keys. You can take the output of "GET /uri/$DIRCAP1?t=json" and
243   use it as the input to "POST /uri/$DIRCAP2?t=set_children" to make DIR2
244-  look very much like DIR1.
245+  look very much like DIR1 (except for any existing children of DIR2 that
246+  were not overwritten, and any existing "tahoe" metadata keys as described
247+  below).
248 
249   When the set_children request contains a child name that already exists in
250   the target directory, this command defaults to overwriting that child with
251hunk ./docs/frontends/webapi.txt 964
252 
253 POST /uri/$DIRCAP/[SUBDIRS../]?t=upload
254 
255- This uploads a file, and attaches it as a new child of the given directory.
256- The file must be provided as the "file" field of an HTML encoded form body,
257- produced in response to an HTML form like this:
258+ This uploads a file, and attaches it as a new child of the given directory,
259+ which must be mutable. The file must be provided as the "file" field of an
260+ HTML-encoded form body, produced in response to an HTML form like this:
261   <form action="." method="POST" enctype="multipart/form-data">
262    <input type="hidden" name="t" value="upload" />
263    <input type="file" name="file" />
264hunk ./docs/frontends/webapi.txt 1009
265 POST /uri/$DIRCAP/[SUBDIRS../]FILENAME?t=upload
266 
267  This also uploads a file and attaches it as a new child of the given
268- directory. It is a slight variant of the previous operation, as the URL
269- refers to the target file rather than the parent directory. It is otherwise
270- identical: this accepts mutable= and when_done= arguments too.
271+ directory, which must be mutable. It is a slight variant of the previous
272+ operation, as the URL refers to the target file rather than the parent
273+ directory. It is otherwise identical: this accepts mutable= and when_done=
274+ arguments too.
275 
276 POST /uri/$FILECAP?t=upload
277 
278hunk ./docs/frontends/webapi.txt 1040
279 
280 POST /uri/$DIRCAP/[SUBDIRS../]?t=delete&name=CHILDNAME
281 
282- This instructs the node to delete a child object (file or subdirectory) from
283- the given directory. Note that the entire subtree is removed. This is
284- somewhat like "rm -rf" (from the point of view of the parent), but other
285- references into the subtree will see that the child subdirectories are not
286- modified by this operation. Only the link from the given directory to its
287- child is severed.
288+ This instructs the node to remove a child object (file or subdirectory) from
289+ the given directory, which must be mutable. Note that the entire subtree is
290+ unlinked from the parent. Unlike deleting a subdirectory in a UNIX local
291+ filesystem, the subtree need not be empty; if it isn't, then other references
292+ into the subtree will see that the child subdirectories are not modified by
293+ this operation. Only the link from the given directory to its child is severed.
294 
295 === Renaming A Child ===
296 
297hunk ./docs/frontends/webapi.txt 1051
298 POST /uri/$DIRCAP/[SUBDIRS../]?t=rename&from_name=OLD&to_name=NEW
299 
300- This instructs the node to rename a child of the given directory. This is
301- exactly the same as removing the child, then adding the same child-cap under
302- the new name. This operation cannot move the child to a different directory.
303+ This instructs the node to rename a child of the given directory, which must
304+ be mutable. This has a similar effect to removing the child, then adding the
305+ same child-cap under the new name, except that it preserves metadata. This
306+ operation cannot move the child to a different directory.
307 
308  This operation will replace any existing child of the new name, making it
309  behave like the UNIX "mv -f" command.
310hunk ./docs/frontends/webapi.txt 1250
311 
312   "path": a list of strings, with the path that is traversed to reach the
313           object
314-  "cap": a writecap for the file or directory, if available, else a readcap
315-  "verifycap": a verifycap for the file or directory
316-  "repaircap": the weakest cap which can still be used to repair the object
317+  "cap": a write-cap URI for the file or directory, if available, else a
318+         read-cap URI
319+  "verifycap": a verify-cap URI for the file or directory
320+  "repaircap": an URI for the weakest cap that can still be used to repair
321+               the object
322   "storage-index": a base32 storage index for the object
323   "check-results": a copy of the dictionary which would be returned by
324                    t=check&output=json, with three top-level keys:
325hunk ./docs/frontends/webapi.txt 1496
326 
327   "path": a list of strings, with the path that is traversed to reach the
328           object
329-  "cap": a writecap for the file or directory, if available, else a readcap
330-  "verifycap": a verifycap for the file or directory
331-  "repaircap": the weakest cap which can still be used to repair the object
332+  "cap": a write-cap URI for the file or directory, if available, else a
333+         read-cap URI
334+  "verifycap": a verify-cap URI for the file or directory
335+  "repaircap": an URI for the weakest cap that can still be used to repair
336+               the object
337   "storage-index": a base32 storage index for the object
338 
339  Note that non-distributed files (i.e. LIT files) will have values of None
340hunk ./docs/frontends/webapi.txt 1731
341 child's name and the child's URI are included in the results of listing the
342 parent directory, so it isn't any harder to use the URI for this purpose.
343 
344+The read and write caps in a given directory node are separate URIs, and
345+can't be assumed to point to the same object even if they were retrieved in
346+the same operation (although the webapi server attempts to ensure this
347+in most cases). If you need to rely on that property, you should explicitly
348+verify it. More generally, you should not make assumptions about the
349+internal consistency of the contents of mutable directories. As a result
350+of the signatures on mutable object versions, it is guaranteed that a given
351+version was written in a single update, but -- as in the case of a file --
352+the contents may have been chosen by a malicious writer in a way that is
353+designed to confuse applications that rely on their consistency.
354+
355 In general, use names if you want "whatever object (whether file or
356 directory) is found by following this name (or sequence of names) when my
357 request reaches the server". Use URIs if you want "this particular object".
358hunk ./src/allmydata/client.py 474
359     # dirnodes. The first takes a URI and produces a filenode or (new-style)
360     # dirnode. The other three create brand-new filenodes/dirnodes.
361 
362-    def create_node_from_uri(self, writecap, readcap=None):
363-        # this returns synchronously.
364-        return self.nodemaker.create_from_cap(writecap, readcap)
365+    def create_node_from_uri(self, write_uri, read_uri=None, deep_immutable=False, name="<unknown name>"):
366+        # This returns synchronously.
367+        # Note that it does *not* validate the write_uri and read_uri; instead we
368+        # may get an opaque node if there were any problems.
369+        return self.nodemaker.create_from_cap(write_uri, read_uri, deep_immutable=deep_immutable, name=name)
370 
371     def create_dirnode(self, initial_children={}):
372         d = self.nodemaker.create_new_mutable_directory(initial_children)
373hunk ./src/allmydata/client.py 483
374         return d
375+
376     def create_immutable_dirnode(self, children, convergence=None):
377         return self.nodemaker.create_immutable_directory(children, convergence)
378 
379hunk ./src/allmydata/control.py 8
380 from twisted.internet import defer
381 from twisted.internet.interfaces import IConsumer
382 from foolscap.api import Referenceable
383-from allmydata.interfaces import RIControlClient
384+from allmydata.interfaces import RIControlClient, IFileNode
385 from allmydata.util import fileutil, mathutil
386 from allmydata.immutable import upload
387 from twisted.python import log
388hunk ./src/allmydata/control.py 70
389         return d
390 
391     def remote_download_from_uri_to_file(self, uri, filename):
392-        filenode = self.parent.create_node_from_uri(uri)
393+        filenode = self.parent.create_node_from_uri(uri, name=filename)
394+        if not IFileNode.providedBy(filenode):
395+            raise AssertionError("The URI does not reference a file.")
396         c = FileWritingConsumer(filename)
397         d = filenode.read(c)
398         d.addCallback(lambda res: filename)
399hunk ./src/allmydata/control.py 204
400             if i >= self.count:
401                 return
402             n = self.parent.create_node_from_uri(self.uris[i])
403+            if not IFileNode.providedBy(n):
404+                raise AssertionError("The URI does not reference a file.")
405             if n.is_mutable():
406                 d1 = n.download_best_version()
407             else:
408hunk ./src/allmydata/dirnode.py 8
409 from twisted.internet import defer
410 from foolscap.api import fireEventually
411 import simplejson
412-from allmydata.mutable.common import NotMutableError
413+from allmydata.mutable.common import NotWriteableError
414 from allmydata.mutable.filenode import MutableFileNode
415hunk ./src/allmydata/dirnode.py 10
416-from allmydata.unknown import UnknownNode
417+from allmydata.unknown import UnknownNode, strip_prefix_for_ro
418 from allmydata.interfaces import IFilesystemNode, IDirectoryNode, IFileNode, \
419      IImmutableFileNode, IMutableFileNode, \
420      ExistingChildError, NoSuchChildError, ICheckable, IDeepCheckable, \
421hunk ./src/allmydata/dirnode.py 14
422-     CannotPackUnknownNodeError
423+     MustBeDeepImmutableError, CapConstraintError
424 from allmydata.check_results import DeepCheckResults, \
425      DeepCheckAndRepairResults
426 from allmydata.monitor import Monitor
427hunk ./src/allmydata/dirnode.py 43
428         new_contents = self.node._pack_contents(children)
429         return new_contents
430 
431+
432 class MetadataSetter:
433     def __init__(self, node, name, metadata):
434         self.node = node
435hunk ./src/allmydata/dirnode.py 79
436         for (name, (child, new_metadata)) in self.entries.iteritems():
437             precondition(isinstance(name, unicode), name)
438             precondition(IFilesystemNode.providedBy(child), child)
439+
440+            # Strictly speaking this is redundant because we would raise the
441+            # error again in pack_children.
442+            child.raise_error()
443+
444             if name in children:
445                 if not self.overwrite:
446                     raise ExistingChildError("child '%s' already exists" % name)
447hunk ./src/allmydata/dirnode.py 132
448         new_contents = self.node._pack_contents(children)
449         return new_contents
450 
451-def _encrypt_rwcap(filenode, rwcap):
452-    assert isinstance(rwcap, str)
453+def _encrypt_rw_uri(filenode, rw_uri):
454+    assert isinstance(rw_uri, str)
455     writekey = filenode.get_writekey()
456     if not writekey:
457         return ""
458hunk ./src/allmydata/dirnode.py 137
459-    salt = hashutil.mutable_rwcap_salt_hash(rwcap)
460+    salt = hashutil.mutable_rwcap_salt_hash(rw_uri)
461     key = hashutil.mutable_rwcap_key_hash(salt, writekey)
462     cryptor = AES(key)
463hunk ./src/allmydata/dirnode.py 140
464-    crypttext = cryptor.process(rwcap)
465+    crypttext = cryptor.process(rw_uri)
466     mac = hashutil.hmac(key, salt + crypttext)
467     assert len(mac) == 32
468     return salt + crypttext + mac
469hunk ./src/allmydata/dirnode.py 147
470     # The MAC is not checked by readers in Tahoe >= 1.3.0, but we still
471     # produce it for the sake of older readers.
472 
473-class MustBeDeepImmutable(Exception):
474-    """You tried to add a non-deep-immutable node to a deep-immutable
475-    directory."""
476-
477 def pack_children(filenode, children, deep_immutable=False):
478     """Take a dict that maps:
479          children[unicode_name] = (IFileSystemNode, metadata_dict)
480hunk ./src/allmydata/dirnode.py 157
481     time.
482 
483     If deep_immutable is True, I will require that all my children are deeply
484-    immutable, and will raise a MustBeDeepImmutable exception if not.
485+    immutable, and will raise a MustBeDeepImmutableError if not.
486     """
487 
488     has_aux = isinstance(children, AuxValueDict)
489hunk ./src/allmydata/dirnode.py 166
490         assert isinstance(name, unicode)
491         entry = None
492         (child, metadata) = children[name]
493-        if deep_immutable and child.is_mutable():
494-            # TODO: consider adding IFileSystemNode.is_deep_immutable()
495-            raise MustBeDeepImmutable("child '%s' is mutable" % (name,))
496+        child.raise_error()
497+        if deep_immutable and not child.is_allowed_in_immutable_directory():
498+            raise MustBeDeepImmutableError("child '%s' is not allowed in an immutable directory" % (name,), name)
499         if has_aux:
500             entry = children.get_aux(name)
501         if not entry:
502hunk ./src/allmydata/dirnode.py 174
503             assert IFilesystemNode.providedBy(child), (name,child)
504             assert isinstance(metadata, dict)
505-            rwcap = child.get_uri() # might be RO if the child is not writeable
506-            if rwcap is None:
507-                rwcap = ""
508-            assert isinstance(rwcap, str), rwcap
509-            rocap = child.get_readonly_uri()
510-            if rocap is None:
511-                rocap = ""
512-            assert isinstance(rocap, str), rocap
513+            rw_uri = child.get_write_uri()
514+            if rw_uri is None:
515+                rw_uri = ""
516+            assert isinstance(rw_uri, str), rw_uri
517+           
518+            # should be prevented by MustBeDeepImmutableError check above
519+            assert not (rw_uri and deep_immutable)
520+
521+            ro_uri = child.get_readonly_uri()
522+            if ro_uri is None:
523+                ro_uri = ""
524+            assert isinstance(ro_uri, str), ro_uri
525             entry = "".join([netstring(name.encode("utf-8")),
526hunk ./src/allmydata/dirnode.py 187
527-                             netstring(rocap),
528-                             netstring(_encrypt_rwcap(filenode, rwcap)),
529+                             netstring(strip_prefix_for_ro(ro_uri, deep_immutable)),
530+                             netstring(_encrypt_rw_uri(filenode, rw_uri)),
531                              netstring(simplejson.dumps(metadata))])
532         entries.append(netstring(entry))
533     return "".join(entries)
534hunk ./src/allmydata/dirnode.py 239
535         plaintext = cryptor.process(crypttext)
536         return plaintext
537 
538-    def _create_node(self, rwcap, rocap):
539-        return self._nodemaker.create_from_cap(rwcap, rocap)
540+    def _create_and_validate_node(self, rw_uri, ro_uri, name):
541+        node = self._nodemaker.create_from_cap(rw_uri, ro_uri,
542+                                               deep_immutable=not self.is_mutable(),
543+                                               name=name)
544+        node.raise_error()
545+        return node
546 
547     def _unpack_contents(self, data):
548         # the directory is serialized as a list of netstrings, one per child.
549hunk ./src/allmydata/dirnode.py 248
550-        # Each child is serialized as a list of four netstrings: (name,
551-        # rocap, rwcap, metadata), in which the name,rocap,metadata are in
552-        # cleartext. The 'name' is UTF-8 encoded. The rwcap is formatted as:
553-        # pack("16ss32s", iv, AES(H(writekey+iv), plaintextrwcap), mac)
554+        # Each child is serialized as a list of four netstrings: (name, ro_uri,
555+        # rwcapdata, metadata), in which the name, ro_uri, metadata are in
556+        # cleartext. The 'name' is UTF-8 encoded. The rwcapdata is formatted as:
557+        # pack("16ss32s", iv, AES(H(writekey+iv), plaintext_rw_uri), mac)
558         assert isinstance(data, str), (repr(data), type(data))
559         # an empty directory is serialized as an empty string
560         if data == "":
561hunk ./src/allmydata/dirnode.py 257
562             return AuxValueDict()
563         writeable = not self.is_readonly()
564+        mutable = self.is_mutable()
565         children = AuxValueDict()
566         position = 0
567         while position < len(data):
568hunk ./src/allmydata/dirnode.py 263
569             entries, position = split_netstring(data, 1, position)
570             entry = entries[0]
571-            (name, rocap, rwcapdata, metadata_s), subpos = split_netstring(entry, 4)
572+            (name, ro_uri, rwcapdata, metadata_s), subpos = split_netstring(entry, 4)
573+            if not mutable and len(rwcapdata) > 0:
574+                raise ValueError("the rwcapdata field of a dirnode in an immutable directory was not empty")
575             name = name.decode("utf-8")
576hunk ./src/allmydata/dirnode.py 267
577-            rwcap = None
578+            rw_uri = ""
579             if writeable:
580hunk ./src/allmydata/dirnode.py 269
581-                rwcap = self._decrypt_rwcapdata(rwcapdata)
582-            if not rwcap:
583-                rwcap = None # rwcap is None or a non-empty string
584-            if not rocap:
585-                rocap = None # rocap is None or a non-empty string
586-            child = self._create_node(rwcap, rocap)
587-            metadata = simplejson.loads(metadata_s)
588-            assert isinstance(metadata, dict)
589-            children.set_with_aux(name, (child, metadata), auxilliary=entry)
590+                rw_uri = self._decrypt_rwcapdata(rwcapdata)
591+
592+            # Since the encryption uses CTR mode, it currently leaks the length of the
593+            # plaintext rw_uri -- and therefore whether it is present, i.e. whether the
594+            # dirnode is writeable (ticket #925). By stripping spaces in Tahoe >= 1.6.0,
595+            # we may make it easier for future versions to plug this leak.
596+            # ro_uri is treated in the same way for consistency.
597+            # rw_uri and ro_uri will be either None or a non-empty string.
598+
599+            rw_uri = rw_uri.strip(' ') or None
600+            ro_uri = ro_uri.strip(' ') or None
601+
602+            try:
603+                child = self._create_and_validate_node(rw_uri, ro_uri, name)
604+                if mutable or child.is_allowed_in_immutable_directory():
605+                    metadata = simplejson.loads(metadata_s)
606+                    assert isinstance(metadata, dict)
607+                    children[name] = (child, metadata)
608+                    children.set_with_aux(name, (child, metadata), auxilliary=entry)
609+                else:
610+                    log.msg(format="mutable cap for child '%(name)s' unpacked from an immutable directory",
611+                                   name=name.encode("utf-8"),
612+                                   facility="tahoe.webish", level=log.UNUSUAL)
613+            except CapConstraintError, e:
614+                log.msg(format="unmet constraint on cap for child '%(name)s' unpacked from a directory:\n"
615+                               "%(message)s", message=e.args[0], name=name.encode("utf-8"),
616+                               facility="tahoe.webish", level=log.UNUSUAL)
617+
618         return children
619 
620     def _pack_contents(self, children):
621hunk ./src/allmydata/dirnode.py 305
622 
623     def is_readonly(self):
624         return self._node.is_readonly()
625+
626     def is_mutable(self):
627         return self._node.is_mutable()
628 
629hunk ./src/allmydata/dirnode.py 309
630+    def is_unknown(self):
631+        return False
632+
633+    def is_allowed_in_immutable_directory(self):
634+        return not self._node.is_mutable()
635+
636+    def raise_error(self):
637+        pass
638+
639     def get_uri(self):
640         return self._uri.to_string()
641 
642hunk ./src/allmydata/dirnode.py 321
643+    def get_write_uri(self):
644+        if self.is_readonly():
645+            return None
646+        return self._uri.to_string()
647+
648     def get_readonly_uri(self):
649         return self._uri.get_readonly().to_string()
650 
651hunk ./src/allmydata/dirnode.py 331
652     def get_cap(self):
653         return self._uri
654+
655     def get_readcap(self):
656         return self._uri.get_readonly()
657hunk ./src/allmydata/dirnode.py 334
658+
659     def get_verify_cap(self):
660         return self._uri.get_verify_cap()
661hunk ./src/allmydata/dirnode.py 337
662+
663     def get_repair_cap(self):
664         if self._node.is_readonly():
665             return None # readonly (mutable) dirnodes are not yet repairable
666hunk ./src/allmydata/dirnode.py 403
667     def set_metadata_for(self, name, metadata):
668         assert isinstance(name, unicode)
669         if self.is_readonly():
670-            return defer.fail(NotMutableError())
671+            return defer.fail(NotWriteableError())
672         assert isinstance(metadata, dict)
673         s = MetadataSetter(self, name, metadata)
674         d = self._node.modify(s.modify)
675hunk ./src/allmydata/dirnode.py 451
676         precondition(isinstance(name, unicode), name)
677         precondition(isinstance(writecap, (str,type(None))), writecap)
678         precondition(isinstance(readcap, (str,type(None))), readcap)
679-        child_node = self._create_node(writecap, readcap)
680-        if isinstance(child_node, UnknownNode):
681-            # don't be willing to pack unknown nodes: we might accidentally
682-            # put some write-authority into the rocap slot because we don't
683-            # know how to diminish the URI they gave us. We don't even know
684-            # if they gave us a readcap or a writecap.
685-            msg = "cannot pack unknown node as child %s" % str(name)
686-            raise CannotPackUnknownNodeError(msg)
687+           
688+        # We now allow packing unknown nodes, provided they are valid
689+        # for this type of directory.
690+        child_node = self._create_and_validate_node(writecap, readcap, name)
691         d = self.set_node(name, child_node, metadata, overwrite)
692         d.addCallback(lambda res: child_node)
693         return d
694hunk ./src/allmydata/dirnode.py 472
695                 writecap, readcap, metadata = e
696             precondition(isinstance(writecap, (str,type(None))), writecap)
697             precondition(isinstance(readcap, (str,type(None))), readcap)
698-            child_node = self._create_node(writecap, readcap)
699-            if isinstance(child_node, UnknownNode):
700-                msg = "cannot pack unknown node as child %s" % str(name)
701-                raise CannotPackUnknownNodeError(msg)
702+           
703+            # We now allow packing unknown nodes, provided they are valid
704+            # for this type of directory.
705+            child_node = self._create_and_validate_node(writecap, readcap, name)
706             a.set_node(name, child_node, metadata)
707         d = self._node.modify(a.modify)
708         d.addCallback(lambda ign: self)
709hunk ./src/allmydata/dirnode.py 488
710         same name.
711 
712         If this directory node is read-only, the Deferred will errback with a
713-        NotMutableError."""
714+        NotWriteableError."""
715 
716         precondition(IFilesystemNode.providedBy(child), child)
717 
718hunk ./src/allmydata/dirnode.py 493
719         if self.is_readonly():
720-            return defer.fail(NotMutableError())
721+            return defer.fail(NotWriteableError())
722         assert isinstance(name, unicode)
723         assert IFilesystemNode.providedBy(child), child
724         a = Adder(self, overwrite=overwrite)
725hunk ./src/allmydata/dirnode.py 505
726     def set_nodes(self, entries, overwrite=True):
727         precondition(isinstance(entries, dict), entries)
728         if self.is_readonly():
729-            return defer.fail(NotMutableError())
730+            return defer.fail(NotWriteableError())
731         a = Adder(self, entries, overwrite=overwrite)
732         d = self._node.modify(a.modify)
733         d.addCallback(lambda res: self)
734hunk ./src/allmydata/dirnode.py 519
735         the operation completes."""
736         assert isinstance(name, unicode)
737         if self.is_readonly():
738-            return defer.fail(NotMutableError())
739+            return defer.fail(NotWriteableError())
740         d = self._uploader.upload(uploadable)
741hunk ./src/allmydata/dirnode.py 521
742-        d.addCallback(lambda results: results.uri)
743-        d.addCallback(self._nodemaker.create_from_cap)
744+        d.addCallback(lambda results:
745+                      self._create_and_validate_node(results.uri, None, name))
746         d.addCallback(lambda node:
747                       self.set_node(name, node, metadata, overwrite))
748         return d
749hunk ./src/allmydata/dirnode.py 532
750         fires (with the node just removed) when the operation finishes."""
751         assert isinstance(name, unicode)
752         if self.is_readonly():
753-            return defer.fail(NotMutableError())
754+            return defer.fail(NotWriteableError())
755         deleter = Deleter(self, name)
756         d = self._node.modify(deleter.modify)
757         d.addCallback(lambda res: deleter.old_child)
758hunk ./src/allmydata/dirnode.py 542
759                             mutable=True):
760         assert isinstance(name, unicode)
761         if self.is_readonly():
762-            return defer.fail(NotMutableError())
763+            return defer.fail(NotWriteableError())
764         if mutable:
765             d = self._nodemaker.create_new_mutable_directory(initial_children)
766         else:
767hunk ./src/allmydata/dirnode.py 564
768         Deferred that fires when the operation finishes."""
769         assert isinstance(current_child_name, unicode)
770         if self.is_readonly() or new_parent.is_readonly():
771-            return defer.fail(NotMutableError())
772+            return defer.fail(NotWriteableError())
773         if new_child_name is None:
774             new_child_name = current_child_name
775         assert isinstance(new_child_name, unicode)
776hunk ./src/allmydata/immutable/filenode.py 20
777 class _ImmutableFileNodeBase(object):
778     implements(IImmutableFileNode, ICheckable)
779 
780+    def get_write_uri(self):
781+        return None
782+
783     def get_readonly_uri(self):
784         return self.get_uri()
785 
786hunk ./src/allmydata/immutable/filenode.py 32
787     def is_readonly(self):
788         return True
789 
790+    def is_unknown(self):
791+        return False
792+
793+    def is_allowed_in_immutable_directory(self):
794+        return True
795+
796+    def raise_error(self):
797+        pass
798+
799     def __hash__(self):
800         return self.u.__hash__()
801     def __eq__(self, other):
802hunk ./src/allmydata/interfaces.py 459
803 class IDirnodeURI(Interface):
804     """I am a URI which represents a dirnode."""
805 
806-
807 class IFileURI(Interface):
808     """I am a URI which represents a filenode."""
809     def get_size():
810hunk ./src/allmydata/interfaces.py 469
811 
812 class IMutableFileURI(Interface):
813     """I am a URI which represents a mutable filenode."""
814+
815 class IDirectoryURI(Interface):
816     pass
817hunk ./src/allmydata/interfaces.py 472
818+
819 class IReadonlyDirectoryURI(Interface):
820     pass
821 
822hunk ./src/allmydata/interfaces.py 476
823-class CannotPackUnknownNodeError(Exception):
824-    """UnknownNodes (using filecaps from the future that we don't understand)
825-    cannot yet be copied safely, so I refuse to copy them."""
826+class CapConstraintError(Exception):
827+    """A constraint on a cap was violated."""
828 
829hunk ./src/allmydata/interfaces.py 479
830-class UnhandledCapTypeError(Exception):
831-    """I recognize the cap/URI, but I cannot create an IFilesystemNode for
832-    it."""
833+class MustBeDeepImmutableError(CapConstraintError):
834+    """Mutable children cannot be added to an immutable directory.
835+    Also, caps obtained from an immutable directory can trigger this error
836+    if they are later found to refer to a mutable object and then used."""
837 
838hunk ./src/allmydata/interfaces.py 484
839-class NotDeepImmutableError(Exception):
840-    """Deep-immutable directories can only contain deep-immutable children"""
841+class MustBeReadonlyError(CapConstraintError):
842+    """Known write caps cannot be specified in a ro_uri field. Also,
843+    caps obtained from a ro_uri field can trigger this error if they
844+    are later found to be write caps and then used."""
845+
846+class MustNotBeUnknownRWError(CapConstraintError):
847+    """Cannot add an unknown child cap specified in a rw_uri field."""
848 
849 # The hierarchy looks like this:
850 #  IFilesystemNode
851hunk ./src/allmydata/interfaces.py 527
852         """
853 
854     def get_uri():
855-        """
856-        Return the URI string that can be used by others to get access to
857-        this node. If this node is read-only, the URI will only offer
858+        """Return the URI string corresponding to the strongest cap associated
859+        with this node. If this node is read-only, the URI will only offer
860         read-only access. If this node is read-write, the URI will offer
861         read-write access.
862 
863hunk ./src/allmydata/interfaces.py 536
864         read-only access with others, use get_readonly_uri().
865         """
866 
867+    def get_write_uri(n):
868+        """Return the URI string that can be used by others to get write
869+        access to this node, if it is writeable. If this is a read-only node,
870+        return None."""
871+
872     def get_readonly_uri():
873         """Return the URI string that can be used by others to get read-only
874         access to this node. The result is a read-only URI, regardless of
875hunk ./src/allmydata/interfaces.py 570
876         file.
877         """
878 
879+    def is_unknown():
880+        """Return True if this is an unknown node."""
881+
882+    def is_allowed_in_immutable_directory():
883+        """Return True if this node is allowed as a child of a deep-immutable
884+        directory. This is true if either the node is of a known-immutable type,
885+        or it is unknown and read-only.
886+        """
887+
888+    def raise_error():
889+        """Raise any error associated with this node."""
890+
891     def get_size():
892         """Return the length (in bytes) of the data this node represents. For
893         directory nodes, I return the size of the backing store. I return
894hunk ./src/allmydata/interfaces.py 927
895         ctime/mtime semantics of traditional filesystems.
896 
897         If this directory node is read-only, the Deferred will errback with a
898-        NotMutableError."""
899+        NotWriteableError."""
900 
901     def set_children(entries, overwrite=True):
902         """Add multiple children (by writecap+readcap) to a directory node.
903hunk ./src/allmydata/interfaces.py 953
904         ctime/mtime semantics of traditional filesystems.
905 
906         If this directory node is read-only, the Deferred will errback with a
907-        NotMutableError."""
908+        NotWriteableError."""
909 
910     def set_nodes(entries, overwrite=True):
911         """Add multiple children to a directory node. Takes a dict mapping
912hunk ./src/allmydata/interfaces.py 2099
913     Tahoe process will typically have a single NodeMaker, but unit tests may
914     create simplified/mocked forms for testing purposes.
915     """
916-    def create_from_cap(writecap, readcap=None):
917+    def create_from_cap(writecap, readcap=None, **kwargs):
918         """I create an IFilesystemNode from the given writecap/readcap. I can
919         only provide nodes for existing file/directory objects: use my other
920         methods to create new objects. I return synchronously."""
921hunk ./src/allmydata/mutable/common.py 11
922                           # creation
923 MODE_READ = "MODE_READ"
924 
925-class NotMutableError(Exception):
926+class NotWriteableError(Exception):
927     pass
928 
929 class NeedMoreDataError(Exception):
930hunk ./src/allmydata/mutable/filenode.py 217
931 
932     def get_uri(self):
933         return self._uri.to_string()
934+
935+    def get_write_uri(self):
936+        if self.is_readonly():
937+            return None
938+        return self._uri.to_string()
939+
940     def get_readonly_uri(self):
941         return self._uri.get_readonly().to_string()
942 
943hunk ./src/allmydata/mutable/filenode.py 236
944 
945     def is_mutable(self):
946         return self._uri.is_mutable()
947+
948     def is_readonly(self):
949         return self._uri.is_readonly()
950 
951hunk ./src/allmydata/mutable/filenode.py 240
952+    def is_unknown(self):
953+        return False
954+
955+    def is_allowed_in_immutable_directory(self):
956+        return not self._uri.is_mutable()
957+
958+    def raise_error(self):
959+        pass
960+
961     def __hash__(self):
962         return hash((self.__class__, self._uri))
963     def __cmp__(self, them):
964hunk ./src/allmydata/nodemaker.py 4
965 import weakref
966 from zope.interface import implements
967 from allmydata.util.assertutil import precondition
968-from allmydata.interfaces import INodeMaker, NotDeepImmutableError
969+from allmydata.interfaces import INodeMaker, MustBeDeepImmutableError
970 from allmydata.immutable.filenode import ImmutableFileNode, LiteralFileNode
971 from allmydata.immutable.upload import Data
972 from allmydata.mutable.filenode import MutableFileNode
973hunk ./src/allmydata/nodemaker.py 47
974     def _create_dirnode(self, filenode):
975         return DirectoryNode(filenode, self, self.uploader)
976 
977-    def create_from_cap(self, writecap, readcap=None):
978+    def create_from_cap(self, writecap, readcap=None, deep_immutable=False, name=u"<unknown name>"):
979         # this returns synchronously. It starts with a "cap string".
980         assert isinstance(writecap, (str, type(None))), type(writecap)
981         assert isinstance(readcap,  (str, type(None))), type(readcap)
982hunk ./src/allmydata/nodemaker.py 51
983+       
984         bigcap = writecap or readcap
985         if not bigcap:
986             # maybe the writecap was hidden because we're in a readonly
987hunk ./src/allmydata/nodemaker.py 57
988             # directory, and the future cap format doesn't have a readcap, or
989             # something.
990-            return UnknownNode(writecap, readcap)
991-        if bigcap in self._node_cache:
992-            return self._node_cache[bigcap]
993-        cap = uri.from_string(bigcap)
994-        node = self._create_from_cap(cap)
995+            return UnknownNode(None, None)  # deep_immutable and name not needed
996+
997+        # The name doesn't matter for caching since it's only used in the error
998+        # attribute of an UnknownNode, and we don't cache those.
999+        memokey = ("I" if deep_immutable else "M") + bigcap
1000+        if memokey in self._node_cache:
1001+            return self._node_cache[memokey]
1002+        cap = uri.from_string(bigcap, deep_immutable=deep_immutable, name=name)
1003+        node = self._create_from_single_cap(cap)
1004         if node:
1005hunk ./src/allmydata/nodemaker.py 67
1006-            self._node_cache[bigcap] = node  # note: WeakValueDictionary
1007+            self._node_cache[memokey] = node  # note: WeakValueDictionary
1008         else:
1009hunk ./src/allmydata/nodemaker.py 69
1010-            node = UnknownNode(writecap, readcap) # don't cache UnknownNode
1011+            # don't cache UnknownNode
1012+            node = UnknownNode(writecap, readcap, deep_immutable=deep_immutable, name=name)
1013         return node
1014 
1015hunk ./src/allmydata/nodemaker.py 73
1016-    def _create_from_cap(self, cap):
1017-        # This starts with a "cap instance"
1018+    def _create_from_single_cap(self, cap):
1019         if isinstance(cap, uri.LiteralFileURI):
1020             return self._create_lit(cap)
1021         if isinstance(cap, uri.CHKFileURI):
1022hunk ./src/allmydata/nodemaker.py 84
1023                             uri.ReadonlyDirectoryURI,
1024                             uri.ImmutableDirectoryURI,
1025                             uri.LiteralDirectoryURI)):
1026-            filenode = self._create_from_cap(cap.get_filenode_cap())
1027+            filenode = self._create_from_single_cap(cap.get_filenode_cap())
1028             return self._create_dirnode(filenode)
1029         return None
1030 
1031hunk ./src/allmydata/nodemaker.py 97
1032         return d
1033 
1034     def create_new_mutable_directory(self, initial_children={}):
1035-        # initial_children must have metadata (i.e. {} instead of None), and
1036-        # should not contain UnknownNodes
1037+        # initial_children must have metadata (i.e. {} instead of None)
1038         for (name, (node, metadata)) in initial_children.iteritems():
1039hunk ./src/allmydata/nodemaker.py 99
1040-            precondition(not isinstance(node, UnknownNode),
1041-                         "create_new_mutable_directory does not accept UnknownNode", node)
1042             precondition(isinstance(metadata, dict),
1043                          "create_new_mutable_directory requires metadata to be a dict, not None", metadata)
1044hunk ./src/allmydata/nodemaker.py 101
1045+            node.raise_error()
1046         d = self.create_mutable_file(lambda n:
1047                                      pack_children(n, initial_children))
1048         d.addCallback(self._create_dirnode)
1049hunk ./src/allmydata/nodemaker.py 111
1050         if convergence is None:
1051             convergence = self.secret_holder.get_convergence_secret()
1052         for (name, (node, metadata)) in children.iteritems():
1053-            precondition(not isinstance(node, UnknownNode),
1054-                         "create_immutable_directory does not accept UnknownNode", node)
1055             precondition(isinstance(metadata, dict),
1056                          "create_immutable_directory requires metadata to be a dict, not None", metadata)
1057hunk ./src/allmydata/nodemaker.py 113
1058-            if node.is_mutable():
1059-                raise NotDeepImmutableError("%s is not immutable" % (node,))
1060+            node.raise_error()
1061+            if not node.is_allowed_in_immutable_directory():
1062+                raise MustBeDeepImmutableError("%s is not immutable" % (node,), name)
1063         n = DummyImmutableFileNode() # writekey=None
1064         packed = pack_children(n, children)
1065         uploadable = Data(packed, convergence)
1066hunk ./src/allmydata/nodemaker.py 120
1067         d = self.uploader.upload(uploadable, history=self.history)
1068-        def _uploaded(results):
1069-            filecap = self.create_from_cap(results.uri)
1070-            return filecap
1071-        d.addCallback(_uploaded)
1072+        d.addCallback(lambda results: self.create_from_cap(None, results.uri))
1073         d.addCallback(self._create_dirnode)
1074         return d
1075hunk ./src/allmydata/scripts/common.py 131
1076     pass
1077 
1078 def get_alias(aliases, path, default):
1079+    from allmydata import uri
1080     # transform "work:path/filename" into (aliases["work"], "path/filename").
1081     # If default=None, then an empty alias is indicated by returning
1082hunk ./src/allmydata/scripts/common.py 134
1083-    # DefaultAliasMarker. We special-case "URI:" to make it easy to access
1084-    # specific files/directories by their read-cap.
1085+    # DefaultAliasMarker. We special-case strings with a recognized cap URI
1086+    # prefix, to make it easy to access specific files/directories by their
1087+    # caps.
1088     path = path.strip()
1089hunk ./src/allmydata/scripts/common.py 138
1090-    if path.startswith("URI:"):
1091+    if uri.has_uri_prefix(path):
1092         # The only way to get a sub-path is to use URI:blah:./foo, and we
1093         # strip out the :./ sequence.
1094         sep = path.find(":./")
1095hunk ./src/allmydata/scripts/tahoe_cp.py 261
1096                 readcap = ascii_or_none(data[1].get("ro_uri"))
1097                 self.children[name] = TahoeFileSource(self.nodeurl, mutable,
1098                                                       writecap, readcap)
1099-            else:
1100-                assert data[0] == "dirnode"
1101+            elif data[0] == "dirnode":
1102                 writecap = ascii_or_none(data[1].get("rw_uri"))
1103                 readcap = ascii_or_none(data[1].get("ro_uri"))
1104                 if writecap and writecap in self.cache:
1105hunk ./src/allmydata/scripts/tahoe_cp.py 279
1106                     if recurse:
1107                         child.populate(True)
1108                 self.children[name] = child
1109+            else:
1110+                # TODO: there should be an option to skip unknown nodes.
1111+                raise TahoeError("Cannot copy unknown nodes (ticket #839). "
1112+                                 "You probably need to use a later version of "
1113+                                 "Tahoe-LAFS to copy this directory.")
1114 
1115 class TahoeMissingTarget:
1116     def __init__(self, url):
1117hunk ./src/allmydata/scripts/tahoe_cp.py 360
1118                                                    urllib.quote(name.encode('utf-8'))])
1119                 self.children[name] = TahoeFileTarget(self.nodeurl, mutable,
1120                                                       writecap, readcap, url)
1121-            else:
1122-                assert data[0] == "dirnode"
1123+            elif data[0] == "dirnode":
1124                 writecap = ascii_or_none(data[1].get("rw_uri"))
1125                 readcap = ascii_or_none(data[1].get("ro_uri"))
1126                 if writecap and writecap in self.cache:
1127hunk ./src/allmydata/scripts/tahoe_cp.py 378
1128                     if recurse:
1129                         child.populate(True)
1130                 self.children[name] = child
1131+            else:
1132+                # TODO: there should be an option to skip unknown nodes.
1133+                raise TahoeError("Cannot copy unknown nodes (ticket #839). "
1134+                                 "You probably need to use a later version of "
1135+                                 "Tahoe-LAFS to copy this directory.")
1136 
1137     def get_child_target(self, name):
1138         # return a new target for a named subdirectory of this dir
1139hunk ./src/allmydata/scripts/tahoe_cp.py 418
1140         set_data = {}
1141         for (name, filecap) in self.new_children.items():
1142             # it just so happens that ?t=set_children will accept both file
1143-            # read-caps and write-caps as ['rw_uri'], and will handle eithe
1144+            # read-caps and write-caps as ['rw_uri'], and will handle either
1145             # correctly. So don't bother trying to figure out whether the one
1146             # we have is read-only or read-write.
1147hunk ./src/allmydata/scripts/tahoe_cp.py 421
1148+            # TODO: think about how this affects forward-compatibility for
1149+            # unknown caps
1150             set_data[name] = ["filenode", {"rw_uri": filecap}]
1151         body = simplejson.dumps(set_data)
1152         POST(url, body)
1153hunk ./src/allmydata/scripts/tahoe_cp.py 783
1154 #  local-file-in-the-way
1155 #   touch proposed
1156 #   tahoe cp -r my:docs/proposed/denver.txt proposed/denver.txt
1157+#  handling of unknown nodes
1158 
1159 # things that maybe should be errors but aren't
1160 #  local-dir-in-the-way
1161hunk ./src/allmydata/scripts/tahoe_put.py 43
1162         #  DIRCAP:./subdir/foo : DIRCAP/subdir/foo
1163         #  MUTABLE-FILE-WRITECAP : filecap
1164 
1165+        # FIXME: this shouldn't rely on a particular prefix.
1166         if to_file.startswith("URI:SSK:"):
1167             url = nodeurl + "uri/%s" % urllib.quote(to_file)
1168         else:
1169hunk ./src/allmydata/test/common.py 54
1170 
1171     def get_uri(self):
1172         return self.my_uri.to_string()
1173+    def get_write_uri(self):
1174+        return None
1175     def get_readonly_uri(self):
1176         return self.my_uri.to_string()
1177     def get_cap(self):
1178hunk ./src/allmydata/test/common.py 108
1179         return False
1180     def is_readonly(self):
1181         return True
1182+    def is_unknown(self):
1183+        return False
1184+    def is_allowed_in_immutable_directory(self):
1185+        return True
1186+    def raise_error(self):
1187+        pass
1188 
1189     def get_size(self):
1190         try:
1191hunk ./src/allmydata/test/common.py 201
1192         return self.my_uri.get_readonly()
1193     def get_uri(self):
1194         return self.my_uri.to_string()
1195+    def get_write_uri(self):
1196+        if self.is_readonly():
1197+            return None
1198+        return self.my_uri.to_string()
1199     def get_readonly(self):
1200         return self.my_uri.get_readonly()
1201     def get_readonly_uri(self):
1202hunk ./src/allmydata/test/common.py 215
1203         return self.my_uri.is_readonly()
1204     def is_mutable(self):
1205         return self.my_uri.is_mutable()
1206+    def is_unknown(self):
1207+        return False
1208+    def is_allowed_in_immutable_directory(self):
1209+        return not self.my_uri.is_mutable()
1210+    def raise_error(self):
1211+        pass
1212     def get_writekey(self):
1213         return "\x00"*16
1214     def get_size(self):
1215hunk ./src/allmydata/test/test_client.py 291
1216         self.failUnless(n.is_readonly())
1217         self.failUnless(n.is_mutable())
1218 
1219-        future = "x-tahoe-crazy://future_cap_format."
1220-        n = c.create_node_from_uri(future)
1221+        unknown_rw = "lafs://from_the_future"
1222+        unknown_ro = "lafs://readonly_from_the_future"
1223+        n = c.create_node_from_uri(unknown_rw, unknown_ro)
1224         self.failUnless(IFilesystemNode.providedBy(n))
1225         self.failIf(IFileNode.providedBy(n))
1226         self.failIf(IImmutableFileNode.providedBy(n))
1227hunk ./src/allmydata/test/test_client.py 299
1228         self.failIf(IMutableFileNode.providedBy(n))
1229         self.failIf(IDirectoryNode.providedBy(n))
1230-        self.failUnlessEqual(n.get_uri(), future)
1231+        self.failUnless(n.is_unknown())
1232+        self.failUnlessEqual(n.get_uri(), unknown_rw)
1233+        self.failUnlessEqual(n.get_write_uri(), unknown_rw)
1234+        self.failUnlessEqual(n.get_readonly_uri(), "ro." + unknown_ro)
1235hunk ./src/allmydata/test/test_dirnode.py 10
1236 from allmydata.client import Client
1237 from allmydata.immutable import upload
1238 from allmydata.interfaces import IImmutableFileNode, IMutableFileNode, \
1239-     ExistingChildError, NoSuchChildError, NotDeepImmutableError, \
1240-     IDeepCheckResults, IDeepCheckAndRepairResults, CannotPackUnknownNodeError
1241+     ExistingChildError, NoSuchChildError, MustNotBeUnknownRWError, \
1242+     MustBeDeepImmutableError, MustBeReadonlyError, \
1243+     IDeepCheckResults, IDeepCheckAndRepairResults
1244 from allmydata.mutable.filenode import MutableFileNode
1245 from allmydata.mutable.common import UncoordinatedWriteError
1246 from allmydata.util import hashutil, base32
1247hunk ./src/allmydata/test/test_dirnode.py 20
1248 from allmydata.test.common import make_chk_file_uri, make_mutable_file_uri, \
1249      ErrorMixin
1250 from allmydata.test.no_network import GridTestMixin
1251-from allmydata.unknown import UnknownNode
1252+from allmydata.unknown import UnknownNode, strip_prefix_for_ro
1253 from allmydata.nodemaker import NodeMaker
1254 from base64 import b32decode
1255 import common_util as testutil
1256hunk ./src/allmydata/test/test_dirnode.py 36
1257         d = c.create_dirnode()
1258         def _done(res):
1259             self.failUnless(isinstance(res, dirnode.DirectoryNode))
1260+            self.failUnless(res.is_mutable())
1261+            self.failIf(res.is_readonly())
1262+            self.failIf(res.is_unknown())
1263+            self.failIf(res.is_allowed_in_immutable_directory())
1264+            res.raise_error()
1265             rep = str(res)
1266             self.failUnless("RW-MUT" in rep)
1267         d.addCallback(_done)
1268hunk ./src/allmydata/test/test_dirnode.py 53
1269         nm = c.nodemaker
1270         setup_py_uri = "URI:CHK:n7r3m6wmomelk4sep3kw5cvduq:os7ijw5c3maek7pg65e5254k2fzjflavtpejjyhshpsxuqzhcwwq:3:20:14861"
1271         one_uri = "URI:LIT:n5xgk" # LIT for "one"
1272+        mut_write_uri = "URI:SSK:vfvcbdfbszyrsaxchgevhmmlii:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq"
1273+        mut_read_uri = "URI:SSK-RO:jf6wkflosyvntwxqcdo7a54jvm:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq"
1274+        future_write_uri = "x-tahoe-crazy://I_am_from_the_future."
1275+        future_read_uri = "x-tahoe-crazy-readonly://I_am_from_the_future."
1276         kids = {u"one": (nm.create_from_cap(one_uri), {}),
1277                 u"two": (nm.create_from_cap(setup_py_uri),
1278                          {"metakey": "metavalue"}),
1279hunk ./src/allmydata/test/test_dirnode.py 60
1280+                u"mut": (nm.create_from_cap(mut_write_uri, mut_read_uri), {}),
1281+                u"fut": (nm.create_from_cap(future_write_uri, future_read_uri), {}),
1282+                u"fro": (nm.create_from_cap(None, future_read_uri), {}),
1283                 }
1284         d = c.create_dirnode(kids)
1285hunk ./src/allmydata/test/test_dirnode.py 65
1286+       
1287         def _created(dn):
1288             self.failUnless(isinstance(dn, dirnode.DirectoryNode))
1289hunk ./src/allmydata/test/test_dirnode.py 68
1290+            self.failUnless(dn.is_mutable())
1291+            self.failIf(dn.is_readonly())
1292+            self.failIf(dn.is_unknown())
1293+            self.failIf(dn.is_allowed_in_immutable_directory())
1294+            dn.raise_error()
1295             rep = str(dn)
1296             self.failUnless("RW-MUT" in rep)
1297             return dn.list()
1298hunk ./src/allmydata/test/test_dirnode.py 77
1299         d.addCallback(_created)
1300+       
1301         def _check_kids(children):
1302hunk ./src/allmydata/test/test_dirnode.py 79
1303-            self.failUnlessEqual(sorted(children.keys()), [u"one", u"two"])
1304+            self.failUnlessEqual(sorted(children.keys()),
1305+                                 [u"fro", u"fut", u"mut", u"one", u"two"])
1306             one_node, one_metadata = children[u"one"]
1307             two_node, two_metadata = children[u"two"]
1308hunk ./src/allmydata/test/test_dirnode.py 83
1309+            mut_node, mut_metadata = children[u"mut"]
1310+            fut_node, fut_metadata = children[u"fut"]
1311+            fro_node, fro_metadata = children[u"fro"]
1312+           
1313             self.failUnlessEqual(one_node.get_size(), 3)
1314hunk ./src/allmydata/test/test_dirnode.py 88
1315-            self.failUnlessEqual(two_node.get_size(), 14861)
1316+            self.failUnlessEqual(one_node.get_uri(), one_uri)
1317+            self.failUnlessEqual(one_node.get_readonly_uri(), one_uri)
1318             self.failUnless(isinstance(one_metadata, dict), one_metadata)
1319hunk ./src/allmydata/test/test_dirnode.py 91
1320+           
1321+            self.failUnlessEqual(two_node.get_size(), 14861)
1322+            self.failUnlessEqual(two_node.get_uri(), setup_py_uri)
1323+            self.failUnlessEqual(two_node.get_readonly_uri(), setup_py_uri)
1324             self.failUnlessEqual(two_metadata["metakey"], "metavalue")
1325hunk ./src/allmydata/test/test_dirnode.py 96
1326+           
1327+            self.failUnlessEqual(mut_node.get_uri(), mut_write_uri)
1328+            self.failUnlessEqual(mut_node.get_readonly_uri(), mut_read_uri)
1329+            self.failUnless(isinstance(mut_metadata, dict), mut_metadata)
1330+           
1331+            self.failUnless(fut_node.is_unknown())
1332+            self.failUnlessEqual(fut_node.get_uri(), future_write_uri)
1333+            self.failUnlessEqual(fut_node.get_readonly_uri(), "ro." + future_read_uri)
1334+            self.failUnless(isinstance(fut_metadata, dict), fut_metadata)
1335+           
1336+            self.failUnless(fro_node.is_unknown())
1337+            self.failUnlessEqual(fro_node.get_uri(), "ro." + future_read_uri)
1338+            self.failUnlessEqual(fut_node.get_readonly_uri(), "ro." + future_read_uri)
1339+            self.failUnless(isinstance(fro_metadata, dict), fro_metadata)
1340         d.addCallback(_check_kids)
1341hunk ./src/allmydata/test/test_dirnode.py 111
1342+
1343         d.addCallback(lambda ign: nm.create_new_mutable_directory(kids))
1344         d.addCallback(lambda dn: dn.list())
1345         d.addCallback(_check_kids)
1346hunk ./src/allmydata/test/test_dirnode.py 115
1347-        future_writecap = "x-tahoe-crazy://I_am_from_the_future."
1348-        future_readcap = "x-tahoe-crazy-readonly://I_am_from_the_future."
1349-        future_node = UnknownNode(future_writecap, future_readcap)
1350-        bad_kids1 = {u"one": (future_node, {})}
1351+
1352+        bad_future_node = UnknownNode(future_write_uri, None)
1353+        bad_kids1 = {u"one": (bad_future_node, {})}
1354         d.addCallback(lambda ign:
1355hunk ./src/allmydata/test/test_dirnode.py 119
1356-                      self.shouldFail(AssertionError, "bad_kids1",
1357-                                      "does not accept UnknownNode",
1358+                      self.shouldFail(MustNotBeUnknownRWError, "bad_kids1",
1359+                                      "cannot attach unknown",
1360                                       nm.create_new_mutable_directory,
1361                                       bad_kids1))
1362         bad_kids2 = {u"one": (nm.create_from_cap(one_uri), None)}
1363hunk ./src/allmydata/test/test_dirnode.py 138
1364         nm = c.nodemaker
1365         setup_py_uri = "URI:CHK:n7r3m6wmomelk4sep3kw5cvduq:os7ijw5c3maek7pg65e5254k2fzjflavtpejjyhshpsxuqzhcwwq:3:20:14861"
1366         one_uri = "URI:LIT:n5xgk" # LIT for "one"
1367-        mut_readcap = "URI:SSK-RO:e3mdrzfwhoq42hy5ubcz6rp3o4:ybyibhnp3vvwuq2vaw2ckjmesgkklfs6ghxleztqidihjyofgw7q"
1368-        mut_writecap = "URI:SSK:vfvcbdfbszyrsaxchgevhmmlii:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq"
1369+        mut_write_uri = "URI:SSK:vfvcbdfbszyrsaxchgevhmmlii:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq"
1370+        mut_read_uri = "URI:SSK-RO:e3mdrzfwhoq42hy5ubcz6rp3o4:ybyibhnp3vvwuq2vaw2ckjmesgkklfs6ghxleztqidihjyofgw7q"
1371+        future_write_uri = "x-tahoe-crazy://I_am_from_the_future."
1372+        future_read_uri = "x-tahoe-crazy-readonly://I_am_from_the_future."
1373         kids = {u"one": (nm.create_from_cap(one_uri), {}),
1374                 u"two": (nm.create_from_cap(setup_py_uri),
1375                          {"metakey": "metavalue"}),
1376hunk ./src/allmydata/test/test_dirnode.py 145
1377+                u"fut": (nm.create_from_cap(None, future_read_uri), {}),
1378                 }
1379         d = c.create_immutable_dirnode(kids)
1380hunk ./src/allmydata/test/test_dirnode.py 148
1381+       
1382         def _created(dn):
1383             self.failUnless(isinstance(dn, dirnode.DirectoryNode))
1384             self.failIf(dn.is_mutable())
1385hunk ./src/allmydata/test/test_dirnode.py 153
1386             self.failUnless(dn.is_readonly())
1387+            self.failIf(dn.is_unknown())
1388+            self.failUnless(dn.is_allowed_in_immutable_directory())
1389+            dn.raise_error()
1390             rep = str(dn)
1391             self.failUnless("RO-IMM" in rep)
1392             cap = dn.get_cap()
1393hunk ./src/allmydata/test/test_dirnode.py 163
1394             self.cap = cap
1395             return dn.list()
1396         d.addCallback(_created)
1397+       
1398         def _check_kids(children):
1399hunk ./src/allmydata/test/test_dirnode.py 165
1400-            self.failUnlessEqual(sorted(children.keys()), [u"one", u"two"])
1401+            self.failUnlessEqual(sorted(children.keys()), [u"fut", u"one", u"two"])
1402             one_node, one_metadata = children[u"one"]
1403             two_node, two_metadata = children[u"two"]
1404hunk ./src/allmydata/test/test_dirnode.py 168
1405+            fut_node, fut_metadata = children[u"fut"]
1406+
1407             self.failUnlessEqual(one_node.get_size(), 3)
1408hunk ./src/allmydata/test/test_dirnode.py 171
1409-            self.failUnlessEqual(two_node.get_size(), 14861)
1410+            self.failUnlessEqual(one_node.get_uri(), one_uri)
1411+            self.failUnlessEqual(one_node.get_readonly_uri(), one_uri)
1412             self.failUnless(isinstance(one_metadata, dict), one_metadata)
1413hunk ./src/allmydata/test/test_dirnode.py 174
1414+
1415+            self.failUnlessEqual(two_node.get_size(), 14861)
1416+            self.failUnlessEqual(two_node.get_uri(), setup_py_uri)
1417+            self.failUnlessEqual(two_node.get_readonly_uri(), setup_py_uri)
1418             self.failUnlessEqual(two_metadata["metakey"], "metavalue")
1419hunk ./src/allmydata/test/test_dirnode.py 179
1420+
1421+            self.failUnless(fut_node.is_unknown())
1422+            self.failUnlessEqual(fut_node.get_uri(), "imm." + future_read_uri)
1423+            self.failUnlessEqual(fut_node.get_readonly_uri(), "imm." + future_read_uri)
1424+            self.failUnless(isinstance(fut_metadata, dict), fut_metadata)
1425         d.addCallback(_check_kids)
1426hunk ./src/allmydata/test/test_dirnode.py 185
1427+       
1428         d.addCallback(lambda ign: nm.create_from_cap(self.cap.to_string()))
1429         d.addCallback(lambda dn: dn.list())
1430         d.addCallback(_check_kids)
1431hunk ./src/allmydata/test/test_dirnode.py 189
1432-        future_writecap = "x-tahoe-crazy://I_am_from_the_future."
1433-        future_readcap = "x-tahoe-crazy-readonly://I_am_from_the_future."
1434-        future_node = UnknownNode(future_writecap, future_readcap)
1435-        bad_kids1 = {u"one": (future_node, {})}
1436+
1437+        bad_future_node1 = UnknownNode(future_write_uri, None)
1438+        bad_kids1 = {u"one": (bad_future_node1, {})}
1439         d.addCallback(lambda ign:
1440hunk ./src/allmydata/test/test_dirnode.py 193
1441-                      self.shouldFail(AssertionError, "bad_kids1",
1442-                                      "does not accept UnknownNode",
1443+                      self.shouldFail(MustNotBeUnknownRWError, "bad_kids1",
1444+                                      "cannot attach unknown",
1445                                       c.create_immutable_dirnode,
1446                                       bad_kids1))
1447hunk ./src/allmydata/test/test_dirnode.py 197
1448-        bad_kids2 = {u"one": (nm.create_from_cap(one_uri), None)}
1449+        bad_future_node2 = UnknownNode(future_write_uri, future_read_uri)
1450+        bad_kids2 = {u"one": (bad_future_node2, {})}
1451         d.addCallback(lambda ign:
1452hunk ./src/allmydata/test/test_dirnode.py 200
1453-                      self.shouldFail(AssertionError, "bad_kids2",
1454-                                      "requires metadata to be a dict",
1455+                      self.shouldFail(MustBeDeepImmutableError, "bad_kids2",
1456+                                      "is not immutable",
1457                                       c.create_immutable_dirnode,
1458                                       bad_kids2))
1459hunk ./src/allmydata/test/test_dirnode.py 204
1460-        bad_kids3 = {u"one": (nm.create_from_cap(mut_writecap), {})}
1461+        bad_kids3 = {u"one": (nm.create_from_cap(one_uri), None)}
1462         d.addCallback(lambda ign:
1463hunk ./src/allmydata/test/test_dirnode.py 206
1464-                      self.shouldFail(NotDeepImmutableError, "bad_kids3",
1465-                                      "is not immutable",
1466+                      self.shouldFail(AssertionError, "bad_kids3",
1467+                                      "requires metadata to be a dict",
1468                                       c.create_immutable_dirnode,
1469                                       bad_kids3))
1470hunk ./src/allmydata/test/test_dirnode.py 210
1471-        bad_kids4 = {u"one": (nm.create_from_cap(mut_readcap), {})}
1472+        bad_kids4 = {u"one": (nm.create_from_cap(mut_write_uri), {})}
1473         d.addCallback(lambda ign:
1474hunk ./src/allmydata/test/test_dirnode.py 212
1475-                      self.shouldFail(NotDeepImmutableError, "bad_kids4",
1476+                      self.shouldFail(MustBeDeepImmutableError, "bad_kids4",
1477                                       "is not immutable",
1478                                       c.create_immutable_dirnode,
1479                                       bad_kids4))
1480hunk ./src/allmydata/test/test_dirnode.py 216
1481+        bad_kids5 = {u"one": (nm.create_from_cap(mut_read_uri), {})}
1482+        d.addCallback(lambda ign:
1483+                      self.shouldFail(MustBeDeepImmutableError, "bad_kids5",
1484+                                      "is not immutable",
1485+                                      c.create_immutable_dirnode,
1486+                                      bad_kids5))
1487         d.addCallback(lambda ign: c.create_immutable_dirnode({}))
1488         def _created_empty(dn):
1489             self.failUnless(isinstance(dn, dirnode.DirectoryNode))
1490hunk ./src/allmydata/test/test_dirnode.py 227
1491             self.failIf(dn.is_mutable())
1492             self.failUnless(dn.is_readonly())
1493+            self.failIf(dn.is_unknown())
1494+            self.failUnless(dn.is_allowed_in_immutable_directory())
1495+            dn.raise_error()
1496             rep = str(dn)
1497             self.failUnless("RO-IMM" in rep)
1498             cap = dn.get_cap()
1499hunk ./src/allmydata/test/test_dirnode.py 245
1500             self.failUnless(isinstance(dn, dirnode.DirectoryNode))
1501             self.failIf(dn.is_mutable())
1502             self.failUnless(dn.is_readonly())
1503+            self.failIf(dn.is_unknown())
1504+            self.failUnless(dn.is_allowed_in_immutable_directory())
1505+            dn.raise_error()
1506             rep = str(dn)
1507             self.failUnless("RO-IMM" in rep)
1508             cap = dn.get_cap()
1509hunk ./src/allmydata/test/test_dirnode.py 273
1510             d.addCallback(_check_kids)
1511             d.addCallback(lambda ign: n.get(u"subdir"))
1512             d.addCallback(lambda sd: self.failIf(sd.is_mutable()))
1513-            bad_kids = {u"one": (nm.create_from_cap(mut_writecap), {})}
1514+            bad_kids = {u"one": (nm.create_from_cap(mut_write_uri), {})}
1515             d.addCallback(lambda ign:
1516hunk ./src/allmydata/test/test_dirnode.py 275
1517-                          self.shouldFail(NotDeepImmutableError, "YZ",
1518+                          self.shouldFail(MustBeDeepImmutableError, "YZ",
1519                                           "is not immutable",
1520                                           n.create_subdirectory,
1521                                           u"sub2", bad_kids, mutable=False))
1522hunk ./src/allmydata/test/test_dirnode.py 283
1523         d.addCallback(_made_parent)
1524         return d
1525 
1526-
1527     def test_check(self):
1528         self.basedir = "dirnode/Dirnode/test_check"
1529         self.set_up_grid()
1530hunk ./src/allmydata/test/test_dirnode.py 416
1531             ro_dn = c.create_node_from_uri(ro_uri)
1532             self.failUnless(ro_dn.is_readonly())
1533             self.failUnless(ro_dn.is_mutable())
1534+            self.failIf(ro_dn.is_unknown())
1535+            self.failIf(ro_dn.is_allowed_in_immutable_directory())
1536+            ro_dn.raise_error()
1537 
1538hunk ./src/allmydata/test/test_dirnode.py 420
1539-            self.shouldFail(dirnode.NotMutableError, "set_uri ro", None,
1540+            self.shouldFail(dirnode.NotWriteableError, "set_uri ro", None,
1541                             ro_dn.set_uri, u"newchild", filecap, filecap)
1542hunk ./src/allmydata/test/test_dirnode.py 422
1543-            self.shouldFail(dirnode.NotMutableError, "set_uri ro", None,
1544+            self.shouldFail(dirnode.NotWriteableError, "set_uri ro", None,
1545                             ro_dn.set_node, u"newchild", filenode)
1546hunk ./src/allmydata/test/test_dirnode.py 424
1547-            self.shouldFail(dirnode.NotMutableError, "set_nodes ro", None,
1548+            self.shouldFail(dirnode.NotWriteableError, "set_nodes ro", None,
1549                             ro_dn.set_nodes, { u"newchild": (filenode, None) })
1550hunk ./src/allmydata/test/test_dirnode.py 426
1551-            self.shouldFail(dirnode.NotMutableError, "set_uri ro", None,
1552+            self.shouldFail(dirnode.NotWriteableError, "set_uri ro", None,
1553                             ro_dn.add_file, u"newchild", uploadable)
1554hunk ./src/allmydata/test/test_dirnode.py 428
1555-            self.shouldFail(dirnode.NotMutableError, "set_uri ro", None,
1556+            self.shouldFail(dirnode.NotWriteableError, "set_uri ro", None,
1557                             ro_dn.delete, u"child")
1558hunk ./src/allmydata/test/test_dirnode.py 430
1559-            self.shouldFail(dirnode.NotMutableError, "set_uri ro", None,
1560+            self.shouldFail(dirnode.NotWriteableError, "set_uri ro", None,
1561                             ro_dn.create_subdirectory, u"newchild")
1562hunk ./src/allmydata/test/test_dirnode.py 432
1563-            self.shouldFail(dirnode.NotMutableError, "set_metadata_for ro", None,
1564+            self.shouldFail(dirnode.NotWriteableError, "set_metadata_for ro", None,
1565                             ro_dn.set_metadata_for, u"child", {})
1566hunk ./src/allmydata/test/test_dirnode.py 434
1567-            self.shouldFail(dirnode.NotMutableError, "set_uri ro", None,
1568+            self.shouldFail(dirnode.NotWriteableError, "set_uri ro", None,
1569                             ro_dn.move_child_to, u"child", rw_dn)
1570hunk ./src/allmydata/test/test_dirnode.py 436
1571-            self.shouldFail(dirnode.NotMutableError, "set_uri ro", None,
1572+            self.shouldFail(dirnode.NotWriteableError, "set_uri ro", None,
1573                             rw_dn.move_child_to, u"child", ro_dn)
1574             return ro_dn.list()
1575         d.addCallback(_ready)
1576hunk ./src/allmydata/test/test_dirnode.py 983
1577         nodemaker = NodeMaker(None, None, None,
1578                               None, None, None,
1579                               {"k": 3, "n": 10}, None)
1580-        writecap = "URI:SSK-RO:e3mdrzfwhoq42hy5ubcz6rp3o4:ybyibhnp3vvwuq2vaw2ckjmesgkklfs6ghxleztqidihjyofgw7q"
1581-        filenode = nodemaker.create_from_cap(writecap)
1582+        write_uri = "URI:SSK-RO:e3mdrzfwhoq42hy5ubcz6rp3o4:ybyibhnp3vvwuq2vaw2ckjmesgkklfs6ghxleztqidihjyofgw7q"
1583+        filenode = nodemaker.create_from_cap(write_uri)
1584         node = dirnode.DirectoryNode(filenode, nodemaker, None)
1585         children = node._unpack_contents(known_tree)
1586         self._check_children(children)
1587hunk ./src/allmydata/test/test_dirnode.py 1057
1588         self.failUnlessIn("lit", packed)
1589 
1590         kids = self._make_kids(nm, ["imm", "lit", "write"])
1591-        self.failUnlessRaises(dirnode.MustBeDeepImmutable,
1592+        self.failUnlessRaises(dirnode.MustBeDeepImmutableError,
1593                               dirnode.pack_children,
1594                               fn, kids, deep_immutable=True)
1595 
1596hunk ./src/allmydata/test/test_dirnode.py 1063
1597         # read-only is not enough: all children must be immutable
1598         kids = self._make_kids(nm, ["imm", "lit", "read"])
1599-        self.failUnlessRaises(dirnode.MustBeDeepImmutable,
1600+        self.failUnlessRaises(dirnode.MustBeDeepImmutableError,
1601                               dirnode.pack_children,
1602                               fn, kids, deep_immutable=True)
1603 
1604hunk ./src/allmydata/test/test_dirnode.py 1068
1605         kids = self._make_kids(nm, ["imm", "lit", "dirwrite"])
1606-        self.failUnlessRaises(dirnode.MustBeDeepImmutable,
1607+        self.failUnlessRaises(dirnode.MustBeDeepImmutableError,
1608                               dirnode.pack_children,
1609                               fn, kids, deep_immutable=True)
1610 
1611hunk ./src/allmydata/test/test_dirnode.py 1073
1612         kids = self._make_kids(nm, ["imm", "lit", "dirread"])
1613-        self.failUnlessRaises(dirnode.MustBeDeepImmutable,
1614+        self.failUnlessRaises(dirnode.MustBeDeepImmutableError,
1615                               dirnode.pack_children,
1616                               fn, kids, deep_immutable=True)
1617 
1618hunk ./src/allmydata/test/test_dirnode.py 1099
1619 
1620     def get_cap(self):
1621         return self.uri
1622+
1623     def get_uri(self):
1624         return self.uri.to_string()
1625hunk ./src/allmydata/test/test_dirnode.py 1102
1626+
1627+    def get_write_uri(self):
1628+        return self.uri.to_string()
1629+
1630     def download_best_version(self):
1631         return defer.succeed(self.data)
1632hunk ./src/allmydata/test/test_dirnode.py 1108
1633+
1634     def get_writekey(self):
1635         return "writekey"
1636hunk ./src/allmydata/test/test_dirnode.py 1111
1637+
1638     def is_readonly(self):
1639         return False
1640hunk ./src/allmydata/test/test_dirnode.py 1114
1641+
1642     def is_mutable(self):
1643         return True
1644hunk ./src/allmydata/test/test_dirnode.py 1117
1645+
1646+    def is_unknown(self):
1647+        return False
1648+
1649+    def is_allowed_in_immutable_directory(self):
1650+        return False
1651+
1652     def modify(self, modifier):
1653         self.data = modifier(self.data, None, True)
1654         return defer.succeed(None)
1655hunk ./src/allmydata/test/test_dirnode.py 1146
1656         self.nodemaker = client.nodemaker
1657 
1658     def test_from_future(self):
1659-        # create a dirnode that contains unknown URI types, and make sure we
1660-        # tolerate them properly. Since dirnodes aren't allowed to add
1661-        # unknown node types, we have to be tricky.
1662+        # Create a mutable directory that contains unknown URI types, and make sure
1663+        # we tolerate them properly.
1664         d = self.nodemaker.create_new_mutable_directory()
1665hunk ./src/allmydata/test/test_dirnode.py 1149
1666-        future_writecap = "x-tahoe-crazy://I_am_from_the_future."
1667-        future_readcap = "x-tahoe-crazy-readonly://I_am_from_the_future."
1668-        future_node = UnknownNode(future_writecap, future_readcap)
1669+        future_write_uri = "x-tahoe-crazy://I_am_from_the_future."
1670+        future_read_uri = "x-tahoe-crazy-readonly://I_am_from_the_future."
1671+        future_imm_uri = "x-tahoe-crazy-immutable://I_am_from_the_future."
1672+        future_node = UnknownNode(future_write_uri, future_read_uri)
1673         def _then(n):
1674             self._node = n
1675             return n.set_node(u"future", future_node)
1676hunk ./src/allmydata/test/test_dirnode.py 1158
1677         d.addCallback(_then)
1678 
1679-        # we should be prohibited from adding an unknown URI to a directory,
1680-        # since we don't know how to diminish the cap to a readcap (for the
1681-        # dirnode's rocap slot), and we don't want to accidentally grant
1682-        # write access to a holder of the dirnode's readcap.
1683+        # We should be prohibited from adding an unknown URI to a directory
1684+        # just in the rw_uri slot, since we don't know how to diminish the cap
1685+        # to a readcap (for the ro_uri slot).
1686         d.addCallback(lambda ign:
1687hunk ./src/allmydata/test/test_dirnode.py 1162
1688-             self.shouldFail(CannotPackUnknownNodeError,
1689+             self.shouldFail(MustNotBeUnknownRWError,
1690                              "copy unknown",
1691hunk ./src/allmydata/test/test_dirnode.py 1164
1692-                             "cannot pack unknown node as child add",
1693+                             "cannot attach unknown rw cap as child",
1694                              self._node.set_uri, u"add",
1695hunk ./src/allmydata/test/test_dirnode.py 1166
1696-                             future_writecap, future_readcap))
1697+                             future_write_uri, None))
1698+
1699+        # However, we should be able to add both rw_uri and ro_uri as a pair of
1700+        # unknown URIs.
1701+        d.addCallback(lambda ign: self._node.set_uri(u"add-pair",
1702+                                                     future_write_uri, future_read_uri))
1703+
1704+        # and to add an URI prefixed with "ro." or "imm." when it is given in a
1705+        # write slot (or URL parameter).
1706+        d.addCallback(lambda ign: self._node.set_uri(u"add-ro",
1707+                                                     "ro." + future_read_uri, None))
1708+        d.addCallback(lambda ign: self._node.set_uri(u"add-imm",
1709+                                                     "imm." + future_imm_uri, None))
1710+
1711         d.addCallback(lambda ign: self._node.list())
1712         def _check(children):
1713hunk ./src/allmydata/test/test_dirnode.py 1182
1714-            self.failUnlessEqual(len(children), 1)
1715+            self.failUnlessEqual(len(children), 4)
1716             (fn, metadata) = children[u"future"]
1717             self.failUnless(isinstance(fn, UnknownNode), fn)
1718hunk ./src/allmydata/test/test_dirnode.py 1185
1719-            self.failUnlessEqual(fn.get_uri(), future_writecap)
1720-            self.failUnlessEqual(fn.get_readonly_uri(), future_readcap)
1721-            # but we *should* be allowed to copy this node, because the
1722-            # UnknownNode contains all the information that was in the
1723-            # original directory (readcap and writecap), so we're preserving
1724-            # everything.
1725+            self.failUnlessEqual(fn.get_uri(), future_write_uri)
1726+            self.failUnlessEqual(fn.get_write_uri(), future_write_uri)
1727+            self.failUnlessEqual(fn.get_readonly_uri(), "ro." + future_read_uri)
1728+
1729+            (fn2, metadata2) = children[u"add-pair"]
1730+            self.failUnless(isinstance(fn2, UnknownNode), fn2)
1731+            self.failUnlessEqual(fn2.get_uri(), future_write_uri)
1732+            self.failUnlessEqual(fn2.get_write_uri(), future_write_uri)
1733+            self.failUnlessEqual(fn2.get_readonly_uri(), "ro." + future_read_uri)
1734+
1735+            (fn3, metadata3) = children[u"add-ro"]
1736+            self.failUnless(isinstance(fn3, UnknownNode), fn3)
1737+            self.failUnlessEqual(fn3.get_uri(), "ro." + future_read_uri)
1738+            self.failUnlessEqual(fn3.get_write_uri(), None)
1739+            self.failUnlessEqual(fn3.get_readonly_uri(), "ro." + future_read_uri)
1740+
1741+            (fn4, metadata4) = children[u"add-imm"]
1742+            self.failUnless(isinstance(fn4, UnknownNode), fn4)
1743+            self.failUnlessEqual(fn4.get_uri(), "imm." + future_imm_uri)
1744+            self.failUnlessEqual(fn4.get_write_uri(), None)
1745+            self.failUnlessEqual(fn4.get_readonly_uri(), "imm." + future_imm_uri)
1746+
1747+            # We should also be allowed to copy the "future" UnknownNode, because
1748+            # it contains all the information that was in the original directory
1749+            # (readcap and writecap), so we're preserving everything.
1750             return self._node.set_node(u"copy", fn)
1751         d.addCallback(_check)
1752hunk ./src/allmydata/test/test_dirnode.py 1212
1753+
1754         d.addCallback(lambda ign: self._node.list())
1755         def _check2(children):
1756hunk ./src/allmydata/test/test_dirnode.py 1215
1757-            self.failUnlessEqual(len(children), 2)
1758+            self.failUnlessEqual(len(children), 5)
1759             (fn, metadata) = children[u"copy"]
1760             self.failUnless(isinstance(fn, UnknownNode), fn)
1761hunk ./src/allmydata/test/test_dirnode.py 1218
1762-            self.failUnlessEqual(fn.get_uri(), future_writecap)
1763-            self.failUnlessEqual(fn.get_readonly_uri(), future_readcap)
1764+            self.failUnlessEqual(fn.get_uri(), future_write_uri)
1765+            self.failUnlessEqual(fn.get_write_uri(), future_write_uri)
1766+            self.failUnlessEqual(fn.get_readonly_uri(), "ro." + future_read_uri)
1767+        d.addCallback(_check2)
1768         return d
1769 
1770hunk ./src/allmydata/test/test_dirnode.py 1224
1771+    def test_unknown_strip_prefix_for_ro(self):
1772+        self.failUnlessEqual(strip_prefix_for_ro("foo",     False), "foo")
1773+        self.failUnlessEqual(strip_prefix_for_ro("ro.foo",  False), "foo")
1774+        self.failUnlessEqual(strip_prefix_for_ro("imm.foo", False), "imm.foo")
1775+        self.failUnlessEqual(strip_prefix_for_ro("foo",     True),  "foo")
1776+        self.failUnlessEqual(strip_prefix_for_ro("ro.foo",  True),  "foo")
1777+        self.failUnlessEqual(strip_prefix_for_ro("imm.foo", True),  "foo")
1778+
1779+    def test_unknownnode(self):
1780+        mut_write_uri = "URI:SSK:vfvcbdfbszyrsaxchgevhmmlii:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq"
1781+        mut_read_uri = "URI:SSK-RO:jf6wkflosyvntwxqcdo7a54jvm:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq"
1782+        lit_uri = "URI:LIT:n5xgk"
1783+
1784+        # This does not attempt to be exhaustive.
1785+        no_no        = [# Opaque node, but not an error.
1786+                        ( 0, UnknownNode(None, None)),
1787+                        ( 1, UnknownNode(None, None, deep_immutable=True)),
1788+                       ]
1789+        unknown_rw   = [# These are errors because we're only given a rw_uri, and we can't
1790+                        # diminish it.
1791+                        ( 2, UnknownNode("foo", None)),
1792+                        ( 3, UnknownNode("foo", None, deep_immutable=True)),
1793+                        ( 4, UnknownNode("ro.foo", None, deep_immutable=True)),
1794+                        ( 5, UnknownNode("ro." + mut_read_uri, None, deep_immutable=True)),
1795+                        ( 6, UnknownNode("URI:SSK-RO:foo", None, deep_immutable=True)),
1796+                        ( 7, UnknownNode("URI:SSK:foo", None)),
1797+                       ]
1798+        must_be_ro   = [# These are errors because a readonly constraint is not met.
1799+                        ( 8, UnknownNode("ro." + mut_write_uri, None)),
1800+                        ( 9, UnknownNode(None, "ro." + mut_write_uri)),
1801+                       ]
1802+        must_be_imm  = [# These are errors because an immutable constraint is not met.
1803+                        (10, UnknownNode(None, "ro.URI:SSK-RO:foo", deep_immutable=True)),
1804+                        (11, UnknownNode(None, "imm.URI:SSK:foo")),
1805+                        (12, UnknownNode(None, "imm.URI:SSK-RO:foo")),
1806+                        (13, UnknownNode("bar", "ro.foo", deep_immutable=True)),
1807+                        (14, UnknownNode("bar", "imm.foo", deep_immutable=True)),
1808+                        (15, UnknownNode("bar", "imm." + lit_uri, deep_immutable=True)),
1809+                        (16, UnknownNode("imm." + mut_write_uri, None)),
1810+                        (17, UnknownNode("imm." + mut_read_uri, None)),
1811+                        (18, UnknownNode("bar", "imm.foo")),
1812+                       ]
1813+        bad_uri      = [# These are errors because the URI is bad once we've stripped the prefix.
1814+                        (19, UnknownNode("ro.URI:SSK-RO:foo", None)),
1815+                        (20, UnknownNode("imm.URI:CHK:foo", None, deep_immutable=True)),
1816+                       ]
1817+        ro_prefixed  = [# These are valid, and the readcap should end up with a ro. prefix.
1818+                        (21, UnknownNode(None, "foo")),
1819+                        (22, UnknownNode(None, "ro.foo")),
1820+                        (32, UnknownNode(None, "ro." + lit_uri)),
1821+                        (23, UnknownNode("bar", "foo")),
1822+                        (24, UnknownNode("bar", "ro.foo")),
1823+                        (32, UnknownNode("bar", "ro." + lit_uri)),
1824+                        (25, UnknownNode("ro.foo", None)),
1825+                        (30, UnknownNode("ro." + lit_uri, None)),
1826+                       ]
1827+        imm_prefixed = [# These are valid, and the readcap should end up with an imm. prefix.
1828+                        (26, UnknownNode(None, "foo", deep_immutable=True)),
1829+                        (27, UnknownNode(None, "ro.foo", deep_immutable=True)),
1830+                        (28, UnknownNode(None, "imm.foo")),
1831+                        (29, UnknownNode(None, "imm.foo", deep_immutable=True)),
1832+                        (31, UnknownNode("imm." + lit_uri, None)),
1833+                        (31, UnknownNode("imm." + lit_uri, None, deep_immutable=True)),
1834+                        (33, UnknownNode(None, "imm." + lit_uri)),
1835+                        (33, UnknownNode(None, "imm." + lit_uri, deep_immutable=True)),
1836+                       ]
1837+        error = unknown_rw + must_be_ro + must_be_imm + bad_uri
1838+        ok = ro_prefixed + imm_prefixed
1839+
1840+        for (i, n) in no_no + error + ok:
1841+            self.failUnless(n.is_unknown(), i)
1842+
1843+        for (i, n) in no_no + error:
1844+            self.failUnless(n.get_uri() is None, i)
1845+            self.failUnless(n.get_write_uri() is None, i)
1846+            self.failUnless(n.get_readonly_uri() is None, i)
1847+
1848+        for (i, n) in no_no + ok:
1849+            n.raise_error()
1850+
1851+        for (i, n) in unknown_rw:
1852+            self.failUnlessRaises(MustNotBeUnknownRWError, lambda: n.raise_error())
1853+
1854+        for (i, n) in must_be_ro:
1855+            self.failUnlessRaises(MustBeReadonlyError, lambda: n.raise_error())
1856+
1857+        for (i, n) in must_be_imm:
1858+            self.failUnlessRaises(MustBeDeepImmutableError, lambda: n.raise_error())
1859+
1860+        for (i, n) in bad_uri:
1861+            self.failUnlessRaises(uri.BadURIError, lambda: n.raise_error())
1862+
1863+        for (i, n) in ok:
1864+            self.failIf(n.get_readonly_uri() is None, i)
1865+
1866+        for (i, n) in ro_prefixed:
1867+            self.failUnless(n.get_readonly_uri().startswith("ro."), i)
1868+
1869+        for (i, n) in imm_prefixed:
1870+            self.failUnless(n.get_readonly_uri().startswith("imm."), i)
1871+
1872+
1873 class DeepStats(unittest.TestCase):
1874     timeout = 240 # It takes longer than 120 seconds on Francois's arm box.
1875     def test_stats(self):
1876hunk ./src/allmydata/test/test_filenode.py 44
1877         self.failUnlessEqual(fn1.get_readcap(), u)
1878         self.failUnlessEqual(fn1.is_readonly(), True)
1879         self.failUnlessEqual(fn1.is_mutable(), False)
1880+        self.failUnlessEqual(fn1.is_unknown(), False)
1881+        self.failUnlessEqual(fn1.is_allowed_in_immutable_directory(), True)
1882+        self.failUnlessEqual(fn1.get_write_uri(), None)
1883         self.failUnlessEqual(fn1.get_readonly_uri(), u.to_string())
1884         self.failUnlessEqual(fn1.get_size(), 1000)
1885         self.failUnlessEqual(fn1.get_storage_index(), u.storage_index)
1886hunk ./src/allmydata/test/test_filenode.py 50
1887+        fn1.raise_error()
1888+        fn2.raise_error()
1889         d = {}
1890         d[fn1] = 1 # exercise __hash__
1891         v = fn1.get_verify_cap()
1892hunk ./src/allmydata/test/test_filenode.py 57
1893         self.failUnless(isinstance(v, uri.CHKFileVerifierURI))
1894         self.failUnlessEqual(fn1.get_repair_cap(), v)
1895+        self.failUnlessEqual(v.is_readonly(), True)
1896+        self.failUnlessEqual(v.is_mutable(), False)
1897 
1898 
1899     def test_literal_filenode(self):
1900hunk ./src/allmydata/test/test_filenode.py 74
1901         self.failUnlessEqual(fn1.get_readcap(), u)
1902         self.failUnlessEqual(fn1.is_readonly(), True)
1903         self.failUnlessEqual(fn1.is_mutable(), False)
1904+        self.failUnlessEqual(fn1.is_unknown(), False)
1905+        self.failUnlessEqual(fn1.is_allowed_in_immutable_directory(), True)
1906+        self.failUnlessEqual(fn1.get_write_uri(), None)
1907         self.failUnlessEqual(fn1.get_readonly_uri(), u.to_string())
1908         self.failUnlessEqual(fn1.get_size(), len(DATA))
1909         self.failUnlessEqual(fn1.get_storage_index(), None)
1910hunk ./src/allmydata/test/test_filenode.py 80
1911+        fn1.raise_error()
1912+        fn2.raise_error()
1913         d = {}
1914         d[fn1] = 1 # exercise __hash__
1915 
1916hunk ./src/allmydata/test/test_filenode.py 114
1917         self.failUnlessEqual(n.get_writekey(), wk)
1918         self.failUnlessEqual(n.get_readkey(), rk)
1919         self.failUnlessEqual(n.get_storage_index(), si)
1920-        # these itmes are populated on first read (or create), so until that
1921+        # these items are populated on first read (or create), so until that
1922         # happens they'll be None
1923         self.failUnlessEqual(n.get_privkey(), None)
1924         self.failUnlessEqual(n.get_encprivkey(), None)
1925hunk ./src/allmydata/test/test_filenode.py 121
1926         self.failUnlessEqual(n.get_pubkey(), None)
1927 
1928         self.failUnlessEqual(n.get_uri(), u.to_string())
1929+        self.failUnlessEqual(n.get_write_uri(), u.to_string())
1930         self.failUnlessEqual(n.get_readonly_uri(), u.get_readonly().to_string())
1931         self.failUnlessEqual(n.get_cap(), u)
1932         self.failUnlessEqual(n.get_readcap(), u.get_readonly())
1933hunk ./src/allmydata/test/test_filenode.py 127
1934         self.failUnlessEqual(n.is_mutable(), True)
1935         self.failUnlessEqual(n.is_readonly(), False)
1936+        self.failUnlessEqual(n.is_unknown(), False)
1937+        self.failUnlessEqual(n.is_allowed_in_immutable_directory(), False)
1938+        n.raise_error()
1939 
1940         n2 = MutableFileNode(None, None, client.get_encoding_parameters(),
1941                              None).init_from_cap(u)
1942hunk ./src/allmydata/test/test_filenode.py 136
1943         self.failUnlessEqual(n, n2)
1944         self.failIfEqual(n, "not even the right type")
1945         self.failIfEqual(n, u) # not the right class
1946+        n.raise_error()
1947         d = {n: "can these be used as dictionary keys?"}
1948         d[n2] = "replace the old one"
1949         self.failUnlessEqual(len(d), 1)
1950hunk ./src/allmydata/test/test_filenode.py 147
1951         self.failUnlessEqual(nro.get_readonly(), nro)
1952         self.failUnlessEqual(nro.get_cap(), u.get_readonly())
1953         self.failUnlessEqual(nro.get_readcap(), u.get_readonly())
1954+        self.failUnlessEqual(nro.is_mutable(), True)
1955+        self.failUnlessEqual(nro.is_readonly(), True)
1956+        self.failUnlessEqual(nro.is_unknown(), False)
1957+        self.failUnlessEqual(nro.is_allowed_in_immutable_directory(), False)
1958         nro_u = nro.get_uri()
1959         self.failUnlessEqual(nro_u, nro.get_readonly_uri())
1960         self.failUnlessEqual(nro_u, u.get_readonly().to_string())
1961hunk ./src/allmydata/test/test_filenode.py 154
1962-        self.failUnlessEqual(nro.is_mutable(), True)
1963-        self.failUnlessEqual(nro.is_readonly(), True)
1964+        self.failUnlessEqual(nro.get_write_uri(), None)
1965         self.failUnlessEqual(nro.get_repair_cap(), None) # RSAmut needs writecap
1966hunk ./src/allmydata/test/test_filenode.py 156
1967+        nro.raise_error()
1968 
1969         v = n.get_verify_cap()
1970         self.failUnless(isinstance(v, uri.SSKVerifierURI))
1971hunk ./src/allmydata/test/test_system.py 20
1972 from allmydata.interfaces import IDirectoryNode, IFileNode, \
1973      NoSuchChildError, NoSharesError
1974 from allmydata.monitor import Monitor
1975-from allmydata.mutable.common import NotMutableError
1976+from allmydata.mutable.common import NotWriteableError
1977 from allmydata.mutable import layout as mutable_layout
1978 from foolscap.api import DeadReferenceError
1979 from twisted.python.failure import Failure
1980hunk ./src/allmydata/test/test_system.py 893
1981             d1.addCallback(lambda res: dirnode.list())
1982             d1.addCallback(self.log, "dirnode.list")
1983 
1984-            d1.addCallback(lambda res: self.shouldFail2(NotMutableError, "mkdir(nope)", None, dirnode.create_subdirectory, u"nope"))
1985+            d1.addCallback(lambda res: self.shouldFail2(NotWriteableError, "mkdir(nope)", None, dirnode.create_subdirectory, u"nope"))
1986 
1987             d1.addCallback(self.log, "doing add_file(ro)")
1988             ut = upload.Data("I will disappear, unrecorded and unobserved. The tragedy of my demise is made more poignant by its silence, but this beauty is not for you to ever know.", convergence="99i-p1x4-xd4-18yc-ywt-87uu-msu-zo -- completely and totally unguessable string (unless you read this)")
1989hunk ./src/allmydata/test/test_system.py 897
1990-            d1.addCallback(lambda res: self.shouldFail2(NotMutableError, "add_file(nope)", None, dirnode.add_file, u"hope", ut))
1991+            d1.addCallback(lambda res: self.shouldFail2(NotWriteableError, "add_file(nope)", None, dirnode.add_file, u"hope", ut))
1992 
1993             d1.addCallback(self.log, "doing get(ro)")
1994             d1.addCallback(lambda res: dirnode.get(u"mydata992"))
1995hunk ./src/allmydata/test/test_system.py 905
1996                            self.failUnless(IFileNode.providedBy(filenode)))
1997 
1998             d1.addCallback(self.log, "doing delete(ro)")
1999-            d1.addCallback(lambda res: self.shouldFail2(NotMutableError, "delete(nope)", None, dirnode.delete, u"mydata992"))
2000+            d1.addCallback(lambda res: self.shouldFail2(NotWriteableError, "delete(nope)", None, dirnode.delete, u"mydata992"))
2001 
2002hunk ./src/allmydata/test/test_system.py 907
2003-            d1.addCallback(lambda res: self.shouldFail2(NotMutableError, "set_uri(nope)", None, dirnode.set_uri, u"hopeless", self.uri, self.uri))
2004+            d1.addCallback(lambda res: self.shouldFail2(NotWriteableError, "set_uri(nope)", None, dirnode.set_uri, u"hopeless", self.uri, self.uri))
2005 
2006             d1.addCallback(lambda res: self.shouldFail2(NoSuchChildError, "get(missing)", "missing", dirnode.get, u"missing"))
2007 
2008hunk ./src/allmydata/test/test_system.py 912
2009             personal = self._personal_node
2010-            d1.addCallback(lambda res: self.shouldFail2(NotMutableError, "mv from readonly", None, dirnode.move_child_to, u"mydata992", personal, u"nope"))
2011+            d1.addCallback(lambda res: self.shouldFail2(NotWriteableError, "mv from readonly", None, dirnode.move_child_to, u"mydata992", personal, u"nope"))
2012 
2013             d1.addCallback(self.log, "doing move_child_to(ro)2")
2014hunk ./src/allmydata/test/test_system.py 915
2015-            d1.addCallback(lambda res: self.shouldFail2(NotMutableError, "mv to readonly", None, personal.move_child_to, u"sekrit data", dirnode, u"nope"))
2016+            d1.addCallback(lambda res: self.shouldFail2(NotWriteableError, "mv to readonly", None, personal.move_child_to, u"sekrit data", dirnode, u"nope"))
2017 
2018             d1.addCallback(self.log, "finished with _got_s2ro")
2019             return d1
2020hunk ./src/allmydata/test/test_uri.py 6
2021 from allmydata import uri
2022 from allmydata.util import hashutil, base32
2023 from allmydata.interfaces import IURI, IFileURI, IDirnodeURI, IMutableFileURI, \
2024-    IVerifierURI
2025+    IVerifierURI, CapConstraintError
2026 
2027 class Literal(unittest.TestCase):
2028     def _help_test(self, data):
2029hunk ./src/allmydata/test/test_uri.py 25
2030         self.failIf(IDirnodeURI.providedBy(u2))
2031         self.failUnlessEqual(u2.data, data)
2032         self.failUnlessEqual(u2.get_size(), len(data))
2033-        self.failUnless(u.is_readonly())
2034-        self.failIf(u.is_mutable())
2035+        self.failUnless(u2.is_readonly())
2036+        self.failIf(u2.is_mutable())
2037+
2038+        u2i = uri.from_string(u.to_string(), deep_immutable=True)
2039+        self.failUnless(IFileURI.providedBy(u2i))
2040+        self.failIf(IDirnodeURI.providedBy(u2i))
2041+        self.failUnlessEqual(u2i.data, data)
2042+        self.failUnlessEqual(u2i.get_size(), len(data))
2043+        self.failUnless(u2i.is_readonly())
2044+        self.failIf(u2i.is_mutable())
2045 
2046         u3 = u.get_readonly()
2047         self.failUnlessIdentical(u, u3)
2048hunk ./src/allmydata/test/test_uri.py 62
2049         fileURI = 'URI:CHK:f5ahxa25t4qkktywz6teyfvcx4:opuioq7tj2y6idzfp6cazehtmgs5fdcebcz3cygrxyydvcozrmeq:3:10:345834'
2050         chk1 = uri.CHKFileURI.init_from_string(fileURI)
2051         chk2 = uri.CHKFileURI.init_from_string(fileURI)
2052+        unk = uri.UnknownURI("lafs://from_the_future")
2053         self.failIfEqual(lit1, chk1)
2054         self.failUnlessEqual(chk1, chk2)
2055         self.failIfEqual(chk1, "not actually a URI")
2056hunk ./src/allmydata/test/test_uri.py 67
2057         # these should be hashable too
2058-        s = set([lit1, chk1, chk2])
2059-        self.failUnlessEqual(len(s), 2) # since chk1==chk2
2060+        s = set([lit1, chk1, chk2, unk])
2061+        self.failUnlessEqual(len(s), 3) # since chk1==chk2
2062 
2063     def test_is_uri(self):
2064         lit1 = uri.LiteralFileURI("some data").to_string()
2065hunk ./src/allmydata/test/test_uri.py 75
2066         self.failUnless(uri.is_uri(lit1))
2067         self.failIf(uri.is_uri(None))
2068 
2069+    def test_is_literal_file_uri(self):
2070+        lit1 = uri.LiteralFileURI("some data").to_string()
2071+        self.failUnless(uri.is_literal_file_uri(lit1))
2072+        self.failIf(uri.is_literal_file_uri(None))
2073+        self.failIf(uri.is_literal_file_uri("foo"))
2074+        self.failIf(uri.is_literal_file_uri("ro.foo"))
2075+        self.failIf(uri.is_literal_file_uri("URI:LITfoo"))
2076+        self.failUnless(uri.is_literal_file_uri("ro.URI:LIT:foo"))
2077+        self.failUnless(uri.is_literal_file_uri("imm.URI:LIT:foo"))
2078+
2079+    def test_has_uri_prefix(self):
2080+        self.failUnless(uri.has_uri_prefix("URI:foo"))
2081+        self.failUnless(uri.has_uri_prefix("ro.URI:foo"))
2082+        self.failUnless(uri.has_uri_prefix("imm.URI:foo"))
2083+        self.failIf(uri.has_uri_prefix(None))
2084+        self.failIf(uri.has_uri_prefix("foo"))
2085+
2086 class CHKFile(unittest.TestCase):
2087     def test_pack(self):
2088         key = "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f"
2089hunk ./src/allmydata/test/test_uri.py 117
2090         self.failUnless(IFileURI.providedBy(u))
2091         self.failIf(IDirnodeURI.providedBy(u))
2092         self.failUnlessEqual(u.get_size(), 1234)
2093-        self.failUnless(u.is_readonly())
2094-        self.failIf(u.is_mutable())
2095+
2096         u_ro = u.get_readonly()
2097         self.failUnlessIdentical(u, u_ro)
2098         he = u.to_human_encoding()
2099hunk ./src/allmydata/test/test_uri.py 137
2100         self.failUnless(IFileURI.providedBy(u2))
2101         self.failIf(IDirnodeURI.providedBy(u2))
2102         self.failUnlessEqual(u2.get_size(), 1234)
2103-        self.failUnless(u2.is_readonly())
2104-        self.failIf(u2.is_mutable())
2105+
2106+        u2i = uri.from_string(u.to_string(), deep_immutable=True)
2107+        self.failUnlessEqual(u.to_string(), u2i.to_string())
2108+        u2ro = uri.from_string(uri.ALLEGED_READONLY_PREFIX + u.to_string())
2109+        self.failUnlessEqual(u.to_string(), u2ro.to_string())
2110+        u2imm = uri.from_string(uri.ALLEGED_IMMUTABLE_PREFIX + u.to_string())
2111+        self.failUnlessEqual(u.to_string(), u2imm.to_string())
2112 
2113         v = u.get_verify_cap()
2114         self.failUnless(isinstance(v.to_string(), str))
2115hunk ./src/allmydata/test/test_uri.py 147
2116+        self.failUnless(v.is_readonly())
2117+        self.failIf(v.is_mutable())
2118+
2119         v2 = uri.from_string(v.to_string())
2120         self.failUnlessEqual(v, v2)
2121         he = v.to_human_encoding()
2122hunk ./src/allmydata/test/test_uri.py 162
2123                                     total_shares=10,
2124                                     size=1234)
2125         self.failUnless(isinstance(v3.to_string(), str))
2126+        self.failUnless(v3.is_readonly())
2127+        self.failIf(v3.is_mutable())
2128 
2129     def test_pack_badly(self):
2130         key = "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f"
2131hunk ./src/allmydata/test/test_uri.py 217
2132         self.failUnlessEqual(readable["UEB_hash"],
2133                              base32.b2a(hashutil.uri_extension_hash(ext)))
2134 
2135-class Invalid(unittest.TestCase):
2136+class Unknown(unittest.TestCase):
2137     def test_from_future(self):
2138         # any URI type that we don't recognize should be treated as unknown
2139         future_uri = "I am a URI from the future. Whatever you do, don't "
2140hunk ./src/allmydata/test/test_uri.py 224
2141         u = uri.from_string(future_uri)
2142         self.failUnless(isinstance(u, uri.UnknownURI))
2143         self.failUnlessEqual(u.to_string(), future_uri)
2144+        self.failUnless(u.get_readonly() is None)
2145+        self.failUnless(u.get_error() is None)
2146+
2147+        u2 = uri.UnknownURI(future_uri, error=CapConstraintError("..."))
2148+        self.failUnlessEqual(u.to_string(), future_uri)
2149+        self.failUnless(u2.get_readonly() is None)
2150+        self.failUnless(isinstance(u2.get_error(), CapConstraintError))
2151 
2152 class Constraint(unittest.TestCase):
2153     def test_constraint(self):
2154hunk ./src/allmydata/test/test_uri.py 271
2155         self.failUnless(IMutableFileURI.providedBy(u2))
2156         self.failIf(IDirnodeURI.providedBy(u2))
2157 
2158+        u2i = uri.from_string(u.to_string(), deep_immutable=True)
2159+        self.failUnless(isinstance(u2i, uri.UnknownURI), u2i)
2160+        u2ro = uri.from_string(uri.ALLEGED_READONLY_PREFIX + u.to_string())
2161+        self.failUnless(isinstance(u2ro, uri.UnknownURI), u2ro)
2162+        u2imm = uri.from_string(uri.ALLEGED_IMMUTABLE_PREFIX + u.to_string())
2163+        self.failUnless(isinstance(u2imm, uri.UnknownURI), u2imm)
2164+
2165         u3 = u2.get_readonly()
2166         readkey = hashutil.ssk_readkey_hash(writekey)
2167         self.failUnlessEqual(u3.fingerprint, fingerprint)
2168hunk ./src/allmydata/test/test_uri.py 288
2169         self.failUnless(IMutableFileURI.providedBy(u3))
2170         self.failIf(IDirnodeURI.providedBy(u3))
2171 
2172+        u3i = uri.from_string(u3.to_string(), deep_immutable=True)
2173+        self.failUnless(isinstance(u3i, uri.UnknownURI), u3i)
2174+        u3ro = uri.from_string(uri.ALLEGED_READONLY_PREFIX + u3.to_string())
2175+        self.failUnlessEqual(u3.to_string(), u3ro.to_string())
2176+        u3imm = uri.from_string(uri.ALLEGED_IMMUTABLE_PREFIX + u3.to_string())
2177+        self.failUnless(isinstance(u3imm, uri.UnknownURI), u3imm)
2178+
2179         he = u3.to_human_encoding()
2180         u3_h = uri.ReadonlySSKFileURI.init_from_human_encoding(he)
2181         self.failUnlessEqual(u3, u3_h)
2182hunk ./src/allmydata/test/test_uri.py 308
2183         self.failUnless(IMutableFileURI.providedBy(u4))
2184         self.failIf(IDirnodeURI.providedBy(u4))
2185 
2186+        u4i = uri.from_string(u4.to_string(), deep_immutable=True)
2187+        self.failUnless(isinstance(u4i, uri.UnknownURI), u4i)
2188+        u4ro = uri.from_string(uri.ALLEGED_READONLY_PREFIX + u4.to_string())
2189+        self.failUnlessEqual(u4.to_string(), u4ro.to_string())
2190+        u4imm = uri.from_string(uri.ALLEGED_IMMUTABLE_PREFIX + u4.to_string())
2191+        self.failUnless(isinstance(u4imm, uri.UnknownURI), u4imm)
2192+
2193         u4a = uri.from_string(u4.to_string())
2194         self.failUnlessEqual(u4a, u4)
2195         self.failUnless("ReadonlySSKFileURI" in str(u4a))
2196hunk ./src/allmydata/test/test_uri.py 357
2197         self.failIf(IFileURI.providedBy(u2))
2198         self.failUnless(IDirnodeURI.providedBy(u2))
2199 
2200+        u2i = uri.from_string(u1.to_string(), deep_immutable=True)
2201+        self.failUnless(isinstance(u2i, uri.UnknownURI))
2202+
2203         u3 = u2.get_readonly()
2204         self.failUnless(u3.is_readonly())
2205         self.failUnless(u3.is_mutable())
2206hunk ./src/allmydata/test/test_uri.py 366
2207         self.failUnless(IURI.providedBy(u3))
2208         self.failIf(IFileURI.providedBy(u3))
2209         self.failUnless(IDirnodeURI.providedBy(u3))
2210+
2211+        u3i = uri.from_string(u2.to_string(), deep_immutable=True)
2212+        self.failUnless(isinstance(u3i, uri.UnknownURI))
2213+
2214         u3n = u3._filenode_uri
2215         self.failUnless(u3n.is_readonly())
2216         self.failUnless(u3n.is_mutable())
2217hunk ./src/allmydata/test/test_uri.py 436
2218         self.failIf(IFileURI.providedBy(u2))
2219         self.failUnless(IDirnodeURI.providedBy(u2))
2220 
2221+        u2i = uri.from_string(u1.to_string(), deep_immutable=True)
2222+        self.failUnlessEqual(u1.to_string(), u2i.to_string())
2223+
2224         u3 = u2.get_readonly()
2225         self.failUnlessEqual(u3.to_string(), u2.to_string())
2226         self.failUnless(str(u3))
2227hunk ./src/allmydata/test/test_uri.py 443
2228 
2229+        u3i = uri.from_string(u2.to_string(), deep_immutable=True)
2230+        self.failUnlessEqual(u2.to_string(), u3i.to_string())
2231+
2232         u2_verifier = u2.get_verify_cap()
2233         self.failUnless(isinstance(u2_verifier,
2234                                    uri.ImmutableDirectoryURIVerifier),
2235hunk ./src/allmydata/test/test_web.py 10
2236 from twisted.web import client, error, http
2237 from twisted.python import failure, log
2238 from nevow import rend
2239-from allmydata import interfaces, uri, webish
2240+from allmydata import interfaces, uri, webish, dirnode
2241 from allmydata.storage.shares import get_share_file
2242 from allmydata.storage_client import StorageFarmBroker
2243 from allmydata.immutable import upload, download
2244hunk ./src/allmydata/test/test_web.py 21
2245 from allmydata.scripts.debug import CorruptShareOptions, corrupt_share
2246 from allmydata.util import fileutil, base32
2247 from allmydata.util.consumer import download_to_data
2248+from allmydata.util.netstring import split_netstring
2249 from allmydata.test.common import FakeCHKFileNode, FakeMutableFileNode, \
2250      create_chk_filenode, WebErrorMixin, ShouldFailMixin, make_mutable_file_uri
2251 from allmydata.interfaces import IMutableFileNode
2252hunk ./src/allmydata/test/test_web.py 739
2253         d = self.PUT(self.public_url + "/foo/new.txt", self.NEWFILE_CONTENTS)
2254         # TODO: we lose the response code, so we can't check this
2255         #self.failUnlessEqual(responsecode, 201)
2256-        d.addCallback(self.failUnlessURIMatchesChild, self._foo_node, u"new.txt")
2257+        d.addCallback(self.failUnlessURIMatchesROChild, self._foo_node, u"new.txt")
2258         d.addCallback(lambda res:
2259                       self.failUnlessChildContentsAre(self._foo_node, u"new.txt",
2260                                                       self.NEWFILE_CONTENTS))
2261hunk ./src/allmydata/test/test_web.py 750
2262                      self.NEWFILE_CONTENTS)
2263         # TODO: we lose the response code, so we can't check this
2264         #self.failUnlessEqual(responsecode, 201)
2265-        d.addCallback(self.failUnlessURIMatchesChild, self._foo_node, u"new.txt")
2266+        d.addCallback(self.failUnlessURIMatchesROChild, self._foo_node, u"new.txt")
2267         d.addCallback(lambda res:
2268                       self.failUnlessChildContentsAre(self._foo_node, u"new.txt",
2269                                                       self.NEWFILE_CONTENTS))
2270hunk ./src/allmydata/test/test_web.py 780
2271             self.failIf(u.is_readonly())
2272             return res
2273         d.addCallback(_check_uri)
2274-        d.addCallback(self.failUnlessURIMatchesChild, self._foo_node, u"new.txt")
2275+        d.addCallback(self.failUnlessURIMatchesRWChild, self._foo_node, u"new.txt")
2276         d.addCallback(lambda res:
2277                       self.failUnlessMutableChildContentsAre(self._foo_node,
2278                                                              u"new.txt",
2279hunk ./src/allmydata/test/test_web.py 800
2280         d = self.PUT(self.public_url + "/foo/bar.txt", self.NEWFILE_CONTENTS)
2281         # TODO: we lose the response code, so we can't check this
2282         #self.failUnlessEqual(responsecode, 200)
2283-        d.addCallback(self.failUnlessURIMatchesChild, self._foo_node, u"bar.txt")
2284+        d.addCallback(self.failUnlessURIMatchesROChild, self._foo_node, u"bar.txt")
2285         d.addCallback(lambda res:
2286                       self.failUnlessChildContentsAre(self._foo_node, u"bar.txt",
2287                                                       self.NEWFILE_CONTENTS))
2288hunk ./src/allmydata/test/test_web.py 825
2289     def test_PUT_NEWFILEURL_mkdirs(self):
2290         d = self.PUT(self.public_url + "/foo/newdir/new.txt", self.NEWFILE_CONTENTS)
2291         fn = self._foo_node
2292-        d.addCallback(self.failUnlessURIMatchesChild, fn, u"newdir/new.txt")
2293+        d.addCallback(self.failUnlessURIMatchesROChild, fn, u"newdir/new.txt")
2294         d.addCallback(lambda res: self.failIfNodeHasChild(fn, u"new.txt"))
2295         d.addCallback(lambda res: self.failUnlessNodeHasChild(fn, u"newdir"))
2296         d.addCallback(lambda res:
2297hunk ./src/allmydata/test/test_web.py 958
2298             self.failUnless(re.search(get_sub, res), res)
2299         d.addCallback(_check)
2300 
2301-        # look at a directory which is readonly
2302+        # look at a readonly directory
2303         d.addCallback(lambda res:
2304                       self.GET(self.public_url + "/reedownlee", followRedirect=True))
2305         def _check2(res):
2306hunk ./src/allmydata/test/test_web.py 1171
2307         return d
2308 
2309     def test_POST_NEWDIRURL_initial_children(self):
2310-        (newkids, filecap1, filecap2, filecap3,
2311-         dircap) = self._create_initial_children()
2312+        (newkids, caps) = self._create_initial_children()
2313         d = self.POST2(self.public_url + "/foo/newdir?t=mkdir-with-children",
2314                        simplejson.dumps(newkids))
2315         def _check(uri):
2316hunk ./src/allmydata/test/test_web.py 1178
2317             n = self.s.create_node_from_uri(uri.strip())
2318             d2 = self.failUnlessNodeKeysAre(n, newkids.keys())
2319             d2.addCallback(lambda ign:
2320-                           self.failUnlessChildURIIs(n, u"child-imm", filecap1))
2321+                           self.failUnlessROChildURIIs(n, u"child-imm",
2322+                                                       caps['filecap1']))
2323+            d2.addCallback(lambda ign:
2324+                           self.failUnlessRWChildURIIs(n, u"child-mutable",
2325+                                                       caps['filecap2']))
2326+            d2.addCallback(lambda ign:
2327+                           self.failUnlessROChildURIIs(n, u"child-mutable-ro",
2328+                                                       caps['filecap3']))
2329             d2.addCallback(lambda ign:
2330hunk ./src/allmydata/test/test_web.py 1187
2331-                           self.failUnlessChildURIIs(n, u"child-mutable",
2332-                                                     filecap2))
2333+                           self.failUnlessROChildURIIs(n, u"unknownchild-ro",
2334+                                                       caps['unknown_rocap']))
2335             d2.addCallback(lambda ign:
2336hunk ./src/allmydata/test/test_web.py 1190
2337-                           self.failUnlessChildURIIs(n, u"child-mutable-ro",
2338-                                                     filecap3))
2339+                           self.failUnlessRWChildURIIs(n, u"unknownchild-rw",
2340+                                                       caps['unknown_rwcap']))
2341             d2.addCallback(lambda ign:
2342hunk ./src/allmydata/test/test_web.py 1193
2343-                           self.failUnlessChildURIIs(n, u"dirchild", dircap))
2344+                           self.failUnlessROChildURIIs(n, u"unknownchild-imm",
2345+                                                       caps['unknown_immcap']))
2346+            d2.addCallback(lambda ign:
2347+                           self.failUnlessRWChildURIIs(n, u"dirchild",
2348+                                                       caps['dircap']))
2349             return d2
2350         d.addCallback(_check)
2351         d.addCallback(lambda res:
2352hunk ./src/allmydata/test/test_web.py 1205
2353         d.addCallback(lambda res: self._foo_node.get(u"newdir"))
2354         d.addCallback(self.failUnlessNodeKeysAre, newkids.keys())
2355         d.addCallback(lambda res: self._foo_node.get(u"newdir"))
2356-        d.addCallback(self.failUnlessChildURIIs, u"child-imm", filecap1)
2357+        d.addCallback(self.failUnlessROChildURIIs, u"child-imm", caps['filecap1'])
2358         return d
2359 
2360     def test_POST_NEWDIRURL_immutable(self):
2361hunk ./src/allmydata/test/test_web.py 1209
2362-        (newkids, filecap1, immdircap) = self._create_immutable_children()
2363+        (newkids, caps) = self._create_immutable_children()
2364         d = self.POST2(self.public_url + "/foo/newdir?t=mkdir-immutable",
2365                        simplejson.dumps(newkids))
2366         def _check(uri):
2367hunk ./src/allmydata/test/test_web.py 1216
2368             n = self.s.create_node_from_uri(uri.strip())
2369             d2 = self.failUnlessNodeKeysAre(n, newkids.keys())
2370             d2.addCallback(lambda ign:
2371-                           self.failUnlessChildURIIs(n, u"child-imm", filecap1))
2372+                           self.failUnlessROChildURIIs(n, u"child-imm",
2373+                                                       caps['filecap1']))
2374+            d2.addCallback(lambda ign:
2375+                           self.failUnlessROChildURIIs(n, u"unknownchild-imm",
2376+                                                       caps['unknown_immcap']))
2377             d2.addCallback(lambda ign:
2378hunk ./src/allmydata/test/test_web.py 1222
2379-                           self.failUnlessChildURIIs(n, u"dirchild-imm",
2380-                                                     immdircap))
2381+                           self.failUnlessROChildURIIs(n, u"dirchild-imm",
2382+                                                       caps['immdircap']))
2383             return d2
2384         d.addCallback(_check)
2385         d.addCallback(lambda res:
2386hunk ./src/allmydata/test/test_web.py 1231
2387         d.addCallback(lambda res: self._foo_node.get(u"newdir"))
2388         d.addCallback(self.failUnlessNodeKeysAre, newkids.keys())
2389         d.addCallback(lambda res: self._foo_node.get(u"newdir"))
2390-        d.addCallback(self.failUnlessChildURIIs, u"child-imm", filecap1)
2391+        d.addCallback(self.failUnlessROChildURIIs, u"child-imm", caps['filecap1'])
2392+        d.addCallback(lambda res: self._foo_node.get(u"newdir"))
2393+        d.addCallback(self.failUnlessROChildURIIs, u"unknownchild-imm", caps['unknown_immcap'])
2394         d.addCallback(lambda res: self._foo_node.get(u"newdir"))
2395hunk ./src/allmydata/test/test_web.py 1235
2396-        d.addCallback(self.failUnlessChildURIIs, u"dirchild-imm", immdircap)
2397+        d.addCallback(self.failUnlessROChildURIIs, u"dirchild-imm", caps['immdircap'])
2398         d.addErrback(self.explain_web_error)
2399         return d
2400 
2401hunk ./src/allmydata/test/test_web.py 1240
2402     def test_POST_NEWDIRURL_immutable_bad(self):
2403-        (newkids, filecap1, filecap2, filecap3,
2404-         dircap) = self._create_initial_children()
2405+        (newkids, caps) = self._create_initial_children()
2406         d = self.shouldFail2(error.Error, "test_POST_NEWDIRURL_immutable_bad",
2407                              "400 Bad Request",
2408hunk ./src/allmydata/test/test_web.py 1243
2409-                             "a mkdir-immutable operation was given a child that was not itself immutable",
2410+                             "needed to be immutable but was not",
2411                              self.POST2,
2412                              self.public_url + "/foo/newdir?t=mkdir-immutable",
2413                              simplejson.dumps(newkids))
2414hunk ./src/allmydata/test/test_web.py 1365
2415         d.addCallback(_check)
2416         return d
2417 
2418-    def failUnlessChildURIIs(self, node, name, expected_uri):
2419+    def failUnlessRWChildURIIs(self, node, name, expected_uri):
2420         assert isinstance(name, unicode)
2421         d = node.get_child_at_path(name)
2422         def _check(child):
2423hunk ./src/allmydata/test/test_web.py 1369
2424+            self.failUnless(child.is_unknown() or not child.is_readonly())
2425             self.failUnlessEqual(child.get_uri(), expected_uri.strip())
2426hunk ./src/allmydata/test/test_web.py 1371
2427+            self.failUnlessEqual(child.get_write_uri(), expected_uri.strip())
2428+            expected_ro_uri = self._make_readonly(expected_uri)
2429+            if expected_ro_uri:
2430+                self.failUnlessEqual(child.get_readonly_uri(), expected_ro_uri.strip())
2431+        d.addCallback(_check)
2432+        return d
2433+
2434+    def failUnlessROChildURIIs(self, node, name, expected_uri):
2435+        assert isinstance(name, unicode)
2436+        d = node.get_child_at_path(name)
2437+        def _check(child):
2438+            self.failUnless(child.is_unknown() or child.is_readonly())
2439+            self.failUnlessEqual(child.get_write_uri(), None)
2440+            self.failUnlessEqual(child.get_uri(), expected_uri.strip())
2441+            self.failUnlessEqual(child.get_readonly_uri(), expected_uri.strip())
2442+        d.addCallback(_check)
2443+        return d
2444+
2445+    def failUnlessURIMatchesRWChild(self, got_uri, node, name):
2446+        assert isinstance(name, unicode)
2447+        d = node.get_child_at_path(name)
2448+        def _check(child):
2449+            self.failUnless(child.is_unknown() or not child.is_readonly())
2450+            self.failUnlessEqual(child.get_uri(), got_uri.strip())
2451+            self.failUnlessEqual(child.get_write_uri(), got_uri.strip())
2452+            expected_ro_uri = self._make_readonly(got_uri)
2453+            if expected_ro_uri:
2454+                self.failUnlessEqual(child.get_readonly_uri(), expected_ro_uri.strip())
2455         d.addCallback(_check)
2456         return d
2457 
2458hunk ./src/allmydata/test/test_web.py 1402
2459-    def failUnlessURIMatchesChild(self, got_uri, node, name):
2460+    def failUnlessURIMatchesROChild(self, got_uri, node, name):
2461         assert isinstance(name, unicode)
2462         d = node.get_child_at_path(name)
2463         def _check(child):
2464hunk ./src/allmydata/test/test_web.py 1406
2465+            self.failUnless(child.is_unknown() or child.is_readonly())
2466+            self.failUnlessEqual(child.get_write_uri(), None)
2467             self.failUnlessEqual(got_uri.strip(), child.get_uri())
2468hunk ./src/allmydata/test/test_web.py 1409
2469+            self.failUnlessEqual(got_uri.strip(), child.get_readonly_uri())
2470         d.addCallback(_check)
2471         return d
2472 
2473hunk ./src/allmydata/test/test_web.py 1420
2474         d = self.POST(self.public_url + "/foo", t="upload",
2475                       file=("new.txt", self.NEWFILE_CONTENTS))
2476         fn = self._foo_node
2477-        d.addCallback(self.failUnlessURIMatchesChild, fn, u"new.txt")
2478+        d.addCallback(self.failUnlessURIMatchesROChild, fn, u"new.txt")
2479         d.addCallback(lambda res:
2480                       self.failUnlessChildContentsAre(fn, u"new.txt",
2481                                                       self.NEWFILE_CONTENTS))
2482hunk ./src/allmydata/test/test_web.py 1431
2483         d = self.POST(self.public_url + "/foo", t="upload",
2484                       file=(filename, self.NEWFILE_CONTENTS))
2485         fn = self._foo_node
2486-        d.addCallback(self.failUnlessURIMatchesChild, fn, filename)
2487+        d.addCallback(self.failUnlessURIMatchesROChild, fn, filename)
2488         d.addCallback(lambda res:
2489                       self.failUnlessChildContentsAre(fn, filename,
2490                                                       self.NEWFILE_CONTENTS))
2491hunk ./src/allmydata/test/test_web.py 1448
2492                       name=filename,
2493                       file=("overridden", self.NEWFILE_CONTENTS))
2494         fn = self._foo_node
2495-        d.addCallback(self.failUnlessURIMatchesChild, fn, filename)
2496+        d.addCallback(self.failUnlessURIMatchesROChild, fn, filename)
2497         d.addCallback(lambda res:
2498                       self.failUnlessChildContentsAre(fn, filename,
2499                                                       self.NEWFILE_CONTENTS))
2500hunk ./src/allmydata/test/test_web.py 1550
2501         d = self.POST(self.public_url + "/foo", t="upload", mutable="true",
2502                       file=("new.txt", self.NEWFILE_CONTENTS))
2503         fn = self._foo_node
2504-        d.addCallback(self.failUnlessURIMatchesChild, fn, u"new.txt")
2505+        d.addCallback(self.failUnlessURIMatchesRWChild, fn, u"new.txt")
2506         d.addCallback(lambda res:
2507                       self.failUnlessMutableChildContentsAre(fn, u"new.txt",
2508                                                              self.NEWFILE_CONTENTS))
2509hunk ./src/allmydata/test/test_web.py 1569
2510                       self.POST(self.public_url + "/foo", t="upload",
2511                                 mutable="true",
2512                                 file=("new.txt", NEWER_CONTENTS)))
2513-        d.addCallback(self.failUnlessURIMatchesChild, fn, u"new.txt")
2514+        d.addCallback(self.failUnlessURIMatchesRWChild, fn, u"new.txt")
2515         d.addCallback(lambda res:
2516                       self.failUnlessMutableChildContentsAre(fn, u"new.txt",
2517                                                              NEWER_CONTENTS))
2518hunk ./src/allmydata/test/test_web.py 1585
2519         NEW2_CONTENTS = NEWER_CONTENTS + "overwrite with PUT\n"
2520         d.addCallback(lambda res:
2521                       self.PUT(self.public_url + "/foo/new.txt", NEW2_CONTENTS))
2522-        d.addCallback(self.failUnlessURIMatchesChild, fn, u"new.txt")
2523+        d.addCallback(self.failUnlessURIMatchesRWChild, fn, u"new.txt")
2524         d.addCallback(lambda res:
2525                       self.failUnlessMutableChildContentsAre(fn, u"new.txt",
2526                                                              NEW2_CONTENTS))
2527hunk ./src/allmydata/test/test_web.py 1714
2528         d = self.POST(self.public_url + "/foo", t="upload",
2529                       file=("bar.txt", self.NEWFILE_CONTENTS))
2530         fn = self._foo_node
2531-        d.addCallback(self.failUnlessURIMatchesChild, fn, u"bar.txt")
2532+        d.addCallback(self.failUnlessURIMatchesROChild, fn, u"bar.txt")
2533         d.addCallback(lambda res:
2534                       self.failUnlessChildContentsAre(fn, u"bar.txt",
2535                                                       self.NEWFILE_CONTENTS))
2536hunk ./src/allmydata/test/test_web.py 1765
2537         fn = self._foo_node
2538         d = self.POST(self.public_url + "/foo", t="upload",
2539                       name="new.txt", file=self.NEWFILE_CONTENTS)
2540-        d.addCallback(self.failUnlessURIMatchesChild, fn, u"new.txt")
2541+        d.addCallback(self.failUnlessURIMatchesROChild, fn, u"new.txt")
2542         d.addCallback(lambda res:
2543                       self.failUnlessChildContentsAre(fn, u"new.txt",
2544                                                       self.NEWFILE_CONTENTS))
2545hunk ./src/allmydata/test/test_web.py 2028
2546         return d
2547 
2548     def test_POST_mkdir_initial_children(self):
2549-        newkids, filecap1, ign, ign, ign = self._create_initial_children()
2550+        (newkids, caps) = self._create_initial_children()
2551         d = self.POST2(self.public_url +
2552                        "/foo?t=mkdir-with-children&name=newdir",
2553                        simplejson.dumps(newkids))
2554hunk ./src/allmydata/test/test_web.py 2037
2555         d.addCallback(lambda res: self._foo_node.get(u"newdir"))
2556         d.addCallback(self.failUnlessNodeKeysAre, newkids.keys())
2557         d.addCallback(lambda res: self._foo_node.get(u"newdir"))
2558-        d.addCallback(self.failUnlessChildURIIs, u"child-imm", filecap1)
2559+        d.addCallback(self.failUnlessROChildURIIs, u"child-imm", caps['filecap1'])
2560         return d
2561 
2562     def test_POST_mkdir_immutable(self):
2563hunk ./src/allmydata/test/test_web.py 2041
2564-        (newkids, filecap1, immdircap) = self._create_immutable_children()
2565+        (newkids, caps) = self._create_immutable_children()
2566         d = self.POST2(self.public_url +
2567                        "/foo?t=mkdir-immutable&name=newdir",
2568                        simplejson.dumps(newkids))
2569hunk ./src/allmydata/test/test_web.py 2050
2570         d.addCallback(lambda res: self._foo_node.get(u"newdir"))
2571         d.addCallback(self.failUnlessNodeKeysAre, newkids.keys())
2572         d.addCallback(lambda res: self._foo_node.get(u"newdir"))
2573-        d.addCallback(self.failUnlessChildURIIs, u"child-imm", filecap1)
2574+        d.addCallback(self.failUnlessROChildURIIs, u"child-imm", caps['filecap1'])
2575+        d.addCallback(lambda res: self._foo_node.get(u"newdir"))
2576+        d.addCallback(self.failUnlessROChildURIIs, u"unknownchild-imm", caps['unknown_immcap'])
2577         d.addCallback(lambda res: self._foo_node.get(u"newdir"))
2578hunk ./src/allmydata/test/test_web.py 2054
2579-        d.addCallback(self.failUnlessChildURIIs, u"dirchild-imm", immdircap)
2580+        d.addCallback(self.failUnlessROChildURIIs, u"dirchild-imm", caps['immdircap'])
2581         return d
2582 
2583     def test_POST_mkdir_immutable_bad(self):
2584hunk ./src/allmydata/test/test_web.py 2058
2585-        (newkids, filecap1, filecap2, filecap3,
2586-         dircap) = self._create_initial_children()
2587+        (newkids, caps) = self._create_initial_children()
2588         d = self.shouldFail2(error.Error, "test_POST_mkdir_immutable_bad",
2589                              "400 Bad Request",
2590hunk ./src/allmydata/test/test_web.py 2061
2591-                             "a mkdir-immutable operation was given a child that was not itself immutable",
2592+                             "needed to be immutable but was not",
2593                              self.POST2,
2594                              self.public_url +
2595                              "/foo?t=mkdir-immutable&name=newdir",
2596hunk ./src/allmydata/test/test_web.py 2120
2597         d.addErrback(self.explain_web_error)
2598         return d
2599 
2600+    def _make_readonly(self, u):
2601+        ro_uri = uri.from_string(u).get_readonly()
2602+        if ro_uri is None:
2603+            return None
2604+        return ro_uri.to_string()
2605+
2606     def _create_initial_children(self):
2607         contents, n, filecap1 = self.makefile(12)
2608         md1 = {"metakey1": "metavalue1"}
2609hunk ./src/allmydata/test/test_web.py 2132
2610         filecap2 = make_mutable_file_uri()
2611         node3 = self.s.create_node_from_uri(make_mutable_file_uri())
2612         filecap3 = node3.get_readonly_uri()
2613+        unknown_rwcap = "lafs://from_the_future"
2614+        unknown_rocap = "ro.lafs://readonly_from_the_future"
2615+        unknown_immcap = "imm.lafs://immutable_from_the_future"
2616         node4 = self.s.create_node_from_uri(make_mutable_file_uri())
2617         dircap = DirectoryNode(node4, None, None).get_uri()
2618hunk ./src/allmydata/test/test_web.py 2137
2619-        newkids = {u"child-imm": ["filenode", {"ro_uri": filecap1,
2620-                                               "metadata": md1, }],
2621-                   u"child-mutable": ["filenode", {"rw_uri": filecap2}],
2622+        newkids = {u"child-imm":        ["filenode", {"rw_uri": filecap1,
2623+                                                      "ro_uri": self._make_readonly(filecap1),
2624+                                                      "metadata": md1, }],
2625+                   u"child-mutable":    ["filenode", {"rw_uri": filecap2,
2626+                                                      "ro_uri": self._make_readonly(filecap2)}],
2627                    u"child-mutable-ro": ["filenode", {"ro_uri": filecap3}],
2628hunk ./src/allmydata/test/test_web.py 2143
2629-                   u"dirchild": ["dirnode", {"rw_uri": dircap}],
2630+                   u"unknownchild-rw":  ["unknown",  {"rw_uri": unknown_rwcap,
2631+                                                      "ro_uri": unknown_rocap}],
2632+                   u"unknownchild-ro":  ["unknown",  {"ro_uri": unknown_rocap}],
2633+                   u"unknownchild-imm": ["unknown",  {"ro_uri": unknown_immcap}],
2634+                   u"dirchild":         ["dirnode",  {"rw_uri": dircap,
2635+                                                      "ro_uri": self._make_readonly(dircap)}],
2636                    }
2637hunk ./src/allmydata/test/test_web.py 2150
2638-        return newkids, filecap1, filecap2, filecap3, dircap
2639+        return newkids, {'filecap1': filecap1,
2640+                         'filecap2': filecap2,
2641+                         'filecap3': filecap3,
2642+                         'unknown_rwcap': unknown_rwcap,
2643+                         'unknown_rocap': unknown_rocap,
2644+                         'unknown_immcap': unknown_immcap,
2645+                         'dircap': dircap}
2646 
2647     def _create_immutable_children(self):
2648         contents, n, filecap1 = self.makefile(12)
2649hunk ./src/allmydata/test/test_web.py 2164
2650         tnode = create_chk_filenode("immutable directory contents\n"*10)
2651         dnode = DirectoryNode(tnode, None, None)
2652         assert not dnode.is_mutable()
2653+        unknown_immcap = "imm.lafs://immutable_from_the_future"
2654         immdircap = dnode.get_uri()
2655hunk ./src/allmydata/test/test_web.py 2166
2656-        newkids = {u"child-imm": ["filenode", {"ro_uri": filecap1,
2657-                                               "metadata": md1, }],
2658-                   u"dirchild-imm": ["dirnode", {"ro_uri": immdircap}],
2659+        newkids = {u"child-imm":        ["filenode", {"ro_uri": filecap1,
2660+                                                      "metadata": md1, }],
2661+                   u"unknownchild-imm": ["unknown",  {"ro_uri": unknown_immcap}],
2662+                   u"dirchild-imm":     ["dirnode",  {"ro_uri": immdircap}],
2663                    }
2664hunk ./src/allmydata/test/test_web.py 2171
2665-        return newkids, filecap1, immdircap
2666+        return newkids, {'filecap1': filecap1,
2667+                         'unknown_immcap': unknown_immcap,
2668+                         'immdircap': immdircap}
2669 
2670     def test_POST_mkdir_no_parentdir_initial_children(self):
2671hunk ./src/allmydata/test/test_web.py 2176
2672-        (newkids, filecap1, filecap2, filecap3,
2673-         dircap) = self._create_initial_children()
2674+        (newkids, caps) = self._create_initial_children()
2675         d = self.POST2("/uri?t=mkdir-with-children", simplejson.dumps(newkids))
2676         def _after_mkdir(res):
2677             self.failUnless(res.startswith("URI:DIR"), res)
2678hunk ./src/allmydata/test/test_web.py 2183
2679             n = self.s.create_node_from_uri(res)
2680             d2 = self.failUnlessNodeKeysAre(n, newkids.keys())
2681             d2.addCallback(lambda ign:
2682-                           self.failUnlessChildURIIs(n, u"child-imm", filecap1))
2683+                           self.failUnlessROChildURIIs(n, u"child-imm",
2684+                                                       caps['filecap1']))
2685+            d2.addCallback(lambda ign:
2686+                           self.failUnlessRWChildURIIs(n, u"child-mutable",
2687+                                                       caps['filecap2']))
2688             d2.addCallback(lambda ign:
2689hunk ./src/allmydata/test/test_web.py 2189
2690-                           self.failUnlessChildURIIs(n, u"child-mutable",
2691-                                                     filecap2))
2692+                           self.failUnlessROChildURIIs(n, u"child-mutable-ro",
2693+                                                       caps['filecap3']))
2694             d2.addCallback(lambda ign:
2695hunk ./src/allmydata/test/test_web.py 2192
2696-                           self.failUnlessChildURIIs(n, u"child-mutable-ro",
2697-                                                     filecap3))
2698+                           self.failUnlessRWChildURIIs(n, u"unknownchild-rw",
2699+                                                       caps['unknown_rwcap']))
2700             d2.addCallback(lambda ign:
2701hunk ./src/allmydata/test/test_web.py 2195
2702-                           self.failUnlessChildURIIs(n, u"dirchild", dircap))
2703+                           self.failUnlessROChildURIIs(n, u"unknownchild-ro",
2704+                                                       caps['unknown_rocap']))
2705+            d2.addCallback(lambda ign:
2706+                           self.failUnlessROChildURIIs(n, u"unknownchild-imm",
2707+                                                       caps['unknown_immcap']))
2708+            d2.addCallback(lambda ign:
2709+                           self.failUnlessRWChildURIIs(n, u"dirchild",
2710+                                                       caps['dircap']))
2711             return d2
2712         d.addCallback(_after_mkdir)
2713         return d
2714hunk ./src/allmydata/test/test_web.py 2210
2715     def test_POST_mkdir_no_parentdir_unexpected_children(self):
2716         # the regular /uri?t=mkdir operation is specified to ignore its body.
2717         # Only t=mkdir-with-children pays attention to it.
2718-        (newkids, filecap1, filecap2, filecap3,
2719-         dircap) = self._create_initial_children()
2720+        (newkids, caps) = self._create_initial_children()
2721         d = self.shouldHTTPError("POST t=mkdir unexpected children",
2722                                  400, "Bad Request",
2723                                  "t=mkdir does not accept children=, "
2724hunk ./src/allmydata/test/test_web.py 2227
2725         return d
2726 
2727     def test_POST_mkdir_no_parentdir_immutable(self):
2728-        (newkids, filecap1, immdircap) = self._create_immutable_children()
2729+        (newkids, caps) = self._create_immutable_children()
2730         d = self.POST2("/uri?t=mkdir-immutable", simplejson.dumps(newkids))
2731         def _after_mkdir(res):
2732             self.failUnless(res.startswith("URI:DIR"), res)
2733hunk ./src/allmydata/test/test_web.py 2234
2734             n = self.s.create_node_from_uri(res)
2735             d2 = self.failUnlessNodeKeysAre(n, newkids.keys())
2736             d2.addCallback(lambda ign:
2737-                           self.failUnlessChildURIIs(n, u"child-imm", filecap1))
2738+                           self.failUnlessROChildURIIs(n, u"child-imm",
2739+                                                          caps['filecap1']))
2740+            d2.addCallback(lambda ign:
2741+                           self.failUnlessROChildURIIs(n, u"unknownchild-imm",
2742+                                                          caps['unknown_immcap']))
2743             d2.addCallback(lambda ign:
2744hunk ./src/allmydata/test/test_web.py 2240
2745-                           self.failUnlessChildURIIs(n, u"dirchild-imm",
2746-                                                     immdircap))
2747+                           self.failUnlessROChildURIIs(n, u"dirchild-imm",
2748+                                                          caps['immdircap']))
2749             return d2
2750         d.addCallback(_after_mkdir)
2751         return d
2752hunk ./src/allmydata/test/test_web.py 2247
2753 
2754     def test_POST_mkdir_no_parentdir_immutable_bad(self):
2755-        (newkids, filecap1, filecap2, filecap3,
2756-         dircap) = self._create_initial_children()
2757+        (newkids, caps) = self._create_initial_children()
2758         d = self.shouldFail2(error.Error,
2759                              "test_POST_mkdir_no_parentdir_immutable_bad",
2760                              "400 Bad Request",
2761hunk ./src/allmydata/test/test_web.py 2251
2762-                             "a mkdir-immutable operation was given a child that was not itself immutable",
2763+                             "needed to be immutable but was not",
2764                              self.POST2,
2765                              "/uri?t=mkdir-immutable",
2766                              simplejson.dumps(newkids))
2767hunk ./src/allmydata/test/test_web.py 2359
2768 
2769         d = client.getPage(url, method="POST", postdata=reqbody)
2770         def _then(res):
2771-            self.failUnlessURIMatchesChild(newuri9, self._foo_node, u"atomic_added_1")
2772-            self.failUnlessURIMatchesChild(newuri10, self._foo_node, u"atomic_added_2")
2773-            self.failUnlessURIMatchesChild(newuri11, self._foo_node, u"atomic_added_3")
2774+            self.failUnlessURIMatchesROChild(newuri9, self._foo_node, u"atomic_added_1")
2775+            self.failUnlessURIMatchesROChild(newuri10, self._foo_node, u"atomic_added_2")
2776+            self.failUnlessURIMatchesROChild(newuri11, self._foo_node, u"atomic_added_3")
2777 
2778         d.addCallback(_then)
2779         d.addErrback(self.dump_error)
2780hunk ./src/allmydata/test/test_web.py 2373
2781     def test_POST_put_uri(self):
2782         contents, n, newuri = self.makefile(8)
2783         d = self.POST(self.public_url + "/foo", t="uri", name="new.txt", uri=newuri)
2784-        d.addCallback(self.failUnlessURIMatchesChild, self._foo_node, u"new.txt")
2785+        d.addCallback(self.failUnlessURIMatchesROChild, self._foo_node, u"new.txt")
2786         d.addCallback(lambda res:
2787                       self.failUnlessChildContentsAre(self._foo_node, u"new.txt",
2788                                                       contents))
2789hunk ./src/allmydata/test/test_web.py 2382
2790     def test_POST_put_uri_replace(self):
2791         contents, n, newuri = self.makefile(8)
2792         d = self.POST(self.public_url + "/foo", t="uri", name="bar.txt", uri=newuri)
2793-        d.addCallback(self.failUnlessURIMatchesChild, self._foo_node, u"bar.txt")
2794+        d.addCallback(self.failUnlessURIMatchesROChild, self._foo_node, u"bar.txt")
2795         d.addCallback(lambda res:
2796                       self.failUnlessChildContentsAre(self._foo_node, u"bar.txt",
2797                                                       contents))
2798hunk ./src/allmydata/test/test_web.py 2611
2799             d.addCallback(lambda res:
2800                           self.failUnlessEqual(res.strip(), new_uri))
2801             d.addCallback(lambda res:
2802-                          self.failUnlessChildURIIs(self.public_root,
2803-                                                    u"foo",
2804-                                                    new_uri))
2805+                          self.failUnlessRWChildURIIs(self.public_root,
2806+                                                      u"foo",
2807+                                                      new_uri))
2808             return d
2809         d.addCallback(_made_dir)
2810         return d
2811hunk ./src/allmydata/test/test_web.py 2630
2812                                  self.public_url + "/foo?t=uri&replace=false",
2813                                  new_uri)
2814             d.addCallback(lambda res:
2815-                          self.failUnlessChildURIIs(self.public_root,
2816-                                                    u"foo",
2817-                                                    self._foo_uri))
2818+                          self.failUnlessRWChildURIIs(self.public_root,
2819+                                                      u"foo",
2820+                                                      self._foo_uri))
2821             return d
2822         d.addCallback(_made_dir)
2823         return d
2824hunk ./src/allmydata/test/test_web.py 2642
2825                                  "400 Bad Request", "PUT to a directory",
2826                                  self.PUT, self.public_url + "/foo?t=BOGUS", "")
2827         d.addCallback(lambda res:
2828-                      self.failUnlessChildURIIs(self.public_root,
2829-                                                u"foo",
2830-                                                self._foo_uri))
2831+                      self.failUnlessRWChildURIIs(self.public_root,
2832+                                                  u"foo",
2833+                                                  self._foo_uri))
2834         return d
2835 
2836     def test_PUT_NEWFILEURL_uri(self):
2837hunk ./src/allmydata/test/test_web.py 3171
2838         d.addErrback(self.explain_web_error)
2839         return d
2840 
2841-    def test_unknown(self):
2842+    def test_unknown(self, immutable=False):
2843         self.basedir = "web/Grid/unknown"
2844hunk ./src/allmydata/test/test_web.py 3173
2845+        if immutable:
2846+            self.basedir = "web/Grid/unknown-immutable"
2847+
2848         self.set_up_grid()
2849         c0 = self.g.clients[0]
2850         self.uris = {}
2851hunk ./src/allmydata/test/test_web.py 3181
2852         self.fileurls = {}
2853 
2854-        future_writecap = "x-tahoe-crazy://I_am_from_the_future."
2855-        future_readcap = "x-tahoe-crazy-readonly://I_am_from_the_future."
2856+        future_write_uri = "x-tahoe-crazy://I_am_from_the_future."
2857+        future_read_uri = "x-tahoe-crazy-readonly://I_am_from_the_future."
2858         # the future cap format may contain slashes, which must be tolerated
2859hunk ./src/allmydata/test/test_web.py 3184
2860-        expected_info_url = "uri/%s?t=info" % urllib.quote(future_writecap,
2861+        expected_info_url = "uri/%s?t=info" % urllib.quote(future_write_uri,
2862                                                            safe="")
2863hunk ./src/allmydata/test/test_web.py 3186
2864-        future_node = UnknownNode(future_writecap, future_readcap)
2865+
2866+        if immutable:
2867+            name = u"future-imm"
2868+            future_node = UnknownNode(None, future_read_uri, deep_immutable=True)
2869+            d = c0.create_immutable_dirnode({name: (future_node, {})})
2870+        else:
2871+            name = u"future"
2872+            future_node = UnknownNode(future_write_uri, future_read_uri)
2873+            d = c0.create_dirnode()
2874 
2875hunk ./src/allmydata/test/test_web.py 3196
2876-        d = c0.create_dirnode()
2877         def _stash_root_and_create_file(n):
2878             self.rootnode = n
2879             self.rooturl = "uri/" + urllib.quote(n.get_uri()) + "/"
2880hunk ./src/allmydata/test/test_web.py 3200
2881             self.rourl = "uri/" + urllib.quote(n.get_readonly_uri()) + "/"
2882-            return self.rootnode.set_node(u"future", future_node)
2883+            if not immutable:
2884+                return self.rootnode.set_node(name, future_node)
2885         d.addCallback(_stash_root_and_create_file)
2886hunk ./src/allmydata/test/test_web.py 3203
2887+
2888         # make sure directory listing tolerates unknown nodes
2889         d.addCallback(lambda ign: self.GET(self.rooturl))
2890hunk ./src/allmydata/test/test_web.py 3206
2891-        def _check_html(res):
2892-            self.failUnlessIn("<td>future</td>", res)
2893-            # find the More Info link for "future", should be relative
2894+        def _check_directory_html(res):
2895+            self.failUnlessIn("<td>%s</td>" % (str(name),), res)
2896+            # find the More Info link for name, should be relative
2897             mo = re.search(r'<a href="([^"]+)">More Info</a>', res)
2898             info_url = mo.group(1)
2899hunk ./src/allmydata/test/test_web.py 3211
2900-            self.failUnlessEqual(info_url, "future?t=info")
2901+            self.failUnlessEqual(info_url, "%s?t=info" % (str(name),))
2902+        d.addCallback(_check_directory_html)
2903 
2904hunk ./src/allmydata/test/test_web.py 3214
2905-        d.addCallback(_check_html)
2906         d.addCallback(lambda ign: self.GET(self.rooturl+"?t=json"))
2907hunk ./src/allmydata/test/test_web.py 3215
2908-        def _check_json(res, expect_writecap):
2909+        def _check_directory_json(res, expect_rw_uri):
2910             data = simplejson.loads(res)
2911             self.failUnlessEqual(data[0], "dirnode")
2912hunk ./src/allmydata/test/test_web.py 3218
2913-            f = data[1]["children"]["future"]
2914+            f = data[1]["children"][name]
2915             self.failUnlessEqual(f[0], "unknown")
2916hunk ./src/allmydata/test/test_web.py 3220
2917-            if expect_writecap:
2918-                self.failUnlessEqual(f[1]["rw_uri"], future_writecap)
2919+            if expect_rw_uri:
2920+                self.failUnlessEqual(f[1]["rw_uri"], future_write_uri)
2921             else:
2922                 self.failIfIn("rw_uri", f[1])
2923hunk ./src/allmydata/test/test_web.py 3224
2924-            self.failUnlessEqual(f[1]["ro_uri"], future_readcap)
2925+            self.failUnlessEqual(f[1]["ro_uri"],
2926+                                 ("imm." if immutable else "ro.") + future_read_uri)
2927             self.failUnless("metadata" in f[1])
2928hunk ./src/allmydata/test/test_web.py 3227
2929-        d.addCallback(_check_json, expect_writecap=True)
2930-        d.addCallback(lambda ign: self.GET(expected_info_url))
2931-        def _check_info(res, expect_readcap):
2932+        d.addCallback(_check_directory_json, expect_rw_uri=not immutable)
2933+
2934+        def _check_info(res, expect_rw_uri, expect_ro_uri):
2935             self.failUnlessIn("Object Type: <span>unknown</span>", res)
2936hunk ./src/allmydata/test/test_web.py 3231
2937-            self.failUnlessIn(future_writecap, res)
2938-            if expect_readcap:
2939-                self.failUnlessIn(future_readcap, res)
2940+            if expect_rw_uri:
2941+                self.failUnlessIn(future_write_uri, res)
2942+            if expect_ro_uri:
2943+                self.failUnlessIn(future_read_uri, res)
2944+            else:
2945+                self.failIfIn(future_read_uri, res)
2946             self.failIfIn("Raw data as", res)
2947             self.failIfIn("Directory writecap", res)
2948             self.failIfIn("Checker Operations", res)
2949hunk ./src/allmydata/test/test_web.py 3242
2950             self.failIfIn("Mutable File Operations", res)
2951             self.failIfIn("Directory Operations", res)
2952-        d.addCallback(_check_info, expect_readcap=False)
2953-        d.addCallback(lambda ign: self.GET(self.rooturl+"future?t=info"))
2954-        d.addCallback(_check_info, expect_readcap=True)
2955+
2956+        # FIXME: these should have expect_rw_uri=not immutable; I don't know
2957+        # why they fail. Possibly related to ticket #922.
2958+
2959+        d.addCallback(lambda ign: self.GET(expected_info_url))
2960+        d.addCallback(_check_info, expect_rw_uri=False, expect_ro_uri=False)
2961+        d.addCallback(lambda ign: self.GET("%s%s?t=info" % (self.rooturl, str(name))))
2962+        d.addCallback(_check_info, expect_rw_uri=False, expect_ro_uri=True)
2963+
2964+        def _check_json(res, expect_rw_uri):
2965+            data = simplejson.loads(res)
2966+            self.failUnlessEqual(data[0], "unknown")
2967+            if expect_rw_uri:
2968+                self.failUnlessEqual(data[1]["rw_uri"], future_write_uri)
2969+            else:
2970+                self.failIfIn("rw_uri", data[1])
2971+            self.failUnlessEqual(data[1]["ro_uri"],
2972+                                 ("imm." if immutable else "ro.") + future_read_uri)
2973+            # TODO: check metadata contents
2974+            self.failUnless("metadata" in data[1])
2975+
2976+        d.addCallback(lambda ign: self.GET("%s%s?t=json" % (self.rooturl, str(name))))
2977+        d.addCallback(_check_json, expect_rw_uri=not immutable)
2978 
2979         # and make sure that a read-only version of the directory can be
2980hunk ./src/allmydata/test/test_web.py 3267
2981-        # rendered too. This version will not have future_writecap
2982+        # rendered too. This version will not have future_write_uri, whether
2983+        # or not future_node was immutable.
2984         d.addCallback(lambda ign: self.GET(self.rourl))
2985hunk ./src/allmydata/test/test_web.py 3270
2986-        d.addCallback(_check_html)
2987+        d.addCallback(_check_directory_html)
2988         d.addCallback(lambda ign: self.GET(self.rourl+"?t=json"))
2989hunk ./src/allmydata/test/test_web.py 3272
2990-        d.addCallback(_check_json, expect_writecap=False)
2991+        d.addCallback(_check_directory_json, expect_rw_uri=False)
2992+
2993+        d.addCallback(lambda ign: self.GET("%s%s?t=json" % (self.rourl, str(name))))
2994+        d.addCallback(_check_json, expect_rw_uri=False)
2995+       
2996+        # TODO: check that getting t=info from the Info link in the ro directory
2997+        # works, and does not include the writecap URI.
2998+        return d
2999+
3000+    def test_immutable_unknown(self):
3001+        return self.test_unknown(immutable=True)
3002+
3003+    def test_mutant_dirnodes_are_omitted(self):
3004+        self.basedir = "web/Grid/mutant_dirnodes_are_omitted"
3005+
3006+        self.set_up_grid()
3007+        c = self.g.clients[0]
3008+        nm = c.nodemaker
3009+        self.uris = {}
3010+        self.fileurls = {}
3011+
3012+        lonely_uri = "URI:LIT:n5xgk" # LIT for "one"
3013+        mut_write_uri = "URI:SSK:vfvcbdfbszyrsaxchgevhmmlii:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq"
3014+        mut_read_uri = "URI:SSK-RO:e3mdrzfwhoq42hy5ubcz6rp3o4:ybyibhnp3vvwuq2vaw2ckjmesgkklfs6ghxleztqidihjyofgw7q"
3015+       
3016+        # This method tests mainly dirnode, but we'd have to duplicate code in order to
3017+        # test the dirnode and web layers separately.
3018+       
3019+        # 'lonely' is a valid LIT child, 'ro' is a mutant child with an SSK-RO readcap,
3020+        # and 'write-in-ro' is a mutant child with an SSK writecap in the ro_uri field.
3021+        # When the directory is read, the mutants should be silently disposed of, leaving
3022+        # their lonely sibling.
3023+        # We don't test the case of a retrieving a cap from the encrypted rw_uri field,
3024+        # because immutable directories don't have a writecap and therefore that field
3025+        # isn't (and can't be) decrypted.
3026+        # TODO: The field still exists in the netstring. Technically we should check what
3027+        # happens if something is put there (it should be ignored), but that can wait.
3028+
3029+        lonely_child = nm.create_from_cap(lonely_uri)
3030+        mutant_ro_child = nm.create_from_cap(mut_read_uri)
3031+        mutant_write_in_ro_child = nm.create_from_cap(mut_write_uri)
3032+
3033+        def _by_hook_or_by_crook():
3034+            return True
3035+        for n in [mutant_ro_child, mutant_write_in_ro_child]:
3036+            n.is_allowed_in_immutable_directory = _by_hook_or_by_crook
3037+
3038+        mutant_write_in_ro_child.get_write_uri    = lambda: None
3039+        mutant_write_in_ro_child.get_readonly_uri = lambda: mut_write_uri
3040+
3041+        kids = {u"lonely":      (lonely_child, {}),
3042+                u"ro":          (mutant_ro_child, {}),
3043+                u"write-in-ro": (mutant_write_in_ro_child, {}),
3044+                }
3045+        d = c.create_immutable_dirnode(kids)
3046+       
3047+        def _created(dn):
3048+            self.failUnless(isinstance(dn, dirnode.DirectoryNode))
3049+            self.failIf(dn.is_mutable())
3050+            self.failUnless(dn.is_readonly())
3051+            # This checks that if we somehow ended up calling dn._decrypt_rwcapdata, it would fail.
3052+            self.failIf(hasattr(dn._node, 'get_writekey'))
3053+            rep = str(dn)
3054+            self.failUnless("RO-IMM" in rep)
3055+            cap = dn.get_cap()
3056+            self.failUnlessIn("CHK", cap.to_string())
3057+            self.cap = cap
3058+            self.rootnode = dn
3059+            self.rooturl = "uri/" + urllib.quote(dn.get_uri()) + "/"
3060+            return download_to_data(dn._node)
3061+        d.addCallback(_created)
3062+
3063+        def _check_data(data):
3064+            # Decode the netstring representation of the directory to check that all children
3065+            # are present. This is a bit of an abstraction violation, but there's not really
3066+            # any other way to do it given that the real DirectoryNode._unpack_contents would
3067+            # strip the mutant children out (which is what we're trying to test, later).
3068+            position = 0
3069+            numkids = 0
3070+            while position < len(data):
3071+                entries, position = split_netstring(data, 1, position)
3072+                entry = entries[0]
3073+                (name, ro_uri, rwcapdata, metadata_s), subpos = split_netstring(entry, 4)
3074+                name = name.decode("utf-8")
3075+                self.failUnless(rwcapdata == "")
3076+                ro_uri = ro_uri.strip()
3077+                if name in kids:
3078+                    self.failIfEqual(ro_uri, "")
3079+                    (expected_child, ign) = kids[name]
3080+                    self.failUnlessEqual(ro_uri, expected_child.get_readonly_uri())
3081+                    numkids += 1
3082+
3083+            self.failUnlessEqual(numkids, 3)
3084+            return self.rootnode.list()
3085+        d.addCallback(_check_data)
3086+       
3087+        # Now when we use the real directory listing code, the mutants should be absent.
3088+        def _check_kids(children):
3089+            self.failUnlessEqual(sorted(children.keys()), [u"lonely"])
3090+            lonely_node, lonely_metadata = children[u"lonely"]
3091+
3092+            self.failUnlessEqual(lonely_node.get_write_uri(), None)
3093+            self.failUnlessEqual(lonely_node.get_readonly_uri(), lonely_uri)
3094+        d.addCallback(_check_kids)
3095+
3096+        d.addCallback(lambda ign: nm.create_from_cap(self.cap.to_string()))
3097+        d.addCallback(lambda n: n.list())
3098+        d.addCallback(_check_kids)  # again with dirnode recreated from cap
3099+
3100+        # Make sure the lonely child can be listed in HTML...
3101+        d.addCallback(lambda ign: self.GET(self.rooturl))
3102+        def _check_html(res):
3103+            self.failIfIn("URI:SSK", res)
3104+            get_lonely = "".join([r'<td>FILE</td>',
3105+                                  r'\s+<td>',
3106+                                  r'<a href="[^"]+%s[^"]+">lonely</a>' % (urllib.quote(lonely_uri),),
3107+                                  r'</td>',
3108+                                  r'\s+<td>%d</td>' % len("one"),
3109+                                  ])
3110+            self.failUnless(re.search(get_lonely, res), res)
3111+
3112+            # find the More Info link for name, should be relative
3113+            mo = re.search(r'<a href="([^"]+)">More Info</a>', res)
3114+            info_url = mo.group(1)
3115+            self.failUnless(info_url.endswith(urllib.quote(lonely_uri) + "?t=info"), info_url)
3116+        d.addCallback(_check_html)
3117+
3118+        # ... and in JSON.
3119+        d.addCallback(lambda ign: self.GET(self.rooturl+"?t=json"))
3120+        def _check_json(res):
3121+            data = simplejson.loads(res)
3122+            self.failUnlessEqual(data[0], "dirnode")
3123+            listed_children = data[1]["children"]
3124+            self.failUnlessEqual(sorted(listed_children.keys()), [u"lonely"])
3125+            ll_type, ll_data = listed_children[u"lonely"]
3126+            self.failUnlessEqual(ll_type, "filenode")
3127+            self.failIf("rw_uri" in ll_data)
3128+            self.failUnlessEqual(ll_data["ro_uri"], lonely_uri)
3129+        d.addCallback(_check_json)
3130         return d
3131 
3132     def test_deep_check(self):
3133hunk ./src/allmydata/test/test_web.py 3443
3134 
3135         # this tests that deep-check and stream-manifest will ignore
3136         # UnknownNode instances. Hopefully this will also cover deep-stats.
3137-        future_writecap = "x-tahoe-crazy://I_am_from_the_future."
3138-        future_readcap = "x-tahoe-crazy-readonly://I_am_from_the_future."
3139-        future_node = UnknownNode(future_writecap, future_readcap)
3140-        d.addCallback(lambda ign: self.rootnode.set_node(u"future",future_node))
3141+        future_write_uri = "x-tahoe-crazy://I_am_from_the_future."
3142+        future_read_uri = "x-tahoe-crazy-readonly://I_am_from_the_future."
3143+        future_node = UnknownNode(future_write_uri, future_read_uri)
3144+        d.addCallback(lambda ign: self.rootnode.set_node(u"future", future_node))
3145 
3146         def _clobber_shares(ignored):
3147             self.delete_shares_numbered(self.uris["sick"], [0,1])
3148hunk ./src/allmydata/unknown.py 1
3149+
3150 from zope.interface import implements
3151 from twisted.internet import defer
3152hunk ./src/allmydata/unknown.py 4
3153-from allmydata.interfaces import IFilesystemNode
3154+from allmydata.interfaces import IFilesystemNode, MustNotBeUnknownRWError, \
3155+    MustBeDeepImmutableError
3156+from allmydata import uri
3157+from allmydata.uri import ALLEGED_READONLY_PREFIX, ALLEGED_IMMUTABLE_PREFIX
3158+
3159+
3160+# See ticket #833 for design rationale of UnknownNodes.
3161+
3162+def strip_prefix_for_ro(ro_uri, deep_immutable):
3163+    """Strip prefixes when storing an URI in a ro_uri slot."""
3164+
3165+    # It is possible for an alleged-immutable URI to be put into a
3166+    # mutable directory. In that case the ALLEGED_IMMUTABLE_PREFIX
3167+    # should not be stripped. In other cases, the prefix can safely
3168+    # be stripped because it is implied by the context.
3169+
3170+    if ro_uri.startswith(ALLEGED_IMMUTABLE_PREFIX):
3171+        if not deep_immutable:
3172+            return ro_uri
3173+        return ro_uri[len(ALLEGED_IMMUTABLE_PREFIX):]
3174+    elif ro_uri.startswith(ALLEGED_READONLY_PREFIX):
3175+        return ro_uri[len(ALLEGED_READONLY_PREFIX):]
3176+    else:
3177+        return ro_uri
3178 
3179 class UnknownNode:
3180     implements(IFilesystemNode)
3181hunk ./src/allmydata/unknown.py 31
3182-    def __init__(self, writecap, readcap):
3183-        assert writecap is None or isinstance(writecap, str)
3184-        self.writecap = writecap
3185-        assert readcap is None or isinstance(readcap, str)
3186-        self.readcap = readcap
3187+
3188+    def __init__(self, given_rw_uri, given_ro_uri, deep_immutable=False,
3189+                 name=u"<unknown name>"):
3190+        assert given_rw_uri is None or isinstance(given_rw_uri, str)
3191+        assert given_ro_uri is None or isinstance(given_ro_uri, str)
3192+        given_rw_uri = given_rw_uri or None
3193+        given_ro_uri = given_ro_uri or None
3194+
3195+        # We don't raise errors when creating an UnknownNode; we instead create an
3196+        # opaque node (with rw_uri and ro_uri both None) that records the error.
3197+        # This avoids breaking operations that never store the opaque node.
3198+        # Note that this means that if a stored dirnode has only a rw_uri, it
3199+        # might be dropped. Any future "write-only" cap formats should have a dummy
3200+        # unusable readcap to stop that from happening.
3201+
3202+        self.error = None
3203+        self.rw_uri = self.ro_uri = None
3204+        if given_rw_uri:
3205+            if deep_immutable:
3206+                if given_rw_uri.startswith(ALLEGED_IMMUTABLE_PREFIX) and not given_ro_uri:
3207+                    # We needed an immutable cap, and were given one. It was given in the
3208+                    # rw_uri slot, but that's fine; we'll move it to ro_uri below.
3209+                    pass
3210+                elif not given_ro_uri:
3211+                    self.error = MustNotBeUnknownRWError("cannot attach unknown rw cap as immutable child",
3212+                                                         name, True)
3213+                    return  # node will be opaque
3214+                else:
3215+                    # We could report either error, but this probably makes more sense.
3216+                    self.error = MustBeDeepImmutableError("cannot attach unknown rw cap as immutable child",
3217+                                                         name)
3218+                    return  # node will be opaque
3219+
3220+            if not given_ro_uri:
3221+                # We were given a single cap argument, or a rw_uri with no ro_uri.
3222+
3223+                if not (given_rw_uri.startswith(ALLEGED_READONLY_PREFIX)
3224+                        or given_rw_uri.startswith(ALLEGED_IMMUTABLE_PREFIX)):
3225+                    # If the single cap is unprefixed, then we cannot tell whether it is a
3226+                    # writecap, and we don't know how to diminish it to a readcap if it is one.
3227+                    # If it didn't *already* have at least an ALLEGED_READONLY_PREFIX, then
3228+                    # prefixing it would be a bad idea because we have been given no reason
3229+                    # to believe that it is a readcap, so we might be letting a client
3230+                    # inadvertently grant excess write authority.
3231+                    self.error = MustNotBeUnknownRWError("cannot attach unknown rw cap as child",
3232+                                                         name, False)
3233+                    return  # node will be opaque
3234+
3235+                # OTOH, if the single cap already had a prefix (which is of the required
3236+                # strength otherwise an error would have been thrown above), then treat it
3237+                # as though it had been given in the ro_uri slot. This has a similar effect
3238+                # to the use for known caps of 'bigcap = writecap or readcap' in
3239+                # nodemaker.py: create_from_cap. It enables copying of unknown readcaps to
3240+                # work in as many cases as we can securely allow.
3241+                given_ro_uri = given_rw_uri
3242+                given_rw_uri = None
3243+            elif given_ro_uri.startswith(ALLEGED_IMMUTABLE_PREFIX):
3244+                # Strange corner case: we were given a cap in both slots, with the ro_uri
3245+                # alleged to be immutable. A real immutable object wouldn't have a writecap.
3246+                self.error = MustBeDeepImmutableError("cannot accept a child entry that specifies "
3247+                                                      "both rw_uri, and ro_uri with an imm. prefix",
3248+                                                      name)
3249+                return  # node will be opaque
3250+
3251+        # If the ro_uri definitely fails the constraint, it should be treated as opaque and
3252+        # the error recorded.
3253+        if given_ro_uri:
3254+            read_cap = uri.from_string(given_ro_uri, deep_immutable=deep_immutable, name=name)
3255+            if isinstance(read_cap, uri.UnknownURI):
3256+                self.error = read_cap.get_error()
3257+                if self.error:
3258+                    assert self.rw_uri is None and self.ro_uri is None
3259+                    return
3260+
3261+        if deep_immutable:
3262+            assert self.rw_uri is None
3263+            # strengthen the constraint on ro_uri to ALLEGED_IMMUTABLE_PREFIX
3264+            if given_ro_uri:
3265+                if given_ro_uri.startswith(ALLEGED_IMMUTABLE_PREFIX):
3266+                    self.ro_uri = given_ro_uri
3267+                elif given_ro_uri.startswith(ALLEGED_READONLY_PREFIX):
3268+                    self.ro_uri = ALLEGED_IMMUTABLE_PREFIX + given_ro_uri[len(ALLEGED_READONLY_PREFIX):]
3269+                else:
3270+                    self.ro_uri = ALLEGED_IMMUTABLE_PREFIX + given_ro_uri
3271+        else:
3272+            # not immutable, so a writecap is allowed
3273+            self.rw_uri = given_rw_uri
3274+            # strengthen the constraint on ro_uri to ALLEGED_READONLY_PREFIX
3275+            if given_ro_uri:
3276+                if (given_ro_uri.startswith(ALLEGED_READONLY_PREFIX) or
3277+                    given_ro_uri.startswith(ALLEGED_IMMUTABLE_PREFIX)):
3278+                    self.ro_uri = given_ro_uri
3279+                else:
3280+                    self.ro_uri = ALLEGED_READONLY_PREFIX + given_ro_uri
3281+
3282+    def get_cap(self):
3283+        return uri.UnknownURI(self.rw_uri or self.ro_uri)
3284+
3285+    def get_readcap(self):
3286+        return uri.UnknownURI(self.ro_uri)
3287+
3288+    def is_readonly(self):
3289+        raise AssertionError("an UnknownNode might be either read-only or "
3290+                             "read/write, so we shouldn't be calling is_readonly")
3291+
3292+    def is_mutable(self):
3293+        raise AssertionError("an UnknownNode might be either mutable or immutable, "
3294+                             "so we shouldn't be calling is_mutable")
3295+
3296+    def is_unknown(self):
3297+        return True
3298+
3299+    def is_allowed_in_immutable_directory(self):
3300+        # An UnknownNode consisting only of a ro_uri is allowed in an
3301+        # immutable directory, even though we do not know that it is
3302+        # immutable (or even read-only), provided that no error was detected.
3303+        return not self.error and not self.rw_uri
3304+
3305+    def raise_error(self):
3306+        if self.error is not None:
3307+            raise self.error
3308+
3309     def get_uri(self):
3310hunk ./src/allmydata/unknown.py 154
3311-        return self.writecap
3312+        return self.rw_uri or self.ro_uri
3313+
3314+    def get_write_uri(self):
3315+        return self.rw_uri
3316+
3317     def get_readonly_uri(self):
3318hunk ./src/allmydata/unknown.py 160
3319-        return self.readcap
3320+        return self.ro_uri
3321+
3322     def get_storage_index(self):
3323         return None
3324hunk ./src/allmydata/unknown.py 164
3325+
3326     def get_verify_cap(self):
3327         return None
3328hunk ./src/allmydata/unknown.py 167
3329+
3330     def get_repair_cap(self):
3331         return None
3332hunk ./src/allmydata/unknown.py 170
3333+
3334     def get_size(self):
3335         return None
3336hunk ./src/allmydata/unknown.py 173
3337+
3338     def get_current_size(self):
3339         return defer.succeed(None)
3340hunk ./src/allmydata/unknown.py 176
3341+
3342     def check(self, monitor, verify, add_lease):
3343         return defer.succeed(None)
3344hunk ./src/allmydata/unknown.py 179
3345+
3346     def check_and_repair(self, monitor, verify, add_lease):
3347         return defer.succeed(None)
3348hunk ./src/allmydata/uri.py 8
3349 from allmydata.storage.server import si_a2b, si_b2a
3350 from allmydata.util import base32, hashutil
3351 from allmydata.interfaces import IURI, IDirnodeURI, IFileURI, IImmutableFileURI, \
3352-    IVerifierURI, IMutableFileURI, IDirectoryURI, IReadonlyDirectoryURI
3353+    IVerifierURI, IMutableFileURI, IDirectoryURI, IReadonlyDirectoryURI, \
3354+    MustBeDeepImmutableError, MustBeReadonlyError, CapConstraintError
3355 
3356hunk ./src/allmydata/uri.py 11
3357-class BadURIError(Exception):
3358+class BadURIError(CapConstraintError):
3359     pass
3360 
3361 # the URI shall be an ascii representation of the file. It shall contain
3362hunk ./src/allmydata/uri.py 75
3363     def init_from_human_encoding(cls, uri):
3364         mo = cls.HUMAN_RE.search(uri)
3365         if not mo:
3366-            raise BadURIError("%s doesn't look like a cap" % (uri,))
3367+            raise BadURIError("'%s' doesn't look like a %s cap" % (uri, cls))
3368         return cls(base32.a2b(mo.group(1)), base32.a2b(mo.group(2)),
3369                    int(mo.group(3)), int(mo.group(4)), int(mo.group(5)))
3370 
3371hunk ./src/allmydata/uri.py 83
3372     def init_from_string(cls, uri):
3373         mo = cls.STRING_RE.search(uri)
3374         if not mo:
3375-            raise BadURIError("%s doesn't look like a cap" % (uri,))
3376+            raise BadURIError("'%s' doesn't look like a %s cap" % (uri, cls))
3377         return cls(base32.a2b(mo.group(1)), base32.a2b(mo.group(2)),
3378                    int(mo.group(3)), int(mo.group(4)), int(mo.group(5)))
3379 
3380hunk ./src/allmydata/uri.py 101
3381 
3382     def is_readonly(self):
3383         return True
3384+
3385     def is_mutable(self):
3386         return False
3387hunk ./src/allmydata/uri.py 104
3388+
3389     def get_readonly(self):
3390         return self
3391 
3392hunk ./src/allmydata/uri.py 140
3393     @classmethod
3394     def init_from_human_encoding(cls, uri):
3395         mo = cls.HUMAN_RE.search(uri)
3396-        assert mo, uri
3397+        if not mo:
3398+            raise BadURIError("'%s' doesn't look like a %s cap" % (uri, cls))
3399         return cls(base32.a2b(mo.group(1)), base32.a2b(mo.group(2)),
3400                    int(mo.group(3)), int(mo.group(4)), int(mo.group(5)))
3401 
3402hunk ./src/allmydata/uri.py 148
3403     @classmethod
3404     def init_from_string(cls, uri):
3405         mo = cls.STRING_RE.search(uri)
3406-        assert mo, (uri, cls, cls.STRING_RE)
3407+        if not mo:
3408+            raise BadURIError("'%s' doesn't look like a %s cap" % (uri, cls))
3409         return cls(si_a2b(mo.group(1)), base32.a2b(mo.group(2)),
3410                    int(mo.group(3)), int(mo.group(4)), int(mo.group(5)))
3411 
3412hunk ./src/allmydata/uri.py 165
3413                  self.total_shares,
3414                  self.size))
3415 
3416+    def is_readonly(self):
3417+        return True
3418+
3419+    def is_mutable(self):
3420+        return False
3421+
3422+    def get_readonly(self):
3423+        return self
3424+
3425+    def get_verify_cap(self):
3426+        return self
3427+
3428 
3429 class LiteralFileURI(_BaseURI):
3430     implements(IURI, IImmutableFileURI)
3431hunk ./src/allmydata/uri.py 193
3432     @classmethod
3433     def init_from_human_encoding(cls, uri):
3434         mo = cls.HUMAN_RE.search(uri)
3435-        assert mo, uri
3436+        if not mo:
3437+            raise BadURIError("'%s' doesn't look like a %s cap" % (uri, cls))
3438         return cls(base32.a2b(mo.group(1)))
3439 
3440     @classmethod
3441hunk ./src/allmydata/uri.py 200
3442     def init_from_string(cls, uri):
3443         mo = cls.STRING_RE.search(uri)
3444-        assert mo, uri
3445+        if not mo:
3446+            raise BadURIError("'%s' doesn't look like a %s cap" % (uri, cls))
3447         return cls(base32.a2b(mo.group(1)))
3448 
3449     def to_string(self):
3450hunk ./src/allmydata/uri.py 209
3451 
3452     def is_readonly(self):
3453         return True
3454+
3455     def is_mutable(self):
3456         return False
3457hunk ./src/allmydata/uri.py 212
3458+
3459     def get_readonly(self):
3460         return self
3461hunk ./src/allmydata/uri.py 215
3462+
3463     def get_storage_index(self):
3464         return None
3465 
3466hunk ./src/allmydata/uri.py 226
3467     def get_size(self):
3468         return len(self.data)
3469 
3470+
3471 class WriteableSSKFileURI(_BaseURI):
3472     implements(IURI, IMutableFileURI)
3473 
3474hunk ./src/allmydata/uri.py 247
3475     def init_from_human_encoding(cls, uri):
3476         mo = cls.HUMAN_RE.search(uri)
3477         if not mo:
3478-            raise BadURIError("'%s' doesn't look like a cap" % (uri,))
3479+            raise BadURIError("'%s' doesn't look like a %s cap" % (uri, cls))
3480         return cls(base32.a2b(mo.group(1)), base32.a2b(mo.group(2)))
3481 
3482     @classmethod
3483hunk ./src/allmydata/uri.py 268
3484 
3485     def abbrev(self):
3486         return base32.b2a(self.writekey[:5])
3487+
3488     def abbrev_si(self):
3489         return base32.b2a(self.storage_index)[:5]
3490 
3491hunk ./src/allmydata/uri.py 274
3492     def is_readonly(self):
3493         return False
3494+
3495     def is_mutable(self):
3496         return True
3497hunk ./src/allmydata/uri.py 277
3498+
3499     def get_readonly(self):
3500         return ReadonlySSKFileURI(self.readkey, self.fingerprint)
3501hunk ./src/allmydata/uri.py 280
3502+
3503     def get_verify_cap(self):
3504         return SSKVerifierURI(self.storage_index, self.fingerprint)
3505 
3506hunk ./src/allmydata/uri.py 284
3507+
3508 class ReadonlySSKFileURI(_BaseURI):
3509     implements(IURI, IMutableFileURI)
3510 
3511hunk ./src/allmydata/uri.py 302
3512     def init_from_human_encoding(cls, uri):
3513         mo = cls.HUMAN_RE.search(uri)
3514         if not mo:
3515-            raise BadURIError("'%s' doesn't look like a cap" % (uri,))
3516+            raise BadURIError("'%s' doesn't look like a %s cap" % (uri, cls))
3517         return cls(base32.a2b(mo.group(1)), base32.a2b(mo.group(2)))
3518 
3519     @classmethod
3520hunk ./src/allmydata/uri.py 309
3521     def init_from_string(cls, uri):
3522         mo = cls.STRING_RE.search(uri)
3523         if not mo:
3524-            raise BadURIError("'%s' doesn't look like a cap" % (uri,))
3525+            raise BadURIError("'%s' doesn't look like a %s cap" % (uri, cls))
3526         return cls(base32.a2b(mo.group(1)), base32.a2b(mo.group(2)))
3527 
3528     def to_string(self):
3529hunk ./src/allmydata/uri.py 323
3530 
3531     def abbrev(self):
3532         return base32.b2a(self.readkey[:5])
3533+
3534     def abbrev_si(self):
3535         return base32.b2a(self.storage_index)[:5]
3536 
3537hunk ./src/allmydata/uri.py 329
3538     def is_readonly(self):
3539         return True
3540+
3541     def is_mutable(self):
3542         return True
3543hunk ./src/allmydata/uri.py 332
3544+
3545     def get_readonly(self):
3546         return self
3547hunk ./src/allmydata/uri.py 335
3548+
3549     def get_verify_cap(self):
3550         return SSKVerifierURI(self.storage_index, self.fingerprint)
3551 
3552hunk ./src/allmydata/uri.py 339
3553+
3554 class SSKVerifierURI(_BaseURI):
3555     implements(IVerifierURI)
3556 
3557hunk ./src/allmydata/uri.py 355
3558     @classmethod
3559     def init_from_human_encoding(cls, uri):
3560         mo = cls.HUMAN_RE.search(uri)
3561-        assert mo, uri
3562+        if not mo:
3563+            raise BadURIError("'%s' doesn't look like a %s cap" % (uri, cls))
3564         return cls(si_a2b(mo.group(1)), base32.a2b(mo.group(2)))
3565 
3566     @classmethod
3567hunk ./src/allmydata/uri.py 362
3568     def init_from_string(cls, uri):
3569         mo = cls.STRING_RE.search(uri)
3570-        assert mo, (uri, cls)
3571+        if not mo:
3572+            raise BadURIError("'%s' doesn't look like a %s cap" % (uri, cls))
3573         return cls(si_a2b(mo.group(1)), base32.a2b(mo.group(2)))
3574 
3575     def to_string(self):
3576hunk ./src/allmydata/uri.py 372
3577         return 'URI:SSK-Verifier:%s:%s' % (si_b2a(self.storage_index),
3578                                            base32.b2a(self.fingerprint))
3579 
3580+    def is_readonly(self):
3581+        return True
3582+
3583+    def is_mutable(self):
3584+        return False
3585+
3586+    def get_readonly(self):
3587+        return self
3588+
3589+    def get_verify_cap(self):
3590+        return self
3591+
3592 class _DirectoryBaseURI(_BaseURI):
3593     implements(IURI, IDirnodeURI)
3594     def __init__(self, filenode_uri=None):
3595hunk ./src/allmydata/uri.py 423
3596 
3597     def abbrev(self):
3598         return self._filenode_uri.to_string().split(':')[2][:5]
3599+
3600     def abbrev_si(self):
3601         return base32.b2a(self._filenode_uri.storage_index)[:5]
3602 
3603hunk ./src/allmydata/uri.py 427
3604-    def get_filenode_cap(self):
3605-        return self._filenode_uri
3606-
3607     def is_mutable(self):
3608         return True
3609 
3610hunk ./src/allmydata/uri.py 430
3611+    def get_filenode_cap(self):
3612+        return self._filenode_uri
3613+
3614     def get_verify_cap(self):
3615         return DirectoryURIVerifier(self._filenode_uri.get_verify_cap())
3616 
3617hunk ./src/allmydata/uri.py 458
3618     def get_readonly(self):
3619         return ReadonlyDirectoryURI(self._filenode_uri.get_readonly())
3620 
3621+
3622 class ReadonlyDirectoryURI(_DirectoryBaseURI):
3623     implements(IReadonlyDirectoryURI)
3624 
3625hunk ./src/allmydata/uri.py 478
3626     def get_readonly(self):
3627         return self
3628 
3629+
3630 class _ImmutableDirectoryBaseURI(_DirectoryBaseURI):
3631     def __init__(self, filenode_uri=None):
3632         if filenode_uri:
3633hunk ./src/allmydata/uri.py 483
3634             assert isinstance(filenode_uri, self.INNER_URI_CLASS), filenode_uri
3635+            assert not filenode_uri.is_mutable()
3636         _DirectoryBaseURI.__init__(self, filenode_uri)
3637 
3638hunk ./src/allmydata/uri.py 486
3639-    def is_mutable(self):
3640-        return False
3641-
3642     def is_readonly(self):
3643         return True
3644 
3645hunk ./src/allmydata/uri.py 489
3646+    def is_mutable(self):
3647+        return False
3648+
3649     def get_readonly(self):
3650         return self
3651 
3652hunk ./src/allmydata/uri.py 495
3653+
3654 class ImmutableDirectoryURI(_ImmutableDirectoryBaseURI):
3655     BASE_STRING='URI:DIR2-CHK:'
3656     BASE_STRING_RE=re.compile('^'+BASE_STRING)
3657hunk ./src/allmydata/uri.py 501
3658     BASE_HUMAN_RE=re.compile('^'+OPTIONALHTTPLEAD+'URI'+SEP+'DIR2-CHK'+SEP)
3659     INNER_URI_CLASS=CHKFileURI
3660+
3661     def get_verify_cap(self):
3662         vcap = self._filenode_uri.get_verify_cap()
3663         return ImmutableDirectoryURIVerifier(vcap)
3664hunk ./src/allmydata/uri.py 512
3665     BASE_STRING_RE=re.compile('^'+BASE_STRING)
3666     BASE_HUMAN_RE=re.compile('^'+OPTIONALHTTPLEAD+'URI'+SEP+'DIR2-LIT'+SEP)
3667     INNER_URI_CLASS=LiteralFileURI
3668+
3669     def get_verify_cap(self):
3670         # LIT caps have no verifier, since they aren't distributed
3671         return None
3672hunk ./src/allmydata/uri.py 517
3673 
3674+
3675 def wrap_dirnode_cap(filecap):
3676     if isinstance(filecap, WriteableSSKFileURI):
3677         return DirectoryURI(filecap)
3678hunk ./src/allmydata/uri.py 527
3679         return ImmutableDirectoryURI(filecap)
3680     if isinstance(filecap, LiteralFileURI):
3681         return LiteralDirectoryURI(filecap)
3682-    assert False, "cannot wrap a dirnode around %s" % filecap.__class__
3683+    assert False, "cannot interpret as a directory cap: %s" % filecap.__class__
3684+
3685 
3686 class DirectoryURIVerifier(_DirectoryBaseURI):
3687     implements(IVerifierURI)
3688hunk ./src/allmydata/uri.py 546
3689     def get_filenode_cap(self):
3690         return self._filenode_uri
3691 
3692+    def is_mutable(self):
3693+        return False
3694+
3695+
3696 class ImmutableDirectoryURIVerifier(DirectoryURIVerifier):
3697     implements(IVerifierURI)
3698     BASE_STRING='URI:DIR2-CHK-Verifier:'
3699hunk ./src/allmydata/uri.py 557
3700     BASE_HUMAN_RE=re.compile('^'+OPTIONALHTTPLEAD+'URI'+SEP+'DIR2-CHK-VERIFIER'+SEP)
3701     INNER_URI_CLASS=CHKFileVerifierURI
3702 
3703+
3704 class UnknownURI:
3705hunk ./src/allmydata/uri.py 559
3706-    def __init__(self, uri):
3707+    def __init__(self, uri, error=None):
3708         self._uri = uri
3709hunk ./src/allmydata/uri.py 561
3710+        self._error = error
3711+
3712     def to_string(self):
3713         return self._uri
3714 
3715hunk ./src/allmydata/uri.py 566
3716-def from_string(s):
3717-    if not isinstance(s, str):
3718-        raise TypeError("unknown URI type: %s.." % str(s)[:100])
3719-    elif s.startswith('URI:CHK:'):
3720-        return CHKFileURI.init_from_string(s)
3721-    elif s.startswith('URI:CHK-Verifier:'):
3722-        return CHKFileVerifierURI.init_from_string(s)
3723-    elif s.startswith('URI:LIT:'):
3724-        return LiteralFileURI.init_from_string(s)
3725-    elif s.startswith('URI:SSK:'):
3726-        return WriteableSSKFileURI.init_from_string(s)
3727-    elif s.startswith('URI:SSK-RO:'):
3728-        return ReadonlySSKFileURI.init_from_string(s)
3729-    elif s.startswith('URI:SSK-Verifier:'):
3730-        return SSKVerifierURI.init_from_string(s)
3731-    elif s.startswith('URI:DIR2:'):
3732-        return DirectoryURI.init_from_string(s)
3733-    elif s.startswith('URI:DIR2-RO:'):
3734-        return ReadonlyDirectoryURI.init_from_string(s)
3735-    elif s.startswith('URI:DIR2-Verifier:'):
3736-        return DirectoryURIVerifier.init_from_string(s)
3737-    elif s.startswith('URI:DIR2-CHK:'):
3738-        return ImmutableDirectoryURI.init_from_string(s)
3739-    elif s.startswith('URI:DIR2-LIT:'):
3740-        return LiteralDirectoryURI.init_from_string(s)
3741-    return UnknownURI(s)
3742+    def get_readonly(self):
3743+        return None
3744+
3745+    def get_error(self):
3746+        return self._error
3747+
3748+    def get_verify_cap(self):
3749+        return None
3750+
3751+
3752+ALLEGED_READONLY_PREFIX = 'ro.'
3753+ALLEGED_IMMUTABLE_PREFIX = 'imm.'
3754+
3755+def from_string(u, deep_immutable=False, name=u"<unknown name>"):
3756+    if not isinstance(u, str):
3757+        raise TypeError("unknown URI type: %s.." % str(u)[:100])
3758+
3759+    # We allow and check ALLEGED_READONLY_PREFIX or ALLEGED_IMMUTABLE_PREFIX
3760+    # on all URIs, even though we would only strictly need to do so for caps of
3761+    # new formats (post Tahoe-LAFS 1.6). URIs that are not consistent with their
3762+    # prefix are treated as unknown. This should be revisited when we add the
3763+    # new cap formats. See <http://allmydata.org/trac/tahoe/ticket/833#comment:31>.
3764+    s = u
3765+    can_be_mutable = can_be_writeable = not deep_immutable
3766+    if s.startswith(ALLEGED_IMMUTABLE_PREFIX):
3767+        can_be_mutable = can_be_writeable = False
3768+        s = s[len(ALLEGED_IMMUTABLE_PREFIX):]
3769+    elif s.startswith(ALLEGED_READONLY_PREFIX):
3770+        can_be_writeable = False
3771+        s = s[len(ALLEGED_READONLY_PREFIX):]
3772+
3773+    error = None
3774+    kind = "cap"
3775+    try:
3776+        if s.startswith('URI:CHK:'):
3777+            return CHKFileURI.init_from_string(s)
3778+        elif s.startswith('URI:CHK-Verifier:'):
3779+            return CHKFileVerifierURI.init_from_string(s)
3780+        elif s.startswith('URI:LIT:'):
3781+            return LiteralFileURI.init_from_string(s)
3782+        elif s.startswith('URI:SSK:'):
3783+            if can_be_writeable:
3784+                return WriteableSSKFileURI.init_from_string(s)
3785+            kind = "URI:SSK file writecap"
3786+        elif s.startswith('URI:SSK-RO:'):
3787+            if can_be_mutable:
3788+                return ReadonlySSKFileURI.init_from_string(s)
3789+            kind = "URI:SSK-RO readcap to a mutable file"
3790+        elif s.startswith('URI:SSK-Verifier:'):
3791+            return SSKVerifierURI.init_from_string(s)
3792+        elif s.startswith('URI:DIR2:'):
3793+            if can_be_writeable:
3794+                return DirectoryURI.init_from_string(s)
3795+            kind = "URI:DIR2 directory writecap"
3796+        elif s.startswith('URI:DIR2-RO:'):
3797+            if can_be_mutable:
3798+                return ReadonlyDirectoryURI.init_from_string(s)
3799+            kind = "URI:DIR2-RO readcap to a mutable directory"
3800+        elif s.startswith('URI:DIR2-Verifier:'):
3801+            return DirectoryURIVerifier.init_from_string(s)
3802+        elif s.startswith('URI:DIR2-CHK:'):
3803+            return ImmutableDirectoryURI.init_from_string(s)
3804+        elif s.startswith('URI:DIR2-LIT:'):
3805+            return LiteralDirectoryURI.init_from_string(s)
3806+        elif s.startswith('x-tahoe-future-test-writeable:') and not can_be_writeable:
3807+            # For testing how future writeable caps would behave in read-only contexts.
3808+            kind = "x-tahoe-future-test-writeable: testing cap"
3809+        elif s.startswith('x-tahoe-future-test-mutable:') and not can_be_mutable:
3810+            # For testing how future mutable readcaps would behave in immutable contexts.
3811+            kind = "x-tahoe-future-test-mutable: testing cap"
3812+        else:
3813+            return UnknownURI(u)
3814+
3815+        # We fell through because a constraint was not met.
3816+        # Prefer to report the most specific constraint.
3817+        if not can_be_mutable:
3818+            error = MustBeDeepImmutableError(kind + " used in an immutable context", name)
3819+        else:
3820+            error = MustBeReadonlyError(kind + " used in a read-only context", name)
3821+           
3822+    except BadURIError, e:
3823+        error = e
3824+
3825+    return UnknownURI(u, error=error)
3826 
3827 def is_uri(s):
3828     try:
3829hunk ./src/allmydata/uri.py 653
3830-        from_string(s)
3831+        from_string(s, deep_immutable=False)
3832         return True
3833     except (TypeError, AssertionError):
3834         return False
3835hunk ./src/allmydata/uri.py 658
3836 
3837-def from_string_dirnode(s):
3838-    u = from_string(s)
3839+def is_literal_file_uri(s):
3840+    if not isinstance(s, str):
3841+        return False
3842+    return (s.startswith('URI:LIT:') or
3843+            s.startswith(ALLEGED_READONLY_PREFIX + 'URI:LIT:') or
3844+            s.startswith(ALLEGED_IMMUTABLE_PREFIX + 'URI:LIT:'))
3845+
3846+def has_uri_prefix(s):
3847+    if not isinstance(s, str):
3848+        return False
3849+    return (s.startswith("URI:") or
3850+            s.startswith(ALLEGED_READONLY_PREFIX + 'URI:') or
3851+            s.startswith(ALLEGED_IMMUTABLE_PREFIX + 'URI:'))
3852+
3853+
3854+# These take the same keyword arguments as from_string above.
3855+
3856+def from_string_dirnode(s, **kwargs):
3857+    u = from_string(s, **kwargs)
3858     assert IDirnodeURI.providedBy(u)
3859     return u
3860 
3861hunk ./src/allmydata/uri.py 682
3862 registerAdapter(from_string_dirnode, str, IDirnodeURI)
3863 
3864-def from_string_filenode(s):
3865-    u = from_string(s)
3866+def from_string_filenode(s, **kwargs):
3867+    u = from_string(s, **kwargs)
3868     assert IFileURI.providedBy(u)
3869     return u
3870 
3871hunk ./src/allmydata/uri.py 689
3872 registerAdapter(from_string_filenode, str, IFileURI)
3873 
3874-def from_string_mutable_filenode(s):
3875-    u = from_string(s)
3876+def from_string_mutable_filenode(s, **kwargs):
3877+    u = from_string(s, **kwargs)
3878     assert IMutableFileURI.providedBy(u)
3879     return u
3880 registerAdapter(from_string_mutable_filenode, str, IMutableFileURI)
3881hunk ./src/allmydata/uri.py 695
3882 
3883-def from_string_verifier(s):
3884-    u = from_string(s)
3885+def from_string_verifier(s, **kwargs):
3886+    u = from_string(s, **kwargs)
3887     assert IVerifierURI.providedBy(u)
3888     return u
3889 registerAdapter(from_string_verifier, str, IVerifierURI)
3890hunk ./src/allmydata/web/common.py 11
3891 from nevow.util import resource_filename
3892 from allmydata.interfaces import ExistingChildError, NoSuchChildError, \
3893      FileTooLargeError, NotEnoughSharesError, NoSharesError, \
3894-     NotDeepImmutableError, EmptyPathnameComponentError
3895+     EmptyPathnameComponentError, MustBeDeepImmutableError, \
3896+     MustBeReadonlyError, MustNotBeUnknownRWError
3897 from allmydata.mutable.common import UnrecoverableFileError
3898 from allmydata.util import abbreviate # TODO: consolidate
3899 
3900hunk ./src/allmydata/web/common.py 185
3901              "failure, or disk corruption. You should perform a filecheck on "
3902              "this object to learn more.")
3903         return (t, http.GONE)
3904-    if f.check(NotDeepImmutableError):
3905-        t = ("NotDeepImmutableError: a mkdir-immutable operation was given "
3906-             "a child that was not itself immutable: %s" % (f.value,))
3907+    if f.check(MustNotBeUnknownRWError):
3908+        name = f.value.args[1]
3909+        immutable = f.value.args[2]
3910+        if immutable:
3911+            t = ("MustNotBeUnknownRWError: an operation to add a child named "
3912+                 "'%s' to a directory was given an unknown cap in a write slot.\n"
3913+                 "If the cap is actually an immutable readcap, then using a "
3914+                 "webapi server that supports a later version of Tahoe may help.\n\n"
3915+                 "If you are using the webapi directly, then specifying an immutable "
3916+                 "readcap in the read slot (ro_uri) of the JSON PROPDICT, and "
3917+                 "omitting the write slot (rw_uri), would also work in this "
3918+                 "case.") % name.encode("utf-8")
3919+        else:
3920+            t = ("MustNotBeUnknownRWError: an operation to add a child named "
3921+                 "'%s' to a directory was given an unknown cap in a write slot.\n"
3922+                 "Using a webapi server that supports a later version of Tahoe "
3923+                 "may help.\n\n"
3924+                 "If you are using the webapi directly, specifying a readcap in "
3925+                 "the read slot (ro_uri) of the JSON PROPDICT, as well as a "
3926+                 "writecap in the write slot if desired, would also work in this "
3927+                 "case.") % name.encode("utf-8")
3928+        return (t, http.BAD_REQUEST)
3929+    if f.check(MustBeDeepImmutableError):
3930+        name = f.value.args[1]
3931+        t = ("MustBeDeepImmutableError: a cap passed to this operation for "
3932+             "the child named '%s', needed to be immutable but was not. Either "
3933+             "the cap is being added to an immutable directory, or it was "
3934+             "originally retrieved from an immutable directory as an unknown "
3935+             "cap." % name.encode("utf-8"))
3936+        return (t, http.BAD_REQUEST)
3937+    if f.check(MustBeReadonlyError):
3938+        name = f.value.args[1]
3939+        t = ("MustBeReadonlyError: a cap passed to this operation for "
3940+             "the child named '%s', needed to be read-only but was not. "
3941+             "The cap is being passed in a read slot (ro_uri), or was retrieved "
3942+             "from a read slot as an unknown cap." % name.encode("utf-8"))
3943         return (t, http.BAD_REQUEST)
3944     if f.check(WebError):
3945         return (f.value.text, f.value.code)
3946hunk ./src/allmydata/web/directory.py 355
3947         charset = get_arg(req, "_charset", "utf-8")
3948         name = name.decode(charset)
3949         replace = boolean_of_arg(get_arg(req, "replace", "true"))
3950-        d = self.node.set_uri(name, childcap, childcap, overwrite=replace)
3951+       
3952+        # We mustn't pass childcap for the readcap argument because we don't
3953+        # know whether it is a read cap. Passing a read cap as the writecap
3954+        # argument will work (it ends up calling NodeMaker.create_from_cap,
3955+        # which derives a readcap if necessary and possible).
3956+        d = self.node.set_uri(name, childcap, None, overwrite=replace)
3957         d.addCallback(lambda res: childcap)
3958         return d
3959 
3960hunk ./src/allmydata/web/directory.py 371
3961             # won't show up in the resulting encoded form.. the 'name'
3962             # field is completely missing. So to allow deletion of an
3963             # empty file, we have to pretend that None means ''. The only
3964-            # downide of this is a slightly confusing error message if
3965+            # downside of this is a slightly confusing error message if
3966             # someone does a POST without a name= field. For our own HTML
3967hunk ./src/allmydata/web/directory.py 373
3968-            # thisn't a big deal, because we create the 'delete' POST
3969+            # this isn't a big deal, because we create the 'delete' POST
3970             # buttons ourselves.
3971             name = ''
3972         charset = get_arg(req, "_charset", "utf-8")
3973hunk ./src/allmydata/web/directory.py 593
3974     def render_title(self, ctx, data):
3975         si_s = abbreviated_dirnode(self.node)
3976         header = ["Tahoe-LAFS - Directory SI=%s" % si_s]
3977-        if self.node.is_readonly():
3978+        if self.node.is_unknown():
3979+            header.append(" (unknown)")
3980+        elif not self.node.is_mutable():
3981+            header.append(" (immutable)")
3982+        elif self.node.is_readonly():
3983             header.append(" (read-only)")
3984         else:
3985             header.append(" (modifiable)")
3986hunk ./src/allmydata/web/directory.py 606
3987     def render_header(self, ctx, data):
3988         si_s = abbreviated_dirnode(self.node)
3989         header = ["Tahoe-LAFS Directory SI=", T.span(class_="data-chars")[si_s]]
3990-        if self.node.is_readonly():
3991+        if self.node.is_unknown():
3992+            header.append(" (unknown)")
3993+        elif not self.node.is_mutable():
3994+            header.append(" (immutable)")
3995+        elif self.node.is_readonly():
3996             header.append(" (read-only)")
3997         return ctx.tag[header]
3998 
3999hunk ./src/allmydata/web/directory.py 619
4000         return T.div[T.a(href=link)["Return to Welcome page"]]
4001 
4002     def render_show_readonly(self, ctx, data):
4003-        if self.node.is_readonly():
4004+        if self.node.is_unknown() or self.node.is_readonly():
4005             return ""
4006         rocap = self.node.get_readonly_uri()
4007         root = get_root(ctx)
4008hunk ./src/allmydata/web/directory.py 646
4009 
4010         root = get_root(ctx)
4011         here = "%s/uri/%s/" % (root, urllib.quote(self.node.get_uri()))
4012-        if self.node.is_readonly():
4013+        if self.node.is_unknown() or self.node.is_readonly():
4014             delete = "-"
4015             rename = "-"
4016         else:
4017hunk ./src/allmydata/web/directory.py 694
4018         ctx.fillSlots("times", times)
4019 
4020         assert IFilesystemNode.providedBy(target), target
4021-        writecap = target.get_uri() or ""
4022-        quoted_uri = urllib.quote(writecap, safe="") # escape slashes too
4023+        target_uri = target.get_uri() or ""
4024+        quoted_uri = urllib.quote(target_uri, safe="") # escape slashes too
4025 
4026         if IMutableFileNode.providedBy(target):
4027             # to prevent javascript in displayed .html files from stealing a
4028hunk ./src/allmydata/web/directory.py 724
4029 
4030         elif IDirectoryNode.providedBy(target):
4031             # directory
4032-            uri_link = "%s/uri/%s/" % (root, urllib.quote(writecap))
4033+            uri_link = "%s/uri/%s/" % (root, urllib.quote(target_uri))
4034             ctx.fillSlots("filename",
4035                           T.a(href=uri_link)[html.escape(name)])
4036             if not target.is_mutable():
4037hunk ./src/allmydata/web/directory.py 811
4038         kids = {}
4039         for name, (childnode, metadata) in children.iteritems():
4040             assert IFilesystemNode.providedBy(childnode), childnode
4041-            rw_uri = childnode.get_uri()
4042+            rw_uri = childnode.get_write_uri()
4043             ro_uri = childnode.get_readonly_uri()
4044             if IFileNode.providedBy(childnode):
4045hunk ./src/allmydata/web/directory.py 814
4046-                if childnode.is_readonly():
4047-                    rw_uri = None
4048                 kiddata = ("filenode", {'size': childnode.get_size(),
4049                                         'mutable': childnode.is_mutable(),
4050                                         })
4051hunk ./src/allmydata/web/directory.py 818
4052             elif IDirectoryNode.providedBy(childnode):
4053-                if childnode.is_readonly():
4054-                    rw_uri = None
4055                 kiddata = ("dirnode", {'mutable': childnode.is_mutable()})
4056             else:
4057                 kiddata = ("unknown", {})
4058hunk ./src/allmydata/web/directory.py 821
4059+
4060             kiddata[1]["metadata"] = metadata
4061hunk ./src/allmydata/web/directory.py 823
4062-            if ro_uri:
4063-                kiddata[1]["ro_uri"] = ro_uri
4064             if rw_uri:
4065                 kiddata[1]["rw_uri"] = rw_uri
4066hunk ./src/allmydata/web/directory.py 825
4067+            if ro_uri:
4068+                kiddata[1]["ro_uri"] = ro_uri
4069             verifycap = childnode.get_verify_cap()
4070             if verifycap:
4071                 kiddata[1]['verify_uri'] = verifycap.to_string()
4072hunk ./src/allmydata/web/directory.py 830
4073+
4074             kids[name] = kiddata
4075hunk ./src/allmydata/web/directory.py 832
4076-        if dirnode.is_readonly():
4077-            drw_uri = None
4078-            dro_uri = dirnode.get_uri()
4079-        else:
4080-            drw_uri = dirnode.get_uri()
4081-            dro_uri = dirnode.get_readonly_uri()
4082+
4083+        drw_uri = dirnode.get_write_uri()
4084+        dro_uri = dirnode.get_readonly_uri()
4085         contents = { 'children': kids }
4086         if dro_uri:
4087             contents['ro_uri'] = dro_uri
4088hunk ./src/allmydata/web/directory.py 845
4089             contents['verify_uri'] = verifycap.to_string()
4090         contents['mutable'] = dirnode.is_mutable()
4091         data = ("dirnode", contents)
4092-        return simplejson.dumps(data, indent=1) + "\n"
4093+        json = simplejson.dumps(data, indent=1) + "\n"
4094+        return json
4095     d.addCallback(_got)
4096     d.addCallback(text_plain, ctx)
4097     return d
4098hunk ./src/allmydata/web/directory.py 852
4099 
4100 
4101-
4102 def DirectoryURI(ctx, dirnode):
4103     return text_plain(dirnode.get_uri(), ctx)
4104 
4105hunk ./src/allmydata/web/directory.py 1143
4106         self.req.write(j+"\n")
4107         return ""
4108 
4109-class UnknownNodeHandler(RenderMixin, rend.Page):
4110 
4111hunk ./src/allmydata/web/directory.py 1144
4112+class UnknownNodeHandler(RenderMixin, rend.Page):
4113     def __init__(self, client, node, parentnode=None, name=None):
4114         rend.Page.__init__(self)
4115         assert node
4116hunk ./src/allmydata/web/directory.py 1149
4117         self.node = node
4118+        self.parentnode = parentnode
4119+        self.name = name
4120 
4121     def render_GET(self, ctx):
4122         req = IRequest(ctx)
4123hunk ./src/allmydata/web/directory.py 1157
4124         t = get_arg(req, "t", "").strip()
4125         if t == "info":
4126             return MoreInfo(self.node)
4127-        raise WebError("GET unknown URI type: can only do t=info, not t=%s" % t)
4128-
4129+        if t == "json":
4130+            if self.parentnode and self.name:
4131+                d = self.parentnode.get_metadata_for(self.name)
4132+            else:
4133+                d = defer.succeed(None)
4134+            d.addCallback(lambda md: UnknownJSONMetadata(ctx, self.node, md))
4135+            return d
4136+        raise WebError("GET unknown URI type: can only do t=info and t=json, not t=%s.\n"
4137+                       "Using a webapi server that supports a later version of Tahoe "
4138+                       "may help." % t)
4139 
4140hunk ./src/allmydata/web/directory.py 1168
4141+def UnknownJSONMetadata(ctx, filenode, edge_metadata):
4142+    rw_uri = filenode.get_write_uri()
4143+    ro_uri = filenode.get_readonly_uri()
4144+    data = ("unknown", {})
4145+    if ro_uri:
4146+        data[1]['ro_uri'] = ro_uri
4147+    if rw_uri:
4148+        data[1]['rw_uri'] = rw_uri
4149+    if edge_metadata is not None:
4150+        data[1]['metadata'] = edge_metadata
4151+    return text_plain(simplejson.dumps(data, indent=1) + "\n", ctx)
4152hunk ./src/allmydata/web/filenode.py 9
4153 from nevow import url, rend
4154 from nevow.inevow import IRequest
4155 
4156-from allmydata.interfaces import ExistingChildError, CannotPackUnknownNodeError
4157+from allmydata.interfaces import ExistingChildError
4158 from allmydata.monitor import Monitor
4159 from allmydata.immutable.upload import FileHandle
4160hunk ./src/allmydata/web/filenode.py 12
4161-from allmydata.unknown import UnknownNode
4162 from allmydata.util import log, base32
4163 
4164 from allmydata.web.common import text_plain, WebError, RenderMixin, \
4165hunk ./src/allmydata/web/filenode.py 22
4166 from allmydata.web.info import MoreInfo
4167 
4168 class ReplaceMeMixin:
4169-
4170     def replace_me_with_a_child(self, req, client, replace):
4171         # a new file is being uploaded in our place.
4172         mutable = boolean_of_arg(get_arg(req, "mutable", "false"))
4173hunk ./src/allmydata/web/filenode.py 56
4174     def replace_me_with_a_childcap(self, req, client, replace):
4175         req.content.seek(0)
4176         childcap = req.content.read()
4177-        childnode = client.create_node_from_uri(childcap, childcap+"readonly")
4178-        if isinstance(childnode, UnknownNode):
4179-            # don't be willing to pack unknown nodes: we might accidentally
4180-            # put some write-authority into the rocap slot because we don't
4181-            # know how to diminish the URI they gave us. We don't even know
4182-            # if they gave us a readcap or a writecap.
4183-            msg = "cannot attach unknown node as child %s" % str(self.name)
4184-            raise CannotPackUnknownNodeError(msg)
4185+        childnode = client.create_node_from_uri(childcap, None, name=self.name)
4186         d = self.parentnode.set_node(self.name, childnode, overwrite=replace)
4187         d.addCallback(lambda res: childnode.get_uri())
4188         return d
4189hunk ./src/allmydata/web/filenode.py 420
4190 
4191 
4192 def FileJSONMetadata(ctx, filenode, edge_metadata):
4193-    if filenode.is_readonly():
4194-        rw_uri = None
4195-        ro_uri = filenode.get_uri()
4196-    else:
4197-        rw_uri = filenode.get_uri()
4198-        ro_uri = filenode.get_readonly_uri()
4199+    rw_uri = filenode.get_write_uri()
4200+    ro_uri = filenode.get_readonly_uri()
4201     data = ("filenode", {})
4202     data[1]['size'] = filenode.get_size()
4203     if ro_uri:
4204hunk ./src/allmydata/web/info.py 24
4205     def get_type(self):
4206         node = self.original
4207         if IDirectoryNode.providedBy(node):
4208+            if not node.is_mutable():
4209+                return "immutable directory"
4210             return "directory"
4211         if IFileNode.providedBy(node):
4212             si = node.get_storage_index()
4213hunk ./src/allmydata/web/info.py 33
4214                 if node.is_mutable():
4215                     return "mutable file"
4216                 return "immutable file"
4217-            return "LIT file"
4218+            return "immutable LIT file"
4219         return "unknown"
4220 
4221     def render_title(self, ctx, data):
4222hunk ./src/allmydata/web/info.py 73
4223 
4224     def render_directory_writecap(self, ctx, data):
4225         node = self.original
4226-        if node.is_readonly():
4227-            return ""
4228         if not IDirectoryNode.providedBy(node):
4229             return ""
4230hunk ./src/allmydata/web/info.py 75
4231+        if node.is_readonly():
4232+            return ""
4233         return ctx.tag[node.get_uri()]
4234 
4235     def render_directory_readcap(self, ctx, data):
4236hunk ./src/allmydata/web/info.py 91
4237             return ""
4238         return ctx.tag[node.get_verify_cap().to_string()]
4239 
4240-
4241     def render_file_writecap(self, ctx, data):
4242         node = self.original
4243         if IDirectoryNode.providedBy(node):
4244hunk ./src/allmydata/web/info.py 95
4245             node = node._node
4246-        if ((IDirectoryNode.providedBy(node) or IFileNode.providedBy(node))
4247-            and node.is_readonly()):
4248-            return ""
4249-        writecap = node.get_uri()
4250-        if not writecap:
4251+        write_uri = node.get_write_uri()
4252+        if not write_uri:
4253             return ""
4254hunk ./src/allmydata/web/info.py 98
4255-        return ctx.tag[writecap]
4256+        return ctx.tag[write_uri]
4257 
4258     def render_file_readcap(self, ctx, data):
4259         node = self.original
4260hunk ./src/allmydata/web/info.py 104
4261         if IDirectoryNode.providedBy(node):
4262             node = node._node
4263-        readcap = node.get_readonly_uri()
4264-        if not readcap:
4265+        read_uri = node.get_readonly_uri()
4266+        if not read_uri:
4267             return ""
4268hunk ./src/allmydata/web/info.py 107
4269-        return ctx.tag[readcap]
4270+        return ctx.tag[read_uri]
4271 
4272     def render_file_verifycap(self, ctx, data):
4273         node = self.original
4274hunk ./src/allmydata/web/root.py 15
4275 from allmydata import get_package_versions_string
4276 from allmydata import provisioning
4277 from allmydata.util import idlib, log
4278-from allmydata.interfaces import IFileNode, UnhandledCapTypeError
4279+from allmydata.interfaces import IFileNode
4280 from allmydata.web import filenode, directory, unlinked, status, operations
4281 from allmydata.web import reliability, storage
4282 from allmydata.web.common import abbreviate_size, getxmlfile, WebError, \
4283hunk ./src/allmydata/web/root.py 88
4284         try:
4285             node = self.client.create_node_from_uri(name)
4286             return directory.make_handler_for(node, self.client)
4287-        except (TypeError, UnhandledCapTypeError, AssertionError):
4288+        except (TypeError, AssertionError):
4289             raise WebError("'%s' is not a valid file- or directory- cap"
4290                            % name)
4291 
4292hunk ./src/allmydata/web/root.py 107
4293         # 'name' must be a file URI
4294         try:
4295             node = self.client.create_node_from_uri(name)
4296-        except (TypeError, UnhandledCapTypeError, AssertionError):
4297+        except (TypeError, AssertionError):
4298             # I think this can no longer be reached
4299             raise WebError("'%s' is not a valid file- or directory- cap"
4300                            % name)
4301}
4302[Miscellaneous documentation, test, and code formatting tweaks.
4303david-sarah@jacaranda.org**20100127070309
4304 Ignore-this: 84ca7e4bb7c64221ae2c61144ef5edef
4305] {
4306hunk ./contrib/fuse/impl_c/blackmatch.py 909
4307 
4308 class TStat(fuse.Stat):
4309     # in fuse 0.2, these are set by fuse.Stat.__init__
4310-    # in fuse 0.2-pre3 (hardy) they are not. badness unsues if they're missing
4311+    # in fuse 0.2-pre3 (hardy) they are not. badness ensues if they're missing
4312     st_mode  = None
4313     st_ino   = 0
4314     st_dev   = 0
4315hunk ./contrib/fuse/impl_c/blackmatch.py 1022
4316     def get_uri(self):
4317         return self.rw_uri or self.ro_uri
4318 
4319+    # TODO: rename to 'is_writeable', or switch sense to 'is_readonly', for consistency with Tahoe code
4320     def writable(self):
4321         return self.rw_uri and self.rw_uri != self.ro_uri
4322 
4323hunk ./docs/frontends/webapi.txt 73
4324 these tasks. In general, everything that can be done with a PUT or DELETE can
4325 also be done with a POST.
4326 
4327-Tahoe's web API is designed for two different consumers. The first is a
4328-program that needs to manipulate the virtual file system. Such programs are
4329+Tahoe's web API is designed for two different kinds of consumer. The first is
4330+a program that needs to manipulate the virtual file system. Such programs are
4331 expected to use the RESTful interface described above. The second is a human
4332 using a standard web browser to work with the filesystem. This user is given
4333 a series of HTML pages with links to download files, and forms that use POST
4334hunk ./docs/frontends/webapi.txt 81
4335 actions to upload, rename, and delete files.
4336 
4337 When an error occurs, the HTTP response code will be set to an appropriate
4338-400-series code (like 404 for an unknown childname, or 400 Gone when a file
4339-is unrecoverable due to insufficient shares), and the HTTP response body will
4340-usually contain a few lines of explanation as to the cause of the error and
4341-possible responses. Unusual exceptions may result in a 500 Internal Server
4342-Error as a catch-all, with a default response body will contain a
4343-Nevow-generated HTML-ized representation of the Python exception stack trace
4344+400-series code (like 404 Not Found for an unknown childname, or 400 Bad Request
4345+when the parameters to a webapi operation are invalid), and the HTTP response
4346+body will usually contain a few lines of explanation as to the cause of the
4347+error and possible responses. Unusual exceptions may result in a
4348+500 Internal Server Error as a catch-all, with a default response body containing
4349+a Nevow-generated HTML-ized representation of the Python exception stack trace
4350 that caused the problem. CLI programs which want to copy the response body to
4351 stderr should provide an "Accept: text/plain" header to their requests to get
4352 a plain text stack trace instead. If the Accept header contains */*, or
4353hunk ./docs/frontends/webapi.txt 111
4354 read- and write- caps, which start with "URI:SSK", and give access to mutable
4355 files.
4356 
4357-(later versions of Tahoe will make these strings shorter, and will remove the
4358+(Later versions of Tahoe will make these strings shorter, and will remove the
4359 unfortunate colons, which must be escaped when these caps are embedded in
4360hunk ./docs/frontends/webapi.txt 113
4361-URLs).
4362+URLs.)
4363 
4364 To refer to any Tahoe object through the web API, you simply need to combine
4365 a prefix (which indicates the HTTP server to use) with the cap (which
4366hunk ./docs/frontends/webapi.txt 124
4367  http://127.0.0.1:3456/uri/ + $CAP
4368 
4369 So, to access the directory named above (which happens to be the
4370-publically-writable sample directory on the Tahoe test grid, described at
4371+publically-writeable sample directory on the Tahoe test grid, described at
4372 http://allmydata.org/trac/tahoe/wiki/TestGrid), the URL would be:
4373 
4374  http://127.0.0.1:3456/uri/URI%3ADIR2%3Adjrdkfawoqihigoett4g6auz6a%3Ajx5mplfpwexnoqff7y5e4zjus4lidm76dcuarpct7cckorh2dpgq/
4375hunk ./docs/frontends/webapi.txt 202
4376 representable as such.
4377 
4378 All Tahoe operations that refer to existing files or directories must include
4379-a suitable read- or write- cap in the URL: the wapi server won't add one
4380+a suitable read- or write- cap in the URL: the webapi server won't add one
4381 for you. If you don't know the cap, you can't access the file. This allows
4382hunk ./docs/frontends/webapi.txt 204
4383-the security properties of Tahoe caps to be extended across the wapi
4384+the security properties of Tahoe caps to be extended across the webapi
4385 interface.
4386 
4387 == Slow Operations, Progress, and Cancelling ==
4388hunk ./docs/frontends/webapi.txt 278
4389    since the operation completed) will remain valid for ten minutes.
4390 
4391 Many "slow" operations can begin to use unacceptable amounts of memory when
4392-operation on large directory structures. The memory usage increases when the
4393+operating on large directory structures. The memory usage increases when the
4394 ophandle is polled, as the results must be copied into a JSON string, sent
4395 over the wire, then parsed by a client. So, as an alternative, many "slow"
4396 operations have streaming equivalents. These equivalents do not use operation
4397hunk ./docs/frontends/webapi.txt 318
4398  retrieve the same contents that were just uploaded. This will create any
4399  necessary intermediate subdirectories.
4400 
4401- To use the /uri/$FILECAP form, $FILECAP be a write-cap for a mutable file.
4402+ To use the /uri/$FILECAP form, $FILECAP must be a write-cap for a mutable file.
4403 
4404  In the /uri/$DIRCAP/[SUBDIRS../]FILENAME form, if the target file is a
4405hunk ./docs/frontends/webapi.txt 321
4406- writable mutable file, that files contents will be overwritten in-place. If
4407+ writeable mutable file, that file's contents will be overwritten in-place. If
4408  it is a read-cap for a mutable file, an error will occur. If it is an
4409  immutable file, the old file will be discarded, and a new one will be put in
4410  its place.
4411hunk ./docs/frontends/webapi.txt 340
4412 PUT /uri
4413 
4414  This uploads a file, and produces a file-cap for the contents, but does not
4415- attach the file into the virtual drive. No directories will be modified by
4416+ attach the file into the filesystem. No directories will be modified by
4417  this operation. The file-cap is returned as the body of the HTTP response.
4418 
4419  If "mutable=true" is in the query arguments, the operation will create a
4420hunk ./docs/frontends/webapi.txt 354
4421 
4422  Create a new empty directory and return its write-cap as the HTTP response
4423  body. This does not make the newly created directory visible from the
4424- virtual drive. The "PUT" operation is provided for backwards compatibility:
4425+ filesystem. The "PUT" operation is provided for backwards compatibility:
4426  new code should use POST.
4427 
4428 POST /uri?t=mkdir-with-children
4429hunk ./docs/frontends/webapi.txt 395
4430             "linkcrtime": 1202777696.7564139,
4431             "linkmotime": 1202777696.7564139,
4432           } } } ]
4433- }
4434+  }
4435 
4436  For forward-compatibility, a mutable directory can also contain caps in
4437  a format that is unknown to the webapi server. When such caps are retrieved
4438hunk ./docs/frontends/webapi.txt 525
4439  the immediate parent directory already has a a child named NAME.
4440 
4441  Note that the name= argument must be passed as a queryarg, because the POST
4442- request body is used for the initial children JSON.
4443+ request body is used for the initial children JSON.
4444 
4445 POST /uri/$DIRCAP/[SUBDIRS../]?t=mkdir-immutable&name=NAME
4446 
4447hunk ./docs/frontends/webapi.txt 628
4448 
4449   Then the rw_uri field will be present in the information about a directory
4450   if and only if you have read-write access to that directory. The verify_uri
4451-  field will be presend if and only if the object has a verify-cap
4452+  field will be present if and only if the object has a verify-cap
4453   (non-distributed LIT files do not have verify-caps).
4454 
4455 ==== About the metadata ====
4456hunk ./docs/frontends/webapi.txt 704
4457   link points.
4458 
4459   4. Also, quite apart from Tahoe, you might be confused about the meaning of
4460-  the 'ctime' in unix local filesystems, which people sometimes think means
4461-  file creation time, but which actually means, in unix local filesystems, the
4462+  the 'ctime' in UNIX local filesystems, which people sometimes think means
4463+  file creation time, but which actually means, in UNIX local filesystems, the
4464   most recent time that the file contents or the file metadata (such as owner,
4465   permission bits, extended attributes, etc.) has changed. Note that although
4466hunk ./docs/frontends/webapi.txt 708
4467-  'ctime' does not mean file creation time in Unix, it does mean link creation
4468+  'ctime' does not mean file creation time in UNIX, it does mean link creation
4469   time in Tahoe, unless the "tahoe backup" command has been used on that link,
4470   in which case it means something about the local filesystem file which
4471   corresponds to the Tahoe file which is pointed at by the link. It means
4472hunk ./docs/frontends/webapi.txt 716
4473   Windows) or file-contents-or-metadata-update-time of the local file (if
4474   "tahoe backup" was run on a different operating system).
4475 
4476-
4477 === Attaching an existing File or Directory by its read- or write- cap ===
4478 
4479 PUT /uri/$DIRCAP/[SUBDIRS../]CHILDNAME?t=uri
4480hunk ./docs/frontends/webapi.txt 739
4481   if there is already an object at the given location, rather than
4482   overwriting the existing object. To allow the operation to overwrite a
4483   file, but return an error when trying to overwrite a directory, use
4484-  "replace=only-files" (this behavior is closer to the traditional unix "mv"
4485+  "replace=only-files" (this behavior is closer to the traditional UNIX "mv"
4486   command). Note that "true", "t", and "1" are all synonyms for "True", and
4487   "false", "f", and "0" are synonyms for "False", and the parameter is
4488   case-insensitive.
4489hunk ./docs/frontends/webapi.txt 743
4490
4491+  Note that this operation does not take its child cap in the form of
4492+  separate "rw_uri" and "ro_uri" fields. Therefore, it cannot accept a
4493+  child cap in a format unknown to the webapi server, because the server
4494+  is not able to attenuate an unknown write cap to a read cap.
4495 
4496 === Adding multiple files or directories to a parent directory at once ===
4497 
4498hunk ./docs/frontends/webapi.txt 809
4499   The object will only become completely unreachable once 1: there are no
4500   reachable directories that reference it, and 2: nobody is holding a read-
4501   or write- cap to the object. (This behavior is very similar to the way
4502-  hardlinks and anonymous files work in traditional unix filesystems).
4503+  hardlinks and anonymous files work in traditional UNIX filesystems).
4504 
4505   This operation will not modify more than a single directory. Intermediate
4506   directories which were implicitly created by PUT or POST methods will *not*
4507hunk ./docs/frontends/webapi.txt 938
4508 POST /uri?t=upload
4509 
4510  This uploads a file, and produces a file-cap for the contents, but does not
4511- attach the file into the virtual drive. No directories will be modified by
4512+ attach the file into the filesystem. No directories will be modified by
4513  this operation.
4514 
4515  The file must be provided as the "file" field of an HTML encoded form body,
4516hunk ./docs/frontends/webapi.txt 1684
4517 
4518 == Static Files in /public_html ==
4519 
4520-The wapi server will take any request for a URL that starts with /static
4521+The webapi server will take any request for a URL that starts with /static
4522 and serve it from a configurable directory which defaults to
4523 $BASEDIR/public_html . This is configured by setting the "[node]web.static"
4524 value in $BASEDIR/tahoe.cfg . If this is left at the default value of
4525hunk ./docs/frontends/webapi.txt 1692
4526 served with the contents of the file $BASEDIR/public_html/subdir/foo.html .
4527 
4528 This can be useful to serve a javascript application which provides a
4529-prettier front-end to the rest of the Tahoe wapi.
4530+prettier front-end to the rest of the Tahoe webapi.
4531 
4532 
4533hunk ./docs/frontends/webapi.txt 1695
4534-== safety and security issues -- names vs. URIs ==
4535+== Safety and security issues -- names vs. URIs ==
4536 
4537 Summary: use explicit file- and dir- caps whenever possible, to reduce the
4538 potential for surprises when the filesystem structure is changed.
4539hunk ./docs/frontends/webapi.txt 1781
4540 
4541 Tahoe nodes implement internal serialization to make sure that a single Tahoe
4542 node cannot conflict with itself. For example, it is safe to issue two
4543-directory modification requests to a single tahoe node's wapi server at the
4544+directory modification requests to a single tahoe node's webapi server at the
4545 same time, because the Tahoe node will internally delay one of them until
4546 after the other has finished being applied. (This feature was introduced in
4547 Tahoe-1.1; back with Tahoe-1.0 the web client was responsible for serializing
4548hunk ./relnotes.txt 1
4549-ANNOUNCING Tahoe, the Lofty-Atmospheric Filesystem, v1.5
4550+ANNOUNCING Tahoe, the Lofty-Atmospheric Filesystem, v1.6
4551 
4552 The Tahoe-LAFS team is pleased to announce the immediate
4553hunk ./relnotes.txt 4
4554-availability of version 1.5 of Tahoe, the Lofty Atmospheric
4555+availability of version 1.6 of Tahoe, the Lofty Atmospheric
4556 File System.
4557 
4558 Tahoe-LAFS is the first cloud storage technology which offers
4559hunk ./relnotes.txt 32
4560 
4561 COMPATIBILITY
4562 
4563-Version 1.5 is fully compatible with the version 1 series of
4564-Tahoe-LAFS. Files written by v1.5 clients can be read by
4565-clients of all versions back to v1.0. v1.5 clients can read
4566-files produced by clients of all versions since v1.0.  v1.5
4567-servers can serve clients of all versions back to v1.0 and v1.5
4568+Version 1.6 is fully compatible with the version 1 series of
4569+Tahoe-LAFS. Files written by v1.6 clients can be read by
4570+clients of all versions back to v1.0. v1.6 clients can read
4571+files produced by clients of all versions since v1.0.  v1.6
4572+servers can serve clients of all versions back to v1.0 and v1.6
4573 clients can use servers of all versions back to v1.0.
4574 
4575hunk ./relnotes.txt 39
4576-This is the sixth release in the version 1 series. The version
4577-1 series of Tahoe-LAFS will be actively supported and
4578+In addition, version 1.6 improves forward-compatibility with
4579+planned future cap formats, allowing updates to a directory
4580+containing both current and future caps, without loss of
4581+information.
4582+
4583+This is the seventh major release in the version 1 series. The
4584+version 1 series of Tahoe-LAFS will be actively supported and
4585 maintained for the forseeable future, and future versions of
4586 Tahoe-LAFS will retain the ability to read and write files
4587 compatible with Tahoe-LAFS v1.
4588hunk ./src/allmydata/dirnode.py 26
4589 from pycryptopp.cipher.aes import AES
4590 from allmydata.util.dictutil import AuxValueDict
4591 
4592+
4593+# TODO: {Deleter,MetadataSetter,Adder}.modify all start by unpacking the
4594+# contents and end by repacking them. It might be better to apply them to
4595+# the unpacked contents.
4596+
4597 class Deleter:
4598     def __init__(self, node, name, must_exist=True):
4599         self.node = node
4600hunk ./src/allmydata/interfaces.py 429
4601         """Return True if the data can be modified by *somebody* (perhaps
4602         someone who has a more powerful URI than this one)."""
4603 
4604+    # TODO: rename to get_read_cap()
4605     def get_readonly():
4606         """Return another IURI instance, which represents a read-only form of
4607         this one. If is_readonly() is True, this returns self."""
4608hunk ./src/allmydata/test/test_web.py 703
4609                              self.PUT, base, "")
4610         return d
4611 
4612+    # TODO: version of this with a Unicode filename
4613     def test_GET_FILEURL_save(self):
4614hunk ./src/allmydata/test/test_web.py 705
4615-        d = self.GET(self.public_url + "/foo/bar.txt?filename=bar.txt&save=true")
4616-        # TODO: look at the headers, expect a Content-Disposition: attachment
4617-        # header.
4618-        d.addCallback(self.failUnlessIsBarDotTxt)
4619+        d = self.GET(self.public_url + "/foo/bar.txt?filename=bar.txt&save=true",
4620+                     return_response=True)
4621+        def _got((res, statuscode, headers)):
4622+            content_disposition = headers["content-disposition"][0]
4623+            self.failUnless(content_disposition == 'attachment; filename="bar.txt"', content_disposition)
4624+            self.failUnlessIsBarDotTxt(res)
4625+        d.addCallback(_got)
4626         return d
4627 
4628     def test_GET_FILEURL_missing(self):
4629hunk ./src/allmydata/test/test_web.py 2265
4630         # Fetch the welcome page.
4631         d = self.GET("/")
4632         def _after_get_welcome_page(res):
4633-            MKDIR_BUTTON_RE=re.compile('<form action="([^"]*)" method="post".*?<input type="hidden" name="t" value="([^"]*)" /><input type="hidden" name="([^"]*)" value="([^"]*)" /><input type="submit" value="Create a directory" />', re.I)
4634+            MKDIR_BUTTON_RE = re.compile(
4635+                '<form action="([^"]*)" method="post".*?'
4636+                '<input type="hidden" name="t" value="([^"]*)" />'
4637+                '<input type="hidden" name="([^"]*)" value="([^"]*)" />'
4638+                '<input type="submit" value="Create a directory" />',
4639+                re.I)
4640             mo = MKDIR_BUTTON_RE.search(res)
4641             formaction = mo.group(1)
4642             formt = mo.group(2)
4643hunk ./src/allmydata/uri.py 14
4644 class BadURIError(CapConstraintError):
4645     pass
4646 
4647-# the URI shall be an ascii representation of the file. It shall contain
4648-# enough information to retrieve and validate the contents. It shall be
4649-# expressed in a limited character set (namely [TODO]).
4650+# The URI shall be an ASCII representation of a reference to the file/directory.
4651+# It shall contain enough information to retrieve and validate the contents.
4652+# It shall be expressed in a limited character set (currently base32 plus ':' and
4653+# capital letters, but future URIs might use a larger charset).
4654+
4655+# TODO:
4656+#  - rename all of the *URI classes/interfaces to *Cap
4657+#  - make variable and method names consistently use _uri for an URI string,
4658+#    and _cap for a Cap object (decoded URI)
4659+#  - remove the human_encoding methods?
4660 
4661 BASE32STR_128bits = '(%s{25}%s)' % (base32.BASE32CHAR, base32.BASE32CHAR_3bits)
4662 BASE32STR_256bits = '(%s{51}%s)' % (base32.BASE32CHAR, base32.BASE32CHAR_1bits)
4663hunk ./src/allmydata/uri.py 32
4664 NUMBER='([0-9]+)'
4665 NUMBER_IGNORE='(?:[0-9]+)'
4666 
4667-# URIs (soon to be renamed "caps") are always allowed to come with a leading
4668+# "human-encoded" URIs are allowed to come with a leading
4669 # 'http://127.0.0.1:(8123|3456)/uri/' that will be ignored.
4670hunk ./src/allmydata/uri.py 34
4671+# Note that nothing in the Tahoe code currently uses the human encoding.
4672 OPTIONALHTTPLEAD=r'(?:https?://(?:[^:/]+)(?::%s)?/uri/)?' % NUMBER_IGNORE
4673 
4674 
4675hunk ./src/allmydata/uri.py 41
4676 class _BaseURI:
4677     def __hash__(self):
4678         return self.to_string().__hash__()
4679+
4680     def __eq__(self, them):
4681         if isinstance(them, _BaseURI):
4682             return self.to_string() == them.to_string()
4683hunk ./src/allmydata/uri.py 47
4684         else:
4685             return False
4686+
4687     def __ne__(self, them):
4688         if isinstance(them, _BaseURI):
4689             return self.to_string() != them.to_string()
4690hunk ./src/allmydata/uri.py 53
4691         else:
4692             return True
4693+
4694     def to_human_encoding(self):
4695         return 'http://127.0.0.1:3456/uri/'+self.to_string()
4696 
4697hunk ./src/allmydata/uri.py 60
4698     def get_storage_index(self):
4699         return self.storage_index
4700 
4701+
4702 class CHKFileURI(_BaseURI):
4703     implements(IURI, IImmutableFileURI)
4704 
4705}
4706[Address comments by Kevan on 833 and add test for stripping spaces
4707david-sarah@jacaranda.org**20100127230642
4708 Ignore-this: de36aeaf4afb3ba05dbeb49a5e9a6b26
4709] {
4710hunk ./docs/frontends/webapi.txt 746
4711   
4712   Note that this operation does not take its child cap in the form of
4713   separate "rw_uri" and "ro_uri" fields. Therefore, it cannot accept a
4714-  child cap in a format unknown to the webapi server, because the server
4715-  is not able to attenuate an unknown write cap to a read cap.
4716+  child cap in a format unknown to the webapi server, unless its URI
4717+  starts with "ro." or "imm.". This restriction is necessary because the
4718+  server is not able to attenuate an unknown write cap to a read cap.
4719+  Unknown URIs starting with "ro." or "imm.", on the other hand, are
4720+  assumed to represent read caps. The client should not prefix a write
4721+  cap with "ro." or "imm." and pass it to this operation, since that
4722+  would result in granting the cap's write authority to holders of the
4723+  directory read cap.
4724 
4725 === Adding multiple files or directories to a parent directory at once ===
4726 
4727hunk ./docs/frontends/webapi.txt 1037
4728 
4729  This attaches a given read- or write- cap "CHILDCAP" to the designated
4730  directory, with a specified child name. This behaves much like the PUT t=uri
4731- operation, and is a lot like a UNIX hardlink.
4732+ operation, and is a lot like a UNIX hardlink. It is subject to the same
4733+ restrictions as that operation on the use of cap formats unknown to the
4734+ webapi server.
4735 
4736  This will create additional intermediate directories as necessary, although
4737  since it is expected to be triggered by a form that was retrieved by "GET
4738hunk ./src/allmydata/dirnode.py 268
4739         while position < len(data):
4740             entries, position = split_netstring(data, 1, position)
4741             entry = entries[0]
4742-            (name, ro_uri, rwcapdata, metadata_s), subpos = split_netstring(entry, 4)
4743+            (name_utf8, ro_uri, rwcapdata, metadata_s), subpos = split_netstring(entry, 4)
4744             if not mutable and len(rwcapdata) > 0:
4745                 raise ValueError("the rwcapdata field of a dirnode in an immutable directory was not empty")
4746hunk ./src/allmydata/dirnode.py 271
4747-            name = name.decode("utf-8")
4748+            name = name_utf8.decode("utf-8")
4749             rw_uri = ""
4750             if writeable:
4751                 rw_uri = self._decrypt_rwcapdata(rwcapdata)
4752hunk ./src/allmydata/dirnode.py 278
4753 
4754             # Since the encryption uses CTR mode, it currently leaks the length of the
4755             # plaintext rw_uri -- and therefore whether it is present, i.e. whether the
4756-            # dirnode is writeable (ticket #925). By stripping spaces in Tahoe >= 1.6.0,
4757-            # we may make it easier for future versions to plug this leak.
4758+            # dirnode is writeable (ticket #925). By stripping trailing spaces in
4759+            # Tahoe >= 1.6.0, we may make it easier for future versions to plug this leak.
4760             # ro_uri is treated in the same way for consistency.
4761             # rw_uri and ro_uri will be either None or a non-empty string.
4762 
4763hunk ./src/allmydata/dirnode.py 283
4764-            rw_uri = rw_uri.strip(' ') or None
4765-            ro_uri = ro_uri.strip(' ') or None
4766+            rw_uri = rw_uri.rstrip(' ') or None
4767+            ro_uri = ro_uri.rstrip(' ') or None
4768 
4769             try:
4770                 child = self._create_and_validate_node(rw_uri, ro_uri, name)
4771hunk ./src/allmydata/dirnode.py 295
4772                     children.set_with_aux(name, (child, metadata), auxilliary=entry)
4773                 else:
4774                     log.msg(format="mutable cap for child '%(name)s' unpacked from an immutable directory",
4775-                                   name=name.encode("utf-8"),
4776+                                   name=name_utf8,
4777                                    facility="tahoe.webish", level=log.UNUSUAL)
4778             except CapConstraintError, e:
4779                 log.msg(format="unmet constraint on cap for child '%(name)s' unpacked from a directory:\n"
4780hunk ./src/allmydata/dirnode.py 299
4781-                               "%(message)s", message=e.args[0], name=name.encode("utf-8"),
4782+                               "%(message)s", message=e.args[0], name=name_utf8,
4783                                facility="tahoe.webish", level=log.UNUSUAL)
4784 
4785         return children
4786hunk ./src/allmydata/test/test_dirnode.py 16
4787 from allmydata.mutable.filenode import MutableFileNode
4788 from allmydata.mutable.common import UncoordinatedWriteError
4789 from allmydata.util import hashutil, base32
4790+from allmydata.util.netstring import split_netstring
4791 from allmydata.monitor import Monitor
4792 from allmydata.test.common import make_chk_file_uri, make_mutable_file_uri, \
4793      ErrorMixin
4794hunk ./src/allmydata/test/test_dirnode.py 52
4795         self.set_up_grid()
4796         c = self.g.clients[0]
4797         nm = c.nodemaker
4798+
4799         setup_py_uri = "URI:CHK:n7r3m6wmomelk4sep3kw5cvduq:os7ijw5c3maek7pg65e5254k2fzjflavtpejjyhshpsxuqzhcwwq:3:20:14861"
4800         one_uri = "URI:LIT:n5xgk" # LIT for "one"
4801         mut_write_uri = "URI:SSK:vfvcbdfbszyrsaxchgevhmmlii:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq"
4802hunk ./src/allmydata/test/test_dirnode.py 120
4803 
4804         bad_future_node = UnknownNode(future_write_uri, None)
4805         bad_kids1 = {u"one": (bad_future_node, {})}
4806+        # This should fail because we don't know how to diminish the future_write_uri
4807+        # cap (given in a write slot and not prefixed with "ro." or "imm.") to a readcap.
4808         d.addCallback(lambda ign:
4809                       self.shouldFail(MustNotBeUnknownRWError, "bad_kids1",
4810                                       "cannot attach unknown",
4811hunk ./src/allmydata/test/test_dirnode.py 140
4812         self.set_up_grid()
4813         c = self.g.clients[0]
4814         nm = c.nodemaker
4815+
4816         setup_py_uri = "URI:CHK:n7r3m6wmomelk4sep3kw5cvduq:os7ijw5c3maek7pg65e5254k2fzjflavtpejjyhshpsxuqzhcwwq:3:20:14861"
4817         one_uri = "URI:LIT:n5xgk" # LIT for "one"
4818         mut_write_uri = "URI:SSK:vfvcbdfbszyrsaxchgevhmmlii:euw4iw7bbnkrrwpzuburbhppuxhc3gwxv26f6imekhz7zyw2ojnq"
4819hunk ./src/allmydata/test/test_dirnode.py 288
4820         d.addCallback(_made_parent)
4821         return d
4822 
4823+    def test_spaces_are_stripped_on_the_way_out(self):
4824+        self.basedir = "dirnode/Dirnode/test_spaces_are_stripped_on_the_way_out"
4825+        self.set_up_grid()
4826+        c = self.g.clients[0]
4827+        nm = c.nodemaker
4828+
4829+        # This test checks that any trailing spaces in URIs are retained in the
4830+        # encoded directory, but stripped when we get them out of the directory.
4831+        # See ticket #925 for why we want that.
4832+
4833+        stripped_write_uri = "lafs://from_the_future\t"
4834+        stripped_read_uri = "lafs://readonly_from_the_future\t"
4835+        spacedout_write_uri = stripped_write_uri + "  "
4836+        spacedout_read_uri = stripped_read_uri + "  "
4837+
4838+        child = nm.create_from_cap(spacedout_write_uri, spacedout_read_uri)
4839+        self.failUnlessEqual(child.get_write_uri(), spacedout_write_uri)
4840+        self.failUnlessEqual(child.get_readonly_uri(), "ro." + spacedout_read_uri)
4841+
4842+        kids = {u"child": (child, {})}
4843+        d = c.create_dirnode(kids)
4844+       
4845+        def _created(dn):
4846+            self.failUnless(isinstance(dn, dirnode.DirectoryNode))
4847+            self.failUnless(dn.is_mutable())
4848+            self.failIf(dn.is_readonly())
4849+            dn.raise_error()
4850+            self.cap = dn.get_cap()
4851+            self.rootnode = dn
4852+            return dn._node.download_best_version()
4853+        d.addCallback(_created)
4854+
4855+        def _check_data(data):
4856+            # Decode the netstring representation of the directory to check that the
4857+            # spaces are retained when the URIs are stored.
4858+            position = 0
4859+            numkids = 0
4860+            while position < len(data):
4861+                entries, position = split_netstring(data, 1, position)
4862+                entry = entries[0]
4863+                (name_utf8, ro_uri, rwcapdata, metadata_s), subpos = split_netstring(entry, 4)
4864+                name = name_utf8.decode("utf-8")
4865+                rw_uri = self.rootnode._decrypt_rwcapdata(rwcapdata)
4866+                self.failUnless(name in kids)
4867+                (expected_child, ign) = kids[name]
4868+                self.failUnlessEqual(rw_uri, expected_child.get_write_uri())
4869+                self.failUnlessEqual("ro." + ro_uri, expected_child.get_readonly_uri())
4870+                numkids += 1
4871+
4872+            self.failUnlessEqual(numkids, 1)
4873+            return self.rootnode.list()
4874+        d.addCallback(_check_data)
4875+       
4876+        # Now when we use the real directory listing code, the trailing spaces
4877+        # should have been stripped (and "ro." should have been prepended to the
4878+        # ro_uri, since it's unknown).
4879+        def _check_kids(children):
4880+            self.failUnlessEqual(sorted(children.keys()), [u"child"])
4881+            child_node, child_metadata = children[u"child"]
4882+
4883+            self.failUnlessEqual(child_node.get_write_uri(), stripped_write_uri)
4884+            self.failUnlessEqual(child_node.get_readonly_uri(), "ro." + stripped_read_uri)
4885+        d.addCallback(_check_kids)
4886+
4887+        d.addCallback(lambda ign: nm.create_from_cap(self.cap.to_string()))
4888+        d.addCallback(lambda n: n.list())
4889+        d.addCallback(_check_kids)  # again with dirnode recreated from cap
4890+        return d
4891+
4892     def test_check(self):
4893         self.basedir = "dirnode/Dirnode/test_check"
4894         self.set_up_grid()
4895hunk ./src/allmydata/test/test_dirnode.py 1198
4896     def is_allowed_in_immutable_directory(self):
4897         return False
4898 
4899+    def raise_error(self):
4900+        pass
4901+
4902     def modify(self, modifier):
4903         self.data = modifier(self.data, None, True)
4904         return defer.succeed(None)
4905hunk ./src/allmydata/test/test_filenode.py 42
4906         self.failUnlessEqual(fn1.get_uri(), u.to_string())
4907         self.failUnlessEqual(fn1.get_cap(), u)
4908         self.failUnlessEqual(fn1.get_readcap(), u)
4909-        self.failUnlessEqual(fn1.is_readonly(), True)
4910-        self.failUnlessEqual(fn1.is_mutable(), False)
4911-        self.failUnlessEqual(fn1.is_unknown(), False)
4912-        self.failUnlessEqual(fn1.is_allowed_in_immutable_directory(), True)
4913+        self.failUnless(fn1.is_readonly())
4914+        self.failIf(fn1.is_mutable())
4915+        self.failIf(fn1.is_unknown())
4916+        self.failUnless(fn1.is_allowed_in_immutable_directory())
4917         self.failUnlessEqual(fn1.get_write_uri(), None)
4918         self.failUnlessEqual(fn1.get_readonly_uri(), u.to_string())
4919         self.failUnlessEqual(fn1.get_size(), 1000)
4920hunk ./src/allmydata/test/test_filenode.py 57
4921         v = fn1.get_verify_cap()
4922         self.failUnless(isinstance(v, uri.CHKFileVerifierURI))
4923         self.failUnlessEqual(fn1.get_repair_cap(), v)
4924-        self.failUnlessEqual(v.is_readonly(), True)
4925-        self.failUnlessEqual(v.is_mutable(), False)
4926+        self.failUnless(v.is_readonly())
4927+        self.failIf(v.is_mutable())
4928 
4929 
4930     def test_literal_filenode(self):
4931hunk ./src/allmydata/test/test_filenode.py 72
4932         self.failUnlessEqual(fn1.get_uri(), u.to_string())
4933         self.failUnlessEqual(fn1.get_cap(), u)
4934         self.failUnlessEqual(fn1.get_readcap(), u)
4935-        self.failUnlessEqual(fn1.is_readonly(), True)
4936-        self.failUnlessEqual(fn1.is_mutable(), False)
4937-        self.failUnlessEqual(fn1.is_unknown(), False)
4938-        self.failUnlessEqual(fn1.is_allowed_in_immutable_directory(), True)
4939+        self.failUnless(fn1.is_readonly())
4940+        self.failIf(fn1.is_mutable())
4941+        self.failIf(fn1.is_unknown())
4942+        self.failUnless(fn1.is_allowed_in_immutable_directory())
4943         self.failUnlessEqual(fn1.get_write_uri(), None)
4944         self.failUnlessEqual(fn1.get_readonly_uri(), u.to_string())
4945         self.failUnlessEqual(fn1.get_size(), len(DATA))
4946hunk ./src/allmydata/test/test_filenode.py 125
4947         self.failUnlessEqual(n.get_readonly_uri(), u.get_readonly().to_string())
4948         self.failUnlessEqual(n.get_cap(), u)
4949         self.failUnlessEqual(n.get_readcap(), u.get_readonly())
4950-        self.failUnlessEqual(n.is_mutable(), True)
4951-        self.failUnlessEqual(n.is_readonly(), False)
4952-        self.failUnlessEqual(n.is_unknown(), False)
4953-        self.failUnlessEqual(n.is_allowed_in_immutable_directory(), False)
4954+        self.failUnless(n.is_mutable())
4955+        self.failIf(n.is_readonly())
4956+        self.failIf(n.is_unknown())
4957+        self.failIf(n.is_allowed_in_immutable_directory())
4958         n.raise_error()
4959 
4960         n2 = MutableFileNode(None, None, client.get_encoding_parameters(),
4961hunk ./src/allmydata/test/test_filenode.py 147
4962         self.failUnlessEqual(nro.get_readonly(), nro)
4963         self.failUnlessEqual(nro.get_cap(), u.get_readonly())
4964         self.failUnlessEqual(nro.get_readcap(), u.get_readonly())
4965-        self.failUnlessEqual(nro.is_mutable(), True)
4966-        self.failUnlessEqual(nro.is_readonly(), True)
4967-        self.failUnlessEqual(nro.is_unknown(), False)
4968-        self.failUnlessEqual(nro.is_allowed_in_immutable_directory(), False)
4969+        self.failUnless(nro.is_mutable())
4970+        self.failUnless(nro.is_readonly())
4971+        self.failIf(nro.is_unknown())
4972+        self.failIf(nro.is_allowed_in_immutable_directory())
4973         nro_u = nro.get_uri()
4974         self.failUnlessEqual(nro_u, nro.get_readonly_uri())
4975         self.failUnlessEqual(nro_u, u.get_readonly().to_string())
4976hunk ./src/allmydata/test/test_web.py 2379
4977     def test_POST_set_children_with_hyphen(self):
4978         return self.test_POST_set_children(command_name="set-children")
4979 
4980-    def test_POST_put_uri(self):
4981+    def test_POST_link_uri(self):
4982         contents, n, newuri = self.makefile(8)
4983         d = self.POST(self.public_url + "/foo", t="uri", name="new.txt", uri=newuri)
4984         d.addCallback(self.failUnlessURIMatchesROChild, self._foo_node, u"new.txt")
4985hunk ./src/allmydata/test/test_web.py 2388
4986                                                       contents))
4987         return d
4988 
4989-    def test_POST_put_uri_replace(self):
4990+    def test_POST_link_uri_replace(self):
4991         contents, n, newuri = self.makefile(8)
4992         d = self.POST(self.public_url + "/foo", t="uri", name="bar.txt", uri=newuri)
4993         d.addCallback(self.failUnlessURIMatchesROChild, self._foo_node, u"bar.txt")
4994hunk ./src/allmydata/test/test_web.py 2397
4995                                                       contents))
4996         return d
4997 
4998-    def test_POST_put_uri_no_replace_queryarg(self):
4999+    def test_POST_link_uri_unknown_bad(self):
5000+        newuri = "lafs://from_the_future"
5001+        d = self.POST(self.public_url + "/foo", t="uri", name="future.txt", uri=newuri)
5002+        d.addBoth(self.shouldFail, error.Error,
5003+                  "POST_link_uri_unknown_bad",
5004+                  "400 Bad Request",
5005+                  "unknown cap in a write slot")
5006+        return d
5007+
5008+    def test_POST_link_uri_unknown_ro_good(self):
5009+        newuri = "ro.lafs://readonly_from_the_future"
5010+        d = self.POST(self.public_url + "/foo", t="uri", name="future-ro.txt", uri=newuri)
5011+        d.addCallback(self.failUnlessURIMatchesROChild, self._foo_node, u"future-ro.txt")
5012+        return d
5013+
5014+    def test_POST_link_uri_unknown_imm_good(self):
5015+        newuri = "imm.lafs://immutable_from_the_future"
5016+        d = self.POST(self.public_url + "/foo", t="uri", name="future-imm.txt", uri=newuri)
5017+        d.addCallback(self.failUnlessURIMatchesROChild, self._foo_node, u"future-imm.txt")
5018+        return d
5019+
5020+    def test_POST_link_uri_no_replace_queryarg(self):
5021         contents, n, newuri = self.makefile(8)
5022         d = self.POST(self.public_url + "/foo?replace=false", t="uri",
5023                       name="bar.txt", uri=newuri)
5024hunk ./src/allmydata/test/test_web.py 2423
5025         d.addBoth(self.shouldFail, error.Error,
5026-                  "POST_put_uri_no_replace_queryarg",
5027+                  "POST_link_uri_no_replace_queryarg",
5028                   "409 Conflict",
5029                   "There was already a child by that name, and you asked me "
5030                   "to not replace it")
5031hunk ./src/allmydata/test/test_web.py 2431
5032         d.addCallback(self.failUnlessIsBarDotTxt)
5033         return d
5034 
5035-    def test_POST_put_uri_no_replace_field(self):
5036+    def test_POST_link_uri_no_replace_field(self):
5037         contents, n, newuri = self.makefile(8)
5038         d = self.POST(self.public_url + "/foo", t="uri", replace="false",
5039                       name="bar.txt", uri=newuri)
5040hunk ./src/allmydata/test/test_web.py 2436
5041         d.addBoth(self.shouldFail, error.Error,
5042-                  "POST_put_uri_no_replace_field",
5043+                  "POST_link_uri_no_replace_field",
5044                   "409 Conflict",
5045                   "There was already a child by that name, and you asked me "
5046                   "to not replace it")
5047hunk ./src/allmydata/test/test_web.py 2704
5048                   "to not replace it")
5049         return d
5050 
5051+    def test_PUT_NEWFILEURL_uri_unknown_bad(self):
5052+        new_uri = "lafs://from_the_future"
5053+        d = self.PUT(self.public_url + "/foo/put-future.txt?t=uri", new_uri)
5054+        d.addBoth(self.shouldFail, error.Error,
5055+                  "POST_put_uri_unknown_bad",
5056+                  "400 Bad Request",
5057+                  "unknown cap in a write slot")
5058+        return d
5059+
5060+    def test_PUT_NEWFILEURL_uri_unknown_ro_good(self):
5061+        new_uri = "ro.lafs://readonly_from_the_future"
5062+        d = self.PUT(self.public_url + "/foo/put-future-ro.txt?t=uri", new_uri)
5063+        d.addCallback(self.failUnlessURIMatchesROChild, self._foo_node,
5064+                      u"put-future-ro.txt")
5065+        return d
5066+
5067+    def test_PUT_NEWFILEURL_uri_unknown_imm_good(self):
5068+        new_uri = "imm.lafs://immutable_from_the_future"
5069+        d = self.PUT(self.public_url + "/foo/put-future-imm.txt?t=uri", new_uri)
5070+        d.addCallback(self.failUnlessURIMatchesROChild, self._foo_node,
5071+                      u"put-future-imm.txt")
5072+        return d
5073+
5074     def test_PUT_NEWFILE_URI(self):
5075         file_contents = "New file contents here\n"
5076         d = self.PUT("/uri", file_contents)
5077hunk ./src/allmydata/test/test_web.py 3407
5078             while position < len(data):
5079                 entries, position = split_netstring(data, 1, position)
5080                 entry = entries[0]
5081-                (name, ro_uri, rwcapdata, metadata_s), subpos = split_netstring(entry, 4)
5082-                name = name.decode("utf-8")
5083+                (name_utf8, ro_uri, rwcapdata, metadata_s), subpos = split_netstring(entry, 4)
5084+                name = name_utf8.decode("utf-8")
5085                 self.failUnless(rwcapdata == "")
5086hunk ./src/allmydata/test/test_web.py 3410
5087-                ro_uri = ro_uri.strip()
5088-                if name in kids:
5089-                    self.failIfEqual(ro_uri, "")
5090-                    (expected_child, ign) = kids[name]
5091-                    self.failUnlessEqual(ro_uri, expected_child.get_readonly_uri())
5092-                    numkids += 1
5093+                self.failUnless(name in kids)
5094+                (expected_child, ign) = kids[name]
5095+                self.failUnlessEqual(ro_uri, expected_child.get_readonly_uri())
5096+                numkids += 1
5097 
5098             self.failUnlessEqual(numkids, 3)
5099             return self.rootnode.list()
5100}
5101
5102Context:
5103
5104[test_runner: cleanup, refactor common code into a non-executable method
5105Brian Warner <warner@lothar.com>**20100127224040
5106 Ignore-this: 4cb4aada87777771f688edfd8129ffca
5107 
5108 Having both test_node() and test_client() (one of which calls the other) felt
5109 confusing to me, so I changed it to have test_node(), test_client(), and a
5110 common do_create() helper method.
5111]
5112[scripts/runner.py: simplify David-Sarah's clever grouped-commands usage trick
5113Brian Warner <warner@lothar.com>**20100127223758
5114 Ignore-this: 70877ebf06ae59f32960b0aa4ce1d1ae
5115]
5116[tahoe backup: skip all symlinks, with warning. Fixes #850, addresses #641.
5117Brian Warner <warner@lothar.com>**20100127223517
5118 Ignore-this: ab5cf05158d32a575ca8efc0f650033f
5119]
5120[NEWS: update with all recent user-visible changes
5121Brian Warner <warner@lothar.com>**20100127222209
5122 Ignore-this: 277d24568018bf4f3fb7736fda64eceb
5123]
5124["tahoe backup": fix --exclude-vcs docs to include Git
5125Brian Warner <warner@lothar.com>**20100127201044
5126 Ignore-this: 756a58dde21bdc65aa62b81803605b5
5127]
5128[docs: fix references to --no-storage, explanation of [storage] section
5129Brian Warner <warner@lothar.com>**20100127200956
5130 Ignore-this: f4be1763a585e1ac6299a4f1b94a59e0
5131]
5132[cli: merge the better version of David-Sarah's split-usage-and-help patch with the earlier version that I mistakenly committed
5133zooko@zooko.com**20100126044559
5134 Ignore-this: 284d188e13b7901013cbb650168e6447
5135]
5136[Split tahoe --help options into groups.
5137david-sarah@jacaranda.org**20100112043935
5138 Ignore-this: 610f9c41b00e6863e3cd047379733e3a
5139]
5140[docs: further CREDITS level-ups for Nils, Kevan, David-Sarah
5141zooko@zooko.com**20100126170021
5142 Ignore-this: 1e513e85cf7b7abf57f056e6d7544b38
5143]
5144[ftpd: clearer error message if Twisted needs a patch (by Nils Durner)
5145zooko@zooko.com**20100126143411
5146 Ignore-this: 440e6831ae6da5135c1edd081c93871f
5147]
5148[Add 'docs/performance.txt', which (for the moment) describes mutable file performance issues
5149Kevan Carstensen <kevan@isnotajoke.com>**20100115204500
5150 Ignore-this: ade4e500217db2509aee35aacc8c5dbf
5151]
5152[docs: more CREDITS for François, Kevan, and David-Sarah
5153zooko@zooko.com**20100126132133
5154 Ignore-this: f37d4977c13066fcac088ba98a31b02e
5155]
5156[tahoe_backup.py: display warnings on errors instead of stopping the whole backup. Fix #729.
5157francois@ctrlaltdel.ch**20100120094249
5158 Ignore-this: 7006ea4b0910b6d29af6ab4a3997a8f9
5159 
5160 This patch displays a warning to the user in two cases:
5161   
5162   1. When special files like symlinks, fifos, devices, etc. are found in the
5163      local source.
5164   
5165   2. If files or directories are not readables by the user running the 'tahoe
5166      backup' command.
5167 
5168 In verbose mode, the number of skipped files and directories is printed at the
5169 end of the backup.
5170 
5171 Exit status returned by 'tahoe backup':
5172 
5173   - 0 everything went fine
5174   - 1 the backup failed
5175   - 2 files were skipped during the backup
5176 
5177]
5178[Message saying that we couldn't find bin/tahoe should say where we looked
5179david-sarah@jacaranda.org**20100116204556
5180 Ignore-this: 1068576fd59ea470f1e19196315d1bb
5181]
5182[Change running.html to describe 'tahoe run'
5183david-sarah@jacaranda.org**20100112044409
5184 Ignore-this: 23ad0114643ce31b56e19bb14e011e4f
5185]
5186[cli: split usage strings into groups (patch by David-Sarah Hopwood)
5187zooko@zooko.com**20100126043921
5188 Ignore-this: 51928d266a7292b873f87f7d53c9a01e
5189]
5190[Add create-node CLI command, and make create-client equivalent to create-node --no-storage (fixes #760)
5191david-sarah@jacaranda.org**20100116052055
5192 Ignore-this: 47d08b18c69738685e13ff365738d5a
5193]
5194[contrib/fuse/runtests.py: Fix #888, configure settings in tahoe.cfg and don't treat warnings as failure
5195francois@ctrlaltdel.ch**20100109123010
5196 Ignore-this: 2590d44044acd7dfa3690c416cae945c
5197 
5198 Fix a few bitrotten pieces in the FUSE test script.  It now configures tahoe
5199 node settings by editing tahoe.cfg which is the new supported method.
5200 
5201 It alos tolerate warnings issued by the mount command, the cause of these
5202 warnings is the same as in #876 (contrib/fuse/runtests.py doesn't tolerate
5203 deprecations warnings).
5204 
5205]
5206[Fix webapi t=mkdir with multpart/form-data, as on the Welcome page. Closes #919.
5207Brian Warner <warner@lothar.com>**20100121065052
5208 Ignore-this: 1f20ea0a0f1f6d6c1e8e14f193a92c87
5209]
5210[Fix boodlegrid use of set_children
5211david-sarah@jacaranda.org**20100126063414
5212 Ignore-this: 3aa2d4836f76303b2bacecd09611f999
5213]
5214[Remove replace= parameter to mkdir-immutable and mkdir-with-children
5215david-sarah@jacaranda.org**20100124224325
5216 Ignore-this: 25207bcc946c0c43d9528718e76ba7b
5217]
5218[Warn about test failures due to setting FLOG* env vars
5219david-sarah@jacaranda.org**20100124220629
5220 Ignore-this: 1c25247ca0f0840390a1b7259a9f4a3c
5221]
5222[Patch to accept t=set-children as well as t=set_children
5223david-sarah@jacaranda.org**20100124030020
5224 Ignore-this: 2c061f12af817cdf77feeeb64098ec3a
5225]
5226[tahoe_add_alias.py: minor refactoring
5227Brian Warner <warner@lothar.com>**20100115064220
5228 Ignore-this: 29910e81ad11209c9e493d65fd2dab9b
5229]
5230[test_dirnode.py: reduce scope of a Client instance, suggested by Kevan.
5231Brian Warner <warner@lothar.com>**20100115062713
5232 Ignore-this: b35efd9e6027e43de6c6f509bfb4ccaa
5233]
5234[test_provisioning: STAN is not always a list. Fix by David-Sarah Hopwood.
5235Brian Warner <warner@lothar.com>**20100115014632
5236 Ignore-this: 9989de7f1e00907706d2b63153138219
5237]
5238[web/directory.py mkdir-immutable: hush pyflakes, add TODO for #903 behavior
5239Brian Warner <warner@lothar.com>**20100114222804
5240 Ignore-this: 717cd3b9a1c8aeee76938c9641db7356
5241]
5242[hush pyflakes-0.4.0 warnings: slightly less-trivial fixes. Closes #900.
5243Brian Warner <warner@lothar.com>**20100114221719
5244 Ignore-this: f774f4637e256ad55502659413a811a8
5245 
5246 This includes one fix (in test_web) which was testing the wrong thing.
5247]
5248[hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900.
5249Brian Warner <warner@lothar.com>**20100114221529
5250 Ignore-this: e96106c8f1a99fbf93306fbfe9a294cf
5251]
5252[tahoe add-alias/create-alias: don't corrupt non-newline-terminated alias
5253Brian Warner <warner@lothar.com>**20100114210246
5254 Ignore-this: 9c994792e53a85159d708760a9b1b000
5255 file. Closes #741.
5256]
5257[change docs and --help to use "grid" instead of "virtual drive": closes #892.
5258Brian Warner <warner@lothar.com>**20100114201119
5259 Ignore-this: a20d4a4dcc4de4e3b404ff72d40fc29b
5260 
5261 Thanks to David-Sarah Hopwood for the patch.
5262]
5263[backupdb.txt: fix ST_CTIME reference
5264Brian Warner <warner@lothar.com>**20100114194052
5265 Ignore-this: 5a189c7a1181b07dd87f0a08ea31b6d3
5266]
5267[client.py: fix/update comments on KeyGenerator
5268Brian Warner <warner@lothar.com>**20100113004226
5269 Ignore-this: 2208adbb3fd6a911c9f44e814583cabd
5270]
5271[Clean up log.err calls, for one of the issues in #889.
5272Brian Warner <warner@lothar.com>**20100112013343
5273 Ignore-this: f58455ce15f1fda647c5fb25d234d2db
5274 
5275 allmydata.util.log.err() either takes a Failure as the first positional
5276 argument, or takes no positional arguments and must be invoked in an
5277 exception handler. Fixed its signature to match both foolscap.logging.log.err
5278 and twisted.python.log.err . Included a brief unit test.
5279]
5280[tidy up DeadReferenceError handling, ignore them in add_lease calls
5281Brian Warner <warner@lothar.com>**20100112000723
5282 Ignore-this: 72f1444e826fd0b9db6d318f89603c38
5283 
5284 Stop checking separately for ConnectionDone/ConnectionLost, since those have
5285 been folded into DeadReferenceError since foolscap-0.3.1 . Write
5286 rrefutil.trap_deadref() in terms of rrefutil.trap_and_discard() to improve
5287 code coverage.
5288]
5289[NEWS: improve "tahoe backup" notes, mention first-backup-after-upgrade duration
5290Brian Warner <warner@lothar.com>**20100111190132
5291 Ignore-this: 10347c590b3375964579ba6c2b0edb4f
5292 
5293 Thanks to Francois Deppierraz for the suggestion.
5294]
5295[test_repairer: add (commented-out) test_each_byte, to see exactly what the
5296Brian Warner <warner@lothar.com>**20100110203552
5297 Ignore-this: 8e84277d5304752edeff052b97821815
5298 Verifier misses
5299 
5300 The results (described in #819) match our expectations: it misses corruption
5301 in unused share fields and in most container fields (which are only visible
5302 to the storage server, not the client). 1265 bytes of a 2753 byte
5303 share (hosting a 56-byte file with an artifically small segment size) are
5304 unused, mostly in the unused tail of the overallocated UEB space (765 bytes),
5305 and the allocated-but-unwritten plaintext_hash_tree (480 bytes).
5306]
5307[repairer: fix some wrong offsets in the randomized verifier tests, debugged by Brian
5308zooko@zooko.com**20100110203721
5309 Ignore-this: 20604a609db8706555578612c1c12feb
5310 fixes #819
5311]
5312[test_repairer: fix colliding basedir names, which caused test inconsistencies
5313Brian Warner <warner@lothar.com>**20100110084619
5314 Ignore-this: b1d56dd27e6ab99a7730f74ba10abd23
5315]
5316[repairer: add deterministic test for #819, mark as TODO
5317zooko@zooko.com**20100110013619
5318 Ignore-this: 4cb8bb30b25246de58ed2b96fa447d68
5319]
5320[contrib/fuse/runtests.py: Tolerate the tahoe CLI returning deprecation warnings
5321francois@ctrlaltdel.ch**20100109175946
5322 Ignore-this: 419c354d9f2f6eaec03deb9b83752aee
5323 
5324 Depending on the versions of external libraries such as Twisted of Foolscap,
5325 the tahoe CLI can display deprecation warnings on stdout.  The tests should
5326 not interpret those warnings as a failure if the node is in fact correctly
5327 started.
5328   
5329 See http://allmydata.org/trac/tahoe/ticket/859 for an example of deprecation
5330 warnings.
5331 
5332 fixes #876
5333]
5334[contrib: fix fuse_impl_c to use new Python API
5335zooko@zooko.com**20100109174956
5336 Ignore-this: 51ca1ec7c2a92a0862e9b99e52542179
5337 original patch by Thomas Delaet, fixed by François, reviewed by Brian, committed by me
5338]
5339[docs: CREDITS: add David-Sarah to the CREDITS file
5340zooko@zooko.com**20100109060435
5341 Ignore-this: 896062396ad85f9d2d4806762632f25a
5342]
5343[mutable/publish: don't loop() right away upon DeadReferenceError. Closes #877
5344Brian Warner <warner@lothar.com>**20100102220841
5345 Ignore-this: b200e707b3f13aa8251981362b8a3e61
5346 
5347 The bug was that a disconnected server could cause us to re-enter the initial
5348 loop() call, sending multiple queries to a single server, provoking an
5349 incorrect UCWE. To fix it, stall the loop() with an eventual.fireEventually()
5350]
5351[immutable/checker.py: oops, forgot some imports. Also hush pyflakes.
5352Brian Warner <warner@lothar.com>**20091229233909
5353 Ignore-this: 4d61bd3f8113015a4773fd4768176e51
5354]
5355[mutable repair: return successful=False when numshares<k (thus repair fails),
5356Brian Warner <warner@lothar.com>**20091229233746
5357 Ignore-this: d881c3275ff8c8bee42f6a80ca48441e
5358 instead of weird errors. Closes #874 and #786.
5359 
5360 Previously, if the file had 0 shares, this would raise TypeError as it tried
5361 to call download_version(None). If the file had some shares but fewer than
5362 'k', it would incorrectly raise MustForceRepairError.
5363 
5364 Added get_successful() to the IRepairResults API, to give repair() a place to
5365 report non-code-bug problems like this.
5366]
5367[node.py/interfaces.py: minor docs fixes
5368Brian Warner <warner@lothar.com>**20091229230409
5369 Ignore-this: c86ad6342ef0f95d50639b4f99cd4ddf
5370]
5371[NEWS: fix 1.4.1 announcement w.r.t. add-lease behavior in older releases
5372Brian Warner <warner@lothar.com>**20091229230310
5373 Ignore-this: bbbbb9c961f3bbcc6e5dbe0b1594822
5374]
5375[checker: don't let failures in add-lease affect checker results. Closes #875.
5376Brian Warner <warner@lothar.com>**20091229230108
5377 Ignore-this: ef1a367b93e4d01298c2b1e6ca59c492
5378 
5379 Mutable servermap updates and the immutable checker, when run with
5380 add_lease=True, send both the do-you-have-block and add-lease commands in
5381 parallel, to avoid an extra round trip time. Many older servers have problems
5382 with add-lease and raise various exceptions, which don't generally matter.
5383 The client-side code was catching+ignoring some of them, but unrecognized
5384 exceptions were passed through to the DYHB code, concealing the DYHB results
5385 from the checker, making it think the server had no shares.
5386 
5387 The fix is to separate the code paths. Both commands are sent at the same
5388 time, but the errback path from add-lease is handled separately. Known
5389 exceptions are ignored, the others (both unknown-remote and all-local) are
5390 logged (log.WEIRD, which will trigger an Incident), but neither will affect
5391 the DYHB results.
5392 
5393 The add-lease message is sent first, and we know that the server handles them
5394 synchronously. So when the checker is done, we can be sure that all the
5395 add-lease messages have been retired. This makes life easier for unit tests.
5396]
5397[test_cli: verify fix for "tahoe get" not creating empty file on error (#121)
5398Brian Warner <warner@lothar.com>**20091227235444
5399 Ignore-this: 6444d52413b68eb7c11bc3dfdc69c55f
5400]
5401[addendum to "Fix 'tahoe ls' on files (#771)"
5402Brian Warner <warner@lothar.com>**20091227232149
5403 Ignore-this: 6dd5e25f8072a3153ba200b7fdd49491
5404 
5405 tahoe_ls.py: tolerate missing metadata
5406 web/filenode.py: minor cleanups
5407 test_cli.py: test 'tahoe ls FILECAP'
5408]
5409[Fix 'tahoe ls' on files (#771). Patch adapted from Kevan Carstensen.
5410Brian Warner <warner@lothar.com>**20091227225443
5411 Ignore-this: 8bf8c7b1cd14ea4b0ebd453434f4fe07
5412 
5413 web/filenode.py: also serve edge metadata when using t=json on a
5414                  DIRCAP/childname object.
5415 tahoe_ls.py: list file objects as if we were listing one-entry directories.
5416              Show edge metadata if we have it, which will be true when doing
5417              'tahoe ls DIRCAP/filename' and false when doing 'tahoe ls
5418              FILECAP'
5419]
5420[tahoe_get: don't create the output file on error. Closes #121.
5421Brian Warner <warner@lothar.com>**20091227220404
5422 Ignore-this: 58d5e793a77ec6e87d9394ade074b926
5423]
5424[webapi: don't accept zero-length childnames during traversal. Closes #358, #676.
5425Brian Warner <warner@lothar.com>**20091227201043
5426 Ignore-this: a9119dec89e1c7741f2289b0cad6497b
5427 
5428 This forbids operations that would implicitly create a directory with a
5429 zero-length (empty string) name, like what you'd get if you did "tahoe put
5430 local /oops/blah" (#358) or "POST /uri/CAP//?t=mkdir" (#676). The error
5431 message is fairly friendly too.
5432 
5433 Also added code to "tahoe put" to catch this error beforehand and suggest the
5434 correct syntax (i.e. without the leading slash).
5435]
5436[CLI: send 'Accept:' header to ask for text/plain tracebacks. Closes #646.
5437Brian Warner <warner@lothar.com>**20091227195828
5438 Ignore-this: 44c258d4d4c7dac0ed58adb22f73331
5439 
5440 The webapi has been looking for an Accept header since 1.4.0, but it treats a
5441 missing header as equal to */* (to honor RFC2616). This change finally
5442 modifies our CLI tools to ask for "text/plain, application/octet-stream",
5443 which seems roughly correct (we either want a plain-text traceback or error
5444 message, or an uninterpreted chunk of binary data to save to disk). Some day
5445 we'll figure out how JSON fits into this scheme.
5446]
5447[Makefile: upload-tarballs: switch from xfer-client to flappclient, closes #350
5448Brian Warner <warner@lothar.com>**20091227163703
5449 Ignore-this: 3beeecdf2ad9c2438ab57f0e33dcb357
5450 
5451 I've also set up a new flappserver on source@allmydata.org to receive the
5452 tarballs. We still need to replace the gutsy buildslave (which is where the
5453 tarballs used to be generated+uploaded) and give it the new FURL.
5454]
5455[misc/ringsim.py: make it deterministic, more detail about grid-is-full behavior
5456Brian Warner <warner@lothar.com>**20091227024832
5457 Ignore-this: a691cc763fb2e98a4ce1767c36e8e73f
5458]
5459[misc/ringsim.py: tool to discuss #302
5460Brian Warner <warner@lothar.com>**20091226060339
5461 Ignore-this: fc171369b8f0d97afeeb8213e29d10ed
5462]
5463[docs: fix helper.txt to describe new config style
5464zooko@zooko.com**20091224223522
5465 Ignore-this: 102e7692dc414a4b466307f7d78601fe
5466]
5467[docs/stats.txt: add TOC, notes about controlling gatherer's listening port
5468Brian Warner <warner@lothar.com>**20091224202133
5469 Ignore-this: 8eef63b0e18db5aa8249c2eafde02c05
5470 
5471 Thanks to Jody Harris for the suggestions.
5472]
5473[Add docs/stats.py, explaining Tahoe stats, the gatherer, and the munin plugins.
5474Brian Warner <warner@lothar.com>**20091223052400
5475 Ignore-this: 7c9eeb6e5644eceda98b59a67730ccd5
5476]
5477[more #859: avoid deprecation warning for unit tests too, hush pyflakes
5478Brian Warner <warner@lothar.com>**20091215000147
5479 Ignore-this: 193622e24d31077da825a11ed2325fd3
5480 
5481 * factor maybe-import-sha logic into util.hashutil
5482]
5483[use hashlib module if available, thus avoiding a DeprecationWarning for importing the old sha module; fixes #859
5484zooko@zooko.com**20091214212703
5485 Ignore-this: 8d0f230a4bf8581dbc1b07389d76029c
5486]
5487[docs: reflow architecture.txt to 78-char lines
5488zooko@zooko.com**20091208232943
5489 Ignore-this: 88f55166415f15192e39407815141f77
5490]
5491[docs: update the about.html a little
5492zooko@zooko.com**20091208212737
5493 Ignore-this: 3fe2d9653c6de0727d3e82bd70f2a8ed
5494]
5495[docs: remove obsolete doc file "codemap.txt"
5496zooko@zooko.com**20091113163033
5497 Ignore-this: 16bc21a1835546e71d1b344c06c61ebb
5498 I started to update this to reflect the current codebase, but then I thought (a) nobody seemed to notice that it hasn't been updated since December 2007, and (b) it will just bit-rot again, so I'm removing it.
5499]
5500[mutable/retrieve.py: stop reaching into private MutableFileNode attributes
5501Brian Warner <warner@lothar.com>**20091208172921
5502 Ignore-this: 61e548798c1105aed66a792bf26ceef7
5503]
5504[mutable/servermap.py: stop reaching into private MutableFileNode attributes
5505Brian Warner <warner@lothar.com>**20091208172608
5506 Ignore-this: b40a6b62f623f9285ad96fda139c2ef2
5507]
5508[mutable/servermap.py: oops, query N+e servers in MODE_WRITE, not k+e
5509Brian Warner <warner@lothar.com>**20091208171156
5510 Ignore-this: 3497f4ab70dae906759007c3cfa43bc
5511 
5512 under normal conditions, this wouldn't cause any problems, but if the shares
5513 are really sparse (perhaps because new servers were added), then
5514 file-modifies might stop looking too early and leave old shares in place
5515]
5516[control.py: fix speedtest: use download_best_version (not read) on mutable nodes
5517Brian Warner <warner@lothar.com>**20091207060512
5518 Ignore-this: 7125eabfe74837e05f9291dd6414f917
5519]
5520[FTP-and-SFTP.txt: fix ssh-keygen pointer
5521Brian Warner <warner@lothar.com>**20091207052803
5522 Ignore-this: bc2a70ee8c58ec314e79c1262ccb22f7
5523]
5524[setup: ignore _darcs in the "test-clean" test and make the "clean" step remove all .egg's in the root dir
5525zooko@zooko.com**20091206184835
5526 Ignore-this: 6066bd160f0db36d7bf60aba405558d2
5527]
5528[remove MutableFileNode.download(), prefer download_best_version() instead
5529Brian Warner <warner@lothar.com>**20091201225438
5530 Ignore-this: 5733eb373a902063e09fd52cc858dec0
5531]
5532[Simplify immutable download API: use just filenode.read(consumer, offset, size)
5533Brian Warner <warner@lothar.com>**20091201225330
5534 Ignore-this: bdedfb488ac23738bf52ae6d4ab3a3fb
5535 
5536 * remove Downloader.download_to_data/download_to_filename/download_to_filehandle
5537 * remove download.Data/FileName/FileHandle targets
5538 * remove filenode.download/download_to_data/download_to_filename methods
5539 * leave Downloader.download (the whole Downloader will go away eventually)
5540 * add util.consumer.MemoryConsumer/download_to_data, for convenience
5541   (this is mostly used by unit tests, but it gets used by enough non-test
5542    code to warrant putting it in allmydata.util)
5543 * update tests
5544 * removes about 180 lines of code. Yay negative code days!
5545 
5546 Overall plan is to rewrite immutable/download.py and leave filenode.read() as
5547 the sole read-side API.
5548]
5549[server.py: undo my bogus 'correction' of David-Sarah's comment fix
5550Brian Warner <warner@lothar.com>**20091201024607
5551 Ignore-this: ff4bb58f6a9e045b900ac3a89d6f506a
5552 
5553 and move it to a better line
5554]
5555[Implement more coherent behavior when copying with dircaps/filecaps (closes #761). Patch by Kevan Carstensen.
5556"Brian Warner <warner@lothar.com>"**20091130211009]
5557[storage.py: update comment
5558"Brian Warner <warner@lothar.com>"**20091130195913]
5559[storage server: detect disk space usage on Windows too (fixes #637)
5560david-sarah@jacaranda.org**20091121055644
5561 Ignore-this: 20fb30498174ce997befac7701fab056
5562]
5563[make status of finished operations consistently "Finished"
5564david-sarah@jacaranda.org**20091121061543
5565 Ignore-this: 97d483e8536ccfc2934549ceff7055a3
5566]
5567[NEWS: update with all user-visible changes since the last release
5568Brian Warner <warner@lothar.com>**20091127224217
5569 Ignore-this: 741da6cd928e939fb6d21a61ea3daf0b
5570]
5571[update "tahoe backup" docs, and webapi.txt's mkdir-with-children
5572Brian Warner <warner@lothar.com>**20091127055900
5573 Ignore-this: defac1fb9a2335b0af3ef9dbbcc67b7e
5574]
5575[Add dirnodes to backupdb and "tahoe backup", closes #606.
5576Brian Warner <warner@lothar.com>**20091126234257
5577 Ignore-this: fa88796fcad1763c6a2bf81f56103223
5578 
5579 * backups now share dirnodes with any previous backup, in any location,
5580   so renames and moves are handled very efficiently
5581 * "tahoe backup" no longer bothers reading the previous snapshot
5582 * if you switch grids, you should delete ~/.tahoe/private/backupdb.sqlite,
5583   to force new uploads of all files and directories
5584]
5585[webapi: fix t=check for DIR2-LIT (i.e. empty immutable directories)
5586Brian Warner <warner@lothar.com>**20091126232731
5587 Ignore-this: 8513c890525c69c1eca0e80d53a231f8
5588]
5589[PipelineError: fix str() on python2.4 . Closes #842.
5590Brian Warner <warner@lothar.com>**20091124212512
5591 Ignore-this: e62c92ea9ede2ab7d11fe63f43b9c942
5592]
5593[test_uri.py: s/NewDirnode/Dirnode/ , now that they aren't "new" anymore
5594Brian Warner <warner@lothar.com>**20091120075553
5595 Ignore-this: 61c8ef5e45a9d966873a610d8349b830
5596]
5597[interface name cleanups: IFileNode, IImmutableFileNode, IMutableFileNode
5598Brian Warner <warner@lothar.com>**20091120075255
5599 Ignore-this: e3d193c229e2463e1d0b0c92306de27f
5600 
5601 The proper hierarchy is:
5602  IFilesystemNode
5603  +IFileNode
5604  ++IMutableFileNode
5605  ++IImmutableFileNode
5606  +IDirectoryNode
5607 
5608 Also expand test_client.py (NodeMaker) to hit all IFilesystemNode types.
5609]
5610[class name cleanups: s/FileNode/ImmutableFileNode/
5611Brian Warner <warner@lothar.com>**20091120072239
5612 Ignore-this: 4b3218f2d0e585c62827e14ad8ed8ac1
5613 
5614 also fix test/bench_dirnode.py for recent dirnode changes
5615]
5616[Use DIR-IMM and t=mkdir-immutable for "tahoe backup", for #828
5617Brian Warner <warner@lothar.com>**20091118192813
5618 Ignore-this: a4720529c9bc6bc8b22a3d3265925491
5619]
5620[web/directory.py: use "DIR-IMM" to describe immutable directories, not DIR-RO
5621Brian Warner <warner@lothar.com>**20091118191832
5622 Ignore-this: aceafd6ab4bf1cc0c2a719ef7319ac03
5623]
5624[web/info.py: hush pyflakes
5625Brian Warner <warner@lothar.com>**20091118191736
5626 Ignore-this: edc5f128a2b8095fb20686a75747c8
5627]
5628[make get_size/get_current_size consistent for all IFilesystemNode classes
5629Brian Warner <warner@lothar.com>**20091118191624
5630 Ignore-this: bd3449cf96e4827abaaf962672c1665a
5631 
5632 * stop caching most_recent_size in dirnode, rely upon backing filenode for it
5633 * start caching most_recent_size in MutableFileNode
5634 * return None when you don't know, not "?"
5635 * only render None as "?" in the web "more info" page
5636 * add get_size/get_current_size to UnknownNode
5637]
5638[ImmutableDirectoryURIVerifier: fix verifycap handling
5639Brian Warner <warner@lothar.com>**20091118164238
5640 Ignore-this: 6bba5c717b54352262eabca6e805d590
5641]
5642[Add t=mkdir-immutable to the webapi. Closes #607.
5643Brian Warner <warner@lothar.com>**20091118070900
5644 Ignore-this: 311e5fab9a5f28b9e8a28d3d08f3c0d
5645 
5646 * change t=mkdir-with-children to not use multipart/form encoding. Instead,
5647   the request body is all JSON. t=mkdir-immutable uses this format too.
5648 * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder,
5649   but let callers override it
5650 * raise NotDeepImmutableError instead of using assert()
5651 * add mutable= argument to DirectoryNode.create_subdirectory(), default True
5652]
5653[move convergence secret into SecretHolder, next to lease secret
5654Brian Warner <warner@lothar.com>**20091118015444
5655 Ignore-this: 312f85978a339f2d04deb5bcb8f511bc
5656]
5657[nodemaker: implement immutable directories (internal interface), for #607
5658Brian Warner <warner@lothar.com>**20091112002233
5659 Ignore-this: d09fccf41813fdf7e0db177ed9e5e130
5660 
5661 * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT
5662 * client.create_immutable_dirnode() is used to create them
5663 * no webapi yet
5664]
5665[stop using IURI()/etc as an adapter
5666Brian Warner <warner@lothar.com>**20091111224542
5667 Ignore-this: 9611da7ea6a4696de2a3b8c08776e6e0
5668]
5669[clean up uri-vs-cap terminology, emphasize cap instances instead of URI strings
5670Brian Warner <warner@lothar.com>**20091111222619
5671 Ignore-this: 93626385f6e7f039ada71f54feefe267
5672 
5673  * "cap" means a python instance which encapsulates a filecap/dircap (uri.py)
5674  * "uri" means a string with a "URI:" prefix
5675  * FileNode instances are created with (and retain) a cap instance, and
5676    generate uri strings on demand
5677  * .get_cap/get_readcap/get_verifycap/get_repaircap return cap instances
5678  * .get_uri/get_readonly_uri return uri strings
5679 
5680 * add filenode.download_to_filename() for control.py, should find a better way
5681 * use MutableFileNode.init_from_cap, not .init_from_uri
5682 * directory URI instances: use get_filenode_cap, not get_filenode_uri
5683 * update/cleanup bench_dirnode.py to match, add Makefile target to run it
5684]
5685[add parser for immutable directory caps: DIR2-CHK, DIR2-LIT, DIR2-CHK-Verifier
5686Brian Warner <warner@lothar.com>**20091104181351
5687 Ignore-this: 854398cc7a75bada57fa97c367b67518
5688]
5689[wui: s/TahoeLAFS/Tahoe-LAFS/
5690zooko@zooko.com**20091029035050
5691 Ignore-this: 901e64cd862e492ed3132bd298583c26
5692]
5693[tests: bump up the timeout on test_repairer to see if 120 seconds was too short for François's ARM box to do the test even when it was doing it right.
5694zooko@zooko.com**20091027224800
5695 Ignore-this: 95e93dc2e018b9948253c2045d506f56
5696]
5697[dirnode.pack_children(): add deep_immutable= argument
5698Brian Warner <warner@lothar.com>**20091026162809
5699 Ignore-this: d5a2371e47662c4bc6eff273e8181b00
5700 
5701 This will be used by DIR2:CHK to enforce the deep-immutability requirement.
5702]
5703[webapi: use t=mkdir-with-children instead of a children= arg to t=mkdir .
5704Brian Warner <warner@lothar.com>**20091026011321
5705 Ignore-this: 769cab30b6ab50db95000b6c5a524916
5706 
5707 This is safer: in the earlier API, an old webapi server would silently ignore
5708 the initial children, and clients trying to set them would have to fetch the
5709 newly-created directory to discover the incompatibility. In the new API,
5710 clients using t=mkdir-with-children against an old webapi server will get a
5711 clear error.
5712]
5713[nodemaker.create_new_mutable_directory: pack_children() in initial_contents=
5714Brian Warner <warner@lothar.com>**20091020005118
5715 Ignore-this: bd43c4eefe06fd32b7492bcb0a55d07e
5716 instead of creating an empty file and then adding the children later.
5717 
5718 This should speed up mkdir(initial_children) considerably, removing two
5719 roundtrips and an entire read-modify-write cycle, probably bringing it down
5720 to a single roundtrip. A quick test (against the volunteergrid) suggests a
5721 30% speedup.
5722 
5723 test_dirnode: add new tests to enforce the restrictions that interfaces.py
5724 claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
5725]
5726[test_dirnode.py: add tests of initial_children= args to client.create_dirnode
5727Brian Warner <warner@lothar.com>**20091017194159
5728 Ignore-this: 2e2da28323a4d5d815466387914abc1b
5729 and nodemaker.create_new_mutable_directory
5730]
5731[update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps
5732Brian Warner <warner@lothar.com>**20091017192829
5733 Ignore-this: b35472285143862a856bf4b361d692f0
5734 
5735 interfaces.py: define INodeMaker, document argument values, change
5736                create_new_mutable_directory() to take dict-of-nodes. Change
5737                dirnode.set_nodes() and dirnode.create_subdirectory() too.
5738 nodemaker.py: use INodeMaker, update create_new_mutable_directory()
5739 client.py: have create_dirnode() delegate initial_children= to nodemaker
5740 dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which
5741                     updates set_nodes() and create_subdirectory()
5742 web/common.py (convert_initial_children_json): create dict-of-nodes
5743 web/directory.py: same
5744 web/unlinked.py: same
5745 test_dirnode.py: update tests to match
5746]
5747[dirnode.py: move pack_children() out to a function, for eventual use by others
5748Brian Warner <warner@lothar.com>**20091017180707
5749 Ignore-this: 6a823fb61f2c180fd38d6742d3196a7a
5750]
5751[move dirnode.CachingDict to dictutil.AuxValueDict, generalize method names,
5752Brian Warner <warner@lothar.com>**20091017180005
5753 Ignore-this: b086933cf429df0fcea16a308d2640dd
5754 improve tests. Let dirnode _pack_children accept either dict or AuxValueDict.
5755]
5756[test/common.py: update FakeMutableFileNode to new contents= callable scheme
5757Brian Warner <warner@lothar.com>**20091013052154
5758 Ignore-this: 62f00a76454a2190d1c8641c5993632f
5759]
5760[The initial_children= argument to nodemaker.create_new_mutable_directory is
5761Brian Warner <warner@lothar.com>**20091013031922
5762 Ignore-this: 72e45317c21f9eb9ec3bd79bd4311f48
5763 now enabled.
5764]
5765[client.create_mutable_file(contents=) now accepts a callable, which is
5766Brian Warner <warner@lothar.com>**20091013031232
5767 Ignore-this: 3c89d2f50c1e652b83f20bd3f4f27c4b
5768 invoked with the new MutableFileNode and is supposed to return the initial
5769 contents. This can be used by e.g. a new dirnode which needs the filenode's
5770 writekey to encrypt its initial children.
5771 
5772 create_mutable_file() still accepts a bytestring too, or None for an empty
5773 file.
5774]
5775[webapi: t=mkdir now accepts initial children, using the same JSON that t=json
5776Brian Warner <warner@lothar.com>**20091013023444
5777 Ignore-this: 574a46ed46af4251abf8c9580fd31ef7
5778 emits.
5779 
5780 client.create_dirnode(initial_children=) now works.
5781]
5782[replace dirnode.create_empty_directory() with create_subdirectory(), which
5783Brian Warner <warner@lothar.com>**20091013021520
5784 Ignore-this: 6b57cb51bcfcc6058d0df569fdc8a9cf
5785 takes an initial_children= argument
5786]
5787[dirnode.set_children: change return value: fire with self instead of None
5788Brian Warner <warner@lothar.com>**20091013015026
5789 Ignore-this: f1d14e67e084e4b2a4e25fa849b0e753
5790]
5791[dirnode.set_nodes: change return value: fire with self instead of None
5792Brian Warner <warner@lothar.com>**20091013014546
5793 Ignore-this: b75b3829fb53f7399693f1c1a39aacae
5794]
5795[dirnode.set_children: take a dict, not a list
5796Brian Warner <warner@lothar.com>**20091013002440
5797 Ignore-this: 540ce72ce2727ee053afaae1ff124e21
5798]
5799[dirnode.set_uri/set_children: change signature to take writecap+readcap
5800Brian Warner <warner@lothar.com>**20091012235126
5801 Ignore-this: 5df617b2d379a51c79148a857e6026b1
5802 instead of a single cap. The webapi t=set_children call benefits too.
5803]
5804[replace Client.create_empty_dirnode() with create_dirnode(), in anticipation
5805Brian Warner <warner@lothar.com>**20091012224506
5806 Ignore-this: cbdaa4266ecb3c6496ffceab4f95709d
5807 of adding initial_children= argument.
5808 
5809 Includes stubbed-out initial_children= support.
5810]
5811[test_web.py: use a less-fake client, making test harness smaller
5812Brian Warner <warner@lothar.com>**20091012222808
5813 Ignore-this: 29e95147f8c94282885c65b411d100bb
5814]
5815[webapi.txt: document t=set_children, other small edits
5816Brian Warner <warner@lothar.com>**20091009200446
5817 Ignore-this: 4d7e76b04a7b8eaa0a981879f778ea5d
5818]
5819[Verifier: check the full cryptext-hash tree on each share. Removed .todos
5820Brian Warner <warner@lothar.com>**20091005221849
5821 Ignore-this: 6fb039c5584812017d91725e687323a5
5822 from the last few test_repairer tests that were waiting on this.
5823]
5824[Verifier: check the full block-hash-tree on each share
5825Brian Warner <warner@lothar.com>**20091005214844
5826 Ignore-this: 3f7ccf6d253f32340f1bf1da27803eee
5827 
5828 Removed the .todo from two test_repairer tests that check this. The only
5829 remaining .todos are on the three crypttext-hash-tree tests.
5830]
5831[Verifier: check the full share-hash chain on each share
5832Brian Warner <warner@lothar.com>**20091005213443
5833 Ignore-this: 3d30111904158bec06a4eac22fd39d17
5834 
5835 Removed the .todo from two test_repairer tests that check this.
5836]
5837[test_repairer: rename Verifier test cases to be more precise and less verbose
5838Brian Warner <warner@lothar.com>**20091005201115
5839 Ignore-this: 64be7094e33338c7c2aea9387e138771
5840]
5841[immutable/checker.py: rearrange code a little bit, make it easier to follow
5842Brian Warner <warner@lothar.com>**20091005200252
5843 Ignore-this: 91cc303fab66faf717433a709f785fb5
5844]
5845[test/common.py: wrap docstrings to 80cols so I can read them more easily
5846Brian Warner <warner@lothar.com>**20091005200143
5847 Ignore-this: b180a3a0235cbe309c87bd5e873cbbb3
5848]
5849[immutable/download.py: wrap to 80cols, no functional changes
5850Brian Warner <warner@lothar.com>**20091005192542
5851 Ignore-this: 6b05fe3dc6d78832323e708b9e6a1fe
5852]
5853[CHK-hashes.svg: cross out plaintext hashes, since we don't include
5854Brian Warner <warner@lothar.com>**20091005010803
5855 Ignore-this: bea2e953b65ec7359363aa20de8cb603
5856 them (until we finish #453)
5857]
5858[docs: a few licensing clarifications requested by Ubuntu
5859zooko@zooko.com**20090927033226
5860 Ignore-this: 749fc8c9aeb6dc643669854a3e81baa7
5861]
5862[setup: remove binary WinFUSE modules
5863zooko@zooko.com**20090924211436
5864 Ignore-this: 8aefc571d2ae22b9405fc650f2c2062
5865 I would prefer to have just source code, or indications of what 3rd-party packages are required, under revision control, and have the build process generate o
5866 r acquire the binaries as needed.  Also, having these in our release tarballs is interfering with getting Tahoe-LAFS uploaded into Ubuntu Karmic.  (Technicall
5867 y, they would accept binary modules as long as they came with the accompanying source so that they could satisfy their obligations under GPL2+ and TGPPL1+, bu
5868 t it is easier for now to remove the binaries from the source tree.)
5869 In this case, the binaries are from the tahoe-w32-client project: http://allmydata.org/trac/tahoe-w32-client , from which you can also get the source.
5870]
5871[setup: remove binary _fusemodule.so 's
5872zooko@zooko.com**20090924211130
5873 Ignore-this: 74487bbe27d280762ac5dd5f51e24186
5874 I would prefer to have just source code, or indications of what 3rd-party packages are required, under revision control, and have the build process generate or acquire the binaries as needed.  Also, having these in our release tarballs is interfering with getting Tahoe-LAFS uploaded into Ubuntu Karmic.  (Technically, they would accept binary modules as long as they came with the accompanying source so that they could satisfy their obligations under GPL2+ and TGPPL1+, but it is easier for now to remove the binaries from the source tree.)
5875 In this case, these modules come from the MacFUSE project: http://code.google.com/p/macfuse/
5876]
5877[doc: add a copy of LGPL2 for documentation purposes for ubuntu
5878zooko@zooko.com**20090924054218
5879 Ignore-this: 6a073b48678a7c84dc4fbcef9292ab5b
5880]
5881[setup: remove a convenience copy of figleaf, to ease inclusion into Ubuntu Karmic Koala
5882zooko@zooko.com**20090924053215
5883 Ignore-this: a0b0c990d6e2ee65c53a24391365ac8d
5884 We need to carefully document the licence of figleaf in order to get Tahoe-LAFS into Ubuntu Karmic Koala.  However, figleaf isn't really a part of Tahoe-LAFS per se -- this is just a "convenience copy" of a development tool.  The quickest way to make Tahoe-LAFS acceptable for Karmic then, is to remove figleaf from the Tahoe-LAFS tarball itself.  People who want to run figleaf on Tahoe-LAFS (as everyone should want) can install figleaf themselves.  I haven't tested this -- there may be incompatibilities between upstream figleaf and the copy that we had here...
5885]
5886[setup: shebang for misc/build-deb.py to fail quickly
5887zooko@zooko.com**20090819135626
5888 Ignore-this: 5a1b893234d2d0bb7b7346e84b0a6b4d
5889 Without this patch, when I ran "chmod +x ./misc/build-deb.py && ./misc/build-deb.py" then it hung indefinitely.  (I wonder what it was doing.)
5890]
5891[docs: Shawn Willden grants permission for his contributions under GPL2+|TGPPL1+
5892zooko@zooko.com**20090921164651
5893 Ignore-this: ef1912010d07ff2ffd9678e7abfd0d57
5894]
5895[docs: Csaba Henk granted permission to license fuse.py under the same terms as Tahoe-LAFS itself
5896zooko@zooko.com**20090921154659
5897 Ignore-this: c61ba48dcb7206a89a57ca18a0450c53
5898]
5899[setup: mark setup.py as having utf-8 encoding in it
5900zooko@zooko.com**20090920180343
5901 Ignore-this: 9d3850733700a44ba7291e9c5e36bb91
5902]
5903[doc: licensing cleanups
5904zooko@zooko.com**20090920171631
5905 Ignore-this: 7654f2854bf3c13e6f4d4597633a6630
5906 Use nice utf-8 © instead of "(c)". Remove licensing statements on utility modules that have been assigned to allmydata.com by their original authors. (Nattraverso was not assigned to allmydata.com -- it was LGPL'ed -- but I checked and src/allmydata/util/iputil.py was completely rewritten and doesn't contain any line of code from nattraverso.)  Add notes to misc/debian/copyright about licensing on files that aren't just allmydata.com-licensed.
5907]
5908[build-deb.py: run darcsver early, otherwise we get the wrong version later on
5909Brian Warner <warner@lothar.com>**20090918033620
5910 Ignore-this: 6635c5b85e84f8aed0d8390490c5392a
5911]
5912[new approach for debian packaging, sharing pieces across distributions. Still experimental, still only works for sid.
5913warner@lothar.com**20090818190527
5914 Ignore-this: a75eb63db9106b3269badbfcdd7f5ce1
5915]
5916[new experimental deb-packaging rules. Only works for sid so far.
5917Brian Warner <warner@lothar.com>**20090818014052
5918 Ignore-this: 3a26ad188668098f8f3cc10a7c0c2f27
5919]
5920[setup.py: read _version.py and pass to setup(version=), so more commands work
5921Brian Warner <warner@lothar.com>**20090818010057
5922 Ignore-this: b290eb50216938e19f72db211f82147e
5923 like "setup.py --version" and "setup.py --fullname"
5924]
5925[test/check_speed.py: fix shbang line
5926Brian Warner <warner@lothar.com>**20090818005948
5927 Ignore-this: 7f3a37caf349c4c4de704d0feb561f8d
5928]
5929[setup: remove bundled version of darcsver-1.2.1
5930zooko@zooko.com**20090816233432
5931 Ignore-this: 5357f26d2803db2d39159125dddb963a
5932 That version of darcsver emits a scary error message when the darcs executable or the _darcs subdirectory is not found.
5933 This error is hidden (unless the --loud option is passed) in darcsver >= 1.3.1.
5934 Fixes #788.
5935]
5936[de-Service-ify Helper, pass in storage_broker and secret_holder directly.
5937Brian Warner <warner@lothar.com>**20090815201737
5938 Ignore-this: 86b8ac0f90f77a1036cd604dd1304d8b
5939 This makes it more obvious that the Helper currently generates leases with
5940 the Helper's own secrets, rather than getting values from the client, which
5941 is arguably a bug that will likely be resolved with the Accounting project.
5942]
5943[immutable.Downloader: pass StorageBroker to constructor, stop being a Service
5944Brian Warner <warner@lothar.com>**20090815192543
5945 Ignore-this: af5ab12dbf75377640a670c689838479
5946 child of the client, access with client.downloader instead of
5947 client.getServiceNamed("downloader"). The single "Downloader" instance is
5948 scheduled for demolition anyways, to be replaced by individual
5949 filenode.download calls.
5950]
5951[tests: double the timeout on test_runner.RunNode.test_introducer since feisty hit a timeout
5952zooko@zooko.com**20090815160512
5953 Ignore-this: ca7358bce4bdabe8eea75dedc39c0e67
5954 I'm not sure if this is an actual timing issue (feisty is running on an overloaded VM if I recall correctly), or it there is a deeper bug.
5955]
5956[stop making History be a Service, it wasn't necessary
5957Brian Warner <warner@lothar.com>**20090815114415
5958 Ignore-this: b60449231557f1934a751c7effa93cfe
5959]
5960[Overhaul IFilesystemNode handling, to simplify tests and use POLA internally.
5961Brian Warner <warner@lothar.com>**20090815112846
5962 Ignore-this: 1db1b9c149a60a310228aba04c5c8e5f
5963 
5964 * stop using IURI as an adapter
5965 * pass cap strings around instead of URI instances
5966 * move filenode/dirnode creation duties from Client to new NodeMaker class
5967 * move other Client duties to KeyGenerator, SecretHolder, History classes
5968 * stop passing Client reference to dirnode/filenode constructors
5969   - pass less-powerful references instead, like StorageBroker or Uploader
5970 * always create DirectoryNodes by wrapping a filenode (mutable for now)
5971 * remove some specialized mock classes from unit tests
5972 
5973 Detailed list of changes (done one at a time, then merged together)
5974 
5975 always pass a string to create_node_from_uri(), not an IURI instance
5976 always pass a string to IFilesystemNode constructors, not an IURI instance
5977 stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri()
5978 client.py: move SecretHolder code out to a separate class
5979 test_web.py: hush pyflakes
5980 client.py: move NodeMaker functionality out into a separate object
5981 LiteralFileNode: stop storing a Client reference
5982 immutable Checker: remove Client reference, it only needs a SecretHolder
5983 immutable Upload: remove Client reference, leave SecretHolder and StorageBroker
5984 immutable Repairer: replace Client reference with StorageBroker and SecretHolder
5985 immutable FileNode: remove Client reference
5986 mutable.Publish: stop passing Client
5987 mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference
5988 MutableChecker: reference StorageBroker and History directly, not through Client
5989 mutable.FileNode: removed unused indirection to checker classes
5990 mutable.FileNode: remove Client reference
5991 client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker
5992 move create_mutable_file() into NodeMaker
5993 test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests.
5994 test_mutable.py: clean up basedir names
5995 client.py: move create_empty_dirnode() into NodeMaker
5996 dirnode.py: get rid of DirectoryNode.create
5997 remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match
5998 stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker
5999 remove Client from NodeMaker
6000 move helper status into History, pass History to web.Status instead of Client
6001 test_mutable.py: fix minor typo
6002]
6003[docs: edits for docs/running.html from Sam Mason
6004zooko@zooko.com**20090809201416
6005 Ignore-this: 2207e80449943ebd4ed50cea57c43143
6006]
6007[docs: install.html: instruct Debian users to use this document and not to go find the DownloadDebianPackages page, ignore the warning at the top of it, and try it
6008zooko@zooko.com**20090804123840
6009 Ignore-this: 49da654f19d377ffc5a1eff0c820e026
6010 http://allmydata.org/pipermail/tahoe-dev/2009-August/002507.html
6011]
6012[docs: relnotes.txt: reflow to 63 chars wide because google groups and some web forms seem to wrap to that
6013zooko@zooko.com**20090802135016
6014 Ignore-this: 53b1493a0491bc30fb2935fad283caeb
6015]
6016[docs: about.html: fix English usage noticed by Amber
6017zooko@zooko.com**20090802050533
6018 Ignore-this: 89965c4650f9bd100a615c401181a956
6019]
6020[docs: fix mis-spelled word in about.html
6021zooko@zooko.com**20090802050320
6022 Ignore-this: fdfd0397bc7cef9edfde425dddeb67e5
6023]
6024[TAG allmydata-tahoe-1.5.0
6025zooko@zooko.com**20090802031303
6026 Ignore-this: 94e5558e7225c39a86aae666ea00f166
6027]
6028Patch bundle hash:
602920c5d329ef51ad6be17e8796387ecd5663a6424d