#1732 closed enhancement (fixed)

consider changes to webapi "Move" API before release

Reported by: warner Owned by: somebody
Priority: critical Milestone: 1.10.0
Component: code-frontend-web Version: 1.9.1
Keywords: forward-compatibility blocker Cc: marcus@…, zooko
Launchpad Bug:

Description

I just finished reviewing and landing #1579, and it looks nice. One thing we should probably consider though, before committing to the API, is whether we're happy with the split "subdirname-vs-dircap" API. In the current code, you can either move a child to a subdir by name:

POST /uri/$DIRCAP?t=move&from_name=OLD&to_dir=SUBDIRNAME&target_type=name

or to a (possibly unrelated) subdir by its dircap:

POST /uri/$DIRCAP?t=move&from_name=OLD&to_dir=NEWDIRCAP&target_type=uri

That feels a bit weird. I'm kind of thinking that we should accept to_dirname= or to_dircap= (and throw an error if you provide both), instead of switching the interpretation of to_dir= according to the presence of that target_type= argument.

thoughts? note this is a blocker for 1.10 (or whatever release is next done on trunk), since we don't want to wind up supporting the wrong API forever.

Change History (57)

comment:1 Changed at 2012-05-10T00:06:43Z by zooko

I agree that to_dirname=/to_dircap= is a better API than to_dir=&target_type=.

comment:2 follow-up: Changed at 2012-05-10T00:43:19Z by davidsarah

  • Keywords forward-compatibility added
  • Type changed from task to defect

I agree as well (in general I don't like overloading of the interpretation of one argument based on another argument, in any API).

Should to_dircap accept a path after the dircap? Note that supporting that would allow changing tahoe mv to use this API, which is a prerequisite to fixing #943 in the way proposed in ticket:943#comment:4 (since that fix depends on knowing the path that was used in the tahoe mv command to specify the destination). If it does accept a path then it should be called something other than to_dircap.

Unix mv can rename a file at the same time as moving it. Should there be a to_name argument (defaulting to from_name)?

[edit: fix wrong bug number for #943]

Last edited at 2012-05-10T00:44:36Z by davidsarah (previous) (diff)

comment:3 Changed at 2012-05-10T15:02:37Z by marcusw

  • Type changed from defect to enhancement

This looks good; the current API is simply a result of the URI target functionality being tacked onto the subdir option. I like warner's API better.

Think I'll change this from "defect" to "enhancement" because my API did in fact work, even if not optimal :p

comment:4 Changed at 2012-05-10T15:03:08Z by marcusw

  • Cc marcus@… added

comment:5 Changed at 2012-05-16T22:10:52Z by zooko

  • Cc zooko added
  • Milestone changed from 1.10.0 to 1.9.2

comment:6 Changed at 2012-05-16T22:11:38Z by zooko

  • Milestone changed from 1.9.2 to 1.10.0

comment:7 Changed at 2012-05-16T22:49:47Z by davidsarah

  • Priority changed from normal to major

Milestone: 1.10.0 is correct because the 'Move' feature is only in trunk, not on the 1.9.2 branch.

comment:8 Changed at 2012-12-13T17:52:54Z by zooko

  • Keywords blocker added

comment:9 Changed at 2012-12-20T17:51:46Z by warner

  • Owner set to warner
  • Status changed from new to assigned

I'll try to write a patch for this over the weekend.

comment:10 in reply to: ↑ 2 Changed at 2013-02-07T17:10:49Z by zooko

Replying to davidsarah:

Should to_dircap accept a path after the dircap? Note that supporting that would allow changing tahoe mv to use this API, which is a prerequisite to fixing #943 in the way proposed in ticket:943#comment:4 (since that fix depends on knowing the path that was used in the tahoe mv command to specify the destination). If it does accept a path then it should be called something other than to_dircap.

There this thing that we use a lot in the WUI/WAPI: a dircap optionally followed by a list of childnames. The list of childnames is clearly a unix (relative) path. But there's no word for "dircap+relpath". I think we need to coin a word for it. I can't think of anything clever and clear, so how about something awkward and explicit and clear, like "dircappluspath"?

Hm, or actually I think we can extend the vocabulary of unix here. Unix already has "absolute paths" and "relative paths". How about if we call these things "rooted paths"?

comment:11 Changed at 2013-02-09T00:22:35Z by davidsarah

I like "rooted path" because it's consistent with the use of "rooted" in graph theory.

comment:12 Changed at 2013-02-10T12:42:47Z by zooko

Great! I think "rooted path" is a good word for this. So in the WAPI, we currently have:

POST /uri/$DIRCAP?t=move&from_name=OLD&to_dir=SUBDIRNAME&target_type=name
POST /uri/$DIRCAP?t=move&from_name=OLD&to_dir=NEWDIRCAP&target_type=uri

And the proposal is to change it to:

POST /uri/$DIRCAP?t=move&from_name=OLD&to_dirname=SUBDIRNAME

and

POST /uri/$DIRCAP?t=move&from_name=OLD&to=NEWROOTEDPATH

?

comment:13 Changed at 2013-02-10T22:17:15Z by davidsarah

Would to_relative and to_rooted be better names for the parameters? (to_dirname seems inaccurate for the case where the target is in the same directory as before.)

comment:14 follow-up: Changed at 2013-02-10T22:20:59Z by davidsarah

Actually, if we allow rooted paths, is it any longer necessary to have distinct parameters? A relative path and a rooted path can be distinguished exactly as they are by the CLI.

comment:15 Changed at 2013-02-10T22:21:27Z by davidsarah

(except for not allowing aliases)

comment:16 in reply to: ↑ 14 Changed at 2013-02-10T23:26:07Z by zooko

Replying to davidsarah:

Actually, if we allow rooted paths, is it any longer necessary to have distinct parameters? A relative path and a rooted path can be distinguished exactly as they are by the CLI.

Well, here is the form that uses a rooted path:

POST /uri/$DIRCAP?t=move&from_name=OLD&to=NEWROOTEDPATH

The only use of the other form -- POST /uri/$DIRCAP?t=move&from_name=OLD&to_dirname=SUBDIRNAME would be to more succinctly express when you want the "to_dirname" to be relative to $DIRCAP instead of relative to a (different) root. So, I guess it is a tiny optimization and not worth the added complexity of API, IMO.

So I think David-Sarah is right, and a single API (POST /uri/$DIRCAP?t=move&from_name=OLD&to=NEWROOTEDPATH) is sufficient.

Last edited at 2013-02-10T23:26:19Z by zooko (previous) (diff)

comment:17 follow-up: Changed at 2013-03-07T17:38:49Z by ClashTheBunny

Rooted path seems to be the correct behavior that we would want to use for a programmatic interface. The only reason to not require the rooted path would be to protect humans from another DIRCAP or a copy paste error. Because we're not trying to protect humans from this, there was agreement on the weekly dev chat that this is the technically correct approach.

We made sure that the WUI does not display or use a move button in 1.9.x, so there are no supported WUI's out there that would be broken. This means that there shouldn't be anybody left in the cold because of a change to this API.

My one thing that I think I understand but want to explicitly state is:

  • If "NEWROOTEDPATH" exists and is a directory, it is considered a destination directory where the file will be moved with the same name.
  • If "NEWROOTEDPATH" exists and is a file, it is an exception and should be raised.
  • If "NEWROOTEDPATH" doesn't exist, but the path above it exists, it's considered the destination with a (possibly new) name.
  • If the path above "NEWROOTEDPATH" doesn't exist, it is an exception and should be raised.
  • If "NEWROOTEDPATH" isn't a rooted path, an exception is raised. This case shall be shielded against for users of the WUI. The WUI will probably default to appending the current DIRCAP.

comment:18 in reply to: ↑ 17 Changed at 2013-03-11T05:29:21Z by zooko

  • Status changed from assigned to new

Thanks a lot for the review, ClashTheBunny! I think your 5-point rules for handling this API call are right. Brian: does that look like a good API to you? Reassigning this ticket to Brian.

One more question, too: I earlier, in comment:16, wrote

POST /uri/$DIRCAP?t=move&from_name=OLD&to=NEWROOTEDPATH

Should it actually be:

POST /uri/$ROOTEDPATH_TO_DIR?t=move&from_name=OLD&to=NEWROOTEDPATH

? That is: does the caller have to supply the actual dircap of the original (source) directory, or can they use a rooted path to that directory?

comment:19 follow-up: Changed at 2013-03-14T16:28:23Z by ClashTheBunny

The rooted path is the most general representation of a file or directory, as it could just be a dircap or a dircap with subdirectories. As such, the correct representation is as zooko mentions:

POST /uri/$ROOTEDPATH_TO_DIR?t=move&from_name=OLD&to=NEWROOTEDPATH

It then adds this slight complication to my 5 point list above:

  • If ROOTEDPATH_TO_DIR actually points to a file, raise an exception.

comment:20 Changed at 2013-03-14T16:38:04Z by davidsarah

I was thinking that zooko's suggested change might be useful to fix #943 (as suggested in ticket:943#comment:4), but it isn't actually necessary -- knowing the full path to the destination is sufficient to fix that ticket. OTOH, we agreed in the Dev Chat that this change was a good one anyway, just for generality and convenience.

comment:21 in reply to: ↑ 19 Changed at 2013-03-14T16:40:41Z by zooko

Replying to ClashTheBunny:

  • If ROOTEDPATH_TO_DIR actually points to a file, raise an exception.

Hey that was a good comment.

comment:22 Changed at 2013-03-14T17:26:46Z by davidsarah

Proposed doc:

Moving A Child
--------------

``POST /uri/$DIRCAP/[SUBDIRS../]?t=move&from_name=OLD&to=$NEWDIRCAP/[NEWSUBDIRS../]NEW``

 This instructs the node to move a child of the given directory to a
 different directory, both of which must be writeable. The to= parameter
 should contain a path to the destination directory (multiple levels of
 descent are supported) ending with the new name. The new name must be
 included even if it is the same as the old name.

 The default behavior is to overwrite any existing object at the given
 location. To prevent this (and make the operation return an error instead
 of overwriting), add a "replace=false" argument. With replace=false, this
 operation will return an HTTP 409 "Conflict" error if there is already an
 object at the given location, rather than overwriting the existing object.
 To allow the operation to overwrite a file, but return an error when trying
 to overwrite a directory, use "replace=only-files" (this behavior is closer
 to the traditional UNIX "mv" command). Note that "true", "t", and "1" are
 all synonyms for "True", and "false", "f", and "0" are synonyms for "False",
 and the parameter is case-insensitive.

 For safety, the child is not unlinked from the old directory until it has
 been successfully added to the new directory.

This includes the "replace=only-files" option to avoid accidentally clobbering a directory, as discussed on the Dev Chat.

Last edited at 2013-03-14T17:34:15Z by davidsarah (previous) (diff)

comment:23 Changed at 2013-03-14T17:40:49Z by davidsarah

Note that this description doesn't define or use the term "rooted path". I've opened #1929 for that.

comment:24 follow-up: Changed at 2013-03-14T17:51:03Z by davidsarah

Improvements to clarify that metadata is preserved and the crash/failure behaviour.

Moving A Child
--------------

``POST /uri/$DIRCAP/[SUBDIRS../]?t=move&from_name=OLD&to=$NEWDIRCAP/[NEWSUBDIRS../]NEW``

 This instructs the node to move a child of the given directory to a
 different directory, both of which must be writeable. Metadata from the
 directory entry is preserved. The to= parameter should contain a path
 to the destination directory (multiple levels of descent are supported)
 ending with the new name. The new name must be included even if it is the
 same as the old name.

 The default behavior is to overwrite any existing object at the given
 location. To prevent this (and make the operation return an error instead
 of overwriting), add a "replace=false" argument. With replace=false, this
 operation will return an HTTP 409 "Conflict" error if there is already an
 object at the given location, rather than overwriting the existing object.
 To allow the operation to overwrite a file, but return an error when trying
 to overwrite a directory, use "replace=only-files" (this behavior is closer
 to the traditional UNIX "mv" command). Note that "true", "t", and "1" are
 all synonyms for "True", and "false", "f", and "0" are synonyms for "False",
 and the parameter is case-insensitive.

 For safety, the child is not unlinked from the old directory until it has
 been successfully added to the new directory. This implies that in case of
 a crash or failure, the child will not be lost, but could be linked at both
 the old and new locations.

comment:25 in reply to: ↑ 24 Changed at 2013-03-14T18:54:11Z by ClashTheBunny

Some possible improvements included below:

  • add 'replace' to top command line including options.
  • add replace=true as being the default value for replace
  • add comments about failure situations in the first paragraph Maybe there's a better response than a 400?
Moving A Child
--------------

``POST /uri/$DIRCAP/[SUBDIRS../]?t=move&from_name=OLD&to=$NEWDIRCAP/[NEWSUBDIRS../]NEW[&replace=true|false|only-files]``

 This instructs the node to move a child of the given directory to a
 different directory, both of which must be writeable. Metadata from the
 directory entry is preserved. The to= parameter should contain a path
 to the destination directory (multiple levels of descent are supported)
 ending with the new name. The new name must be included even if it is the
 same as the old name.  This operation will return an HTTP 400 "Bad Request"
 if ``$DIRCAP/[SUBDIRS../]`` refers to a file or doesn't exist or if 
 ``$NEWDIRCAP/[NEWSUBDIRS../]`` doesn't exist.

 The default behavior is to overwrite any existing object at the given
 location (replace=true). To prevent this (and make the operation return an error instead
 of overwriting), add a "replace=false" argument. With replace=false, this
 operation will return an HTTP 409 "Conflict" error if there is already an
 object at the given location, rather than overwriting the existing object.
 To allow the operation to overwrite a file, but return an error when trying
 to overwrite a directory, use "replace=only-files" (this behavior is closer
 to the traditional UNIX "mv" command). Note that "true", "t", and "1" are
 all synonyms for "True", and "false", "f", and "0" are synonyms for "False",
 and the parameter is case-insensitive.

 For safety, the child is not unlinked from the old directory until it has
 been successfully added to the new directory. This implies that in case of
 a crash or failure, the child will not be lost, but could be linked at both
 the old and new locations.

comment:26 Changed at 2013-03-15T06:15:17Z by davidsarah

The correct response code would be 404 Not Found if the source doesn't exist, or 403 Forbidden if it is a file. (You could also make a case for 418 ;-)

comment:27 Changed at 2013-03-18T14:31:29Z by zooko

We agreed in the dev chat of 2013-03-14 that this API won't support an implicit final name, so you can't say POST /uri/$OLDDIRCAP/$OLDNAME/?t=move&from_name=$OLD&to=$NEWDIRCAP/$NEWNAME/ to mean that the new name should be $NEWDIRCAP/$NEWNAME/$OLD. This is something that unix mv supports ("mv foo/bar baz/"), and tahoe mv might or might not support it, but this API won't. The reason not to is that if it did, then it becomes harder to disambiguate if the user means to add to the directory $NEWDIRCAP/$NEWNAME a link under the name $OLD, or to add to the directory $NEWDIRCAP a link under the name $NEWNAME.

Now I'm confused about why we spell it $CAP/$NAME/&arg=$FINALNAME in the source but $CAP/$NAME/$FINALNAME in the destination? It's not consistent. In the interests of consistency, how about one of these two APIs:

POST /uri/$DIRCAP/[SUBDIRS../]OLD?t=move&to=$NEWDIRCAP/[NEWSUBDIRS../]NEW[&replace=true|false|only-files]

or

POST /uri/$DIRCAP/[SUBDIRS../]?t=move&from_name=OLD&to=$NEWDIRCAP/[NEWSUBDIRS../]&to_name=NEW[&replace=true|false|only-files]

Between these two, the first one has the advantage of being less wordy.

comment:28 Changed at 2013-03-18T17:17:49Z by davidsarah

I can think of two arguments for the second alternative:

a) Consistency with ?t=rename.

b) The object modified by the POST is $DIRCAP/[SUBDIRS../]. $DIRCAP/[SUBDIRS../]OLD is not modified (unless it happens to be aliased to the source or destination directory).

a) is not a strong argument; I would prefer conciseness over consistency with ?t=rename.

b) is a stronger argument, although the operation also modifies the destination directory which is not the object of the request. Still, I suppose this does mean that the second alternative is more consistent with REST, so I'm -0 on switching to the first alternative. I'm not sure which is more important, that or internal consistency.

Last edited at 2013-03-18T17:33:15Z by davidsarah (previous) (diff)

comment:29 Changed at 2013-03-18T17:31:06Z by davidsarah

BTW, ?t=move has the potential to leak filenames and/or cap URIs into logs, similar to ?t=rename and ?t=rename-form in #1904.

comment:30 follow-up: Changed at 2013-03-18T18:04:26Z by warner

Talking with zooko at pycon, we think the following might be best:

POST /uri/$DIRCAP/[SUBDIRS../]?t=move&from_name=OLD&to_dir=$NEWDIRCAP/[NEWSUBDIRS../]&to_name=NEW[&replace=true|false|only-files]

Zooko's first spelling (.../OLD?t=move&to=...NEW) would be nice(r), but the webapi code uses twisted.web Resource traversal to find the dir/file object referenced by the URL, and to accomodate .../OLD) we'd need to change that code to remember the penultimate Resource (the directory), which would be a lot of work. (parsing to= is easier: we could just regexp off the last slash).

comment:31 in reply to: ↑ 30 Changed at 2013-03-18T20:01:57Z by davidsarah

Replying to warner:

Talking with zooko at pycon, we think the following might be best:

POST /uri/$DIRCAP/[SUBDIRS../]?t=move&from_name=OLD&to_dir=$NEWDIRCAP/[NEWSUBDIRS../]&to_name=NEW[&replace=true|false|only-files]

Ah yes, I see the consistency argument for that. In that case perhaps to_name should default to from_name after all.

Zooko's first spelling (.../OLD?t=move&to=...NEW) would be nice(r), but the webapi code uses twisted.web Resource traversal to find the dir/file object referenced by the URL, and to accomodate .../OLD) we'd need to change that code to remember the penultimate Resource (the directory), which would be a lot of work. (parsing to= is easier: we could just regexp off the last slash).

That seems like a compelling argument. So the choice is now between comment:25 and comment:30.

comment:32 Changed at 2013-03-18T20:33:07Z by davidsarah

Oh, I've had another idea. What if we just generalize t=rename so that it can take an optional to_dir parameter? I.e.:

Moving or Renaming a Child
--------------------------

``POST /uri/$DIRCAP/[SUBDIRS../]?t=rename&from_name=OLD
       [&to_dir=$NEWDIRCAP/[NEWSUBDIRS../][&to_name=NEW][&replace=true|false|only-files]``

 This instructs the node to move or rename a child of the given directory,
 which must be writeable. If {{{to_dir}}} is present, the child is moved
 into the directory specified by {{{to_dir}}}, which must also be writeable.
 If {{{to_dir}}} is not present, the child is renamed within the same directory.
 If {{{to_name}}} is not present then it defaults to {{{from_name}}}.
 If the destination directory and name are the same as the source, the
 operation has no effect.

 Metadata from the source directory entry is preserved. Multiple levels of
 descent in the source and destination paths are supported.

 This operation will return an HTTP 404 "Not Found" error if
 ``$DIRCAP/[SUBDIRS../]``, the child being moved, or {{{to_dir}}} (if given)
 does not exist. It will return an HTTP 400 "Bad Request" error if any
 entry in the source or destination paths is not a directory.

 The default behavior is to overwrite any existing object at the given
 location (replace=true). To prevent this (and make the operation return
 an error instead of overwriting), add a "replace=false" argument. With
 replace=false, this operation will return an HTTP 409 "Conflict" error if
 there is already an object at the given location, rather than overwriting
 the existing object. To allow the operation to overwrite a file, but
 return an HTTP 409 error when trying to overwrite a directory, use
 "replace=only-files" (this behavior is closer to the traditional UNIX "mv"
 command). Note that "true", "t", and "1" are all synonyms for "True";
 "false", "f", and "0" are synonyms for "False"; and the parameter is
 case-insensitive.

 When moving to a different directory, for safety, the child is not unlinked
 from the old directory until it has been successfully added to the new
 directory. This implies that in case of a crash or failure, the child will
 not be lost, but could be linked at both the old and new locations.

 Prior to Tahoe-LAFS v1.10, the {{{to_dir}}} parameter and the
 {{{replace=only-files}}} option were not supported.
Last edited at 2013-03-19T01:59:21Z by davidsarah (previous) (diff)

comment:33 Changed at 2013-03-19T03:01:42Z by davidsarah

IRC conversation, edited for formatting:

davidsarah: did you see my suggestion on #1732 ?

warner: yeah

warner: I'm uncertain about changing t=rename. Basically t=rename currently addresses a directory, and tells it to do something to itself. It's a single atomic operation (well, as atomic as changing any mutable file)

davidsarah: well, the operation we want is a strict generalization of rename

warner: whereas t=move addresses one directory and tells it to talk to a (possibly different) directory

warner: from a filesystem point of view, yeah

davidsarah: but only possibly different

warner: but in terms of dirnodes, that generalization would increase the number of actors from one to two

davidsarah: if the destination directory for t=move is the same as the source, then it should be atomic (from the filesystem point of view)

warner: true

davidsarah: so the complexity is still there

warner: yeah. I guess I think of a lot of the webapi in terms of speaking to a specific node and telling it to do something

davidsarah: basically the only difference between t=rename in my suggestion and t=move in https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1732#comment:30 is the defaults for to_dir and to_name. I.e. to_dir defaults to the source dir, and to_name defaults to from_name

warner: from the outside, yeah

davidsarah: I'm all about eliminating redundancy :-)

warner: but the current t=rename is quasi-atomic (there's no way to wind up with multiple copies of the child), and that would change it to be sometimes non-atomic (if interrupted, it might add to the target but not remove from the original)

warner: I dunno

davidsarah: [It would be non-atomic] only if to_dir is provided

warner: I guess we could argue that having a t=move with the proposed syntax means it'd be cleaner to modify t=rename instead

davidsarah: besides, there was a suggestion to deprecate t=rename if we added t=move

warner: hm. ok, so I think "rename" shouldn't be used to move something long distances, whereas "move" is a fine verb for that. And I guess my attitude suggests that, if we only have one command, then it should be "move"

davidsarah: hmm. I don't place much significance on the operation name

warner: I dunno. [...] let's pester zooko to think about this too

davidsarah: well, we don't have the option of adding move and immediately removing rename... if we want to retain backward compatibility

warner: yeah, true. So if we keep rename around, then it suggests that "move" should be something different

davidsarah: I think we need Zooko as a casting vote :-) (it is actually quite useful having 3 of us deciding these things)

davidsarah: here's another advantage of my suggestion: the existing tests for t=rename will test the same-directory case, and so the new tests only need to cover the different-directory case

comment:34 Changed at 2013-03-20T21:31:56Z by warner

  • Priority changed from major to critical

marking "critical" to indicate this is a blocker for 1.10

comment:35 follow-up: Changed at 2013-03-21T18:20:47Z by warner

I mentioned to zooko this morning that I don't think I'm picky about this API, and am happy go along with whatever everyone else likes. But then I thought of two constraints that I think are important:

  • don't remove an existing API (so t=rename must stick around)
  • don't add parameters to an existing API that can't be discovered by callers, since our webapi practice is to ignore arguments we don't recognize

The latter constraint precludes adding some new argument to t=rename that would change behavior, because then a new client (who knows about the new argument) who submits such a request to an older gateway (who ignores the new argument) will get a very different behavior than they were expecting. It's the same reason that python functions reject unrecognized keyword-arguments: the use of kwargs allows new callees to add new kwargs over time, but still get a clear error when a new caller calls an older callee, instead of silently ignoring the new arg and doing something completely different.

So I guess that means I *am* picky about it, and I'm voting for a new t=move. :)

comment:36 Changed at 2013-03-21T19:00:12Z by ClashTheBunny

Zooko and I talked about this during the dev talk and I wanted to share my two points.

I first felt that not having the full rooted cap to the file to be moved was punting in that it was hard to implement the other way, so we would do it this way. Zooko helped me understand that the real operation happens at the directory, in that the directory is a file allocation table that is being edited. He told me about the history with the command line 'rm' being confusing in that it doesn't remove a file, but just removes the entry in the directory's file allocation table. Because of this, I'm good with the new verbage of doing a move operation on the containing directory. It's also a win for people who want to just replace "rename" with the new API and things will just work.

The second thing we talked about is that his new API should really be called "relink" in that it's not operating on (moving) a file, but it's creating a link for a file in a new place, and then deleting the old link (or editing the link in the current directory). That brings me to my proposal for a slight alteration to comment 30's definition of the new API:

POST /uri/$DIRCAP/[SUBDIRS../]?t=relink&from_name=OLD&to_dir=$NEWDIRCAP/[NEWSUBDIRS../]&to_name=NEW[&replace=true|false|only-files]

This way people understand that the operation doesn't change a file in any way, but it actually edits links to the file. It operates on the directory, and not the file too, which is reflected in this. It also leave rename the same, but allows for people who want to update to the new API from 'rename' to just replace 'rename' with 'relink'.

comment:37 Changed at 2013-03-21T21:29:36Z by warner

I'm cool with that.

comment:38 in reply to: ↑ 35 Changed at 2013-03-21T23:30:32Z by davidsarah

Replying to warner: [...]

  • don't add parameters to an existing API that can't be discovered by callers, since our webapi practice is to ignore arguments we don't recognize

The latter constraint precludes adding some new argument to t=rename that would change behavior, because then a new client (who knows about the new argument) who submits such a request to an older gateway (who ignores the new argument) will get a very different behavior than they were expecting.

Oh, I am dense for not seeing that. OK, that rules out just generalizing t=rename. +1 for calling it t=relink.

comment:39 follow-up: Changed at 2013-03-22T00:16:03Z by davidsarah

OK, I think this is what we have now:

Relinking a Child
-----------------

``POST /uri/$DIRCAP/[SUBDIRS../]?t=relink&from_name=OLD``
       ``&to_dir=$NEWDIRCAP/[NEWSUBDIRS../]&to_name=NEW``
       ``[&replace=true|false|only-files]``    (Tahoe >= v1.10)

 This instructs the node to relink a child of the given source directory,
 into a different directory and/or to a different name. The source and
 destination directories must be writeable. {{{to_dir}}} may be the same
 as the source directory and {{{to_name}}} may be the same as {{{from_name}}},
 but {{{to_dir}}} and {{{to_name}}} are always required. If the destination
 link (directory and name) is the same as the source link, the operation has
 no effect.

 Metadata from the source directory entry is preserved. Multiple levels of
 descent in the source and destination paths are supported.

 This operation will return an HTTP 404 "Not Found" error if
 ``$DIRCAP/[SUBDIRS../]``, the child being moved, or the destination
 directory does not exist. It will return an HTTP 400 "Bad Request" error
 if any entry in the source or destination paths is not a directory.

 The default behavior is to overwrite any existing object at the destination
 (replace=true). To prevent this (and make the operation return an error
 instead of overwriting), add a "replace=false" argument. With replace=false,
 this operation will return an HTTP 409 "Conflict" error if there is already
 an object at the destination, rather than overwriting the existing object.
 To allow the operation to overwrite a file, but return an HTTP 409 error
 when trying to overwrite a directory, use "replace=only-files" (this behavior
 is closer to the traditional UNIX "mv" command). Note that "true", "t", and
 "1" are all synonyms for "True"; "false", "f", and "0" are synonyms for
 "False"; and the parameter is case-insensitive.

 When relinking into a different directory, for safety, the child is not
 unlinked from the old directory until it has been successfully linked into
 the new directory. This implies that in case of a crash or failure, the
 child will not be lost, but could be linked at both the old and new
 locations.

 The source link should not be the same as any link (directory and child name)
 in the ``to_dir`` path. This restriction is not enforced, but it may be
 enforced in a future version. If it were violated then the result would be
 to create a cycle in the directory structure that is not necessarily reachable
 from the root of the destination path (``$NEWDIRCAP``), which could result in
 data loss, as described in ticket `#943`_.

.. _`#943`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/943

(I just added the bit about #943.)

Last edited at 2013-03-22T00:16:34Z by davidsarah (previous) (diff)

comment:40 in reply to: ↑ 39 Changed at 2013-03-22T06:15:24Z by ClashTheBunny

I believe from_name is always required, which would be nice to include in the text. Maybe the first paragraph could look like this:

This instructs the node to relink a child of the given source directory, into a different directory and/or to a different name. The source and destination directories must be writeable. from_name must exist and must be be supplied. to_dir may be the same as the source directory and to_name may be the same as from_name, but to_dir and to_name are always required. If the destination link (directory and name) is the same as the source link, the operation has no effect.

and again as a quote:

This instructs the node to relink a child of the given source directory,
into a different directory and/or to a different name. The source and
destination directories must be writeable. {{{from_name}}} must exist
and must be be supplied.  {{{to_dir}}} may be the same as the source
directory and {{{to_name}}} may be the same as {{{from_name}}}, but
{{{to_dir}}} and {{{to_name}}} are always required. If the destination
link (directory and name) is the same as the source link, the operation
has no effect.

But even without that, I think this is great. +1

comment:41 Changed at 2013-03-22T21:13:46Z by zooko

I've thought about this. I'm +1 on comment:39 and comment:40, with the following comments:

  • to_name could be optional, and if omitted it defaults to from_name. The reason we earlier agreed to require it was that it could be ambiguous what was the name and what was the path. Now that the to_name lives in a separate field from the to_dir, there's no ambiguity.
  • Let's not deprecate the old rename at this time, but I'd like to revisit that in the future…

comment:42 Changed at 2013-03-23T18:03:48Z by warner

I'm vaguely -0 on anything being optional, but it's a really weak -0.. I wouldn't object to it landing in that form.

comment:43 follow-up: Changed at 2013-03-23T20:07:52Z by ClashTheBunny

My personal feeling is that this API would benefit from having a total compatibility with rename such that if rename were deprecated in the future, it would be a one word change from 'rename' to 'relink'. You could even do it with a hex editor on a binary since it's the same number of bits. I don't care as much about things being optional as I care about things being compatible. If we require to_dir, if people want to jump from rename to relink, that much more work will be required and that much more chance of people messing up.

Philosophically, I'm for APIs that easy to use, but only if they are unambiguous and easy to not misuse. I don't think that having either to_dir or to_name be required makes the API ambiguous or easy to misuse. A good programmer with an attention to details would probably fill them both in anyhow.

comment:44 in reply to: ↑ 43 Changed at 2013-03-24T09:18:06Z by davidsarah

Replying to ClashTheBunny:

My personal feeling is that this API would benefit from having a total compatibility with rename such that if rename were deprecated in the future, it would be a one word change from 'rename' to 'relink'. You could even do it with a hex editor on a binary since it's the same number of bits. I don't care as much about things being optional as I care about things being compatible. If we require to_dir, if people want to jump from rename to relink, that much more work will be required and that much more chance of people messing up.

Well, if that were a concern then we could just not deprecate rename. It's not really imposing significant complexity, since it will be implementable by delegating to relink.

I'm with warner: -0 on having either to_dir or to_name be optional. Do we have concensus on the comment:39 spec then? If so I will start implementing it tomorrow.

comment:45 follow-up: Changed at 2013-03-25T01:33:40Z by warner

Ok, I think we have consensus. to_name= is optional, per zooko, to_dir= is optional, per ClashTheBunny, and we leave t=rename untouched.

Thanks davidsarah for implementing this!

comment:46 in reply to: ↑ 45 ; follow-up: Changed at 2013-03-26T16:49:47Z by zooko

Replying to warner:

Ok, I think we have consensus. to_name= is optional, per zooko, to_dir= is optional, per ClashTheBunny, and we leave t=rename untouched.

Agreed!

This means we need a sensible error message if both to_name= and to_dir= are omitted. ☺

comment:48 Changed at 2013-04-04T16:47:43Z by daira

  • Owner changed from warner to daira
  • Status changed from new to assigned

comment:50 Changed at 2013-04-05T05:15:05Z by daira

  • Keywords review-needed added
  • Owner changed from daira to zooko
  • Status changed from assigned to new

comment:51 Changed at 2013-04-05T11:59:08Z by zooko

  • Keywords reviewed added; review-needed removed
  • Owner changed from zooko to daira

+1! ☺

comment:52 Changed at 2013-04-05T18:42:04Z by daira

  • Resolution set to fixed
  • Status changed from new to closed

Fixed in 35f37cc5b8646540.

Last edited at 2013-04-05T18:42:35Z by daira (previous) (diff)

comment:53 in reply to: ↑ 46 Changed at 2013-04-05T18:43:51Z by daira

Replying to zooko:

This means we need a sensible error message if both to_name= and to_dir= are omitted. ☺

If both are omitted then the operation succeeds with no effect.

comment:54 Changed at 2013-04-09T12:16:20Z by zooko

  • Resolution fixed deleted
  • Status changed from closed to reopened

If I understand the github WUI correctly, Brian made some docs changes that I don't agree with:

https://github.com/tahoe-lafs/tahoe-lafs/commit/f9335892f2008198bd8210b98dd953665d8a5e08

comment:55 Changed at 2013-04-09T13:09:55Z by zooko

  • Keywords review-needed added; reviewed removed
  • Owner changed from daira to somebody
  • Status changed from reopened to new

Here's an attempt at fixing the issues I have with these docs while preserving Brian's apparent idea of making it discoverable to people who think in terms of "moving" the child: https://github.com/tahoe-lafs/tahoe-lafs/pull/36 Please review!

comment:56 Changed at 2013-04-09T17:57:58Z by warner

  • Keywords review-needed removed
  • Resolution set to fixed
  • Status changed from new to closed

I like those changes.. landed.

comment:57 Changed at 2013-04-10T02:06:51Z by daira

Reviewed, +1 for zooko's changes.

Note: See TracTickets for help on using tickets.