See also OneHundredYearCryptography.

= Goals for new filecaps =

This is a place to record desiderata for the next version of our
mutable/immutable filecaps. Many of the design requirements are spread out
across separate tickets: this page is here to consolidate them. We should not
release a new filecap format without checking it against everything on this
list.

There will be a related pair of new encoding designs/protocols (or equivalent;
it has also been suggested that a single protocol could support both mutable and
immutable files). The NewImmutableEncodingDesign and NewMutableEncodingDesign
pages will hold those design discussions.

Ticket #432 was the starting point: it contained a list of features.

[query:keywords~=newcaps|newurls All tickets tagged 'newcaps' or 'newurls'] ([[TicketQuery(keywords~=newcaps|newurls, count)]])

== make them real URIs ==

Kevin Reid points out that the Tahoe calls URIs are not actually URIs (in the
established sense). To make them real, we need to:

 * make them start with {{{x-tahoe:}}} or {{{tahoe:}}} (or {{{lafs:}}}),
   register {{{tahoe:}}} with IANA (#418) (#683)
 * understand how URI/URL/URNs are built, decide about hierarchical segments
   vs non-hierarchical segments. What's magical about a leading double-slash?
   Do we need one?
 * according to #683, a URI '''identifies''' a resource, but does not
   necessarily provide enough information to actually access it (i.e. if you
   have a URI and somebody pointed you at a file, you could confidently tell
   them whether or not it was the right file, but if you only have the URI,
   then you might not be able to find the file without additional
   information). If the cap has both identifying and location information,
   it's called a URL.
 * Tahoe filecaps are meant to be URLs (they are intended to provide location
   information), but to really make that work, you also need to define which
   grid you're talking about. So far this has always been implicit, but that
   has caused us problems. #403 talks about making an explicit "gridid" and
   would provide a procedure to get from a gridid string to a set of storage
   servers. The existing tahoe codebase could use the introducer FURL as a
   gridid, if there were a good place to put it in the filecap (#683 touches
   on this).
  * from the point of view of a web browser, you also need a gateway service
    (the Tahoe client node with a webapi frontend). The tahoe URLs that we've
    been passing around so far always reference one of these, either by
    assuming that {{{http://localhost:8123}}} is a suitable gateway or by
    explicitly referencing an external gateway like testgrid.allmydata.org
    (with deleterious effects on security and availability). I hope that our
    new filecaps are defined independently of a webapi gateway used to access
    them, and that we have a clear procedure for starting with a filecap and
    a gateway HTTP URL, and ending with the contents of the file.

== Make them shorter, prettier, and easier to use ==

 * Short and not so ugly. This is important to enable
   cut-and-paste (see below), but also just because people are
   suspicious and averse to long and ugly URLs. See #882 for
   notes in which dozens of people have spontaneously complained
   about the current URLs. By contrast, tiny URLs such as
   tinyurl.com, bit.ly, etc. are ubiquitous nowadays; users have
   no problem with those -- see Twitter.
  * I (warner) am curious about where the suspicion comes from. Do long URLs
    make people think they're being attacked, some sort of browser buffer
    overrun thing? Or that they're being phished, with a URL that a human
    would evaluate differently than their browser? I agree that people
    (including me) don't like long URLs, but I've never pushed anyone to
    explain the "suspicion" aspect. One comment in #217 says "smells a bit
    spammy", and a later one says "Spooks me every time".
   * It's likely because it's difficult for a human to verify there isn't hidden information
     in there, or a hidden URL, that they're sending out or visiting that they therefore
     can't anticipate or intelligently control. When people see a long hex string, perhaps it represents
     information that the person crafting it wants to hide from the person using it.
     I totally understand the skepticism; however, in this case there's nothing to be done, I think.
     -midnightmagic
 * Enable convenient cut-and-paste. If caps are too long they'll wrap in
   email. If they contain lots of word-breaking characters then you have to
   drag after you've double clicked (this is probably ok). If the word-broken
   sections are small and at the beginning or end then you have to be very
   precise about that drag. The best design would be a single short
   non-word-breaking string. The next best will be to have a large
   non-word-breaking string at the start and end, with smaller segments (if
   necessary) in the middle. Note that {{{tahoe:}}} is an easy target, but
   {{{x-tahoe:}}} is not (you'd have to double-click on the "x").
 * Usable in a browser. Specifically, it should be easy to actually use a
   filecap that you get in email or IM, and many email/IM clients will look
   for http URLs and make them clickable. If tahoe filecaps start with
   {{{http:}}}, then they'll be made clickable. This is at odds with the
   IANA-friendly {{{tahoe:}}} prefix. Clients may make {{{tahoe:}}} URIs
   clickable too (I've seen them make other letters-then-colon strings
   clickable, even when the letters are not "http"), so perhaps a reasonable
   solution is to provide an OS-level URI handler for the {{{tahoe:}}}
   scheme, which could embed the filecap in an http URL and submit it to a
   webbrowser (i.e. when you click on {{{tahoe:foo}}}, a helper program is
   launched with {{{tahoe:foo}}}, and that in turn launches your web browser
   with {{{http://localhost:8123/foo}}}). (#52)

== make them long enough to be secure ==

We want filecaps to be as short as possible, but no shorter. There are
several lower bounds on the length:

 * confidentiality: A large computing effort should not be able
   to obtain the plaintext of a tahoe file without knowing the
   readcap. We require reasonable margin against improvements in
   hardware speed and organizational efficiency/motivation of
   distributed efforts (e.g. could a million PS3 owners break a
   filecap?). This currently implies a 128 bit confidentiality
   field.
 * integrity: a large computing effort should not be able to
   produce shares which will be accepted by the readcap holder
   but which do not result in the same file as created the
   original uploader (and retrieved by other downloaders). We
   desire all three of the standard hash properties (collision
   resistance, first-pre-image resistance, second-pre-image
   resistance) to also apply to tahoe immutable files and their
   filecaps. This currently implies a 128bit (or 256bit?)
   integrity field.
  * variable-length integrity field (#102, comment 16+17),
    allowing users to decide between short caps and strong
    integrity guarantees
 * storage collision resistance (#753): a Tahoe grid should be
   able to store trillions of files and still have a vanishingly
   small chance of two files using the same storage-index (and
   thus confusing each other's shares). The storage-index is
   generally compressed out of the filecap, by deriving it with
   various hashing stages on the other filecap parameters. The
   shortest value in this derivation chain must be at least
   128bits long, and preferably about 192bits long.


== other features ==

 * Self-identifying. It should be visually clear what sort of filecap the
   string represents: read-write or read-only, mutable-or-immutable,
   file-or-directory. This is especially important when sharing tahoe objects
   over out-of-band channels like IM and email: it should be easy for the
   user to tell whether they're giving away readonly access or read-write
   access. We've considered prefixes like {{{DWM..}}} for "Directory
   Writeable Mutable" and {{{FRI..}}} for "File Readonly Immutable" (#102
   comment 12). If these are jammed against the (base62) crypto bits it may
   be difficult to tell where the prefix ends and the crypto bits begin,
   especially because the crypto bits will be using the same character set
   ({{{FRIDWM...}}}). It might be a good idea to separate the type prefix
   from the cryptobits: {{{FRI-cryptobits}}} or {{{FRI/cryptobits}}}.
  * in addition, tahoe URIs should be distinguishable from local filenames by
    a CLI tool, so that {{{tahoe cp $CAP local/foo.txt}}} is unambiguous.
    (unfortunately, the current practice of using "tahoe:" as a default alias
    name collides with this badly, but perhaps if the new URIs include the
    double-slash, this won't be a problem:
    {{{tahoe cp tahoe://CAP local/foo.txt}}} copies from a specific URI,
    while {{{tahoe cp tahoe:blah local/foo.txt}}} copies from a child of
    the "tahoe:" alias).
  * I'd like to make it easy to layer uses on top of one another: since
    directories are just a specific way of interpreting the contents of a
    (mutable) file, let's make the directory cap be closely related to the
    underlying filecap. For example, if we end up using
    {{{tahoe://MR/cryptobits}}} to describe a read-only mutable file
    referenced by "cryptobits", then we could use
    {{{tahoe://D/MR/cryptobits}}} for the directory that uses it as a backing
    store. The rule would be that {{{tahoe://D/$A}}} would be handled by
    fetching {{{tahoe://$A}}} and then interpreting its contents as a
    directory structure. Then reading immutable-dirnodes (#607) would be
    trivial. Another way to think about this is that if our filecaps were
    verbose s-expressions, these caps could be expressed as "(readonly
    (mutable cryptobits))" and "(directory (readonly (mutable cryptobits)))".
 * provide for verifycaps, repaircaps, and traversalcaps (#308, #217).
   Repaircaps in particular may require a grant of storage authority, which
   might entail a cap format that can accept arbitrary extra non-hierarchical
   fields. Appendcaps or "drop-box" writecaps might fall into this same
   space. But remember that URIs should identify objects, not the action that
   you want to do on it: a webapi scheme may use a POST/PUT/DELETE method, or
   append a t=json adverb, or alternatively encode the verb/adverb into the
   HTTP url (think {{{GET .../filecap/json}}} or
   {{{PUT unlinked/ciphertext}}}), but these are independent of the underlying
   filecap. Exotic cap types do not need to be short.
 * provide ciphertext access. Reading from a verifycap should give you
   ciphertext. It should be possible to upload ciphertext directly.
 * provide for a grid-identifier, possibly on the MSB end, e.g.
   {{{tahoe://grid1234/IR/cryptobits}}}. Perhaps let some contexts define a
   "default grid id", such that {{{tahoe://IR/cryptobits}}} is expanded to
   mean {{{tahoe://grid1234/IR/cryptobits}}}. Something like
   {{{tahoe://grid1234/D/MR/cryptobits}}} should reference
   {{{tahoe://grid1234/MR/cryptobits}}}. (#403)
 * permit multiple encodings of the same file (same k, different N) to use
   each other's shares (#678)
 * be derived from a hash of the plaintext to detect bugs in the decryption
   or erasure-decoding (#453) and possibly also to be able to easily give
   users a hash of the plaintext if they want it (#280)

== Can caps have Forward Secrecy? ==