This is a place to share our work on the Accounting task.

Accounting

Tahoe-1.1 is lacking a significant feature: the ability for the admin of a storage server to keep track of how much disk space is being used by various clients. Friendnet operators can use this information to see how much their friends are using (and perhaps ask them to use less). Commercial grid operators can use this to bill customers according to how much space they use, or to limit them to a pre-defined plan ("1GB maximum"). (related tickets: #119)

Tahoe needs better handling of disk-full situations (related tickets: #390), but this is a server-wide issue, whereas Accounting is specifically per-user.

Potential Requirements

Here is a list of potential requirements, in no particular order of when (or if) we actually need them to be completed:

1. only grant space to approved clients: "Larry the Leech" should not be able to upload files or cause existing files to be retained
2. be able to answer the question "How much space is Bob using?" 2a. asking this question about a single server (friendnet) 2b. asking this question system-wide: summed across all storage servers (commercial grid)
3. prior restraint: prevent Bob from consuming more than X bytes per server
4. disable a previously-allowed account / revocation
5. expiration: revoke permission after some amount of time, unless explicitly renewed
6. delegation: if Bob has permission, he can grant some to Little Bobby Junior 6a. subdivision / resellers : commercial grid operator grants space to a business partner 6b. repair caps: clients delegate limited upload authority to a repairer 6c. renewal cap: clients delegate lease-renewal authority 6d. helper: clients enable the helper to upload files for them
7. auditing: who owns this share, how did they get permission to upload it
8. reconcilliation / garbage collection : which shares does Bob own?
9. measure traffic: how many bytes did Bob upload or download (as opposed to how much is he currently storing)

Immediate Goals

We've established a smaller set of goals for the next few releases:

• 1.2 (august 08):
  • be able to answer "how much does Bob store?"
  • deny service to Larry The Leech
  • enable accounting in both friendnet and commercial grids
  • enable accounting in webapi and helper interfaces

• 1.3 (september?)
  • be able to enumerate Bob's shares, for reconcilliation

• next after that
  • more generalized delegation

Design Overview

As touched upon in source:docs/proposed/accounts-pubkey.txt (and source:docs/proposed/accounts-introducer.txt), each share on a storage server is kept alive (in a garbage-collection sense) by one or more "leases", and each lease is assigned to a given "account"/"user"/"owner". The server has an imaginary "lease table" (imaginary in the snese that it is not actually implemented as a giant table: instead the data is broken up into more efficient/robust pieces). This two-dimensional lease table has Storage Index along one axis, and Account on the other, and each cell of the table represents a potential lease.

Each account-owner gets control over their column of the table: they can add leases to existing shares, upload new shares (which immediately acquire new leases), cancel their lease on a share (possibly causing the share to be garbage-collected), or get a list of all of their leases (for reconcilliation).

Some clients may be "super-powered", meaning that they may have the authority to affect more than one row of the table. It may be necessary to give a Repairer this sort of authority to let it keep files alive when the original uploading client is not participating in the maintenance process. POLA dictates that we try to avoid needing this sort of authority inflation, so superpower delegation is just a fallback plan.

The admin of each storage server decides their own policy by configuring their server with various certificates and public keys: fundamentally, storage authority originates with the server, and is delegated outwards to the clients. Clients are configured with certificates and private keys that allow them to use some portion of the server's authority.

Each time a client uploads a file (or otherwise makes use of storage authority), they must demonstrate their authority to each server, through a negotiation protocol. The client.upload() API will be modified to accept a new argument, tenatively named "cred=", that represents this authority. The webapi will also acquire such an argument, allowing the HTTP client to pass its authority to the webapi server so the server can perform the upload.

Design Pieces

• Add cred= to upload API
  • client.upload() needs a cred= argument
  • the webapi PUT/POST commands need a cred= argument
  • the javascript-based webfront program (used by allmydata.com) needs cred
  • the human-oriented "wui" needs a way (cookies? sessions?) to express storage authority

• Define how to configure clients with their storage authority

• define how to create these credentials
  • certificate-signing tools
  • "tahoe sign" subcommand

• define how to configure servers with their certificates
• changes to Introduction
  • advertise accepted pubkeys in the storage-v2 announcements?
• changes to peer selection
• furlification process, persistence/optimization
• label format: how should leases be labeled
• usage-table management: databases, size totals, what to store in each lease
• Usage/Aggregator? service
  • web interface
  • petname database / display