make tahoe Tor- and I2P-friendly

[warner]

Jake Appelbaum and I were talking at the last hackfest about what it would take to run Tahoe safely through the Tor (anonymizing onion-router) proxy.

I figured it wouldn't take much: just removing the automatically-added local IP addresses from the advertised FURLs. You'd treat the tubid as a pseudonym (i.e. never run this node without Tor). Listeners would be a complete loss (that is, other nodes would not be able to establish connections to yours, until 1: we get SOCKS4A client-side support into Twisted, 2: make sure Foolscap can use .onion names in connection hints, and 3: add a Foolscap Listener that establishes itself on a Tor hidden-service port).

But beyond that, it should just be a question of running Tahoe under 'tsocks', so that all of its outbound connections go through the socks proxy and then through Tor to the other servers.

So the task for this ticket: provide a configuration knob to override the default "find all my IP addresses and add them to the connection hints (via Tub.setLocation)" behavior, and instead provide a hard-coded list of hints instead.

[warner]

fixed misspelling of Jake's name.. oops.

[warner]

9976bd439ac50be5 adds "tub.location" to tahoe.cfg . I haven't tested it at all, but I think this ought to handle the main issue. I think all that's left is for someone Tor-knowledgeable to test it out and write up a short guide on running tahoe under tsocks.

[zooko]

To close this ticket write up documentation of how to use Tahoe-LAFS with Tor. Also use Tahoe-LAFS with Tor and make sure that it works. :-)

[zooko]

See the docs about tub.location in configuration.txt.

[ioerror]

Should the Tor documentation show how to run Tahoe transparently? Should it be for a Tahoe storage node? For a Tahoe client connecting to the test grid?

What would be useful?

[ioerror]

It seems like we have a few possible interesting use cases:

1. Tahoe as a client to any grid where all connections leave the Tor network
2. Tahoe storage nodes that advertise their address as their respective .onion address
   1. All outgoing TCP connections from the storage node must only connect through Tor
   2. The storage node should not leak it's own known IP
3. A full Tahoe grid that is only available over Tor
   1. The introducer should reject listing all peers that aren't .onions
   2. All nodes should be reachable as .onions (thus we solve the NAT problem)
   3. It's super slow and everyone loves the idea but the practice is full of shame

I think that 1 is currently possible with tsocks and a functional Tor client. 2 seems to be easy enough to do with the tub.location patch and iptables wizardry. 3 seems likely best solved with transparent Tor proxying and manual tub.location stuffing.

As a side note, it's probably the case that Tahoe should weight reconstruction and not use blocks from .onions as their first choice for streaming data.

Is there anything else that's missing? What other ways should we use Tor and Tahoe together?

[sid77]

Maybe an initial setup could be a tahoe running machine transparent proxied through Tor, just to see if it work.

[leif]

Running a grid on Tor hidden services works for me (by setting tub.location to an onion address and running tahoe under usewithtor), but I've noticed two IP address leaks so far (#1942 and #1947).

[leif]

Currently Tahoe is usable with tor via the usewithtor/torsocks LD_PRELOAD tool, but it would be nice to have less hacky proxy support built-in.

Over at ​http://foolscap.lothar.com/trac/ticket/217 david415 is working on adding twisted endpoint support to foolscap. Once that is done, I was initially thinking that Tahoe should load (as yet unwritten) twisted plugins which register the txsocksx library as endpoints called "socks" and "tor" (the latter of which would automatically use the tor socks port 9050, and perhaps also fall back to the other common tor port 9150). After simply loading these plugins, we could use furls in the form of "tor:foo.onion:nnnn" or "socks:example.com:nnnn:sockshost=127.0.0.1:socksport=9050". However, there are two problems with this approach:

• It would only allow a user to use tor when furls in announcements received from the introducer tell it to.
• In the case that the whole grid is expected to be using tor: furls, anyone could announce a non-tor storage furl which would cause clients to connect without tor and reveal their IP address.

So, endpoint support alone is not enough to make Tahoe tor-friendly. But, I still think adding endpoints to foolscap seems like the correct thing to do.

My current thought is that Tahoe should have an "always_use_tor" option which causes all furls to be rewritten as tor: endpoints before being passed to foolscap. If this option is not enabled, tor connections will be made if a tor endpoint is used, but not otherwise. So, users of mixed tor/non-tor grids can leave it disabled if they prefer faster connections to non-tor nodes.

This is almost enough, but still leaves a problem for users who want to connect via Tor to all storage nodes *except* for their own. I think the solution here is to use david415's introducerless branch (cf. #467) and add a new per-server option to make exceptions to the always_use_tor directive.

[leif]

Related ticket: #1942 (replace google chart in wui with d3.js: it leaks information)

[dawuud]

I've stopped commenting in foolscap trac ticket #217 because I think that we can simply convert old connection hints into twisted connection endpoints. It should be obvious from Leif's comment ( ​https://tahoe-lafs.org/trac/tahoe-lafs/ticket/517#comment:15 ) that we should not construct endpoint descriptors from location hint 3-tuples (type, host, port).

I have commented on some issues with getting twisted endpoints to work in the client side of foolscap here in foolscap trac ticket #203:

​http://foolscap.lothar.com/trac/ticket/203#comment:30 ​http://foolscap.lothar.com/trac/ticket/203#comment:31

[str4d]

zooko, warner and I (and others who I unfortunately don't know handles for) had a lengthy discussion about this at RWC. Proper notes to follow, either here or on ​http://foolscap.lothar.com/trac/ticket/203

[dawuud]

Greetings!

I have written Tor client and server endpoint parsers...

Here's the tor client endpoint I wrote... based on txsocksx. It has retry logic for guessing the system tor socks port: ​https://github.com/david415/txsocksx/tree/endpoint_parsers_retry_socks

• should we instead use txtorcon to launch a new tor proc with which to speak socks to instead of trying to connect to the system tor proc?

• or maybe if it fails to socks connect to all the tor ports in it's list... it fails-back to launching it's own tor proc with txtorcon

Here's the tor server endpoint I rewrote to launch tor in it's listen method... so that it remains compatible with Twisted IStreamServerEndpoint interface... also I wrote a parser for it : ​https://github.com/david415/txtorcon/tree/endpoint_parser_plugin-rewrite3

• Meejah and I were discussing the prospect of using the controlPort of the system tor proc to create Tor Hidden Services :

10:40 < dawuud> meejah: so your earlier point was that we could use the controlPort to configure a hidden service on the system tor? 13:36 < meejah> dawuud: yes, we can use controlPort to configure a new hiddenservice in a running system Tor *however* 13:37 < meejah> for how, e.g., it's installed on debian it won't work for most people (unless you're running as 'debian-tor' user) 13:37 < meejah> there is a nearly-fixed tor bug adding a "group readable" option for hidden-service dirs, and then this will all work fine 13:45 < coderman> oh fuuuu i'm the worst Tor contrib ever ​https://trac.torproject.org/projects/tor/ticket/11291 meejah ? 14:07 < meejah> coderman: yes, that's the one :)

• and then the question become *when* to launch tor to create a hidden service? I think perhaps as a fallback to first attempting to use the system tor controlPort to create the hidden service.

[dawuud]

I've made lots of progress in the past few months.

Meejah and I released the new Tor Hidden Service endpoint and parser: ​https://github.com/meejah/txtorcon

The Tor client endpoint is also in a working state: ​https://github.com/david415/txsocksx

I have written a 100% backwards compatible fork of Foolscap which uses Twisted endpoint descriptor strings for both the client and server side: ​https://github.com/david415/foolscap/tree/endpoint_descriptors_server-rewrite6

My original Tahoe-LAFS "Tor only mode" feature is here: ​https://github.com/david415/tahoe-lafs/tree/ticket517

However my second rough draft is higher quality: ​https://github.com/david415/tahoe-lafs/tree/david-truckee

I have tried out this branch of Tahoe-LAFS and found it functional for the client side... The server side has not yet been tested and I believe there to be an issue:

Foolscap needs to become even more endpoint agnostic... in order for the server side of Tahoe-LAFS to use Tor with the txtorcon endpoint... but to do this Twisted needs a new plugin system to serialize IListeningPort objects into endpoint descriptor strings. I will soon attempt to either discuss this on the Twisted mailing list or open a Twisted trac ticket.

[dawuud]

This trac ticket requires that Tahoe trac ticket #1010 also be resolved; It will benefit both Tor and I2p users.

Specifically the design specified in comment 37 by Zooko:

comment:37:ticket:1010

[dawuud]

We need to also resolve this sub-ticket which is concerned with randomizing client IDs: ​https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2384

[dawuud]

I recently opened this related txtorcon ticket: fire endpoint listen deferred when HS desc is desseminated ​https://github.com/meejah/txtorcon/issues/107

here's the related tor trac ticket: ​https://trac.torproject.org/projects/tor/ticket/3523

[warner]

FYI, ​http://foolscap.lothar.com/trac/ticket/236 is a plan that dawuud and I came up with for making Foolscap handle Tor/i2p Listeners and connection-hints cleanly.

[dawuud]

I attended the Tahoe-LAFS nuts and bolts meeting yesterday and Daira suggested for the Tahoe-LAFS portion of this integration project we should write documentation first. Zooko and I worked on this last year... here it is:

​https://github.com/david415/tahoe-lafs/blob/tor-i2p-documentation/docs/anonymity-configuration.rst

Perhaps this document needs some review and corrections.

What's the next step for the Tahoe-LAFS portion of the integration? Implement some of the features as specified in the above documentation?

Maybe this ticket should be resolved first? ​https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1010

Ahh furthermore we may have to fix up some changes in txtorcon for this project... such as: ​https://github.com/meejah/txtorcon/issues/123

[daira]

Rebased docs to master: ​https://github.com/tahoe-lafs/tahoe-lafs/commits/517.tor-i2p-documentation.0

[warner]

From today's meeting, we're aiming for a static approach, in which all allocation (of hidden services and regular TCP ports) takes place during the execution of tahoe create-node, and the runtime code (tahoe start) just has to follow the instructions left behind in tahoe.cfg.

This would simplify Tahoe's main allmydata.node.Node constructor to be completely synchronous, removing the awkward code that waits for the Tub to start running before it can register FURLs for whatever services are being exposed.

It would force server operators to choose a TCP port and a connect-hint hostname (or IP address) when they create the server (or modify the config file). New nodes would no longer use IP-address autodetection (but old ones would continue to work).

The necessary tasks (many of which can be parallelized) are:

• Get ​http://foolscap.lothar.com/trac/ticket/236 done
• Move the controlport/logport to a separate Tub, with its own private key, defaulting to only listening on 127.0.0.1 with a dynamically-allocated port, and add some config knobs to control this
• Replace the remaining Tub.setLocationAutomatically() (in the keyserver and stats-gatherer) with something else
• Replace the asynchronous IP-address-autodetect code used when tub.location=AUTO with a synchronous form (using stdlib's subprocess module). Move this call from Node._setup_tub into Node.create_tub, which runs during the constructor. Move the Tub.setLocation up there too. Replace when_tub_ready() with synchronous calls, and remove the whole async _startService method.
• change tahoe create-client to make a tahoe.cfg that doesn't listen on anything
• change "tahoe create-node" to require --listen and --location
• change "tahoe create-node" to handle the new Tor arguments, once we figure out how they should be spelled
• add Tor support to the main tahoe runtime: launch the Tor executable if necessary, configure and install the Foolscap plugins if necessary
• finally remove the code that writes to Node._portnumfile

We also need to figure out the override-specific-servers mechanism (to use TCP for some whitelisted servers, even though we're using Tor for everything else). I think this will involve a Foolscap plugin that can either use Tor or TCP for TCP hints, depending upon some lookup function that it calls at runtime.

[dawuud]

track progress of tahoe CLI changes here: ​https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2490

[dawuud]

tracking progress of synchronous node startup here: ​https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2491

[daira]

Replying to warner:

• change "tahoe create-node" to require --listen and --location

I'm concerned that this will be a significant usability regression.

[str4d]

Replying to warner:

From today's meeting, we're aiming for a static approach, in which all allocation (of hidden services and regular TCP ports) takes place during the execution of tahoe create-node, and the runtime code (tahoe start) just has to follow the instructions left behind in tahoe.cfg.

This would simplify Tahoe's main allmydata.node.Node constructor to be completely synchronous, removing the awkward code that waits for the Tub to start running before it can register FURLs for whatever services are being exposed.

This all fits well with I2P too. Like dawuud has said elsewhere, it's easy to have a static HS / Destination, and let Tor / I2P handle the network side. An I2P Destination can be set up automatically in the same way that a Tor HS now also can be, although right now txi2p assumes a running router. It wouldn't be hard to implement an auto-start feature for a configured router location, but this is usability semantics that is irrelevant to the current approach.

It would force server operators to choose a TCP port and a connect-hint hostname (or IP address) when they create the server (or modify the config file). New nodes would no longer use IP-address autodetection (but old ones would continue to work).

I'm ambivalent on this, so whatever makes it easier for you. I tend to think that servers aren't really going to be moving around too much, and it's obvious enough to the sysop when they do and this breaks, and simple enough to fix.

The necessary tasks (many of which can be parallelized) are:

• Get ​http://foolscap.lothar.com/trac/ticket/236 done
• Move the controlport/logport to a separate Tub, with its own private key, defaulting to only listening on 127.0.0.1 with a dynamically-allocated port, and add some config knobs to control this
• Replace the remaining Tub.setLocationAutomatically() (in the keyserver and stats-gatherer) with something else
• Replace the asynchronous IP-address-autodetect code used when tub.location=AUTO with a synchronous form (using stdlib's subprocess module). Move this call from Node._setup_tub into Node.create_tub, which runs during the constructor. Move the Tub.setLocation up there too. Replace when_tub_ready() with synchronous calls, and remove the whole async _startService method.
• change tahoe create-client to make a tahoe.cfg that doesn't listen on anything
• change "tahoe create-node" to require --listen and --location
• change "tahoe create-node" to handle the new Tor arguments, once we figure out how they should be spelled

The mailing list post referred to a bunch of Tor-specific arguments. Functionally, I assume the Tor-specific arguments will be passed to the Tor plugin, the I2P-specific arguments will be passed to I2P, etc.

I think from a usability perspective we need to make sure it is obvious to the user which arguments do what (the --tor-* arguments definitely achieve this), but also we don't want to overwhelm the user with configuration options. I can see at least the following distinct setups being viable:

• Clearnet-only
• Tor-only
• I2P-only
• Tor + I2P

and potentially:

• Clearnet + Tor
• Clearnet + I2P
• Clearnet + Tor + I2P

The --tor-only and --listen-on-tor -style arguments IMHO aren't going to scale nicely. I don't expect more than Tor and I2P realistically (in the near future), but if each combination is being given its own flag... Maybe instead there could be a single --listen-on= argument that takes a comma-separated list of plugin identifiers, ie. --listen-on=tcp,tor,i2p. This would also then simplify the logic for plugin-specific arguments: --PLUGINIDENTIFIER-* gets passed directly to PLUGIN.

• add Tor support to the main tahoe runtime: launch the Tor executable if necessary, configure and install the Foolscap plugins if necessary

• add I2P support to the main tahoe runtime ;)

• finally remove the code that writes to Node._portnumfile

We also need to figure out the override-specific-servers mechanism (to use TCP for some whitelisted servers, even though we're using Tor for everything else). I think this will involve a Foolscap plugin that can either use Tor or TCP for TCP hints, depending upon some lookup function that it calls at runtime.

Again, this needs to be planned to handle more than just Tor.

[warner]

From today's meeting, folks seemed comfortable with the following tahoe.cfg syntax to configure the client side:

[node]
anonymous = true

[connections]
tor.socks-proxy = HOST:PORT
tor.(otherstuff)=??

tcp.socks-proxy = HOST:PORT

i2p.connect-proxy = URL

The presence of [connections]tor.socks-proxy= will cause Tahoe to install a Foolscap connection plugin that handles FURL connection-hints which begin with tor: by sending them to the given SOCKS5 proxy. The presence of tcp.socks-proxy= makes it remove all plugins and then install a similar plugin that handles tcp: connection hints. And the i2p plugin is similar, but uses an HTTP "CONNECT" proxy instead of SOCKS, since apparently that's how the I2P client does it.

If [node]anonymous is true, we'll enforce two things, as per #1010:

• [node]tub.location does not contain "AUTO"
• [connections]tcp.socks-proxy is set

(We could make this stronger: require tub.location to only contain tor: or i2p: hints, require the hostnames in tub.location to end in .onion or .i2p, and maybe require the tcp.socks-proxy to point at 127.0.0.1 instead of some potentially-exterior host. But for now I think these two are sufficient)

We'd like to allow tub.location to be empty. We need to make sure Foolscap can handle that (on both server and client sides), and then eventually we want tahoe create-client to leave tub.location and tub.port empty (meaning "don't listen at all"). Note that this will be spelled tub.location= (empty string), and not #tub.location= (missing key), because the latter triggers the current default of "AUTO".

For this first step, we'll just do the client side, and only in tahoe.cfg. This should be enough to:

• use a previously-configured (and running) Tor/I2P client
• manually configure Tor/I2P to forward inbound connections to a tahoe server
• advertise tor: or i2p: connection hints (via tub.listen)
• utilize tor: and i2p: connection hints from others

In a second phase, we'll use the tor.(otherstuff) fields (which might not want to live in [connections], I dunno) to launch a Tor daemon at node startup (maybe using a pre-configured base directory under ~/.tahoe/, maybe with a config file that's built from the tahoe.cfg options). Then in the third phase, we make changes to the tahoe create-client/tahoe create-node arguments, to configure that Tor daemon (and allocate onion servers, etc).

But for now, I think these tahoe.cfg changes will enable manual Tor/I2P setup without needing to figure out the --tor-only / --listen-on-tor / etc arguments just yet.

[warner]

Or, for history, one syntax which got a -0 (for being kinda YAGNI) was to declare that any [connections] key that matched (\w+)\.socks-proxy would cause a Foolscap plugin to get installed that routes $1-type connection hints to the given SOCKS proxy (and a similar thing for CONNECT proxies). Then Tahoe wouldn't know anything special about Tor or I2P, it'd just be the names of the tahoe.cfg keys that mattered. That would let users add new connection types in the future without changing the Tahoe code. We felt that people aren't adding new anonymous-router systems very often (for good reasons), and there are other Tor/I2P-specific things that probably *will* require special handling (like launching a Tor daemon when Tahoe starts up), so the benefit wouldn't really be worth it.

[str4d]

Replying to warner:

From today's meeting, folks seemed comfortable with the following tahoe.cfg syntax to configure the client side:

[node]
anonymous = true

[connections]
tor.socks-proxy = HOST:PORT
tor.(otherstuff)=??

tcp.socks-proxy = HOST:PORT

i2p.connect-proxy = URL

The presence of [connections]tor.socks-proxy= will cause Tahoe to install a Foolscap connection plugin that handles FURL connection-hints which begin with tor: by sending them to the given SOCKS5 proxy. The presence of tcp.socks-proxy= makes it remove all plugins and then install a similar plugin that handles tcp: connection hints. And the i2p plugin is similar, but uses an HTTP "CONNECT" proxy instead of SOCKS, since apparently that's how the I2P client does it.

That is how I2P does it now, using duck's HTTP proxy patch. But that was only because it was the simplest patch; as I believe was raised in that ticket's thread, Tahoe / Foolscap isn't really an HTTP protocol. And I2P could just as easily use a SOCKS proxy. But this is only useful for clients.

This line of discussion raises the question: for this initial client-only phase, is the intention to just use plain SOCKS (via txsocksx) and HTTP (via something like duck's I2P patch) proxies, rather than txtorcon and txi2p?

If txi2p is going to be used, then the option needs to be i2p.sam-api=. I assume that tor.socks-proxy and i2p.sam-api would be allowed to be empty, which would mean "use the default value".

If [node]anonymous is true, we'll enforce two things, as per #1010:

• [node]tub.location does not contain "AUTO"
• [connections]tcp.socks-proxy is set

Should this be "one of tcp.socks-proxy, tor.socks-proxy, or i2p.sam-api"?

(We could make this stronger: require tub.location to only contain tor: or i2p: hints, require the hostnames in tub.location to end in .onion or .i2p, and maybe require the tcp.socks-proxy to point at 127.0.0.1 instead of some potentially-exterior host. But for now I think these two are sufficient)

We'd like to allow tub.location to be empty. We need to make sure Foolscap can handle that (on both server and client sides), and then eventually we want tahoe create-client to leave tub.location and tub.port (meaning "don't listen at all").

For this first step, we'll just do the client side, and only in tahoe.cfg. This should be enough to:

• use a previously-configured (and running) Tor/I2P client
• manually configure Tor/I2P to forward inbound connections to a tahoe server
• advertise tor: or i2p: connection hints (via tub.listen)

I assume that tub.location is meant, which would be empty for clients. And while only the SOCKS/HTTP proxy side is implemented, then with [node]anonymous = True, tub.listen would be empty because there would be no way to create a Tor HS or I2P Destination.

• utilize tor: and i2p: connection hints from others

In a second phase, we'll use the tor.(otherstuff) fields (which might not want to live in [connections], I dunno) to launch a Tor daemon at node startup (maybe using a pre-configured base directory under ~/.tahoe/, maybe with a config file that's built from the tahoe.cfg options). Then in the third phase, we make changes to the tahoe create-client/tahoe create-node arguments, to configure that Tor daemon (and allocate onion servers, etc).

But for now, I think these tahoe.cfg changes will enable manual Tor/I2P setup without needing to figure out the --tor-only / --listen-on-tor / etc arguments just yet.

+1.

[daira]

Replying to warner:

From today's meeting, we're aiming for a static approach, in which all allocation (of hidden services and regular TCP ports) takes place during the execution of tahoe create-node, and the runtime code (tahoe start) just has to follow the instructions left behind in tahoe.cfg.

This would simplify Tahoe's main allmydata.node.Node constructor to be completely synchronous, removing the awkward code that waits for the Tub to start running before it can register FURLs for whatever services are being exposed.

It would force server operators to choose a TCP port and a connect-hint hostname (or IP address) when they create the server (or modify the config file). New nodes would no longer use IP-address autodetection (but old ones would continue to work).

I definitely feel like I'm missing something here. If old nodes continue to work --with AUTO meaning to autodetect at node startup-- how can the Node constructor be completely synchronous and tahoe start just have to follow the instructions left behind in tahoe.cfg?

[warner]

It's because we can get our IP address synchronously, via blocking subprocess.check_output(["ifconfig"]). We'll do this before Twisted has taken over the SIGCHLD handler (which happens in a reactor "during-startup" event trigger, so after reactor.run, so after allmydata.node.Node is constructed).

We can also allocate a port synchronously. I just landed some code in Foolscap to do this (in foolscap/util.py):

def allocate_tcp_port():
    """Return an (integer) available TCP port on localhost. This briefly
    listens on the port in question, then closes it right away."""
    # We want to bind() the socket but not listen(). Twisted (in
    # tcp.Port.createInternetSocket) would do several other things:
    # non-blocking, close-on-exec, and SO_REUSEADDR. We don't need
    # non-blocking because we never listen on it, and we don't need
    # close-on-exec because we close it right away. So just add SO_REUSEADDR.
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    if platformType == "posix" and sys.platform != "cygwin":
        s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    s.bind(("127.0.0.1", 0))
    port = s.getsockname()[1]
    s.close()
    return port

(mind you, I'd like to confirm that function works correctly on windows.. if you've got a windows box handy, could you run Foolscap's current trunk unit tests and see if test_util.AllocatePort passes?)

Although as I think we talked about before, we probably don't need to use allocate_tcp_port() in tahoe's startup code: the only case where a node starts up with tub.port = tcp:0 and doesn't have a NODEDIR/client.port file is if you created the node with the older version of tahoe create-node, but never actually started it with that version, and now you're trying to start it with the newer version.

The newer version of tahoe create-node will allocate the port before writing out tahoe.cfg, so the runtime will never see a tcp:0 anywhere.

[daira]

Replying to warner:

It's because we can get our IP address synchronously, via blocking subprocess.check_output(["ifconfig"]). We'll do this before Twisted has taken over the SIGCHLD handler (which happens in a reactor "during-startup" event trigger, so after reactor.run, so after allmydata.node.Node is constructed).

We can also allocate a port synchronously. I just landed some code in Foolscap to do this (in foolscap/util.py)

Okay, but aren't these changes independent of the other Tor/I2P changes? I don't see how they motivate the potential breakage of backward compatibility in making AUTO no longer be the default (for non-anonymous nodes that listen at all).

[warner]

Replying to str4d:

That is how I2P does it now, using duck's HTTP proxy patch. But that was only because it was the simplest patch; as I believe was raised in that ticket's thread, Tahoe / Foolscap isn't really an HTTP protocol. And I2P could just as easily use a SOCKS proxy. But this is only useful for clients.

Ah, great, if I2P can do SOCKS instead, that will make the plugin much easier to write. I don't know much about I2P :).

This line of discussion raises the question: for this initial client-only phase, is the intention to just use plain SOCKS (via txsocksx) and HTTP (via something like duck's I2P patch) proxies, rather than txtorcon and txi2p?

I figured that the Foolscap plugin would be implemented by using txsocksx. For just client connectivity, we don't need full control of the Tor/I2P node yet.

Later, to set up a server, we'll certainly want to use txtorcon and txi2p.

If [node]anonymous is true, we'll enforce two things, as per #1010:

• [node]tub.location does not contain "AUTO"
• [connections]tcp.socks-proxy is set

Should this be "one of tcp.socks-proxy, tor.socks-proxy, or i2p.sam-api"?

It'd need to be tcp.something, because that's what will control what happens when the Tub connects with a TCP hint. tor.socks-proxy controls what happens when it connects to a Tor hint, and i2p.* will control I2P hints. The anonymity safety flag must limit TCP.. it doesn't care so much about what happens with hints that have to use a safe connection mode anyways.

[warner]

Updates from today's meeting:

• We established four phases:
• Phase 1: By adding [connections] entries to tahoe.cfg (and installing txsocksx and/or txi2p), Tahoe clients will become capable of connecting to Tor/I2P servers. Additional [connections] entries will make it use Tor for normal servers too. tub.location= will make the client not advertise a listening port. [node]anonymous = true will enforce all this.
• users can manually configure a Tor/I2P server to route inbound connections to a localhost listening port, then set tub.port=tcp:12345:interface="127.0.0.1" and tub.location=tor:abc123.onion:80 (or I2P equivalent) to offer hidden storage service. These hidden servers will only be useful to clients who have made the other phase1 changes, to enable tor/i2p connections
• Phase 2: tub.port= will make the client not even open a listening port. The logport/controlport will be moved off to a separate Tub (which defaults to listening on 127.0.0.1, but can be configured otherwise), so it will listen even though the main Tub does not. [node]anonymous might enforce this.
• Phase 3: Add CLI arguments to tahoe create-client and tahoe create-node to add the right [connections] settings. Maybe tor.socks-proxy=?
• Phase 4: Automate the listen-on-Tor/I2P server settings: add CLI arguments to tahoe create-node that mean "please do whatever is necessary to listen on Tor, and/or I2P, and/or disable normal TCP listening". This will allocate services (keys, descriptors, etc) during create-node, then write whatever decisions have been made into tahoe.cfg. The implementation will not impose any new runtime requirements on the post-reactor-startup phase of node startup (i.e. the onion address must be determined by create-node and written into tahoe.cfg, so it will be available when tahoe run first constructs the Node object).

Also, str4d cleared up my confusion about how I2P connections are made. str4d is completely right, the Foolscap plugin for "i2p:" hints should just ask txi2p to create an Endpoint that does whatever I2P-specific handshaking is necessary, then turns into a regular TCP socket (which means it should handle .startTLS too). There's no need for HTTP CONNECT proxies or SOCKS proxies, txi2p has its own protocol for talking to the I2P daemon, and knows what needs to be done.

txi2p will also be used in phase 4 to allocate the i2p address: tahoe create-node will create an I2P listening endpoint (storing key material into a NODEDIR/private/ directory), tell it to start listening, use .getHost() to learn the .i2p address, then shuts it all down. The address is used to write the correct connection hint into tahoe.cfg tub.location, and the key material directory is used to create a server endpoint string to write into tub.port. As long as the i2p server-endpoint-parser plugin is available at node startup, Foolscap will just ask Twisted to parse tub.port, and that should create the right i2p server endpoint.

I mentioned that, instead, it'd also be ok (in fact it might be faster) to have tahoe create-node call a txi2p convenience function that allocates the address without actually creating an endpoint. The normal endpoint.listen() process creates a tunnel, which involves a lot of network traffic. At create-node time, all we really need to do is generate the keys: we can put off creating the real Endpoint until the node is launched.

[zooko]

Dear Brian: Great! The plan in comment:41 seems like a good plan, and I'm excited about us having consensus and a clear plan with reasonably-sized steps that we can chew. ☺

One detail:

• Phase 1: By adding [connections] entries to tahoe.cfg (and installing txsocksx and/or txi2p), Tahoe clients will become capable of connecting to Tor/I2P servers. Additional [connections] entries will make it use Tor for normal servers too. tub.location= will make the client not advertise a listening port. [node]anonymous = true will enforce all this.

Could we please spell this as tub.location=AUTODETECT instead of tub.location=? That's what I asked for in comment:40:ticket:1010, and I think str4d already implemented something like it (perhaps spelled AUTO instead of AUTODETECT), according to his comment in comment:43:ticket:1010.

[warner]

The current trunk behavior (established in 1.10.1 from 15-Jun-2015) looks like:

• #tub.location=: same as tub.location = AUTO. This is the create-node default.
• no tub.location in the config file at all: same as above
• tub.location = AUTO: use just the autodetected address
• tub.location = hint1,AUTO,hint2: use three hints, replace AUTO with autodetected address
• tub.location = hint1: use an explicit hint, disable autodetection
• tub.location =: use no connection hints at all

Does that change your question? I was claiming that tub.location= will *disable* autodetect, and your question sounded like you wanted it to *enable* autodetect. Are you maybe looking for a tub.location = DON'T LISTEN ON ANYTHING sort of explicit flag, to avoid the subtle confusion between tub.location= and #tub.location=?

To avoid breaking existing nodes, we're stuck with #tub.location= meaning autodetect. Currently, tahoe create-client and create-node write #tub.location= into the new tahoe.cfg file, but we can change that without impacting compatibility.

I'm cool with making don't-listen the default. I'm slightly less cool with adding an explicit don't-listen token (but not totally against it), because then we'd be scanning the config file for two different special values, increasing potential problems for things like "what if my actual hostname matches the special value", etc. But maybe an explicit negative token is clearest / most-usable option.

[str4d]

Replying to warner:

txi2p will also be used in phase 4 to allocate the i2p address: tahoe create-node will create an I2P listening endpoint (storing key material into a NODEDIR/private/ directory), tell it to start listening, use .getHost() to learn the .i2p address, then shuts it all down. The address is used to write the correct connection hint into tahoe.cfg tub.location, and the key material directory is used to create a server endpoint string to write into tub.port. As long as the i2p server-endpoint-parser plugin is available at node startup, Foolscap will just ask Twisted to parse tub.port, and that should create the right i2p server endpoint.

I mentioned that, instead, it'd also be ok (in fact it might be faster) to have tahoe create-node call a txi2p convenience function that allocates the address without actually creating an endpoint. The normal endpoint.listen() process creates a tunnel, which involves a lot of network traffic. At create-node time, all we really need to do is generate the keys: we can put off creating the real Endpoint until the node is launched.

I have ​added an API method that takes a reactor and a path to a keyfile, saves a new Destination to the keyfile, and returns an IAddress containing the information required to connect to that Destination (the same I2PAddress that .getHost() returns). It only requires that the SAM API be accessible.

[str4d]

I've written a simple I2P plugin implementing IConnectionHintHandler, and with a trivial 3-line diff I have successfully run the foolscap calculator example over I2P (manually running a server Destination for the server side, as is done now with the I2P-specific Tahoe build). So I can confirm that foolscap trunk is ready for use in implementing comment:41.

[str4d]

I've rebased a bunch of anon-config documentation commits here:

​https://github.com/tahoe-lafs/tahoe-lafs/pull/189

Some of the content (both in my PR and the document in general) needs revision in light of recent developments above, but I want to get my year-old improvements in first :)

[warner]

str4d: that's great! Are there any changes you'd like to see to the foolscap (client-side) API before I make a new release? Barring any, I plan to cut it later today, so we can make progress on the Tahoe side in tomorrow's meeting.

[str4d]

Replying to warner:

str4d: that's great! Are there any changes you'd like to see to the foolscap (client-side) API before I make a new release? Barring any, I plan to cut it later today, so we can make progress on the Tahoe side in tomorrow's meeting.

For the in-client API, I think it is great. Once I remembered that only the client side was implemented :P it was a trivial patch (see above).

For the plugin API, I have a few thoughts:

• Implementing the API feels a little duplicate-y. It's only four lines (match regex, check for match, grab groups, fallback to None), but...
• There is currently no restriction on the format of the location hints, so e.g. I could require different syntax in my I2P one than others. Is this flexibility that we want to allow?
• By requiring plugins to check for themselves if a string is for them, it allows earlier plugins to intercept the strings of later plugins (but not to modify them).

All three points would be "solved" by having a hint-parsing method that turns hints into groups, looking up an IConnectionHintHandler corresponding to the parsed hint type (via a dict that is filled when addConnectionHintHandler() is called), and passing the parsed groups to hint_to_endpoint() as kwargs. But this is basically re-implementing the Twisted endpoint string API. Is this worth the effort? Or am I over-engineering?

[warner]

Yeah, the intention is that the plugin should make its decision based solely upon the "type", defined as "everything before the first colon". And everything after the first colon is up for grabs by the type: in particular, ipv6 raw addresses will look like tcp:[fe80::1234:5678]:80, so the type parser won't necessarily just split on colons. That rules out a signature of hint_to_endpoint(pieces, reactor). The best we could probably do is hint_to_endpoint(everything_after_the_colon, reactor), which then feels weird because we're no longer passing full hints around.

Even if we continue passing full hints into the plugins, we might clean things up with the type-to-plugin dictionary you described. I'm not super happy about having the plugin return "None" to mean "decline to handle this hint". Let's see, that would obligate us to properly move the legacy "host:port" handling up out of the DefaultTCP plugin (since there'd be no way to register a hint type that would match it), but that's a good idea anyways. It makes it slightly more work to make a plugin that handles multiple types (like a combined "tor:/tcp:"-over-tor plugin), but really you just have to register a single plugin multiple times, no big deal.

I was originally thinking tub.register(plugin) and then plugin.hint_type would be a statically-defined string, but now I'm in favor of tub.register(hint_type, plugin).

So that would change the hint_to_plugin implementation: no more fallback, and if the regexp fails then the hint was invalid and can be ignored (which is distinct from the previous "I don't claim this one, let some other plugin give it a try"). Ah, so we also need an "invalid hint" error which the TubConnector can catch/log/ignore.

I'll see if I can implement the latter before making the release. Thanks for the feedback!

[warner]

Further notes in ​http://foolscap.lothar.com/trac/ticket/236#comment:16

[str4d]

Replying to warner:

Yeah, the intention is that the plugin should make its decision based solely upon the "type", defined as "everything before the first colon". And everything after the first colon is up for grabs by the type: in particular, ipv6 raw addresses will look like tcp:[fe80::1234:5678]:80, so the type parser won't necessarily just split on colons. That rules out a signature of hint_to_endpoint(pieces, reactor). The best we could probably do is hint_to_endpoint(everything_after_the_colon, reactor), which then feels weird because we're no longer passing full hints around.

Fair enough. I guess then we just have to clearly document that plugins will definitely get "type:" at the start of the string, but everything else is allowed to be flexible.

Even if we continue passing full hints into the plugins, we might clean things up with the type-to-plugin dictionary you described. I'm not super happy about having the plugin return "None" to mean "decline to handle this hint". Let's see, that would obligate us to properly move the legacy "host:port" handling up out of the DefaultTCP plugin (since there'd be no way to register a hint type that would match it), but that's a good idea anyways. It makes it slightly more work to make a plugin that handles multiple types (like a combined "tor:/tcp:"-over-tor plugin), but really you just have to register a single plugin multiple times, no big deal.

I was originally thinking tub.register(plugin) and then plugin.hint_type would be a statically-defined string, but now I'm in favor of tub.register(hint_type, plugin).

The idea I had is that the plugin must have a type attribute, like Twisted endpoint plugins have the prefix attribute used to identify them. If we wanted to support handlers with multiple types, then allow prefix to be either a string or a list of strings, and then add the handler to the dict in multiple places. Alternatively, having tub.register(hint_type, plugin) means that the user can optionally use a handler to handle only one or a few of the types it supports.

This raises another point: if moving to the dict-style configuration, how do we handle the case where two plugins are registered handling the same type? Twisted simply uses the first plugin it finds that matches, ignoring others. The current setup does the same thing - the first plugin to match is used. But in a dict-style system, I expect it would be the last foolscap plugin added that is used, because it would replace existing ones. The alternative would be that once a handler for a type is added, subsequent adds are ignored until tub.removeAllConnectionHandlers() is called.

So that would change the hint_to_plugin implementation: no more fallback, and if the regexp fails then the hint was invalid and can be ignored (which is distinct from the previous "I don't claim this one, let some other plugin give it a try"). Ah, so we also need an "invalid hint" error which the TubConnector can catch/log/ignore.

Sounds good.

I'll see if I can implement the latter before making the release. Thanks for the feedback!

[warner]

Having type attributes would also make it easier for these to be proper plugins: that is, some sort of scan-your-filesystem thing happens at startup, without an application being aware of it, and every plugin that's found gets registered automatically. It's harder to do that when the registration function must be told the type to use. Automatic registration is handy for deployment, sometimes, but it's also slightly creepy, and in this instance I think I'd feel more comfortable with non-expliclt registration (especially because we have an immediate use case in which two different plugins, tor and tcp, both want to be registered for the "tcp:" type).

I ended up going with last-one-wins for the dict. I'd be ok with "second registration throws error" too, which we could add to a subsequent release without much kerfluffle. I'd be nervous about "second registration is silently ignored".

Changes are landed, tests and docs are done.. starting the release process now. Thanks!

[str4d]

Example I2P client plugin, and patches to use it for foolscap calculator example

[daira]

Reviewed ​https://github.com/tahoe-lafs/tahoe-lafs/pull/189.

[warner]

Notes from today's meeting:

• We'll land str4d's docs from PR-189, plus some changes to make it clear that this is not-yet-implemented work, and that tub.location needs to have actual pre-generated .onion addresses
• We'll add a foolscap server plugin scheme, so tub.registerServerPlugin("foo", plugin) means that tub.listenOn("foo:stuff") will use endpoint = plugin.make_endpoint("foo:stuff"). If there's no plugin, the Tub will always fall back to Twisted's server-endpoint parser. This plugin will have the opportunity to add arguments to the endpoint being created.
• We'll define tahoe.cfg keys for the [connections] section to control how a tor/i2p daemon is configured. Some potential values:
  • tor.launch-global
  • tor.control-port
  • tor.socks-port
  • tor.bin-path
  • i2p.sam-port
  • i2p.sam-host
  • i2p.bin-path
• Tahoe startup has three phases
  • first phase: read tub.location, call tub.setLocation(), register FURLs
  • second phase: read [connections], launch tor/i2p daemon if necessary, build and install the foolscap client and server plugins.
  • third phase: read tub.port, call tub.listenOn(), start the tub
• tub.port can be onion:privkey=$KEYMATERIAL, or maybe privkey=private/onion.key (to keep secrets out of tahoe.cfg). During tahoe create-node we generate the key, store the private key into that file, store the public .onion address in tub.location. i2p private keys are roughly 900 base64 characters, we don't know how large tor keys are (only relevant if we store them directly in tahoe.cfg).

We *might* do verification: parse tub.location to find the .onion address, read the private key, compute the associated public address, make sure they match. It's an open question as to how much damage the tahoe config directory can tolerate before we can't recover usefully. It'd be nice if a deleted private key could be regenerated, at least manually. OTOH tahoe can be much more agnostic about tub.port and tub.location if we don't do this verification.

An externally-managed tor daemon will use tub.port = tcp:12345:interface=127.0.0.1 and tub.location = tor:foo.onion:80. An automatically-managed tor daemon will use tub.port = onion:privkey=.. and tub.location = tor:foo.onion:80, and inside tub.listenOn(), the txtorcon server endpoint parser will do some setup work (maybe launch tor, ask it to register the pre-configured onion service), then return a tcp/localhost endpoint for Foolscap to listen on.

One concern that should be noted in the docs: even if you don't advertise a real IP address, your node might be listening on something which could be probed externally, for a confirmation attack. Like if you use tub.port = tcp:12345 (without the interface=127.0.0.1), and configure tor to forward foo.onion to that, then someone might scan the entire internet for ports that react the same way as foo.onion, and find yours. The attack is made harder by NAT, also by restricting the socket to the local interface, but the only real defense would be for tor/i2p to learn to send some kind of secret handshake at the beginning of the connection, and for the local TCP endpoint to check the handshake before passing the socket up to Foolscap.

txtorcon has a useful "global" feature, where it remembers a process-wide singleton object that can be used for all connections. So if tahoe tells txtorcon to create this during startup phase 1, then it will be used for the server endpoint in phase 3 (without any extra work, and more importantly without additional config strings in tub.port). I2P doesn't have a corresponding feature, so the foolscap server plugin for i2p will need to add the extra arguments when building the server endpoint.

[warner]

Foolscap server plugins are in ​foolscap#243.

[dawuud]

rough draft SOCKS client and Tor client plugins here in my dev branch: ​https://github.com/david415/foolscap/tree/tor-plugin.0

These plugins are untested... although the Tor client endpoint code was mostly copy pasted from my old txtorsocksx github repo: ​https://github.com/david415/txtorsocksx/blob/master/txtorsocksx/endpoints.py

I'd like to test this plugin with foolscap and tahoe-lafs... by connecting to the onion grid.

[dawuud]

warner made a proper client-side foolscap plugin here: ​https://github.com/warner/foolscap/tree/tor

so the next step is to get Tahoe-LAFS to load foolscap plugins on startup?

[dawuud]

i made a simple txsocksx based tor client here: ​https://github.com/david415/foolscap/tree/tor-client-plugin.0 ​http://foolscap.lothar.com/trac/ticket/242

i'd like to test it with tahoe but i'm not sure how since tahoe currently does not work with foolscap-0.9.1 and later...

what's the next step? how can i use tahoe with the latest foolscap?

[daira]

1.10.2.post21 [master: b5222e36798086567efca6b0b436976c6b8598fe] definitely passes tests (on Linux) with foolscap 0.9.1.

[dawuud]

ah ok thanks daira. i've fixed my build system's problems.

since then i've gotten some work done:

• written unit tests for the Tor client foolscap plugin
• tested the foolscap tor plugin in a branch that activate only this plugin...

i used that testing branch of foolscap with Tahoe-LAFS and successfully connected to an onion grid.

Next I would like to add a tahoe.cfg configuration section for loading foolscap transport client plugins and specifying options; specifically socks host and port for the tor plugin.

After that we can iterate on the Tor plugin until we've got several with different properties... such as launching tor and saving tor state to a private/ directory... etc.

[dawuud]

i've now gotten my socks client foolscap transport plugin to work with my tahoe dev branch using these configuration settings:

[connections]
tor.socks_host = 127.0.0.1
tor.socks_port = 9050
tcp.socks_host = 127.0.0.1
tcp.socks_port = 9050

this is my dev branch: ​https://github.com/david415/tahoe-lafs/tree/517.add-client-plugin-config.0

i've also added unit tests... to make sure the socks plugin is loaded correctly and that the port is converted from str to int etc.

my foolscap socks client plugin is in my dev branch here: ​https://github.com/david415/foolscap/tree/tor-client-plugin.0

please review.

[leif]

Thanks, David! It looks like the foolscap side has a usable SOCKS plugin now, and the other plugins should be relatively easy to write!

On the Tahoe side, though, I think that it would be better to avoid requiring that Tahoe explicitly support specific foolscap plugins and their options (even if it does learn about some of them later to implement the anonymity options discussed in #1010).

I can think of at least 5 or 6 possible client-side plugins so far; TCP (default), SOCKS, Tor (socks w/ automatic port selection: SocksSocket? (unix domain socket), 9050, 9150, or discovered-via-control-port), HTTP Proxy, I2P (HTTP proxy with automatic port selection?), and perhaps UNIX domain sockets or other twisted endpoints - and I would like to be able to use any combination of them simultaneously.

I'm not sure if I'd prefer that Tahoe specify (via a tahoe.cfg option) which plugins to load or if they should be loaded automatically like Twisted plugins, but in either case I think that Tahoe shouldn't need plugin-specific code.

As previously discussed, using options obtained from connection hints would be dangerous when we are receiving the hints from announcements on the network. But, it is also a convenient way to communicate arbitrary options to the plugin when the hints are coming from a local config file.

So, I think we *should* allow arbitary options in connection hint strings (eg, a proxy servers's host, port, username, password, etc) but we should sanitize the hints we receive from announcements: only the type, host, and port are ever used from announcements.

I propose eventually allowing per-server plugin configuration using my and David's "introducerless" branch (based on warner's work) discussed in ticket #68 which I hope to have rebased to master soon.

Without that branch merged, we can implement less flexible plugin support (without per-server local configuration overrides) by mapping announcements' connection hint types to known plugins as discussed previously in this ticket. This would even allow connecting to mixed tcp, tor, and i2p networks IFF the servers are announcing new-style connection hints with types, as the tor and i2p plugins should work with only a host and port.

To enable connecting to an existing all-tor or all-i2p grids where some servers have not been upgraded, and also to provide the mode that connects to both onion and non-onion addresses over tor, and also to provide the ability to use arbitary socks or HTTP proxies with whatever options are desired, I propose an "announcement_rewrite" option which can accept a value like tor:%h:%p or socks5:%h:%p:socksserver=127.0.0.1:socksport=1080 or i2p:%h or even foo:%h:%p:origtype=%t.

So, to recap:

• Without the introducerless branch, it will be possible to (a) connect to heterogeneous grids of new servers, or (b) connect to homogeneous grids of old-and-new servers and route all connections over a specific proxy.

• With the introducerless branch, it is possible to disregard announcements and specify unique connection plugin options for specific servers. (While still having a rewrite rule to coerce newly-announced servers to some default proxy connection.)

[daira]

We agreed in today's Nuts & Bolts meeting to bump better Tor/I2P support out to 1.11.0.

[dawuud]

Yesterday in our pairing session Daira code reviewed my Foolscap and Tahoe-LAFS changes specified in the above comment: ​https://tahoe-lafs.org/trac/tahoe-lafs/ticket/517#comment:62

Daira added a small code change to make the connection configuration section of tahoe.cfg more flexible. I fixed a minor bug where we were failing to catch the exception of the connections configuration section does not exist... this caused many unit tests to fail.

Latest Foolscap and Tahoe-LAFS Tor integration development branches, here:

​https://github.com/david415/tahoe-lafs/tree/517.add-client-plugin-config.1

and here:

​https://github.com/david415/foolscap/tree/tor-client-plugin.1

[daira]

Latest branches at:

• ​https://github.com/tahoe-lafs/tahoe-lafs/tree/517.add-client-plugin-config.1
• ​https://github.com/daira/foolscap/tree/tor-client-plugin.2

[warner]

Milestone renamed

[dawuud]

Latest branches at:

• ​https://github.com/david415/tahoe-lafs/tree/517.add-client-plugin-config.2
• ​https://github.com/david415/foolscap/tree/tor-client-plugin.3

Code review needed.

[leif]

Replying to dawuud:

Latest branches at:

• ​https://github.com/david415/tahoe-lafs/tree/517.add-client-plugin-config.2
• ​https://github.com/david415/foolscap/tree/tor-client-plugin.3

Code review needed.

Cool, glad to see this is progressing!

The design as implemented above allows for some flexibility but not enough in my opinion.

It does allow multiple proxies named foo and bar to be configured by implicitly defining new connection hint types foo and bar with {foo,bar}.socks_{host,port} in Tahoe's connections configuration section; those types could then be used from locally-specified connection hints via the #467 introless branch, or could even be used in announcements if there is coordination between servers and clients as to what the names of the proxies are.

But what is unfortunately not yet possible (except perhaps with the hacky solution below) is the use case I keep coming back to: I want to connect to announced servers over tor by default (which, in the most recent branch means setting tcp.socks_{host,port}) but I want to use the introless config in #467 to specify somehow that I want to connect to certain servers directly (perhaps because they are on my LAN or a VPN).

There is a hacky way that this could technically be possible using dawuud's latest branches: I could run a local socks server to facilitate non-Tor connections. But, I'd have to give it some name, and if the introducer (or any server connecting to the introducer) guessed the name they could send me connection hints using that name (which would cause me to make non-tor connections to them).

I still like my connection-hint-rewriting plan I described in a previous comment, which would allow Tahoe to not have any knowledge of various foolscap plugins (in its code - it would in its config). Perhaps there is a better solution, but it seems to me that blindly passing connection hints from announcements into foolscap is at odds with my desire to both (a) default to tor (read: not let malicious servers cause me to make non-tor connections) and (b) have the ability to make non-tor connections to servers of my choosing.

[leif]

During the last Nuts and Bolts meeting I came to realize that my connection-hint-rewriting plan was a lousy idea (with implementation pitfalls leading to potential security bugs, even), and we've now arrived at what I think is a quite good design which Brian has written up in #2759 and which dawuud is presently implementing (on top of the introless-multiintro branch from #68 and #467).

[meejah]

I tried running with the two branches above but either get import errors (during unit-tests) like:

from foolscap.connection_plugins import SOCKS5
exceptions.ImportError: cannot import name SOCKS5

...which leads me to believe it's picking up the wrong foolscap. But if I pip install -e src/foolscap which is at tor-client-plugin.3 then I get version errors from tahoe like:

    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (foolscap 0.6.4+169.g18e6dd9 (/home/mike/src/foolscap/src), Requirement.parse('foolscap>=0.10.1'))

Do I have the wrong branches checked out?

(Got it, I either I was using the wrong remote or it was a bit out of date). But ... works now (or at least, the import error is gone).

[dawuud]

Meejah let's discuss this in #2759

[dawuud]

#2788 is our latest ticket describing our Tor integration design

[warner]

moving most tickets from 1.12 to 1.13 so we can release 1.12 with magic-folders

[warner]

Meejah and I were just working out what the tahoe.cfg syntax ought to be for controlling how Tor gets started (especially on the client side). Here's our current idea:

tahoe.cfg acquires a [tor] section, with the following keys (and their defaults):

• enable = true
• socks.port = (empty)
• control.port = (empty, but uses unix:/var/run/tor/control if launch is false)
• launch = false
• tor.executable = (empty)
• data.directory = (empty, or a subdir of tahoe's NODEDIR)

The highest-priority setting is enable. If enable is false, all other settings are ignored, and nothing uses Tor.

The next-highest priority is socks.port. If set, the client attempts to import txsocksx at startup, and if that is successful, it installs a Foolscap connection handler for tor: hints which route them to a SOCKS handler with the given port. (note that txtorcon depends on txsocksx)

The lowest-priority settings are control.port and launch (tor.executable and data.directory are only used if launch = true).

If control.port is set, and launch is false, the client will attempt to import txtorcon at startup, and if that is successful, installs a connection handler that:

• uses txtorcon to connect to the pre-running Tor process via the control port
• asks that Tor for its SOCKS port
• routes the tor: hint through the SOCKS port

If control.port is empty and launch is false, the client uses control.port = unix:/var/run/tor/control. This will Just Work (opportunistically) for debian/fedora systems where the tor package has been installed and the user is a member of the appropriate unix system group (e.g. debian-tor).

If control.port is empty and launch is true, the client uses txtorcon to start a new Tor process and allocate a suitable control port. Connections are made through the new Tor's SOCKS port as above. tor.executable can be used to specify the exact Tor binary to run (if empty, the default behavior is to search $PATH for tor). data.directory= can be used to provide a persistent datadir, which reduces startup time (from tens of seconds to a few seconds).

If control.port is set and launch is true, txtorcon is asked to launch a Tor process as above, but the process is told to use control.port for it's control port, instead of allocating a new one.

(we haven't decided what data.directory should default to: maybe a subdirectory of the Tahoe node directory? or maybe a new temp directory?)

So, as far as use cases go:

• Leaving everything set to the defaults will Just Work when the tor package is installed (and thus a Tor process is running), txtorcon is installed, and the user is in the right unix group
• If you know you have a Tor running, set socks.port= to use it, and install at least txsocksx. Use this if you want more control over the Tor config.
• if you have Tor installed but not running, use launch = true, and maybe tor.executable= if your Tor lives in an unusual place
• If you have Tor/txtorcon installed, but for whatever reason (company policy?) do *not* want to use Tor, set enable = false.

We'll need an additional flag to say "use Tor for TCP hints too", which is checked by the anonymous = true safety flag.

On the server side, #2773 describes the CLI options for tahoe create-node. You'd provide --location and --port to use a pre-configured Tor hidden service (and would probably want to set socks.port= to point at the same Tor, for outbound connections, along with the "use Tor for TCP too" and anonymous flags). Or providing --listen-tor would tell tahoe create-node to get a Tor control port and create any necessary hidden-service keys, and write the --location and --port (and other how-to-start-Tor reminders) into tahoe.cfg. We might need CLI options for all of the [tor] controls listed above, since we'll have to start a Tor during create-node, before the user has a chance to edit their tahoe.cfg.

[warner]

In talking with Leif and David today, we found out that they're discouraging the use of the debian-tor unix group (and of opening the Tor control port to non-admin users in general). They're concerned that it exposes to much authority to programs that don't really need it.

Instead, they recommended that we just try SOCKS on both localhost port 9050 (which is where a system Tor usually listens) and port 9150 (where the Tor Browser Bundle listens). Not coincidentally, this is exactly what txtorcon.endpoints.TorClientEndpoint does by default. We had previously worried that this would enable some other local user to camp out on these ports and intercept the connections (whereas they couldn't camp out on a unix socket that lives in /var/run without being root). But Leif pointed out that if somebody malicious is already running enough code on your box to listen on a local socket, then you've already lost (for example they could make their own outbound connection and leak your IP address) (note that Foolscap is still encrypting the connection and verifying the TLS cert, so this local attacker couldn't violate your confidentiality, but the one thing they *can* violate is your anonymity)

So the new "Hopefully Just Works" default will be to use SOCKS on 9050/9150, rather than to use a control port to discover a SOCKS port. That changes the above plan as follows:

• enable = true
• socks.port = (empty, but uses 9050/9150 if control.port is empty and launch is false)
• control.port = (empty)
• launch = false
• tor.executable = (empty)
• data.directory = (empty, or a subdir of tahoe's NODEDIR)

The highest-priority setting is enable. If enable is false, all other settings are ignored, and nothing uses Tor.

The next-highest priority is socks.port. If set, the client attempts to import foolscap.handlers.socks (which needs txsocksx) at startup, and if that is successful, it installs a Foolscap connection handler for tor: hints which route them to a SOCKS handler with the given port (note that txtorcon depends on txsocksx). This doesn't enable the use of multiple SOCKS port: the socks.port value is a twisted client endpoint descriptor, which only permits specifying a single SOCKS port. The 9050/9150 fallback behavior is buried inside TorClientEndpoint and not exposed in the API (in fact, TorClientEndpoint doesn't accept an arbitrary endpoint for the SOCKS port: just a hostname and integer portnum, so we may need to define socks.port to be strictly tcp:HOST:PORT).

(if control.port or launch are set too, we'll still launch a new Tor, but outbound Tor connections will use socks.port rather than anything involving the new Tor instance. However we might use the new Tor for *listening* purposes)

If socks.port is not set, but control.port is empty and and launch is false, then we'll use TorClientEndpoint's default behavior (use SOCKS on 9050, if that fails fall back to 9150).

If socks.port is not set and control.port *is* set, then the client attempts to import foolscap.handlers.tor (which needs txtorcon), and if that is successful, installs a connection handler that:

• uses txtorcon to connect to a Tor process via the control port
• asks that Tor for its SOCKS port
• routes the tor: hint through the SOCKS port

If launch is true, the client uses txtorcon to start a new Tor process. If control.port is empty, it asks Tor to allocate a suitable control port, otherwise it tells Tor to use the specified port. Connections are made through the new Tor's SOCKS port as above. tor.executable can be used to specify the exact Tor binary to run (if empty, the default behavior is to search $PATH for tor). data.directory= can be used to provide a persistent datadir, which reduces startup time (from tens of seconds to a few seconds).

enable	socks.port	control.port	launch
false	*	*	*	no Tor
(true)	(empty)	(empty)	(false)	SOCKS on 9050/9150
(true)	(empty)	(empty)	true	launch Tor, ask new Tor to allocate a control port, ask it for SOCKS port
(true)	(empty)	set	(false)	connect to pre-configured Tor on control.port, ask it for SOCKS port
(true)	(empty)	set	true	launch Tor, tell new Tor to use control.port, ask it for SOCKS port
(true)	HOST:PORT	(empty)	(false)	SOCKS on HOST:PORT
(true)	HOST:PORT	(empty)	true	SOCKS on HOST:PORT, launch Tor, ask new Tor to allocate a control port (for server)
(true)	HOST:PORT	set	(false)	SOCKS on HOST:PORT, connect to pre-configured Tor on control.port (for server)
(true)	HOST:PORT	set	true	SOCKS on HOST:PORT, launch Tor, tell new Tor to use control.port (for server)

Note that all of this merely controls *how* we use Tor, not *when* we use it. By default (unless enable = false), Tor-based connection hints (for external servers, which we learn through the Introducer) will use Tor, as specified by this config. To make regular TCP hints use Tor too, see the connections.yaml "Connection Policy" in #2788. To make the local storage server listen on a Tor hidden service, we'll use something else (#2733). To make the connection to the introducer(s) use Tor, see something in #2788.

So, as far as use cases go:

• Leaving everything set to the defaults will Just Work (i.e. outbound tor: hints should connect) when:
  • the (system) tor package is installed (and thus a Tor process is running), or the Tor Browser Bundle is running
  • and (python) txsocksx is installed
• If you know you have some other Tor running, set socks.port= to use it, and install at least txsocksx. Use this if you want more control over the Tor config.
• If you have Tor installed but not running, use launch = true, and maybe tor.executable= if your Tor lives in an unusual place
• If you have Tor/txtorcon installed, but for whatever reason (company policy?) do *not* want to use Tor, set enable = false.

[warner]

Note that this work depends upon a new Foolscap release which provides the socks/tor connection handlers in the right place. ​foolscap#242 and ​foolscap#246 are tracking these.

For the server side, we're still looking at tahoe create-server --tor (which is mutually-exclusive with --location/--port/--hostname from #2773), but with enough CLI arguments to let you set these tahoe.cfg values right away, because create-server --tor will need to use txtorcon to spin up a hidden service (using some "file system onion service" class it does/will provide), extract the private key and onion address, then write both into some as-yet-undefined tahoe.cfg fields, where they can be read again at tahoe run time to re-create the server endpoint.

[str4d]

Note that the above will not be an issue with I2P / txi2p, because the SAM API only allows the creation and control of tunnels, and only those created by that specific client. The primary issue will be that currently SAM is not enabled by default, but we may change that in a future release. The secondary issue is the lack of auto-launch capability in txi2p, which I will probably have to get around to adding at some point.

I will parse the above and figure out the equivalent setup for I2P this weekend.

[str4d]

I have made ​foolscap#261 to track the needed I2P connection handler in Foolscap.

[dawuud]

The tor SOCKS port and control port should both be accessible via UNIX domain socket, tor currently supports this usage. Ideally tor onion services wouldn't even run with the privilege to use an IP stack... only UNIX domain sockets.

[warner]

I'm starting to sketch out the foolscap-side Tor connection handler, and I'm thinking that we should simply reject all the weird combinations of these arguments. What do you think about using this table instead?:

enable	socks.port	control.port	launch
false	*	*	*	(1) no Tor
(true)	(empty)	(empty)	(false)	(2) SOCKS on 9050/9150, no server
(true)	(empty)	(empty)	true	(3) launch Tor, ask new Tor to allocate a control port, ask it for SOCKS port
(true)	(empty)	set	(false)	(4) connect to pre-configured Tor on control.port, ask it for SOCKS port
(true)	(empty)	set	true	INVALID
(true)	HOST:PORT	(empty)	(false)	(5) SOCKS on HOST:PORT, no server
(true)	HOST:PORT	(empty)	true	INVALID
(true)	HOST:PORT	set	(false)	INVALID
(true)	HOST:PORT	set	true	INVALID

That'd give us 5 cases. Case 5 uses the txsocksx-based handler. Case 2+3+4 use the txtorcon one (with case 2 using the default behavior). Only cases 3 and 4 enable automatically-configured hidden-service listeners (2+5 can do outbound, but don't have a control port so they don't have a way to ask Tor to create a new onion service).

Can anyone think of a good reason to use the other combinations? I'm especially looking to avoid parsing control.port to figure out how to tell a launched Tor to listen on it.

[warner]

Oh, and we should make socks.port be an endpoint descriptor, instead of HOST:PORT, right? To enable unix-domain socks ports? So tcp:localhost:9050 instead of just localhost:9050.

[str4d]

+1 on simplifying the combinations. Regarding the currently-INVALID combinations:

• I could imagine control.port+launch or socks.port+launch being used as an indicator to try to launch an existing Tor in case it isn't already running, but then the semantics are slightly different, because the other usages of launch imply a new Tor process (which is reasonable given that the process is client-only).
  • In I2P's case, running multiple I2P routers on the same machine is not recommended; thus launch for it would mean try to launch an existing I2P installation if it isn't already running.
• Given that you can request the SOCKS port over the control port, the only reason for being able to set socks.port+control.port would be if the user wanted to use separate tor processes for outgoing and incoming connections, which I can't see a decent justification for. The launch flag is irrelevant here.

I'd certainly recommend making socks.port an endpoint descriptor from the start, but given that it was mentioned above that TorClientEndpoint doesn't accept an arbitrary endpoint for the SOCKS port (just a hostname and integer portnum), you'd have to parse it regardless.

[str4d]

Here is my equivalent to the above for I2P.

---

tahoe.cfg acquires an [i2p] section, with the following keys (and their defaults):

• enable = true
• sam.port = (empty, uses tcp:127.0.0.1:7656 by default)
• launch = false
• i2p.executable = (empty, looks for common installation locations by default)
• i2p.configdir = (empty, looks for common config directories by default)

The highest-priority setting is enable. If enable is false, all other settings are ignored, and nothing uses I2P. If enable is true, the client attempts to import foolscap.handlers.i2p (which needs txi2p) at startup, and if that is successful, it installs a Foolscap connection handler for i2p: hints that route them to a SAMI2PStreamClientEndpoint.

The next-highest priority is sam.port. The sam.port value is a Twisted client endpoint descriptor, pointing to a SAM API. If sam.port is not set, then we'll use tcp:127.0.0.1:7656. The tcp:127.0.0.1:7656 fallback behavior is part of the txi2p client and server endpoint string syntax, but is not part of the SAMI2PStreamClientEndpoint and SAMI2PStreamServerEndpoint APIs (which require that the caller pass in an IStreamClientEndpoint that will connect to the SAM API).

If launch is true, the client uses txi2p to start the given existing I2P installation if it is not already running. Connections are made through the I2P installation's SAM port as above. i2p.executable can be used to specify the exact I2P binary to run (if empty, the default behavior is to search $PATH and common installation locations for i2prouter).

If i2p.configdir is set, the configuration files within will be parsed to obtain the current SAM port.

enable	sam.port	launch	i2p.configdir
false	*	*	*	(1) No I2P
(true)	(empty)	(false)	(empty)	(2) SAM on tcp:127.0.0.1:7656
(true)	(empty)	true	(empty)	(3) Launch I2P, try to parse config for SAM port, fall back on tcp:127.0.0.1:7656
(true)	(empty)	(false)	path/to/.i2p	(4) Parse config for SAM port
(true)	(empty)	true	path/to/.i2p	(5) Launch I2P, parse config for SAM port
(true)	tcp:HOST:PORT	(false)	*	(6) SAM on tcp:HOST:PORT
(true)	tcp:HOST:PORT	true	*	INVALID

Note that all of this merely controls how we use I2P, not when we use it. By default (unless enable = false), I2P-based connection hints (for external servers, which we learn through the Introducer) will use I2P, as specified by this config. To make the local storage server listen on the I2P destination (most likely the same one outgoing connections are made on), we'll use something else (#2773, although we can reuse at least part of this config, because listening happens through the same SAM API). To make the connection to the introducer(s) use I2P, see something in #2788.

So, as far as use cases go:

• Leaving everything set to the defaults will Just Work (i.e. outbound i2p: hints should connect) when:
  • an existing I2P process is running locally
  • and (Python) txi2p is installed
• If you know you have some other I2P running (local or remote), set sam.port= to use it, and install at least txi2p. Use this if you want to leverage a 24/7 router on another machine (with the SAM port either port-forwarded, or directly-accessible and protected with SSL and authentication).
• If you have I2P installed but not running, use launch = true, and maybe i2p.executable= if your I2P lives in an unusual place.
• If you have I2P/txi2p installed, but for whatever reason (company policy?) do *not* want to use I2P, set enable = false.

Note that txi2p currently does not have the functionality to implement launch, i2p.executable or i2p.configdir. Until that is implemented, only cases 1, 2 and 6 will work. I'm also not entirely sure about cases 3 and 4 yet; that will crystallize after implementation.

[Brian Warner <warner@…>]

In 73d5376b/trunk:

Implement basic connection handlers (tor/i2p)

The node now attempts to create Tor/I2P connection handlers (if the
right libraries are available), and will use them for tor/i2p FURL hints
by default. For now it only creates default handlers: there is not yet
any code to interpret the [tor]/[i2p] sections of tahoe.cfg which
would let you override that process.

The node also parses the [connections] section, allowing tcp: tor to
use Tor for all outbound TCP connections. It defaults to tcp: tcp, of
course.

Static storage-server connections will now honor the connections:
overrides in servers.yaml, allowing specific servers to use TCP where
they would normally be restricted to Tor.

refs ticket:2788
refs ticket:517

[Brian Warner <warner@…>]

In 0951201/trunk:

Merge 2788-connection-handlers: add Tor/I2P support

This adds client-side support for automatically connecting to Tor and
I2P -hosted servers (when the server advertises a "tor:" or "i2p:"
-style connection hint, and when the necessary extra libraries are
installed, and when a Tor/I2P daemon is running and reachable at the
default location).

The new [connections] section of tahoe.cfg can include tcp: tor to
make all TCP connections use Tor instead. This ought to hide the
client's IP address from servers and the Introducer.

The new [tor] and [i2p] sections of tahoe.cfg will control how
Tor/I2P connections are made (whether to spin up a new Tor daemon, what
port to use to connect to an existing one, etc). configuration.rst
explains the possible syntax. Note that only the default I2P connection
method is implemented in Foolscap-0.12.2, but not the alternatives, so
many options that are legal syntax for tahoe.cfg will not work yet.

It also enables the connections section of private/servers.yaml to
override the connection-handler mapping.

refs ticket:517
closes ticket:2788

[warner]

With the landing of #2788 yesterday, I think we've finished most of the work of this ticket. I've re-read the history, and I think this is the set of related tickets (not all of which need to be finished to declare success, but they should all be examined to be sure):

• #1942: google chart in WUI leaks information
• #1010: anonymous client mode (tahoe.cfg anonymous=true safety flag)
• #2384: randomized client Tub IDs
• #2490: anonymous-friendly create-node/create-client
• #2491: synchronous node startup
• #2773: create-node --location/--port/--hostname
• #2794: move controlport/logport to separate Tub
• #68: introless
• #467: static upload server selection

Remaining work that should be done before the 1.12.0 release:

• #2815: document how to manually configure a tahoe server on an .onion address (#2490 will be the automatic form of this, related to #2773)
• #2816: make sure tub.location can be empty, and make tub.port= (empty) mean "don't listen". This is for clients. Maybe make the tahoe create-client initial tahoe.cfg file use tub.port=(empty).
• maybe resolve #1942 (the google-chart leak from the WUI page), or as a temporary measure, make anonymous=true disable that chart (stop serving the IMG tag that renders it)
• #1010: define/implement the anonymous=true safety flag, specifically how much it ought to constrain, and whether it should allow running a server or not. For sure it should require:
  • [node]tub.location= does not contain AUTO (even by default)
  • [connections]tcp=tor is set
  • maybe we should implement #2384 (randomized client IDs) and require it, but this is a deeper question of whether we're being anonymous or pseudonymous, and whether we're trying to hide linkability from the servers too, or just the internet as large
  • maybe constrain tub.location to advertise an .onion/.i2p hint, to protect a user against accidentally manually typing their real IP address into that
  • maybe constrain tor.socks_port to point at localhost
  • maybe require tub.port= is empty? for clients this is what you want, but for servers that run behind Tor, tub.port= will listen on localhost, and that's fine

[Brian Warner <warner@…>]

In cb35473/trunk:

Merge branch '517-tor-docs'

refs ticket:517
closes tahoe-lafs/tahoe-lafs#323

[warner]

Ticket/dependency updates:

The chart in #1942 has been removed (the ticket remains open to track a new implementation, but it is no longer privacy-relevant). We added [node] reveal-IP-address = false to #1010 (instead of anonymous = true. We aren't worried about randomizing tubids (since we don't share tubs anymore, and scaled back "anonymous" to be just "hide the IP address"), and added a --hide-ip CLI arg to tahoe create-node.

We're working on #2773 (which I want in this release, because I think --location/--port/--hostname is an important server-setup step, and it makes manual setup of a Tor-based server easier). I think we can use #2490 to track --listen=tor, for the automatic server setup, but I'm not concerned about getting that part into 1.12.

#68 is probably a good nice-to-have, and I haven't yet figured out how this should interact with #467 and/or #573 (the two "static server selection" tickets), but I don't think either are necessary to declare our tor/i2p support complete.

So I'm gonna says that 1.12 is blocked on #2773, but not on #2490. And this ticket (#517) can be closed when:

• #2773 is closed (create-node --location/--port/--hostname, for manual server config)
• #2490 is closed (create-node --listen=tor, for automatic server config)
• and we have docs about both manual and automatic configuration

[warner]

looks like we'll be able to do this in 1.12

[Brian Warner <warner@…>]

In 5a195e2/trunk:

Merge branch '2490-tor-2'

This adds --listen=tor to create-node and create-server, along with
.onion-address allocation at creation time, and onion-service
starting (launching or connecting to tor as necessary) as node startup
time.

closes ticket:2490
refs ticket:2773
refs ticket:1010
refs ticket:517

[warner]

Str4d's I2P work has landed too (#2838), so I think we can close this now. yay!