tcp hole-punching!

[zooko]

Did you know that it is actually possible to do tcp hole-punching (for some NAT situations) and that there exists source code which is an almost-drop-in-replacement for connectTCP() and listenTCP(), and that this source code is going to be open sourced soon?

[ghazel]

As a matter of fact I did! Now if I could only make a Trac ticket email me every weekend to remind me to do it...

[zooko]

I just reminded ghazel.

This might make a fun project for The Hack Fest next Friday...

[ghazel]

I didn't get to it at the hack fest, but I did get Tahoe running, woo! I've been reviewing the timeout cases in Stunted, to make sure it won't retry forever. Perhaps I'll put up a Google Code page about it, so you can check it out.

[zooko]

What if I want it to keep retrying forever? (Or until I tell it to stop.)

[jsgf]

Any progress on this? I'd like to look at this if it still needs doing.

ghazel: Do you have a library available?

[zooko]

Please post to this ticket and/or tahoe-dev if you do anything on this.

[zooko]

Assigning to ghazel to answer jsgf's question. :-)

[zooko]

If you like this ticket, you may also like #49 (UPnP), #50 (STUNT/ICE), #445 (implement relay: allow storage servers behind NAT), and #754 (merge manually specified tub location with autodetected tub location).

[davidsarah]

The Google Summer of Code project would be "make Tahoe work through firewalls", not necessarily this specific ticket. That might require use of more than one technique.

[exarkun]

I wonder what code zooko was referring to. It sounds like Vertex but I don't recall there ever being a time when Vertex *wasn't* open source, so maybe not. In any case, Vertex *is* open source and ... might still work? I don't know. Even if it does I suppose in 2020 it might not be the best option anyway.

[maylee]

Choosing this ticket at random. Is this something that fits into any of our current milestones, or should we make a milestone this can be bucketed into?

[lpirl]

Maybe "Grid Management", if any? As in: "It makes a grid easier to manage when servers need no extra firewall/NAT configuration". Although this might be more an operation issue but oh well. $ .02

[exarkun]

To choose a milestone, let's work backwards from an end-user-facing goal. What is that in this case? Real servers rarely have to deal with NAT. They just get public IP addresses. Perhaps the goal is related to operation by "home" users? That is, to allow me to run a listening Tahoe node (so, not just a storage client - a storage server and/or an introducer) on my consumer-grade network connection where I have no publicly routeable address to make my servers listen on.

Oops, wait. Is that really relevant? Can I TCP hole-punch through carrier-grade NAT (CGN)? Wikipedia vaguely nods towards "yes" (no citations). All of my hole-punching experience is with consumer-grade routers so I can't say first-hand. You have to be able to hole-punch through CGN to meaningfully improve the home user experience now, I think.

Well ... maybe there are mild ease-of-use improvements from just defeating consumer-grade router NAT. Tahoe could deal with UPnP instead of making me click buttons on my router.

Suppose we can do some or all of those things. The end result is that Tahoe storage nodes and introducers can run on my low-end network connection. This helps out ... friendnet operation, perhaps? A group of "friends" are more likely to have a NAT'd server among them (it's not until you have two that you actually need help here. still.).

If you wanted a remote control for your client node then the same requirement would apply, I suppose. You want the remote control to be able to connect back to your client node regardless of intervening network topology. There's a lot of other work required before we know if/how we can have such remote controls though.

I think "Grid Management" is essentially orthogonal. It's actually about efforts related to ticket:2916 which is a very specific kind of grid management.

If we had some kind of "friendnet" milestone it might make sense there. If we had some kind of "remote control" milestone it might make sense there. It might even make sense if we sliced the problem differently and had a "maximum network compatibility" milestone that included all of the hole punching, UPnP, STUN, etc tickets.

None of those milestones exist yet, though, and I wouldn't suggest creating them just to have a place to put this ticket. I think we need to plan a roadmap first and then we'll see what milestones are on it and we can sort tickets into them. There will be *lots* of tickets that don't fall into a milestone in the near-term future based on such a roadmap and that's fine (or at least, that's life).