[tahoe-dev] [tahoe-lafs] #867: use ipv6

[Randall Mason]

A quick note about this:

> Avoiding mess is always good.  What happens currently with 169.254.0.0/16
> addresses in Tahoe?  What about RFC1918 addresses?  What about 127.0.0.0/8?
> Are they deprioritized so that connections happen to them in the last case?
> What is the delay if the connection times out?  Does Tahoe only connect in
> serial, as apposed to starting to open up x different connections and take
> the first one that connects?  Does Tahoe use the order of the address in the
> furl?  What's the current algorithm for IPv4 addresses and the justification
> for it?

Looking at foolscap, it does start a connection to every single
address in the furl and takes the first one that succeeds.  All the
interesting stuff happens in negotiate.py:

Each hint in the furl has a connection opened to it:

1376     def connectToAll(self):
1377         while self.remainingLocations:
1378             location = self.remainingLocations.pop()
1379             if location in self.attemptedLocations:
1380                 continue
1381             self.attemptedLocations.append(location)
1382             host, port = location
1383             lp = self.log("connectTCP to %s" % (location,))
1384             f = TubConnectorClientFactory(self, host, lp)
1385             c = reactor.connectTCP(host, port, f)
1386             self.pendingConnections[f] = c
1387             # the tcp.Connector that we get back from
reactor.connectTCP will
1388             # retain a reference to the transport that it
creates, so we can
1389             # use it to disconnect the established (but not yet
negotiated)
1390             # connection
1391             if
self.tub.options.get("debug_stall_second_connection"):
1392                 # for unit tests, hold off on making the second
connection
1393                 # for a moment. This allows the first connection
to get to a
1394                 # known state.
1395                 reactor.callLater(0.1, self.connectToAll)
1396                 return
1397         self.checkForFailure()

Once a connection succeeds, it calls negotiationComplete:

1173         # if we were created as a client, we'll have a
TubConnector. Let them
1174         # know that this connection has succeeded, so they can
stop any other
1175         # connection attempts still in progress.
1176         if self.isClient:
1177             self.connector.negotiationComplete(self.factory)

Go through all the other connections and kill them:

1437     def negotiationComplete(self, factory):
1438         # 'factory' has just completed negotiation, so abandon
all the other
1439         # connection attempts
1440         self.log("negotiationComplete, %s won" % factory)
1441         self.active = False
1442         if self.timer:
1443             self.timer.cancel()
1444             self.timer = None
1445         del self.pendingConnections[factory] # this one succeeded
1446         for f in self.pendingConnections.keys(): # abandon the
others
1447             # for connections that are not yet established, this
will trigger
1448             # clientConnectionFailed. For connections that are
established
1449             # (and exchanging negotiation messages), this does
1450             # loseConnection() and will thus trigger
negotiationFailed.
1451             f.disconnect()
1452         self.checkForIdle()

Because of this, it doesn't seem that removing any addresses seems
like a win.  It can only lose.  A simple change within foolscap will
allow for each fe80::/10 address to be connected to with each
interface that's up.  This will result in more connections, but it
will succeed in connecting in side cases.  The change would go in
negotiate.py here:

1316         for h in self.target.getLocations():
1317             if h[0] == "ipv4" or h[0] == "ipv6":
1318                 (host, port) = h[1:]
1319                 hints.append( (host, port) )

which is part of the already proposed changes to foolscap.  If we can
just add a re that finds fe80::/10 addresses and loop over the
interfaces on the host appending each %if to each fe80 address.  This
way all addresses possible are used.  None of the gymnastics of
figuring out what port the fe80:: address is on, just try a connect()
on all interfaces.  This will still take the lowest latency link to
connect (at least on average).  Even without any change, there is not
much of a performance penalty.  connectAll() is only called if there
is a "redirect" and on initial startup.  It does not jump around with
connections once it has a good link, so it should be the best
performance too.  It does not prioritize either the order or the
protocol, even though I list ipv6 addresses first in the furl, I end
up with an ipv4 connection much of the time.  I've also ended up with
different ones of my universally addressable IPv6 addresses too when
only on IPv6.

-Randall