teach magic-folder unit tests to time travel

[dawuud]

It makes sense to have our unit tests fast forward the clock to avoiding blocking on sleep; and also we really shouldn't be using a global reactor.

​https://twistedmatrix.com/documents/current/core/development/policy/test-standard.html

"""Most unit tests should also avoid waiting for real time to pass. Unit tests which construct and advance a twisted.internet.task.Clock are fast and deterministic."""

and

"""Since unit tests are avoiding real I/O and real time, they can usually avoid using a real reactor. The only exceptions to this are unit tests for a real reactor implementation."""

[daira]

We can't avoid using a real reactor for the magic folder tests, because they make use of the real web API over loopback HTTP. There is already provision for passing a Clock to the constructors for things that have timeouts/delays; the tests just need to be changed to use that.

[dawuud]

I'd like to report minor progress here in my dev branch: ​https://github.com/david415/tahoe-lafs/tree/2500.unit-test-clock-fast.0

• Here I've taught the magic-folder class to accept a clock variable which is then

used internally for deferLater calls...

• unsuccessfully taught test_alice_bob to use it's own clock... but there's some buggy destructor code that we think prevents this from working properly

[dawuud]

Switching efforts to this dev branch that contains newer code from ​https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1710#comment:30

dev branch here: ​https://github.com/david415/tahoe-lafs/tree/2500.unit-test-clock-fast.2

test_alice_bob still fails...

[dawuud]

the MockTest.test_alice_bob fixed here: ​https://github.com/david415/tahoe-lafs/tree/2500.unit-test-clock-fast.2

the RealTest.test_alice_bob still fails... but why!?

[dawuud]

progress here in my dev branch: ​https://github.com/david415/tahoe-lafs/tree/2500.unit-test-clock-fast.3

i got the mock test_alice_bob test passing... however the real test still fails; it's locked... seems like a dead lock.

[daira]

Reviewing ​https://github.com/tahoe-lafs/tahoe-lafs/pull/191.

[daira]

Replying to daira:

Reviewing ​https://github.com/tahoe-lafs/tahoe-lafs/pull/191.

Thanks meejah, but I think there should be a less disruptive way of solving this. The approach in this commit makes me nervous because it's effectively testing different code than actually runs in production. It should be possible to just set the 'processed' hook earlier if the problem is as described in the ​comment at line 375 of test_magic_folder.py.

So, -1 on the review.

[daira]

Yeah, it looks like the problem is that Bob's 'processed' hook needs to be set before Alice writes the file. Then it should be possible to set _turn_delay to 0 without problems (if that is the only race condition in the test).

[meejah]

See: ​https://github.com/tahoe-lafs/tahoe-lafs/pull/195 which uses the Clock() approach.