Opened at 2020-08-30T15:23:01Z
Closed at 2021-01-15T21:01:02Z
#3399 closed enhancement (fixed)
Evaluate adding mypy checks to code checks
| Reported by: | jaraco | Owned by: | GitHub <noreply@…> |
|---|---|---|---|
| Priority: | normal | Milestone: | Support Python 3 |
| Component: | unknown | Version: | n/a |
| Keywords: | review-needed | Cc: | |
| Launchpad Bug: |
Description
The mypy project (and type checking more generally) promises to improve code quality and catch type errors statically. Recent versions of mypy have fairly robust support for inferring types for many functions. The Python 3 porting team would like to evaluate the viability of employing mypy checks during the code check phase to catch such errors early and perhaps elucidate issues with binary vs. text during the porting effort.
Change History (30)
comment:1 Changed at 2020-09-11T19:37:44Z by jaraco
- Status changed from new to assigned
comment:2 Changed at 2020-09-11T19:40:54Z by jaraco
comment:3 Changed at 2020-09-11T20:00:03Z by jaraco
Taking a look at the interfaces, I see that they're using foolscap.api.RemoteInterface, which explicitly states that the conventional "self" arguments are disallowed.
So it seems at first blush that mypy and foolscap are mutually incompatible.
comment:4 Changed at 2020-09-11T20:04:58Z by jaraco
I imagine there are a few options forward:
- Amend mypy to support the foolscap interface(s).
- Update foolscap to be compatible with mypy conventions.
- Disable mypy checks for code that touches foolscap interfaces.
Options (1) and (2) are likely to be expensive and possibly intractable efforts. It's uncertain what the scope of (3) would be or what effect that would have on the checks.
I was unable to find any references in foolscap referencing mypy, so I suspect this issue has not been approached previously.
comment:5 Changed at 2020-09-11T20:05:44Z by jaraco
- Owner jaraco deleted
- Status changed from assigned to new
comment:6 Changed at 2020-09-11T21:18:24Z by wearpants
I'll need to look into this when I have a bit more brain, but as a quick check, does zope.interface have the same behavior? If so, maybe we can piggyback off what the mypy zope plugin is already doing to support that.
Otherwise, I agree that option 3 is the best choice, probably by explicitly disabling specific checks on a line-by-line basis. My only hesitancy about doing that would be if it impedes the effectiveness of mypy elsewhere in the code.
comment:7 Changed at 2020-09-18T19:52:38Z by jaraco
Fantastic suggestion. Digging deeper into the interfaces implementations, I noticed that many of them are in fact zope interfaces. In 471c88c70, I've simply added that plugin and enabled it and the errors dropped from 242 to 142 (exactly 100? that's suspicious o_O).
typechecks installed: mypy==0.782,mypy-extensions==0.4.3,mypy-zope==0.2.7,typed-ast==1.4.1,typing-extensions==3.7.4.3,zope.event==4.5.0,zope.interface==5.1.0,zope.schema==6.0.0
typechecks run-test-pre: PYTHONHASHSEED='2059477720'
typechecks run-test: commands[0] | mypy src
src/allmydata/util/pollmixin.py:26: error: Need type annotation for '_poll_should_ignore_these_errors' (hint: "_poll_should_ignore_these_errors: List[<type>] = ...")
src/allmydata/test/test_python3.py:50: error: "Callable[[Python3PortingEffortTests], Any]" has no attribute "todo"
src/allmydata/test/check_load.py:47: error: Need type annotation for 'last_stats' (hint: "last_stats: Dict[<type>, <type>] = ...")
src/allmydata/test/check_load.py:56: error: Incompatible types in assignment (expression has type "float", target has type "int")
src/allmydata/interfaces.py:59: error: Method must have at least one argument
src/allmydata/interfaces.py:67: error: Method must have at least one argument
src/allmydata/interfaces.py:110: error: Method must have at least one argument
src/allmydata/interfaces.py:683: error: Cannot determine consistent method resolution order (MRO) for "IVerifierURI"
src/allmydata/interfaces.py:750: error: Interface methods should not have 'self' argument
src/allmydata/interfaces.py:758: error: Interface methods should not have 'self' argument
src/allmydata/interfaces.py:2845: error: Method must have at least one argument
src/allmydata/interfaces.py:2863: error: Method must have at least one argument
src/allmydata/interfaces.py:2880: error: Method must have at least one argument
src/allmydata/interfaces.py:2883: error: Method must have at least one argument
src/allmydata/interfaces.py:2889: error: Method must have at least one argument
src/allmydata/interfaces.py:2896: error: Method must have at least one argument
src/allmydata/interfaces.py:2909: error: Method must have at least one argument
src/allmydata/interfaces.py:2939: error: Method must have at least one argument
src/allmydata/web/private.py:64: error: zope.interface.implementer accepts interface, not allmydata.web.private.IToken.
src/allmydata/web/private.py:64: error: Make sure you have stubs for all packages that provide interfaces for allmydata.web.private.IToken class hierarchy.
src/allmydata/introducer/interfaces.py:45: error: Method must have at least one argument
src/allmydata/codec.py:23: error: 'CRSEncoder' is missing following 'ICodecEncoder' interface members: encode_proposal.
src/allmydata/util/fileutil.py:312: error: Name '_getfullpathname' already defined on line 310
src/allmydata/storage/immutable.py:193: error: zope.interface.implementer accepts interface, not allmydata.interfaces.RIBucketWriter.
src/allmydata/storage/immutable.py:193: error: Make sure you have stubs for all packages that provide interfaces for allmydata.interfaces.RIBucketWriter class hierarchy.
src/allmydata/storage/immutable.py:292: error: zope.interface.implementer accepts interface, not allmydata.interfaces.RIBucketReader.
src/allmydata/storage/immutable.py:292: error: Make sure you have stubs for all packages that provide interfaces for allmydata.interfaces.RIBucketReader class hierarchy.
src/allmydata/storage/crawler.py:22: error: Name 'pickle' already defined (possibly by an import)
src/allmydata/introducer/client.py:20: error: zope.interface.implementer accepts interface, not allmydata.introducer.interfaces.RIIntroducerSubscriberClient_v2.
src/allmydata/introducer/client.py:20: error: Make sure you have stubs for all packages that provide interfaces for allmydata.introducer.interfaces.RIIntroducerSubscriberClient_v2 class hierarchy.
src/allmydata/introducer/client.py:159: error: Signature of "IntroducerClient" incompatible with "subscribe_to" of supertype "IIntroducerClient"
src/allmydata/introducer/client.py:200: error: Signature of "IntroducerClient" incompatible with "publish" of supertype "IIntroducerClient"
src/allmydata/stats.py:126: error: zope.interface.implementer accepts interface, not allmydata.interfaces.RIStatsProvider.
src/allmydata/stats.py:126: error: Make sure you have stubs for all packages that provide interfaces for allmydata.interfaces.RIStatsProvider class hierarchy.
src/allmydata/stats.py:179: error: zope.interface.implementer accepts interface, not allmydata.interfaces.RIStatsGatherer.
src/allmydata/stats.py:179: error: Make sure you have stubs for all packages that provide interfaces for allmydata.interfaces.RIStatsGatherer class hierarchy.
src/allmydata/node.py:712: error: Need type annotation for 'GENERATED_FILES' (hint: "GENERATED_FILES: List[<type>] = ...")
src/allmydata/windows/fixups.py:220: error: Name 'unichr' is not defined
src/allmydata/test/test_iputil.py:136: error: "Callable[[ListAddresses], Any]" has no attribute "timeout"
src/allmydata/web/status.py:1327: error: Name 'cmp' is not defined
src/allmydata/web/status.py:1335: error: Name 'cmp' is not defined
src/allmydata/test/test_stats.py:9: error: Incompatible types in assignment (expression has type "float", base class "CPUUsageMonitor" defined the type as "int")
src/allmydata/storage/server.py:39: error: zope.interface.implementer accepts interface, not allmydata.interfaces.RIStorageServer.
src/allmydata/storage/server.py:39: error: Make sure you have stubs for all packages that provide interfaces for allmydata.interfaces.RIStorageServer class hierarchy.
src/allmydata/uri.py:491: error: '_DirectoryBaseURI' is missing following 'IURI' interface members: is_readonly, get_readonly.
src/allmydata/uri.py:726: error: Incompatible types in assignment (expression has type "Type[CHKFileVerifierURI]", base class "DirectoryURIVerifier" defined the type as "Type[SSKVerifierURI]")
src/allmydata/immutable/literal.py:10: error: '_ImmutableFileNodeBase' is missing following 'allmydata.interfaces.IFilesystemNode' interface members: get_storage_index, get_size, get_cap, get_readcap, get_repair_cap, get_verify_cap, get_uri, get_current_size.
src/allmydata/immutable/literal.py:10: error: '_ImmutableFileNodeBase' is missing following 'allmydata.interfaces.IReadable' interface members: download_to_data, read.
src/allmydata/immutable/literal.py:10: error: '_ImmutableFileNodeBase' is missing following 'allmydata.interfaces.IFileNode' interface members: get_best_readable_version, download_best_version, get_size_of_best_version.
src/allmydata/immutable/literal.py:10: error: '_ImmutableFileNodeBase' is missing following 'ICheckable' interface members: check, check_and_repair.
src/allmydata/immutable/encode.py:78: error: 'Encoder' is missing following 'IEncoder' interface members: set_size.
src/allmydata/scripts/run_common.py:70: error: List item 0 has incompatible type "Tuple[str, str, None, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/debug.py:21: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/debug.py:419: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/debug.py:613: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/debug.py:661: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/debug.py:875: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/create_node.py:147: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/create_node.py:154: error: List item 0 has incompatible type "Tuple[str, str, None, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/create_node.py:155: error: List item 1 has incompatible type "Tuple[str, str, None, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/create_node.py:156: error: List item 2 has incompatible type "Tuple[str, str, str, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/create_node.py:158: error: List item 3 has incompatible type "Tuple[str, str, None, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/create_node.py:160: error: List item 4 has incompatible type "Tuple[str, None, int, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/create_node.py:161: error: List item 5 has incompatible type "Tuple[str, None, int, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/create_node.py:162: error: List item 6 has incompatible type "Tuple[str, None, int, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/create_node.py:163: error: List item 7 has incompatible type "Tuple[str, None, None, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/create_node.py:199: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/immutable/upload.py:319: error: 'PeerSelector' is missing following 'IPeerSelector' interface members: confirm_share_allocation, add_peers.
src/allmydata/immutable/upload.py:1411: error: zope.interface.implementer accepts interface, not allmydata.interfaces.RIEncryptedUploadable.
src/allmydata/immutable/upload.py:1411: error: Make sure you have stubs for all packages that provide interfaces for allmydata.interfaces.RIEncryptedUploadable class hierarchy.
src/allmydata/control.py:58: error: zope.interface.implementer accepts interface, not allmydata.interfaces.RIControlClient.
src/allmydata/control.py:58: error: Make sure you have stubs for all packages that provide interfaces for allmydata.interfaces.RIControlClient class hierarchy.
src/allmydata/scripts/tahoe_start.py:20: error: List item 0 has incompatible type "Tuple[str, str, None, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/stats_gatherer.py:15: error: List item 0 has incompatible type "Tuple[str, None, None, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/stats_gatherer.py:16: error: List item 1 has incompatible type "Tuple[str, None, None, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/stats_gatherer.py:17: error: List item 2 has incompatible type "Tuple[str, None, None, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/stats_gatherer.py:27: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/mutable/filenode.py:554: error: Signature of "MutableFileNode" incompatible with "upload" of supertype "IMutableFileNode"
src/allmydata/mutable/filenode.py:702: error: 'MutableFileVersion' is missing following 'IMutableFileVersion' interface members: get_servermap.
src/allmydata/mutable/filenode.py:941: error: Signature of "MutableFileVersion" incompatible with "download_to_data" of supertype "allmydata.interfaces.IReadable"
src/allmydata/immutable/offloaded.py:125: error: zope.interface.implementer accepts interface, not allmydata.interfaces.RICHKUploadHelper.
src/allmydata/immutable/offloaded.py:125: error: Make sure you have stubs for all packages that provide interfaces for allmydata.interfaces.RICHKUploadHelper class hierarchy.
src/allmydata/immutable/offloaded.py:450: error: 'LocalCiphertextReader' is missing following 'IEncryptedUploadable' interface members: set_upload_status.
src/allmydata/immutable/offloaded.py:486: error: zope.interface.implementer accepts interface, not allmydata.interfaces.RIHelper.
src/allmydata/immutable/offloaded.py:486: error: Make sure you have stubs for all packages that provide interfaces for allmydata.interfaces.RIHelper class hierarchy.
src/allmydata/frontends/ftpd.py:2: error: Module 'types' has no attribute 'NoneType'
src/allmydata/dirnode.py:557: error: Signature of "DirectoryNode" incompatible with "set_uri" of supertype "IDirectoryNode"
src/allmydata/web/directory.py:1217: error: Incompatible types in assignment (expression has type "str", base class "MultiFormatResource" defined the type as "None")
src/allmydata/web/directory.py:1274: error: Incompatible types in assignment (expression has type "str", base class "MultiFormatResource" defined the type as "None")
src/allmydata/scripts/cli.py:54: error: List item 0 has incompatible type "Tuple[str, None, None, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/cli.py:65: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:75: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:84: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:88: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:107: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:150: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:155: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:168: error: List item 0 has incompatible type "Tuple[str, None, None, str]"; expected "List[Optional[str]]"
src/allmydata/scripts/cli.py:186: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:198: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:226: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:261: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:270: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:289: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:375: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:392: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:408: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:420: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:435: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/scripts/cli.py:452: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None")
src/allmydata/frontends/sftpd.py:3: error: Module 'types' has no attribute 'NoneType'
src/allmydata/scripts/runner.py:65: error: Unsupported operand types for + ("List[List[Union[Type[BaseOptions], str, None]]]" and "List[Tuple[str, None, Type[InviteOptions], str]]")
src/allmydata/scripts/runner.py:110: error: Module has no attribute "dispatch"
src/allmydata/introducer/server.py:130: error: zope.interface.implementer accepts interface, not allmydata.introducer.interfaces.RIIntroducerPublisherAndSubscriberService_v2.
src/allmydata/introducer/server.py:130: error: Make sure you have stubs for all packages that provide interfaces for allmydata.introducer.interfaces.RIIntroducerPublisherAndSubscriberService_v2 class hierarchy.
src/allmydata/test/storage_plugin.py:41: error: Method must have at least one argument
src/allmydata/test/storage_plugin.py:49: error: zope.interface.implementer accepts interface, not allmydata.interfaces.IFoolscapStoragePlugin.
src/allmydata/test/storage_plugin.py:49: error: Make sure you have stubs for all packages that provide interfaces for allmydata.interfaces.IFoolscapStoragePlugin class hierarchy.
src/allmydata/test/storage_plugin.py:106: error: zope.interface.implementer accepts interface, not allmydata.test.storage_plugin.RIDummy.
src/allmydata/test/storage_plugin.py:106: error: Make sure you have stubs for all packages that provide interfaces for allmydata.test.storage_plugin.RIDummy class hierarchy.
src/allmydata/test/storage_plugin.py:117: error: 'DummyStorageClient' is missing following 'IStorageServer' interface members: get_version, allocate_buckets, add_lease, renew_lease, get_buckets, slot_readv, slot_testv_and_readv_and_writev, advise_corrupt_share.
src/allmydata/test/common.py:395: error: 'FakeCHKFileNode' is missing following 'allmydata.interfaces.IFilesystemNode' interface members: get_readcap.
src/allmydata/test/common.py:533: error: 'FakeMutableFileNode' is missing following 'IMutableFileNode' interface members: download_version, upload.
src/allmydata/test/test_storage_client.py:508: error: On Python 3 '{}'.format(b'abc') produces "b'abc'"; use !r if this is a desired behavior
src/allmydata/test/test_helper.py:88: error: Need type annotation for 'introducer_clients' (hint: "introducer_clients: List[<type>] = ...")
src/allmydata/test/no_network.py:142: error: 'NoNetworkServer' is missing following 'IServer' interface members: start_connecting.
src/allmydata/test/no_network.py:180: error: 'NoNetworkStorageBroker' is missing following 'IStorageBroker' interface members: get_all_connections, get_all_connectors, get_all_peerids, get_all_connections_for, get_permuted_peers.
src/allmydata/test/check_memory.py:495: error: Module has no attribute "maxint"
src/allmydata/test/check_memory.py:497: error: Module has no attribute "maxint"
src/allmydata/test/check_memory.py:501: error: Module has no attribute "maxint"
src/allmydata/test/test_sftp.py:17: error: Name 'conch_interfaces' already defined on line 12
src/allmydata/test/test_sftp.py:18: error: Name 'sftp' already defined on line 13
src/allmydata/test/test_sftp.py:19: error: Incompatible import of "sftpd" (imported name has type Module, local name has type "object")
src/allmydata/test/test_sftp.py:23: error: Incompatible types in assignment (expression has type "None", variable has type "ImportError")
src/allmydata/test/test_encode.py:176: error: Incompatible types in assignment (expression has type "float", variable has type "int")
src/allmydata/test/test_dirnode.py:1529: error: 'FakeMutableFile' is missing following 'allmydata.interfaces.IFilesystemNode' interface members: get_readcap, get_repair_cap, get_verify_cap, get_readonly_uri, get_storage_index, get_size, get_current_size.
src/allmydata/test/test_dirnode.py:1529: error: 'FakeMutableFile' is missing following 'allmydata.interfaces.IFileNode' interface members: get_best_readable_version, get_size_of_best_version.
src/allmydata/test/test_dirnode.py:1529: error: 'FakeMutableFile' is missing following 'IMutableFileNode' interface members: get_best_mutable_version, overwrite, get_servermap, download_version, upload, get_version.
src/allmydata/test/test_deepcheck.py:919: error: Name 'unicode' is not defined
src/allmydata/test/test_checker.py:65: error: 'FakeServer' is missing following 'IServer' interface members: start_connecting, get_rref, get_storage_server.
src/allmydata/test/test_checker.py:78: error: 'FakeCheckResults' is missing following 'ICheckResults' interface members: get_uri, get_happiness, get_encoding_needed, get_encoding_expected, get_share_counter_good, get_share_counter_wrong, get_incompatible_shares, get_servers_responding, get_host_counter_good_shares, get_version_counter_recoverable, get_version_counter_unrecoverable, get_sharemap, get_report.
src/allmydata/test/test_checker.py:109: error: 'FakeCheckAndRepairResults' is missing following 'ICheckAndRepairResults' interface members: get_storage_index_string.
Found 142 errors in 48 files (checked 291 source files)
ERROR: InvocationError for command /Users/jaraco/code/public/tahoe-lafs/.tox/typechecks/bin/mypy src (exited with code 1)
___________________________________ summary ____________________________________
ERROR: typechecks: commands failed
Still some errors remain about "Method must have at least one argument.", suggesting that the interface detection is being missed on some interfaces, but now at a more manageable level.
comment:8 Changed at 2020-09-18T20:41:08Z by jaraco
I've filed Shoobx/mypy-zope#21 to seek some help with addressing the remaining issues with "Method must have at least one argument".
comment:9 Changed at 2020-10-31T16:57:36Z by jaraco
After proposing Shoobx/mypy-zope#24 and warner/foolscap#76, I realized I could test the approach against the Tahoe-LAFS test suite to validate the fix, but to my dismay, the mypy checks still fail with the same 142 errors.
comment:10 Changed at 2020-10-31T18:57:14Z by jaraco
After some troubleshooting, I've found that I can validate this code using those branches:
from foolscap.api import RemoteInterface, IntegerConstraint, StringConstraint Number = IntegerConstraint(8) # 2**(8*8) == 16EiB ~= 18e18 ~= 18 exabytes Offset = Number ShareData = StringConstraint(None) class RIBucketWriter(RemoteInterface): """ Objects of this kind live on the server side. """ def write(offset=Offset, data=ShareData): return None
...which is almost verbatim the code that leads to the mypy error in allmydata.interfaces. So there's still a gap to be bridged.
comment:11 Changed at 2020-10-31T19:08:42Z by jaraco
Interesting - testing that file using the typechecks tox environment triggers the failure.
Aha! I had an outstanding change in foolscap that's still needed. Incorporating that change allows the RemoteInterface? objects to pass type checks.
comment:12 Changed at 2020-10-31T19:19:24Z by jaraco
With the latest push to foolscap#76, the errors around at least one method to RemoteInterface are gone. Only 113 errors remain.
comment:13 Changed at 2020-10-31T19:48:29Z by jaraco
But you'll just have to take my word for it, because the CI builders don't seem to be aware of my branch until I submit a PR. I guess I should start that.
comment:14 Changed at 2020-11-20T17:17:38Z by jaraco
I'm working on #886 to capture the progress of the mypy work, but unfortunately, the Github checks don't seem to include Circle CI and even in Circle CI, my tests appear to be unauthorized.
comment:15 Changed at 2020-11-20T18:31:37Z by jaraco
While working on private.py, I get this error:
src/allmydata/web/private.py:64: error: zope.interface.implementer accepts interface, not allmydata.web.private.IToken. src/allmydata/web/private.py:64: error: Make sure you have stubs for all packages that provide interfaces for allmydata.web.private.IToken class hierarchy.
I distilled the issue and filed Shoobx/mypy-zope#26 to capture what appears to be an underlying issue.
comment:16 Changed at 2020-11-23T18:45:49Z by jaraco
Investigating in that issue, I learned quite a bit about how mypy works. It seems that because Twisted isn't typed, it requires stubs to effectively communicate its interfaces (and not trip up on the decorator syntax). I ended up trying two approaches:
- 3399.mypy.workaround applies a clumsy but effective workaround that addresses this one error.
- 3399.mypy.twisted-stubs creates twisted stubs, first from twisted trunk, then from the latest stable. However, this approach creates many more errors than it satisfies, probably because adding the stubs removes the mask of errors from ignore_missing_imports.
Based on that, I'm going to try to proceed with addressing the type errors without twisted stubs for now and see how far I can get without relying on twisted stubs, a non-trivial effort.
comment:17 Changed at 2020-11-23T19:07:55Z by jaraco
As I'm working through the errors, I see this one: src/allmydata/interfaces.py:685: error: Cannot determine consistent method resolution order (MRO) for "IVerifierURI". That class derives from Interface as well as IURI, which derives from Interface. That seems like an unnecessary double-base, so I'm removing it.
comment:18 Changed at 2020-11-23T20:04:00Z by jaraco
Now I'm looking at this error:
src/allmydata/storage/immutable.py:205: error: 'BucketWriter' is missing following 'RIBucketWriter' interface members: abort, close, write. src/allmydata/storage/immutable.py:304: error: 'BucketReader' is missing following 'RIBucketReader' interface members: advise_corrupt_share, read.
Indeed, the RIBucketWriter and RIBucketReader define those methods, but the BucketReader/Writer do not. Is that a feature of the RemoteInterface? What is the rationale behind this arrangement?
comment:19 Changed at 2020-11-29T16:31:27Z by jaraco
Exarkun added some clarity to the question by pointing out that BucketWriter has those methods, just prefixed by remote_. So it seems that there's some protocol in RemoteInterfaces that expects interface methods to have that prefix. Probably mypy.zope or foolscap will need to implement a mypy plugin to honor that convention.
comment:20 Changed at 2020-11-29T17:29:45Z by jaraco
Today, I attempted to replicate the issue in isolation, but failed:
test $ ls
mypy.ini remote.py
test $ cat mypy.ini
[mypy]
ignore_missing_imports = True
plugins=mypy_zope:plugin
test $ cat remote.py
__requires__ = [
'mypy-zope@git+https://github.com/jaraco/mypy-zope@bugfix/21-InterfaceClass-subclass',
'foolscap@git+https://github.com/jaraco/foolscap@bugfix/75-use-metaclass',
]
from foolscap.api import RemoteInterface, Referenceable
from zope.interface import implementer
class RIBucketWriter(RemoteInterface):
""" Objects of this kind live on the server side. """
def write(offset=None, data=None):
return None
@implementer(RIBucketWriter)
class BucketWriter(Referenceable):
def remote_write(self, offset, data):
pass
test $ pip-run --use-pep517 -- -m mypy remote.py
Collecting mypy-zope@ git+https://github.com/jaraco/mypy-zope@bugfix/21-InterfaceClass-subclass
Cloning https://github.com/jaraco/mypy-zope (to revision bugfix/21-InterfaceClass-subclass) to /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-0kbosrxs/mypy-zope_a3857a33b4424ad28fc9a092a32deb1f
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting foolscap@ git+https://github.com/jaraco/foolscap@bugfix/75-use-metaclass
Cloning https://github.com/jaraco/foolscap (to revision bugfix/75-use-metaclass) to /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-0kbosrxs/foolscap_7a99538f390f4567a7f8091539bf4071
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting mypy==0.790
Using cached mypy-0.790-py3-none-any.whl (2.4 MB)
Collecting zope.interface
Using cached zope.interface-5.2.0-cp39-cp39-macosx_10_9_x86_64.whl (194 kB)
Collecting zope.schema
Using cached zope.schema-6.0.0-py2.py3-none-any.whl (85 kB)
Collecting six
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Processing /Users/jaraco/Library/Caches/pip/wheels/e5/5c/53/f56b69010340b883474a456e8ee34b546e27f78f01b36701e3/Twisted-20.3.0-cp39-cp39-macosx_10_9_x86_64.whl
Collecting pyOpenSSL
Using cached pyOpenSSL-20.0.0-py2.py3-none-any.whl (54 kB)
Collecting typed-ast<1.5.0,>=1.4.0
Using cached typed_ast-1.4.1-cp39-cp39-macosx_10_15_x86_64.whl (225 kB)
Collecting mypy-extensions<0.5.0,>=0.4.3
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting typing-extensions>=3.7.4
Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting setuptools
Using cached setuptools-50.3.2-py3-none-any.whl (785 kB)
Collecting zope.event
Using cached zope.event-4.5.0-py2.py3-none-any.whl (6.8 kB)
Collecting constantly>=15.1
Using cached constantly-15.1.0-py2.py3-none-any.whl (7.9 kB)
Collecting attrs>=19.2.0
Using cached attrs-20.3.0-py2.py3-none-any.whl (49 kB)
Collecting incremental>=16.10.1
Using cached incremental-17.5.0-py2.py3-none-any.whl (16 kB)
Collecting hyperlink>=17.1.1
Using cached hyperlink-20.0.1-py2.py3-none-any.whl (48 kB)
Collecting Automat>=0.3.0
Using cached Automat-20.2.0-py2.py3-none-any.whl (31 kB)
Collecting PyHamcrest!=1.10.0,>=1.9.0
Using cached PyHamcrest-2.0.2-py3-none-any.whl (52 kB)
Collecting service-identity>=18.1.0
Using cached service_identity-18.1.0-py2.py3-none-any.whl (11 kB)
Collecting idna!=2.3,>=0.6
Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting cryptography>=3.2
Using cached cryptography-3.2.1-cp35-abi3-macosx_10_10_x86_64.whl (1.8 MB)
Collecting pyasn1
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting pyasn1-modules
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting cffi!=1.11.3,>=1.8
Using cached cffi-1.14.4-cp39-cp39-macosx_10_9_x86_64.whl (177 kB)
Collecting pycparser
Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Building wheels for collected packages: mypy-zope, foolscap
Building wheel for mypy-zope (PEP 517) ... done
Created wheel for mypy-zope: filename=mypy_zope-0.2.9.dev0-py3-none-any.whl size=29442 sha256=5d416c94987932d4c62552cce217a14f007896c9fc8cec702b6474f390aa960b
Stored in directory: /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-ephem-wheel-cache-5rfaa5xe/wheels/6e/93/57/0be05da77af6d6324f8375d9f7a1f91d5b6cc20b91e6c28c80
Building wheel for foolscap (PEP 517) ... done
Created wheel for foolscap: filename=foolscap-20.4.0+5.gd70d8ec-py2.py3-none-any.whl size=312412 sha256=ef46bd7f18023168363e657dfbda7085f2464756064532135a3dd1f1115cbde5
Stored in directory: /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-ephem-wheel-cache-5rfaa5xe/wheels/c5/da/84/b38eac115cb1fc7f17f674229e212cabfa64975c03dc178ba5
Successfully built mypy-zope foolscap
Installing collected packages: pycparser, six, setuptools, pyasn1, idna, cffi, attrs, zope.interface, PyHamcrest, pyasn1-modules, incremental, hyperlink, cryptography, constantly, Automat, zope.event, typing-extensions, typed-ast, twisted, service-identity, pyOpenSSL, mypy-extensions, zope.schema, mypy, mypy-zope, foolscap
Successfully installed Automat-20.2.0 PyHamcrest-2.0.2 attrs-20.3.0 cffi-1.14.4 constantly-15.1.0 cryptography-3.2.1 foolscap-20.4.0+5.gd70d8ec hyperlink-20.0.1 idna-2.10 incremental-17.5.0 mypy-0.790 mypy-extensions-0.4.3 mypy-zope-0.2.9.dev0 pyOpenSSL-20.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.20 service-identity-18.1.0 setuptools-50.3.2 six-1.15.0 twisted-20.3.0 typed-ast-1.4.1 typing-extensions-3.7.4.3 zope.event-4.5.0 zope.interface-5.2.0 zope.schema-6.0.0
Success: no issues found in 1 source file
No issues found even with the pre-release versions of those branches.
But, using the same reproducer but running with mypy as found in the tahoe project, the error does emerge:
test $ ~/p/tahoe-lafs/.tox/typechecks/bin/mypy remote.py remote.py:18: error: 'BucketWriter' is missing following 'RIBucketWriter' interface members: write. Found 1 error in 1 file (checked 1 source file)
comment:21 Changed at 2020-11-29T17:34:29Z by jaraco
Attempting to bisect these two behaviors, I'm first going to create a new environment and eliminate pip-run from the equation.
test $ python -m venv env
test $ env/bin/pip install mypy-zope@git+https://github.com/jaraco/mypy-zope@bugfix/21-InterfaceClass-subclass foolscap@git+https://
github.com/jaraco/foolscap@bugfix/75-use-metaclass
Collecting mypy-zope@ git+https://github.com/jaraco/mypy-zope@bugfix/21-InterfaceClass-subclass
Cloning https://github.com/jaraco/mypy-zope (to revision bugfix/21-InterfaceClass-subclass) to /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-77g6an3b/mypy-zope_f6b54b0e1c91451abed30cf7a1265976
Collecting foolscap@ git+https://github.com/jaraco/foolscap@bugfix/75-use-metaclass
Cloning https://github.com/jaraco/foolscap (to revision bugfix/75-use-metaclass) to /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-install-77g6an3b/foolscap_bb7193a6b86a4326b0d66ba21edbd79f
Collecting mypy==0.790
Using cached mypy-0.790-py3-none-any.whl (2.4 MB)
Collecting mypy-extensions<0.5.0,>=0.4.3
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting typed-ast<1.5.0,>=1.4.0
Using cached typed_ast-1.4.1-cp39-cp39-macosx_10_15_x86_64.whl (225 kB)
Collecting typing-extensions>=3.7.4
Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Processing /Users/jaraco/Library/Caches/pip/wheels/e5/5c/53/f56b69010340b883474a456e8ee34b546e27f78f01b36701e3/Twisted-20.3.0-cp39-cp39-macosx_10_9_x86_64.whl
Collecting constantly>=15.1
Using cached constantly-15.1.0-py2.py3-none-any.whl (7.9 kB)
Collecting service-identity>=18.1.0
Using cached service_identity-18.1.0-py2.py3-none-any.whl (11 kB)
Collecting incremental>=16.10.1
Using cached incremental-17.5.0-py2.py3-none-any.whl (16 kB)
Collecting attrs>=19.2.0
Using cached attrs-20.3.0-py2.py3-none-any.whl (49 kB)
Collecting PyHamcrest!=1.10.0,>=1.9.0
Using cached PyHamcrest-2.0.2-py3-none-any.whl (52 kB)
Collecting Automat>=0.3.0
Using cached Automat-20.2.0-py2.py3-none-any.whl (31 kB)
Collecting hyperlink>=17.1.1
Using cached hyperlink-20.0.1-py2.py3-none-any.whl (48 kB)
Collecting idna!=2.3,>=0.6
Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting pyOpenSSL
Using cached pyOpenSSL-20.0.0-py2.py3-none-any.whl (54 kB)
Collecting cryptography
Using cached cryptography-3.2.1-cp35-abi3-macosx_10_10_x86_64.whl (1.8 MB)
Collecting zope.interface
Using cached zope.interface-5.2.0-cp39-cp39-macosx_10_9_x86_64.whl (194 kB)
Requirement already satisfied: setuptools in ./env/lib/python3.9/site-packages (from zope.interface->mypy-zope@ git+https://github.com/jaraco/mypy-zope@bugfix/21-InterfaceClass-subclass) (49.2.1)
Collecting six
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting pyasn1-modules
Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting pyasn1
Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting cffi!=1.11.3,>=1.8
Using cached cffi-1.14.4-cp39-cp39-macosx_10_9_x86_64.whl (177 kB)
Collecting pycparser
Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Collecting zope.schema
Using cached zope.schema-6.0.0-py2.py3-none-any.whl (85 kB)
Requirement already satisfied: setuptools in ./env/lib/python3.9/site-packages (from zope.interface->mypy-zope@ git+https://github.com/jaraco/mypy-zope@bugfix/21-InterfaceClass-subclass) (49.2.1)
Collecting zope.event
Using cached zope.event-4.5.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: setuptools in ./env/lib/python3.9/site-packages (from zope.interface->mypy-zope@ git+https://github.com/jaraco/mypy-zope@bugfix/21-InterfaceClass-subclass) (49.2.1)
Using legacy 'setup.py install' for mypy-zope, since package 'wheel' is not installed.
Using legacy 'setup.py install' for foolscap, since package 'wheel' is not installed.
Installing collected packages: pycparser, six, pyasn1, idna, cffi, attrs, zope.interface, PyHamcrest, pyasn1-modules, incremental, hyperlink, cryptography, constantly, Automat, zope.event, typing-extensions, typed-ast, twisted, service-identity, pyOpenSSL, mypy-extensions, zope.schema, mypy, mypy-zope, foolscap
Running setup.py install for mypy-zope ... done
Running setup.py install for foolscap ... done
Successfully installed Automat-20.2.0 PyHamcrest-2.0.2 attrs-20.3.0 cffi-1.14.4 constantly-15.1.0 cryptography-3.2.1 foolscap-20.4.0+5.gd70d8ec hyperlink-20.0.1 idna-2.10 incremental-17.5.0 mypy-0.790 mypy-extensions-0.4.3 mypy-zope-0.2.9.dev0 pyOpenSSL-20.0.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.20 service-identity-18.1.0 six-1.15.0 twisted-20.3.0 typed-ast-1.4.1 typing-extensions-3.7.4.3 zope.event-4.5.0 zope.interface-5.2.0 zope.schema-6.0.0
WARNING: You are using pip version 20.2.3; however, version 20.2.4 is available.
You should consider upgrading via the '/Users/jaraco/code/public/tahoe-lafs/test/env/bin/python -m pip install --upgrade pip' command.
test $ env/bin/mypy remote.py
remote.py:18: error: 'BucketWriter' is missing following 'RIBucketWriter' interface members: write.
Found 1 error in 1 file (checked 1 source file)
Right, so I'm reminded of python/mypy#9678. I just need to stay away from pip-run for now. At least, I've replicated the issue now in an isolated environment. I'll take that to the foolscap project.
comment:22 Changed at 2020-11-29T18:27:38Z by jaraco
I've filed warner/foolscap#78 to capture the weakness in foolscap. For this project, I'm going to see if there's any way I can bypass these checks.
comment:23 Changed at 2020-11-29T18:35:44Z by jaraco
Looks like it's straightforward to avoid typechecks on types failing due to Referenceable. I'll be checking all of these errors to see if the workaround applies:
tahoe-lafs 3399.mypy $ tox -e typechecks | grep 'interface members' src/allmydata/storage/immutable.py:205: error: 'BucketWriter' is missing following 'RIBucketWriter' interface members: abort, close, write. src/allmydata/storage/immutable.py:304: error: 'BucketReader' is missing following 'RIBucketReader' interface members: advise_corrupt_share, read. src/allmydata/uri.py:493: error: '_DirectoryBaseURI' is missing following 'IURI' interface members: get_readonly, is_readonly. src/allmydata/immutable/literal.py:23: error: '_ImmutableFileNodeBase' is missing following 'allmydata.interfaces.IReadable' interface members: download_to_data, read. src/allmydata/immutable/literal.py:23: error: '_ImmutableFileNodeBase' is missing following 'allmydata.interfaces.IFilesystemNode' interface members: get_size, get_storage_index, get_cap, get_current_size, get_readcap, get_repair_cap, get_uri, get_verify_cap. src/allmydata/immutable/literal.py:23: error: '_ImmutableFileNodeBase' is missing following 'allmydata.interfaces.IFileNode' interface members: download_best_version, get_best_readable_version, get_size_of_best_version. src/allmydata/immutable/literal.py:23: error: '_ImmutableFileNodeBase' is missing following 'ICheckable' interface members: check, check_and_repair. src/allmydata/immutable/encode.py:91: error: 'Encoder' is missing following 'IEncoder' interface members: set_size. src/allmydata/immutable/upload.py:333: error: 'PeerSelector' is missing following 'IPeerSelector' interface members: add_peers, confirm_share_allocation. src/allmydata/immutable/upload.py:1426: error: 'RemoteEncryptedUploadable' is missing following 'RIEncryptedUploadable' interface members: __remote_name__, close, get_all_encoding_parameters, read_encrypted. src/allmydata/mutable/filenode.py:715: error: 'MutableFileVersion' is missing following 'IMutableFileVersion' interface members: get_servermap. src/allmydata/immutable/offloaded.py:144: error: 'CHKUploadHelper' is missing following 'RICHKUploadHelper' interface members: __remote_name__, get_version, upload. src/allmydata/immutable/offloaded.py:468: error: 'LocalCiphertextReader' is missing following 'IEncryptedUploadable' interface members: set_upload_status. src/allmydata/immutable/offloaded.py:505: error: 'Helper' is missing following 'RIHelper' interface members: __remote_name__, get_version, upload_chk. src/allmydata/test/storage_plugin.py:110: error: 'DummyStorageServer' is missing following 'RIDummy' interface members: __remote_name__, just_some_method. src/allmydata/test/storage_plugin.py:119: error: 'DummyStorageClient' is missing following 'IStorageServer' interface members: add_lease, advise_corrupt_share, allocate_buckets, get_buckets, get_version, renew_lease, slot_readv, slot_testv_and_readv_and_writev. src/allmydata/test/common.py:395: error: 'FakeCHKFileNode' is missing following 'allmydata.interfaces.IFilesystemNode' interface members: get_readcap. src/allmydata/test/common.py:533: error: 'FakeMutableFileNode' is missing following 'IMutableFileNode' interface members: download_version, upload. src/allmydata/test/no_network.py:179: error: 'NoNetworkServer' is missing following 'IServer' interface members: start_connecting. src/allmydata/test/no_network.py:217: error: 'NoNetworkStorageBroker' is missing following 'IStorageBroker' interface members: get_all_connections, get_all_connections_for, get_all_connectors, get_all_peerids, get_permuted_peers. src/allmydata/test/test_dirnode.py:1529: error: 'FakeMutableFile' is missing following 'allmydata.interfaces.IFilesystemNode' interface members: get_current_size, get_readcap, get_readonly_uri, get_repair_cap, get_size, get_storage_index, get_verify_cap. src/allmydata/test/test_dirnode.py:1529: error: 'FakeMutableFile' is missing following 'allmydata.interfaces.IFileNode' interface members: get_best_readable_version, get_size_of_best_version. src/allmydata/test/test_dirnode.py:1529: error: 'FakeMutableFile' is missing following 'IMutableFileNode' interface members: download_version, get_best_mutable_version, get_servermap, get_version, overwrite, upload. src/allmydata/test/test_checker.py:65: error: 'FakeServer' is missing following 'IServer' interface members: get_rref, get_storage_server, start_connecting. src/allmydata/test/test_checker.py:78: error: 'FakeCheckResults' is missing following 'ICheckResults' interface members: get_encoding_expected, get_encoding_needed, get_happiness, get_host_counter_good_shares, get_incompatible_shares, get_report, get_servers_responding, get_share_counter_good, get_share_counter_wrong, get_sharemap, get_uri, get_version_counter_recoverable, get_version_counter_unrecoverable. src/allmydata/test/test_checker.py:109: error: 'FakeCheckAndRepairResults' is missing following 'ICheckAndRepairResults' interface members: get_storage_index_string.
comment:24 Changed at 2020-11-29T18:49:34Z by jaraco
9ab3c4023 works around the errors in the few places where the issue occurred. Interestingly, there are still some Referenceable subclasses that don't appear to be affected (e.g. control:59).
comment:25 Changed at 2020-11-29T21:37:38Z by jaraco
Good news. With aggressive suppression of errors and a few improvements, the package now passes all checks on mypy:
tahoe-lafs 3399.mypy $ git-id d2d3f1f4a tahoe-lafs 3399.mypy $ tox -e typechecks typechecks installed: attrs==20.3.0,Automat==20.2.0,cffi==1.14.3,constantly==15.1.0,cryptography==3.2.1,foolscap @ git+https://github.com/jaraco/foolscap@d70d8ecee926d3693b9fac55aebe13573d4f71c9,hyperlink==20.0.1,idna==2.10,incremental==17.5.0,mypy==0.790,mypy-extensions==0.4.3,mypy-zope @ git+https://github.com/jaraco/mypy-zope@48f912cc3cdb4ee37d4d3783ef466a7dce8bf0a9,pyasn1==0.4.8,pyasn1-modules==0.2.8,pycparser==2.20,PyHamcrest==2.0.2,pyOpenSSL==19.1.0,service-identity==18.1.0,six==1.15.0,Twisted==20.3.0,typed-ast==1.4.1,typing-extensions==3.7.4.3,zope.event==4.5.0,zope.interface==5.2.0,zope.schema==6.0.0 typechecks run-test-pre: PYTHONHASHSEED='974849251' typechecks run-test: commands[0] | mypy src Success: no issues found in 293 source files _____________________________________________________________ summary ______________________________________________________________ typechecks: commands succeeded congratulations :)
Next steps:
- (optional) create releases of the code in upstream branches
- validate the CI integration
- review and merge the PR
comment:26 Changed at 2020-11-29T21:50:49Z by jaraco
- Keywords review-needed added
comment:27 Changed at 2020-12-04T15:26:03Z by jaraco
I've replaced the pull request with a new one from a branch in the main repo, #915. Most of the tests are passing and the ones that are failing look to be unrelated to the change.
comment:28 Changed at 2020-12-15T18:16:43Z by exarkun
- Keywords review-needed removed
comment:29 Changed at 2021-01-09T17:48:08Z by jaraco
- Keywords review-needed added
I've addressed the comments in the ticket and upstream issues. Please review again.
comment:30 Changed at 2021-01-15T21:01:02Z by GitHub <noreply@…>
- Owner set to GitHub <noreply@…>
- Resolution set to fixed
- Status changed from new to closed
In cd06f1c/trunk:
Note: See
TracTickets for help on using
tickets.
I've started work in this branch. Just adding mypy checks reveals 200+ errors with the bulk in interfaces.py:
typechecks installed: mypy==0.782,mypy-extensions==0.4.3,typed-ast==1.4.1,typing-extensions==3.7.4.3 typechecks run-test-pre: PYTHONHASHSEED='8406647' typechecks run-test: commands[0] | mypy src src/allmydata/introducer/interfaces.py:45: error: Method must have at least one argument src/allmydata/introducer/interfaces.py:106: error: Method must have at least one argument src/allmydata/interfaces.py:59: error: Method must have at least one argument src/allmydata/interfaces.py:67: error: Method must have at least one argument src/allmydata/interfaces.py:110: error: Method must have at least one argument src/allmydata/interfaces.py:318: error: Method must have at least one argument src/allmydata/interfaces.py:436: error: Method must have at least one argument src/allmydata/interfaces.py:457: error: Method must have at least one argument src/allmydata/interfaces.py:468: error: Method must have at least one argument src/allmydata/interfaces.py:473: error: Method must have at least one argument src/allmydata/interfaces.py:484: error: Method must have at least one argument src/allmydata/interfaces.py:488: error: Method must have at least one argument src/allmydata/interfaces.py:492: error: Method must have at least one argument src/allmydata/interfaces.py:502: error: Method must have at least one argument src/allmydata/interfaces.py:508: error: Method must have at least one argument src/allmydata/interfaces.py:528: error: Method must have at least one argument src/allmydata/interfaces.py:545: error: Method must have at least one argument src/allmydata/interfaces.py:548: error: Method must have at least one argument src/allmydata/interfaces.py:551: error: Method must have at least one argument src/allmydata/interfaces.py:560: error: Method must have at least one argument src/allmydata/interfaces.py:570: error: Method must have at least one argument src/allmydata/interfaces.py:599: error: Method must have at least one argument src/allmydata/interfaces.py:627: error: Method must have at least one argument src/allmydata/interfaces.py:642: error: Method must have at least one argument src/allmydata/interfaces.py:657: error: Method must have at least one argument src/allmydata/interfaces.py:661: error: Method must have at least one argument src/allmydata/interfaces.py:666: error: Method must have at least one argument src/allmydata/interfaces.py:670: error: Method must have at least one argument src/allmydata/interfaces.py:678: error: Method must have at least one argument src/allmydata/interfaces.py:690: error: Method must have at least one argument src/allmydata/interfaces.py:701: error: Method must have at least one argument src/allmydata/interfaces.py:772: error: Method must have at least one argument src/allmydata/interfaces.py:782: error: Method must have at least one argument src/allmydata/interfaces.py:790: error: Method must have at least one argument src/allmydata/interfaces.py:793: error: Method must have at least one argument src/allmydata/interfaces.py:888: error: Method must have at least one argument src/allmydata/interfaces.py:922: error: Method must have at least one argument src/allmydata/interfaces.py:925: error: Method must have at least one argument src/allmydata/interfaces.py:930: error: Method must have at least one argument src/allmydata/interfaces.py:1001: error: Method must have at least one argument src/allmydata/interfaces.py:1007: error: Method must have at least one argument src/allmydata/interfaces.py:1011: error: Method must have at least one argument src/allmydata/interfaces.py:1019: error: Method must have at least one argument src/allmydata/interfaces.py:1027: error: Method must have at least one argument src/allmydata/interfaces.py:1037: error: Method must have at least one argument src/allmydata/interfaces.py:1042: error: Method must have at least one argument src/allmydata/interfaces.py:1051: error: Method must have at least one argument src/allmydata/interfaces.py:1056: error: Method must have at least one argument src/allmydata/interfaces.py:1062: error: Method must have at least one argument src/allmydata/interfaces.py:1071: error: Method must have at least one argument src/allmydata/interfaces.py:1074: error: Method must have at least one argument src/allmydata/interfaces.py:1080: error: Method must have at least one argument src/allmydata/interfaces.py:1084: error: Method must have at least one argument src/allmydata/interfaces.py:1092: error: Method must have at least one argument src/allmydata/interfaces.py:1101: error: Method must have at least one argument src/allmydata/interfaces.py:1127: error: Method must have at least one argument src/allmydata/interfaces.py:1201: error: Method must have at least one argument src/allmydata/interfaces.py:1294: error: Method must have at least one argument src/allmydata/interfaces.py:1304: error: Method must have at least one argument src/allmydata/interfaces.py:1349: error: Method must have at least one argument src/allmydata/interfaces.py:1358: error: Method must have at least one argument src/allmydata/interfaces.py:1367: error: Method must have at least one argument src/allmydata/interfaces.py:1517: error: Method must have at least one argument src/allmydata/interfaces.py:1542: error: Method must have at least one argument src/allmydata/interfaces.py:1619: error: Method must have at least one argument src/allmydata/interfaces.py:1622: error: Method must have at least one argument src/allmydata/interfaces.py:1630: error: Method must have at least one argument src/allmydata/interfaces.py:1779: error: Method must have at least one argument src/allmydata/interfaces.py:1892: error: Method must have at least one argument src/allmydata/interfaces.py:1930: error: Method must have at least one argument src/allmydata/interfaces.py:1955: error: Method must have at least one argument src/allmydata/interfaces.py:1968: error: Method must have at least one argument src/allmydata/interfaces.py:1992: error: Method must have at least one argument src/allmydata/interfaces.py:1995: error: Method must have at least one argument src/allmydata/interfaces.py:2011: error: Method must have at least one argument src/allmydata/interfaces.py:2022: error: Method must have at least one argument src/allmydata/interfaces.py:2044: error: Method must have at least one argument src/allmydata/interfaces.py:2050: error: Method must have at least one argument src/allmydata/interfaces.py:2074: error: Method must have at least one argument src/allmydata/interfaces.py:2118: error: Method must have at least one argument src/allmydata/interfaces.py:2134: error: Method must have at least one argument src/allmydata/interfaces.py:2147: error: Method must have at least one argument src/allmydata/interfaces.py:2161: error: Method must have at least one argument src/allmydata/interfaces.py:2164: error: Method must have at least one argument src/allmydata/interfaces.py:2167: error: Method must have at least one argument src/allmydata/interfaces.py:2172: error: Method must have at least one argument src/allmydata/interfaces.py:2175: error: Method must have at least one argument src/allmydata/interfaces.py:2178: error: Method must have at least one argument src/allmydata/interfaces.py:2186: error: Method must have at least one argument src/allmydata/interfaces.py:2189: error: Method must have at least one argument src/allmydata/interfaces.py:2205: error: Method must have at least one argument src/allmydata/interfaces.py:2208: error: Method must have at least one argument src/allmydata/interfaces.py:2345: error: Method must have at least one argument src/allmydata/interfaces.py:2348: error: Method must have at least one argument src/allmydata/interfaces.py:2351: error: Method must have at least one argument src/allmydata/interfaces.py:2354: error: Method must have at least one argument src/allmydata/interfaces.py:2359: error: Method must have at least one argument src/allmydata/interfaces.py:2366: error: Method must have at least one argument src/allmydata/interfaces.py:2369: error: Method must have at least one argument src/allmydata/interfaces.py:2372: error: Method must have at least one argument src/allmydata/interfaces.py:2375: error: Method must have at least one argument src/allmydata/interfaces.py:2379: error: Method must have at least one argument src/allmydata/interfaces.py:2390: error: Method must have at least one argument src/allmydata/interfaces.py:2395: error: Method must have at least one argument src/allmydata/interfaces.py:2400: error: Method must have at least one argument src/allmydata/interfaces.py:2408: error: Method must have at least one argument src/allmydata/interfaces.py:2416: error: Method must have at least one argument src/allmydata/interfaces.py:2420: error: Method must have at least one argument src/allmydata/interfaces.py:2424: error: Method must have at least one argument src/allmydata/interfaces.py:2431: error: Method must have at least one argument src/allmydata/interfaces.py:2434: error: Method must have at least one argument src/allmydata/interfaces.py:2445: error: Method must have at least one argument src/allmydata/interfaces.py:2448: error: Method must have at least one argument src/allmydata/interfaces.py:2451: error: Method must have at least one argument src/allmydata/interfaces.py:2457: error: Method must have at least one argument src/allmydata/interfaces.py:2462: error: Method must have at least one argument src/allmydata/interfaces.py:2466: error: Method must have at least one argument src/allmydata/interfaces.py:2478: error: Method must have at least one argument src/allmydata/interfaces.py:2482: error: Method must have at least one argument src/allmydata/interfaces.py:2495: error: Method must have at least one argument src/allmydata/interfaces.py:2499: error: Method must have at least one argument src/allmydata/interfaces.py:2509: error: Method must have at least one argument src/allmydata/interfaces.py:2520: error: Method must have at least one argument src/allmydata/interfaces.py:2524: error: Method must have at least one argument src/allmydata/interfaces.py:2560: error: Method must have at least one argument src/allmydata/interfaces.py:2564: error: Method must have at least one argument src/allmydata/interfaces.py:2569: error: Method must have at least one argument src/allmydata/interfaces.py:2575: error: Method must have at least one argument src/allmydata/interfaces.py:2610: error: Method must have at least one argument src/allmydata/interfaces.py:2706: error: Method must have at least one argument src/allmydata/interfaces.py:2711: error: Method must have at least one argument src/allmydata/interfaces.py:2714: error: Method must have at least one argument src/allmydata/interfaces.py:2718: error: Method must have at least one argument src/allmydata/interfaces.py:2723: error: Method must have at least one argument src/allmydata/interfaces.py:2726: error: Method must have at least one argument src/allmydata/interfaces.py:2732: error: Method must have at least one argument src/allmydata/interfaces.py:2736: error: Method must have at least one argument src/allmydata/interfaces.py:2741: error: Method must have at least one argument src/allmydata/interfaces.py:2745: error: Method must have at least one argument src/allmydata/interfaces.py:2748: error: Method must have at least one argument src/allmydata/interfaces.py:2752: error: Method must have at least one argument src/allmydata/interfaces.py:2764: error: Method must have at least one argument src/allmydata/interfaces.py:2767: error: Method must have at least one argument src/allmydata/interfaces.py:2772: error: Method must have at least one argument src/allmydata/interfaces.py:2779: error: Method must have at least one argument src/allmydata/interfaces.py:2783: error: Method must have at least one argument src/allmydata/interfaces.py:2788: error: Method must have at least one argument src/allmydata/interfaces.py:2793: error: Method must have at least one argument src/allmydata/interfaces.py:2796: error: Method must have at least one argument src/allmydata/interfaces.py:2800: error: Method must have at least one argument src/allmydata/interfaces.py:2805: error: Method must have at least one argument src/allmydata/interfaces.py:2808: error: Method must have at least one argument src/allmydata/interfaces.py:2845: error: Method must have at least one argument src/allmydata/interfaces.py:2863: error: Method must have at least one argument src/allmydata/interfaces.py:2880: error: Method must have at least one argument src/allmydata/interfaces.py:2883: error: Method must have at least one argument src/allmydata/interfaces.py:2889: error: Method must have at least one argument src/allmydata/interfaces.py:2896: error: Method must have at least one argument src/allmydata/interfaces.py:2909: error: Method must have at least one argument src/allmydata/interfaces.py:2939: error: Method must have at least one argument src/allmydata/interfaces.py:2966: error: Method must have at least one argument src/allmydata/interfaces.py:2977: error: Method must have at least one argument src/allmydata/util/pollmixin.py:26: error: Need type annotation for '_poll_should_ignore_these_errors' (hint: "_poll_should_ignore_these_errors: List[<type>] = ...") src/allmydata/test/test_python3.py:50: error: "Callable[[Python3PortingEffortTests], Any]" has no attribute "todo" src/allmydata/test/check_load.py:47: error: Need type annotation for 'last_stats' (hint: "last_stats: Dict[<type>, <type>] = ...") src/allmydata/test/check_load.py:53: error: Incompatible types in assignment (expression has type "int", variable has type "str") src/allmydata/test/check_load.py:56: error: Incompatible types in assignment (expression has type "float", target has type "int") src/allmydata/monitor.py:44: error: Method must have at least one argument src/allmydata/monitor.py:49: error: Method must have at least one argument src/allmydata/monitor.py:60: error: Method must have at least one argument src/allmydata/monitor.py:77: error: Method must have at least one argument src/allmydata/monitor.py:81: error: Method must have at least one argument src/allmydata/monitor.py:86: error: Method must have at least one argument src/allmydata/util/fileutil.py:312: error: Name '_getfullpathname' already defined on line 310 src/allmydata/storage/crawler.py:22: error: Name 'pickle' already defined (possibly by an import) src/allmydata/node.py:712: error: Need type annotation for 'GENERATED_FILES' (hint: "GENERATED_FILES: List[<type>] = ...") src/allmydata/windows/fixups.py:220: error: Name 'unichr' is not defined src/allmydata/test/test_iputil.py:136: error: "Callable[[ListAddresses], Any]" has no attribute "timeout" src/allmydata/web/status.py:1327: error: Name 'cmp' is not defined src/allmydata/web/status.py:1335: error: Name 'cmp' is not defined src/allmydata/test/test_stats.py:9: error: Incompatible types in assignment (expression has type "float", base class "CPUUsageMonitor" defined the type as "int") src/allmydata/uri.py:726: error: Incompatible types in assignment (expression has type "Type[CHKFileVerifierURI]", base class "DirectoryURIVerifier" defined the type as "Type[SSKVerifierURI]") src/allmydata/scripts/run_common.py:70: error: List item 0 has incompatible type "Tuple[str, str, None, str]"; expected "List[Optional[str]]" src/allmydata/scripts/debug.py:21: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/debug.py:419: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/debug.py:613: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/debug.py:661: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/debug.py:875: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/create_node.py:147: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/create_node.py:154: error: List item 0 has incompatible type "Tuple[str, str, None, str]"; expected "List[Optional[str]]" src/allmydata/scripts/create_node.py:155: error: List item 1 has incompatible type "Tuple[str, str, None, str]"; expected "List[Optional[str]]" src/allmydata/scripts/create_node.py:156: error: List item 2 has incompatible type "Tuple[str, str, str, str]"; expected "List[Optional[str]]" src/allmydata/scripts/create_node.py:158: error: List item 3 has incompatible type "Tuple[str, str, None, str]"; expected "List[Optional[str]]" src/allmydata/scripts/create_node.py:160: error: List item 4 has incompatible type "Tuple[str, None, int, str]"; expected "List[Optional[str]]" src/allmydata/scripts/create_node.py:161: error: List item 5 has incompatible type "Tuple[str, None, int, str]"; expected "List[Optional[str]]" src/allmydata/scripts/create_node.py:162: error: List item 6 has incompatible type "Tuple[str, None, int, str]"; expected "List[Optional[str]]" src/allmydata/scripts/create_node.py:163: error: List item 7 has incompatible type "Tuple[str, None, None, str]"; expected "List[Optional[str]]" src/allmydata/scripts/create_node.py:199: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/tahoe_start.py:20: error: List item 0 has incompatible type "Tuple[str, str, None, str]"; expected "List[Optional[str]]" src/allmydata/scripts/stats_gatherer.py:15: error: List item 0 has incompatible type "Tuple[str, None, None, str]"; expected "List[Optional[str]]" src/allmydata/scripts/stats_gatherer.py:16: error: List item 1 has incompatible type "Tuple[str, None, None, str]"; expected "List[Optional[str]]" src/allmydata/scripts/stats_gatherer.py:17: error: List item 2 has incompatible type "Tuple[str, None, None, str]"; expected "List[Optional[str]]" src/allmydata/scripts/stats_gatherer.py:27: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/frontends/ftpd.py:2: error: Module 'types' has no attribute 'NoneType' src/allmydata/web/directory.py:1217: error: Incompatible types in assignment (expression has type "str", base class "MultiFormatResource" defined the type as "None") src/allmydata/web/directory.py:1274: error: Incompatible types in assignment (expression has type "str", base class "MultiFormatResource" defined the type as "None") src/allmydata/scripts/cli.py:54: error: List item 0 has incompatible type "Tuple[str, None, None, str]"; expected "List[Optional[str]]" src/allmydata/scripts/cli.py:65: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:75: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:84: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:88: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:107: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:150: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:155: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:168: error: List item 0 has incompatible type "Tuple[str, None, None, str]"; expected "List[Optional[str]]" src/allmydata/scripts/cli.py:186: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:198: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:226: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:261: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:270: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:289: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:375: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:392: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:408: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:420: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:435: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/scripts/cli.py:452: error: Incompatible types in assignment (expression has type "str", base class "BaseOptions" defined the type as "None") src/allmydata/frontends/sftpd.py:3: error: Module 'types' has no attribute 'NoneType' src/allmydata/scripts/runner.py:65: error: Unsupported operand types for + ("List[List[Union[Type[BaseOptions], str, None]]]" and "List[Tuple[str, None, Type[InviteOptions], str]]") src/allmydata/scripts/runner.py:110: error: Module has no attribute "dispatch" src/allmydata/test/storage_plugin.py:41: error: Method must have at least one argument src/allmydata/test/test_storage_client.py:508: error: On Python 3 '{}'.format(b'abc') produces "b'abc'"; use !r if this is a desired behavior src/allmydata/test/test_helper.py:88: error: Need type annotation for 'introducer_clients' (hint: "introducer_clients: List[<type>] = ...") src/allmydata/test/check_memory.py:495: error: Module has no attribute "maxint" src/allmydata/test/check_memory.py:497: error: Module has no attribute "maxint" src/allmydata/test/check_memory.py:501: error: Module has no attribute "maxint" src/allmydata/test/test_sftp.py:17: error: Name 'conch_interfaces' already defined on line 12 src/allmydata/test/test_sftp.py:18: error: Name 'sftp' already defined on line 13 src/allmydata/test/test_sftp.py:19: error: Incompatible import of "sftpd" (imported name has type Module, local name has type "object") src/allmydata/test/test_sftp.py:23: error: Incompatible types in assignment (expression has type "None", variable has type "ImportError") src/allmydata/test/test_encode.py:176: error: Incompatible types in assignment (expression has type "float", variable has type "int") src/allmydata/test/test_deepcheck.py:919: error: Name 'unicode' is not defined Found 242 errors in 31 files (checked 291 source files) ERROR: InvocationError for command /Users/jaraco/code/public/tahoe-lafs/.tox/typechecks/bin/mypy src (exited with code 1) ___________________________________ summary ____________________________________ ERROR: typechecks: commands failed