try switching from coveralls.io to codecov.io

[daira]

The codecov.io reports apparently include branch coverage, which I consider essential, but ​coveralls does not support. The ​changes needed seem completely straightforward.

The only disadvantage would be that the historical coverage data wouldn't be copied across, but I think that's fine.

[zooko]

When doing this, please see if you can figure out how to elicit information about which lines had changes in their coverage between versions. Coveralls just reports the change in the number of lines that are covered, which is not really useful information.

[daira]

"This application will be able to read and write all public and private [github] repo data."

Scratch that then.

[exarkun]

​https://www.tahoe-lafs.org/trac/tahoe-lafs/ticket/3385 was a duplicate of this.

[exarkun]

coveralls.io no longer requests the "repo" scope (I believe that is the scope that corresponds to the description "This application will be able to read and write all public and private [github] repo data.".

Instead, it requests:

• Read org and team membership, read org projects
• Access commit status
• Access user email addresses (read-only)

This is substantially less invasive so it seems reasonable to consider this switch again.

My current interest stems from codecov behavior of reporting results early, before all jobs have run, which invariably makes the report look extremely bad. I've tried fixing this with "wait_for_ci: true" (​https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3568) but to no avail.

[exarkun]

Here's a setback. To support collecting coverage data from more than one job, you need to manually deliver a "finished" notification to coveralls.io. With some annoying manual work it's possible to do this correctly for _either_ CircleCI _or_ GitHub? Actions but delivering the finished hook only after _both_ CI providers are done is a lot harder.

I guess you'd need a job on one of them that could find the corresponding jobs on the other and wait? And only after that job (and all other local jobs) finish would you deliver the finished notification.

I don't feel like working on this any more today.

[exarkun]

For reference ​https://docs.coveralls.io/parallel-build-webhook

[exarkun]

Aha. I just realized I read the ticket summary backwards. sigh.

[exarkun]

coveralls.io has branch coverage reporting now. It's not ideal but all the information seems to be there, at least.

[exarkun]

Alright I've made enough of a mess on this ticket already, I'm going to close it again, sorry about the noise.