darcs failures due to new SSL certificate

[warner]

Many buildslaves and many users are reporting failures during "darcs get", due to some sort of SSL cert problem. Most are seeing errors like:

darcs failed:  Not a repository: https://tahoe-lafs.org/source/tahoe-lafs/trunk
 (Failed to download URL https://tahoe-lafs.org/source/tahoe-lafs/trunk/_darcs/inventory:
 Peer certificate cannot be authenticated with known CA certificates)

The _darcs/inventory part is a red herring (that file is supposed to be missing), but the CA problem seems accurate. From what we can tell so far, some distributions or packages of darcs (or libcurl) do not include enough (or any) CA roots.

[warner]

marcusw's CA list

[marcusw]

That's the default cygwin list. It proves that the curl in darcs for windows (pre-built) does not use the cygwin certificates because it includes the root cert in the chain which the tahoe-lafs.org cert is signed by (the intermediate link being RapidSSL). From this, it can be concluded that either the curl in the windows darcs uses an internal cert list or it's using a windows system certificate list, and either way, neither the GeoTrust? nor RapidSSL cert is not included in that list.

The workaround in the first case would be to ​build darcs from scratch, and in the latter case would be to simply add it to the windows list. Note that the windows list I'm referring to may not actually exist, and I haven't checked whether it does yet.

[zooko]

This problem occurs on ​NetBSD and ​Sickness's OpenBSD. They both say:

darcs failed:  Not a repository: https://tahoe-lafs.org/source/tahoe-lafs/trunk (Failed to download URL https://tahoe-lafs.org/source/tahoe-lafs/trunk/_darcs/inventory: Peer certificate cannot be authenticated with known CA certificates)

A problem with different symptoms occurs on ​Cygwin, ​CentOS, ​Eugen's Lenny.

They all say:

darcs failed:  Not a repository: https://tahoe-lafs.org/source/tahoe-lafs/trunk (Failed to download URL https://tahoe-lafs.org/source/tahoe-lafs/trunk/_darcs/inventory: HTTP 301 error getting https://tahoe-lafs.org/source/tahoe-lafs/trunk/_darcs/inventory)

The following buildslaves are able to darcs get over https: ​Zandr's Ubuntu, ​Brian's Ubuntu, ​Fedora, ​Arthur's Lenny, ​FreeBSD, ​Kyle's OpenBSD.

[zooko]

Here are the versions of darcs being used. Failing with "Peer certificate cannot be authenticated":

• MM's NetBSD: darcs 2.2.0 -- darcs-exact-version: darcs compiled on Jul 21 2010, at 20:15:30 # configured Thu Jan 15 14:31:24 PST 2009 ./configure /usr/local/share/config.site /usr/local/etc/config.site Context: TAG 2.2.0 Petr Rockai <me@…>20090115150916]
• Sickness's OpenBSD: darcs 2.5.2 -- darcs compiled on Oct 31 2011, at 08:15:20 Context: TAG 2.5.2 Ganesh Sittampalam <ganesh@…>20110313223504 Ignore-this: f3f57f3eacb2fdd4cdafc581c05058e3 ] Compiled with: array-0.3.0.2 base-4.3.1.0 bytestring-0.9.1.10 containers-0.4.0.0 directory-1.1.0.0 extensible-exceptions-0.1.1.2 filepath-1.2.0.0 hashed-storage-0.5.4 haskeline-0.6.4.0 html-1.0.1.2 mmap-0.5.6 mtl-2.0.1.0 old-time-1.0.0.6 parsec-3.1.1 process-1.0.1.5 random-1.0.0.3 regex-compat-0.93.1 tar-0.3.1.0 terminfo-0.3.1.3 text-0.11.0.6 unix-2.4.2.0 zlib-0.5.3.1

Failing with "HTTP 301 error":

• Cygwin: darcs 2.5.2 -- darcs compiled on Nov 22 2011, at 00:35:33 Context: TAG 2.5.2 Ganesh Sittampalam <ganesh@…>20110313223504 Ignore-this: f3f57f3eacb2fdd4cdafc581c05058e3 Compiled with: HTTP-4000.1.1 array-0.3.0.2 base-4.3.1.0 bytestring-0.9.1.10 containers-0.4.0.0 directory-1.1.0.0 extensible-exceptions-0.1.1.2 filepath-1.2.0.0 hashed-storage-0.5.9 haskeline-0.6.4.5 html-1.0.1.2 mtl-2.0.1.0 network-2.3.0.2 old-time-1.0.0.6 parsec-3.1.1 process-1.0.1.5 random-1.0.0.3 regex-compat-0.93.1 regex-posix-0.94.4 tar-0.3.1.0 text-0.11.0.6 unix-compat-0.3 zlib-0.5.3.1
• CentOS: darcs 2.4.3 -- darcs-exact-version: darcs compiled on May 9 2010, at 18:45:56 Context: TAG 2.4.3 Eric Kow <kowey@…>20100509132315 Ignore-this: 1a920525d0cd01c352d5a2893bbea10d ]
• Eugen's Lenny: darcs 2.5.0 -- darcs-exact-version: darcs compiled on Nov 2 2010, at 15:01:47 Context: TAG 2.5 Reinier Lamers <tux_rocker@…>20101024151805 Ignore-this: 1561ce30bfb1950a440c03371e0e2f20 ] Compiled with: array-0.3.0.1 base-4.2.0.2 bytestring-0.9.1.7 containers-0.3.0.0 directory-1.0.1.1 extensible-exceptions-0.1.1.1 filepath-1.1.0.4 hashed-storage-0.5.3 haskeline-0.6.3.1 html-1.0.1.2 mmap-0.5.7 mtl-1.1.0.2 old-time-1.0.0.5 parsec-2.1.0.1 process-1.0.1.3 random-1.0.0.2 regex-compat-0.93.1 tar-0.3.1.0 terminfo-0.3.1.3 text-0.10.0.0 unix-2.4.0.2 zlib-0.5.2.0

Succeeding:

• Zandr's Ubuntu: darcs 2.3.0 -- darcs-exact-version: darcs compiled on Feb 8 2010, at 13:19:04 Context: TAG 2.3.0 Petr Rockai <me@…>20090723115125 Ignore-this: e326d4ddff92c578e8fe8a3c23d00193 ]
• Brian's Ubuntu: darcs 2.4.4 -- darcs-exact-version: darcs compiled on Nov 12 2010, at 11:11:03 Context: TAG 2.4.4 Eric Kow <kowey@…>20100515090819 Ignore-this: 7d1a0e6a17c2be314f6ab1607bbcac13 ]
• Fedora: darcs 2.5.2 -- darcs-exact-version: darcs compiled on Oct 25 2011, at 08:21:14 Context: TAG 2.5.2 Ganesh Sittampalam <ganesh@…>20110313223504 Ignore-this: f3f57f3eacb2fdd4cdafc581c05058e3 ] Compiled with: array-0.3.0.2 base-4.3.1.0 bytestring-0.9.1.10 containers-0.4.0.0 directory-1.1.0.0 extensible-exceptions-0.1.1.2 filepath-1.2.0.0 hashed-storage-0.5.9 haskeline-0.6.4.3 html-1.0.1.2 mmap-0.5.7 mtl-2.0.1.0 old-time-1.0.0.6 parsec-3.1.1 process-1.0.1.5 random-1.0.0.3 regex-compat-0.93.1 tar-0.3.1.0 terminfo-0.3.2.2 text-0.11.0.6 unix-2.4.2.0 zlib-0.5.3.1
• Arthur's Lenny: darcs 2.0.2 -- darcs-exact-version: darcs compiled on Sep 30 2008, at 08:02:55 # configured Mon Jun 23 18:19:52 PDT 2008 ./configure /usr/local/share/config.site /usr/local/etc/config.site Context: TAG 2.0.2 David Roundy <droundy@…>20080624012041]
• Randy's FreeBSD: darcs 2.5.1 -- darcs-exact-version: darcs compiled on Jun 17 2011, at 23:02:16 Context: TAG 2.5.1 Ganesh Sittampalam <ganesh@…>20110210233529 Ignore-this: 9e4daeb4ceaf1c2743235c68564c519b ] Compiled with: array-0.3.0.2 base-4.3.1.0 bytestring-0.9.1.10 containers-0.4.0.0 directory-1.1.0.0 extensible-exceptions-0.1.1.2 filepath-1.2.0.0 hashed-storage-0.5.5 haskeline-0.6.3.2 html-1.0.1.2 mtl-2.0.1.0 old-time-1.0.0.6 parsec-3.1.1 process-1.0.1.5 random-1.0.0.3 regex-compat-0.93.1 tar-0.3.1.0 text-0.11.0.6 unix-2.4.2.0 zlib-0.5.3.1
• Kyle's OpenBSD: darcs 2.4.4 -- darcs-exact-version: darcs compiled on Aug 11 2010, at 03:17:21 Context: TAG 2.4.4 Eric Kow <kowey@…>20100515090819 Ignore-this: 7d1a0e6a17c2be314f6ab1607bbcac13 ]

[zooko]

	works	fails CA	fails 301
Lenny	2.0.2		2.5.0
Ubuntu	2.3.0, 2.4.4
Fedora	2.5.2
FreeBSD	2.5.1
OpenBSD	2.4.4	2.5.2
CentOS			2.4.3
NetBSD		2.2.0
Cygwin			2.5.3

[dcoder]

Temporary workaround (works for Windows, at least):

• Get ca-bundle.crt attached up there, and rename it curl-ca-bundle.crt
• Get an SSL-capable curl binary from ​http://curl.haxx.se/download.html
• Put both curl.exe and curl-ca-bundle.crt somewhere in %PATH%
• Set the environment variable DARCS_GET_HTTPS=curl.exe

Now darcs should use curl.exe, and checkout should work, albeit significantly slower.

[marcusw]

I'd like to point out that the "Peer certificate cannot be authenticated with known CA certificates" error is caused by a lack of CA certs in a libcurl-enabled darcs, and the "HTTP 301 error" is caused by a non-curl darcs rewriting the https URL to http and entering a redirect loop.

Dcoder's workaround works for me.

[zooko]

Heffalump from the #darcs channel on IRC says that the Haskell "HTTP" library doesn't do SSL, and silently tries http when you asked it to do https, so that probably explains all of the 301 errors. The only solution is not to use a version of darcs that uses the HTTP library and instead use a version of darcs that uses libcurl. The CA errors are, if I understand correctly, that some builds of darcs have a libcurl builtin which doesn't have a full CA cert bundle compiled in, nor does it load a CA cert bundle from the filesystem, or if it does then the one that it loads doesn't have the right root CA in. The work-around for that is as described by dcoder in comment:5.

[zooko]

Here is the issue ticket about the Haskell HTTP library silently sending requests in the clear when asked to send requests over https: ​https://github.com/haskell/HTTP/issues/16

[davidsarah]

This is no longer relevant to the buildslaves, which are not using darcs. Leaving it open until we (and LAE) have completely switched to git. OTOH, git seems to be having some similar failures re: the github certificate -- do we need a new ticket for that?

[zooko]

We worked-around the problems with git/https by configuring it to use git: instead of https:. Let's close this ticket as fixed.