I've been working with Jamf Support on this but we haven't been able to find a solution so far.
I have a MacOS 10.11.6 server running Server.app 5.2. JSS is 9.6
When trying to run a "install package" policy through Self Service (haven't tested automatic policy) it fails with the following debug error:
NSURLSession/NSURLConnection HTTP load failed (kCFStreamErrorDomainSSL, -9813)
Based on this error, I assumed it was a problem with the certificate. So I recreated it:
We have an internal certificate authority, then an intermediate, and then the certificate installed on the server. I created the certificate's key as a JKS, then created the CSR, requested the signed cert, downloaded root CAs, imported the Root CAs into the JKS, then imported the signed cert into the JKS.
Once everything was created in the JKS, I exported it as a P12 using openssl.
I then imported that certificate into Server.app, it imported the key, cert, and two additional certs (the intermediate and root)
I then configured the default https web site to use the certificate.
That didn't fix it.
A few random things I've tested (each inidividual)
Turn off proxy (all our computers use auto proxy) - failed.
Proxy the Mac through Charles SSL Proxy to try to diagnose - succeeded (because of self-signed cert being installed and trusted?)
The Root CA was originally installed in the login keychain on the client Mac. Moved the root CA to the system keychain - failed.
Tested the https server using nscurl to make sure I'm not running into App Transport Security issues. It passed all tests.
