Encrypt Communications using Self-signed Certificates in Go

There are a few reasons our customers ask to implement this:

1. Simplify deployment: To no longer require a reverse proxy/web server or a tool like Certbot to obtain certs, and no need to add DNS records.
2. Automate deployment: The client/agent program can easily generate itself a certificate. There are no caveats like when solving ACME or DNS challenges. This also means that you don't need to open port 80 and 443.
3. Privacy concerns: They don't want their infrastructure domains to show up on certificate transparency records (try it yourself! crt.sh). This can be prevented by using wildcard certificates but dealing with them is a hassle. Cool if you manage to get one but what happens when you renew it on one place and now need to distribute it to 20 different servers?

By no means this is a new way of doing things, however there are a lot of misconceptions/ambiguity in the existing articles on the internet (Implementing mTLS? Creating CAs? No, you don't need any of this). We would like to clarify the steps with our blog post.

In this blog post we will create:

• An agent/client program which will create and use self-signed certificates. The created certificate will be saved to disk to later be imported in the server program.
• A server program that will trust and use the agent's certificate when connecting to agent's API endpoints.

The Agent

When generating a self-signed certificate, you actually don't need to create a CA first. Simply sign the generated certificate with itself. For more information, see crypto/x509#CreateCertificate

The code for generating an SSL/TLS certificate and saving to file in PEM format:

package agent

import (
    "bytes"
    "crypto/rand"
    "crypto/rsa"
    "crypto/tls"
    "crypto/x509"
    "encoding/pem"
    "math/big"
    "net"
    "os"
    "time"
)

func generateSelfSignedCert(certLoc, keyLoc, host string) (tls.Certificate, error) {
    // Below certificate will be valid for 10 years from current time.
    // You can provide more information such as county or company name however
    // those fields are not mandatory.
    cert := &x509.Certificate{
        SerialNumber:          big.NewInt(0),
        NotBefore:             time.Now(),
        NotAfter:              time.Now().AddDate(10, 0, 0),
        ExtKeyUsage:           []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
        KeyUsage:              x509.KeyUsageDigitalSignature,
        BasicConstraintsValid: true,
    }

    // You must provide an IP address (IPAddresses) or a hostname (DNSNames).
    ip := net.ParseIP(host)
    if ip != nil {
        cert.IPAddresses = []net.IP{ip}
    } else {
        cert.DNSNames = []string{host}
    }

    // Create private and public keys.
    certPrivKey, err := rsa.GenerateKey(rand.Reader, 4096)
    if err != nil {
        return tls.Certificate{}, err
    }

    // Sign the certificate with itself.
    certBytes, err := x509.CreateCertificate(rand.Reader, cert, cert, &certPrivKey.PublicKey, certPrivKey)
    if err != nil {
        return tls.Certificate{}, err
    }

    // PEM encode and save to file
    certPEM := new(bytes.Buffer)
    pem.Encode(certPEM, &pem.Block{
        Type:  "CERTIFICATE",
        Bytes: certBytes,
    })
    certPrivKeyPEM := new(bytes.Buffer)
    pem.Encode(certPrivKeyPEM, &pem.Block{
        Type:  "RSA PRIVATE KEY",
        Bytes: x509.MarshalPKCS1PrivateKey(certPrivKey),
    })
    if err := os.WriteFile(certLoc, certPEM.Bytes(), 0600); err != nil {
        return tls.Certificate{}, err
    }
    if err := os.WriteFile(keyLoc, certPrivKeyPEM.Bytes(), 0600); err != nil {
        return tls.Certificate{}, err
    }

    // Return tls.Certificate from the cert we just created.
    // We will pass this to http.Server instead of re-reading the certificate
    // from file.
    serverCert, err := tls.X509KeyPair(certPEM.Bytes(), certPrivKeyPEM.Bytes())
    if err != nil {
        return tls.Certificate{}, err
    }
    return serverCert, err
}

And this is how you can use your newly generated certificate with Go's HTTP server:

    certLoc := filepath.Join(config.CertDir, "cert.pem")
    keyLoc := filepath.Join(config.CertDir, "key.pem")

    // Generate the certificate using our function.
    selfCert, err := generateSelfSignedCert(certLoc, keyLoc, config.Host)
    if err != nil {
        log.Fatal().Err(err).Msg("Failed to generate new self-signed certificate")
    }
    log.Info().Str("cert location", certLoc).
        Msg("Generated new self-signed certificate.")

    r := chi.NewRouter()
    r.Get("/hello", getHello)

    // Pass the certificate here.
    tlsConfig := &tls.Config{
        Certificates: []tls.Certificate{selfCert},
    }
    server := http.Server{
        Addr:      ":3000",
        Handler:   r,
        TLSConfig: tlsConfig,
    }

    // Start the server. We have already given it the certificate so we don't need
    // to provide paths to the certificate files here.
    server.ListenAndServeTLS("", "")

The Server

Firstly, we need to figure out is where to save the certificate of the agent so we can trust it when connecting. You can have an endpoint for enrolling new agents where you receive the certificate and save it to a database. That part is totally up to you.

After saving your certificates in PEM format (the same format the agent program used to save to file), you can import them into Go's HTTP client this way.

    rootCAs, _ := x509.SystemCertPool()

    // Read your certificate from somewhere. This time it is of string type
    // stored in a database row.
    if ok := rootCAs.AppendCertsFromPEM([]byte(cert)); !ok {
        return nil, fmt.Errorf("failed to parse self-signed certificate")
    }
    config := &tls.Config{
        RootCAs: rootCAs,
    }
    tr := &http.Transport{TLSClientConfig: config}
    client := &http.Client{Transport: tr}

And that's it! Keep in mind, this isn't mTLS or another form of authentication, an attacker can still choose to proceed without trusting the agent certificate. For that you will need to implement authentication on top of this.