Implementing mTLS in Go

development golang microservices

Introduction

Mutual authentication with Transport Layer Security (mTLS) is a method for mutual authentication. It is often used in securing network communication between two services and ensures that the parties at each end of this communication are who they claim to be by verifying that they both have the correct private key. It has been a standard security and authentication mechanism in a service mesh like Istio, Linkerd, AWS App Mesh and others. It also encrypts communications between services just like TLS does in HTTPS requests. In TLS, only the client validates the servers certificate by using the certificate authority, but in mTLS, both client and server both verify the identity of each other.

In mTLS, both the client and server have a certificate, and both sides authenticate using their public/private key pair. Typically, the steps involved in mTLS is as follows:

Steps in mTLS authentication

In mTLS, the organization implementing it will act as it’s own certificate authority unlike in TLS where certificate authority is an external organization that checks if the certificate owner legitimately owns the associated domain. Therefore, the organization will have to create a root TLS certificate that will be used to sign the certificates of the services that communicate with each other. Now that we know how mTLS works, let’s try to test this out by implementing it in a simple Go server.

Certificate Creation and Validation

In order to use mTLS, we will require a root certificate that acts as the certificate authority. The organization that owns the server will use this root certificate’s public and private key to sign the client and the server certificates. The organization has to keep the root certificate’s private key safe and should NOT disclose it to anybody outside the organization.

The organization will generate the server certificate and then will need the certificate’s public and private key along with root certificate’s public key when starting the HTTP server.

The organization will also create the client’s public certificate and the private key. The organization will have to share the root certificate’s public key, client certificate’s pubic and private key with the client that will connect to the server. The client will need to keep the client certificate’s private key safe and NOT share it with anybody outside the team.

Let’s start by creating the root certificate’s public and private key using openssl.


$ mkdir certs

$ openssl req -newkey rsa:2048 \
  -nodes -x509 \
  -days 3650 \
  -out certs/ca.pem \
  -keyout certs/ca.key \
  -subj "/C=US/ST=California/L=San Francisco/O=ayada/OU=dev/CN=localhost"

Here the common name in the certificate is set to localhost since we will be testing this on our laptop’s. The ca.pem is the root certificate’s public key and ca.key is the root certificate’s private key. Both are stored under directory certs. The expiry of the root certificate is set to 10 years.

Now that we have our root certificate, let’s automate the creation of client and server certificates in Go. The function GenerateAndSignCertificate will be used to generate the client and server certificates. Not that the Subject and DNSNames fields in cert object needs to be updated as per your needs. You can also use a configuration file to determine the values for it. Not that the server will only allow requests from domains that match the domain names in DNSNames. Validity of the certificates generated here is set to 10 years as per the NotAfter field in cert object.


// GenerateAndSignCertificate method will use the root certificate's public and private key 
// to generate a certificate and sign it. The certificate's public and private keys will be 
// stored in the files provided as argument to this function.
func GenerateAndSignCertificate(root *KeyPair, publicKeyFile, privateKeyFile string) error {
	cert := &x509.Certificate{
		SerialNumber: big.NewInt(1658),
		Subject: pkix.Name{
			Organization: []string{"ayada"},
			Country:      []string{"US"},
			Province:     []string{"California"},
			Locality:     []string{"San Francisco"},
			CommonName:   "localhost",
		},
		DNSNames:     []string{"localhost", "ayada.dev", "*.ayada.dev"},
		IPAddresses:  []net.IP{net.IPv4(127, 0, 0, 1), net.IPv6loopback},
		NotBefore:    time.Now(),
		NotAfter:     time.Now().AddDate(10, 0, 0),
		SubjectKeyId: []byte{1, 2, 3, 4, 6},
		ExtKeyUsage:  []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth, x509.ExtKeyUsageServerAuth},
		KeyUsage:     x509.KeyUsageDigitalSignature,
	}

	certPrivKey, err := rsa.GenerateKey(rand.Reader, 4096)
	if err != nil {
		return err
	}

	// create the certificate and use the root certificate to sign it.
	certBytes, err := x509.CreateCertificate(rand.Reader, cert, root.PublicKey, &certPrivKey.PublicKey, root.PrivateKey)
	if err != nil {
		return err
	}

	certPEM := new(bytes.Buffer)
	pem.Encode(certPEM, &pem.Block{
		Type:  "CERTIFICATE",
		Bytes: certBytes,
	})

	certPrivKeyPEM := new(bytes.Buffer)
	pem.Encode(certPrivKeyPEM, &pem.Block{
		Type:  "RSA PRIVATE KEY",
		Bytes: x509.MarshalPKCS1PrivateKey(certPrivKey),
	})

	if err := ioutil.WriteFile(publicKeyFile, certPEM.Bytes(), 0644); err != nil {
		return fmt.Errorf("failed to write to public key file: %w", err)
	}

	if err := ioutil.WriteFile(privateKeyFile, certPrivKeyPEM.Bytes(), 0644); err != nil {
		return fmt.Errorf("failed to write to private key file: %w", err)
	}

	return nil
}

// Public and Private key pair for the certificate.
type KeyPair struct {
	PublicKey  *x509.Certificate
	PrivateKey *rsa.PrivateKey
}

We will also need couple of helper functions to read certificates from files stored on disk. We will need two functions; ReadCertificateAuthority to read and parse the root certificate that was generated using openssl cli and ReadCertificate to read and parse the client/server certificates that was generated using the above GenerateAndSignCertificate function.


// ReadCertificate reads and parses the certificates from files provided as 
// argument to this function.
func ReadCertificate(publicKeyFile, privateKeyFile string) (*KeyPair, error) {
	cert := new(KeyPair)

	privKey, errRead := ioutil.ReadFile(privateKeyFile)
	if errRead != nil {
		return nil, fmt.Errorf("failed to read private key: %w", errRead)
	}

	privPemBlock, _ := pem.Decode(privKey)

	// Note that we use PKCS1 to parse the private key here.
	parsedPrivKey, errParse := x509.ParsePKCS1PrivateKey(privPemBlock.Bytes)
	if errParse != nil {
		return nil, fmt.Errorf("failed to parse private key: %w", errParse)
	}

	cert.PrivateKey = parsedPrivKey

	pubKey, errRead := ioutil.ReadFile(publicKeyFile)
	if errRead != nil {
		return nil, fmt.Errorf("failed to read public key: %w", errRead)
	}

	publicPemBlock, _ := pem.Decode(pubKey)

	parsedPubKey, errParse := x509.ParseCertificate(publicPemBlock.Bytes)
	if errParse != nil {
		return nil, fmt.Errorf("failed to parse public key: %w", errParse)
	}

	cert.PublicKey = parsedPubKey

	return cert, nil
}

// ReadCertificateAuthority reads and parses the root certificate from files provided 
// as argument to this function.
func ReadCertificateAuthority(publicKeyFile, privateKeyFile string) (*KeyPair, error) {
	root := new(KeyPair)

	rootKey, errRead := ioutil.ReadFile(privateKeyFile)
	if errRead != nil {
		return nil, fmt.Errorf("failed to read private key: %w", errRead)
	}

	privPemBlock, _ := pem.Decode(rootKey)

	// Note that we use PKCS8 to parse the private key here.
	rootPrivKey, errParse := x509.ParsePKCS8PrivateKey(privPemBlock.Bytes)
	if errParse != nil {
		return nil, fmt.Errorf("failed to parse private key: %w", errParse)
	}

	root.PrivateKey = rootPrivKey.(*rsa.PrivateKey)

	rootCert, errRead := ioutil.ReadFile(publicKeyFile)
	if errRead != nil {
		return nil, fmt.Errorf("failed to read public key: %w", errRead)
	}

	publicPemBlock, _ := pem.Decode(rootCert)

	rootPubCrt, errParse := x509.ParseCertificate(publicPemBlock.Bytes)
	if errParse != nil {
		return nil, fmt.Errorf("failed to parse public key: %w", errParse)
	}

	root.PublicKey = rootPubCrt

	return root, nil
}

Let’s also write a helper function to verify the generated client and server certificates using the root certificate.


// Verification function will verify the client and server certificates using the 
// root certificate that was provided in the function arguments. If generateCertificate 
// is set to true, it will also generate new client and server certificates that is 
// signed by root certificate.
func Verification(serverName string, clientPubKey, clientPrivKey, serverPubKey, 
    serverPrivKey, rootPubKey, rootPrivKey string, generateCertificate bool) {
	ca, errCA := ReadCertificateAuthority(rootPubKey, rootPrivKey)
	if errCA != nil {
		log.Fatalf("failed to read ca certificate: %v", errCA)
	}

	if generateCertificate {
		// generate and sign client certificate using root certificate.
		if err := GenerateAndSignCertificate(ca, clientPubKey, clientPrivKey); err != nil {
			log.Fatalf("failed to generate client certificate: %v", err)
		}

		// generate and sign server certificate using root certificate.
		if err := GenerateAndSignCertificate(ca, serverPubKey, serverPrivKey); err != nil {
			log.Fatalf("failed to generate server certificate: %v", err)
		}
	}

	// read the client certificate from file and parse it.
	clientCert, errCert := ReadCertificate(clientPubKey, clientPrivKey)
	if errCert != nil {
		log.Fatalf("failed to read certificate: %v", errCert)
	}

	// read the server certificate from file and parse it.
	serverCert, errCert := ReadCertificate(serverPubKey, serverPrivKey)
	if errCert != nil {
		log.Fatalf("failed to read certificate: %v", errCert)
	}

	roots := x509.NewCertPool()
	roots.AddCert(ca.PublicKey)

	opts := x509.VerifyOptions{
		Roots:         roots,
		Intermediates: x509.NewCertPool(),
		DNSName:       serverName,
	}

	// verify client certificate; return err on failure.
	if _, err := clientCert.PublicKey.Verify(opts); err != nil {
		log.Fatalf("failed to verify client certificate: %v", err)
	}

	// verify server certificate; return err on failure.
	if _, err := serverCert.PublicKey.Verify(opts); err != nil {
		log.Fatalf("failed to verify server certificate: %v", err)
	}

	log.Print("client and server cert verification succeeded")
}

HTTP Client and Server

Now let’s implement a simple HTTP server and client to demonstrate the use mTLS. All we need is a httptest server that will receive HTTP request from a client. We will set the appropriate certificates in the HTTP client and server. Let’s start with the server. The server will need server certificate’s public and private key files. It will also need the root certificate’s public key file. We’ll pass them as argument to the HTTPServer function here.


// HTTPServer will start a mTLS enabled httptest server and return the test server.
// It requires server certificate's public and private key files and root certificate's 
// public key file as arguments.
func HTTPServer(serverPublicKey, serverPrivateKey, rootPublicKey string) *httptest.Server {
	// server certificate.
	serverCert, err := tls.LoadX509KeyPair(serverPublicKey, serverPrivateKey)
	if err != nil {
		return nil
	}

	// root certificate.
	rootCert, err := ioutil.ReadFile(rootPublicKey)
	if err != nil {
		log.Fatalf("failed to read root public key: %v", err)
	}
	rootCertPool := x509.NewCertPool()
	rootCertPool.AppendCertsFromPEM(rootCert)

	// httptest server with TLS config to enable mTLS.
	server := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintln(w, "success!")
	}))
	server.TLS = &tls.Config{
		Certificates: []tls.Certificate{serverCert},
		ClientCAs:    rootCertPool,
		ClientAuth:   tls.RequireAndVerifyClientCert,
	}
	server.StartTLS()
	return server
}

Now let’s implement the HTTP client that has mTLS enabled. The SendRequest function will accept the server URL, client certificate’s public and private key files, and the root certificate’s public key file as arguments. It will then send an mTLS enabled GET request to the server. If the authentication is successful, you will see success in the response body. If the certificate is absent in the request or if it is signed by a different certificate authority, the request will fail and the error you see will be http: TLS handshake error.


// SendRequest function will send a GET request to the server URL provided in the argument.
// It also requires client certificate's public and private key files and root certificate's 
// public key file.
func SendRequest(serverURL, clientPublicKey, clientPrivateKey, rootPublicKey string) {
	// client certificates.
	cert, err := tls.LoadX509KeyPair(clientPublicKey, clientPrivateKey)
	if err != nil {
		log.Fatalf("failed to load client certificate: %v", err)
	}

	// root certificate public key
	rootCert, errRead := ioutil.ReadFile(rootPublicKey)
	if errRead != nil {
		log.Fatalf("failed to read public key: %v", errRead)
	}
	publicPemBlock, _ := pem.Decode(rootCert)
	rootPubCrt, errParse := x509.ParseCertificate(publicPemBlock.Bytes)
	if errParse != nil {
		log.Fatalf("failed to parse public key: %v", errParse)
	}

	rootCertpool := x509.NewCertPool()
	rootCertpool.AddCert(rootPubCrt)

	// http client with root and client certificates.
	client := http.Client{
		Transport: &http.Transport{
			TLSClientConfig: &tls.Config{
				RootCAs:      rootCertpool,
				Certificates: []tls.Certificate{cert},
			},
		},
	}

	resp, err := client.Get(serverURL)
	if err != nil {
		log.Printf("failed to GET: %v", err)
		return
	}

	body, errRead := ioutil.ReadAll(resp.Body)
	if errRead != nil {
		log.Printf("failed to read body: %v", err)
		return
	}
	defer resp.Body.Close()

	log.Printf("successful GET: %s", string(body))
}

Let’s run this code from main.go.


const (
	clientPublicKey  = "certs/client.pem"
	clientPrivateKey = "certs/client.key"

	serverPublicKey  = "certs/server.pem"
	serverPrivateKey = "certs/server.key"

	rootPublicKey  = "certs/ca.pem"
	rootPrivateKey = "certs/ca.key"
)

func main() {
	certificate.Verification(
		"localhost", 
		clientPublicKey, clientPrivateKey, 
		serverPublicKey, serverPrivateKey, 
		rootPublicKey, rootPrivateKey, 
		false,
	)

	srv := server.HTTPServer(serverPublicKey, serverPrivateKey, rootPublicKey)
	defer srv.Close()

	server.SendRequest(srv.URL, clientPublicKey, clientPrivateKey, rootPublicKey)
}

When the certificates are valid, the output will look like this:


$ go run main.go
2021/12/09 15:32:29 client and server cert verification succeeded
2021/12/09 15:32:29 successful GET: success!

Let’s say the client certificates are not included in the HTTP request, the output will look like this:


$ go run main.go
2021/12/09 15:28:52 client and server cert verification succeeded
2021/12/09 15:28:53 failed to GET: Get "https://127.0.0.1:50536": x509: certificate signed by unknown authority
2021/12/09 15:28:53 http: TLS handshake error from 127.0.0.1:50537: remote error: tls: bad certificate

The full source code is available in the Github repository https://github.com/abvarun226/blog-source-code/tree/master/implementing-mtls-in-go

I hope this was helpful and if you find any errors or issues in this post, please do let me know in the comments section below.

In case you would like to get notified about more articles like this, please subscribe to my substack.



Related Posts

Create Queues in AWS SQS using Go
Background Tasks in Go
Search operation in Go
Delete Queues in AWS SQS using Go
Strings in Go

Get new posts by email

Comments

comments powered by Disqus