Zero Downtime nginx Letsencrypt Certificate Renewals Without the nginx Plugin

Written by Rob Peck · 4 minute read

Quick post on how to install Letsencrypt certificates into nginx without using the official plugin. There may be some cases where you don’t want to use the official plugin (which until recently was still marked as “experimental.”) The concepts here could be theoretically applied to any webserver software.

Basics of a ACME Challenge

Letsencrypt is based on a technology called ACME, which stands for Automated Certificate Management Environment. It’s a way for a certificate issuer to verify your ownership of a domain and issue you a certificate without requiring any manual intervention. And while there are a number of ways for this to happen, by far the most common is via a webserver.

The ACME client places a file in the /.well-known/acme-challenges/ directory. This will usually look something like /.well-known/acme-challenge/LYORRg3BLMyxa8_WYUa27QHofvO2M2GfvoPkLV5H-7I. The certificate issuer attempts to download this file. If it was successful, new certificates are generated, downloaded by the client, and installed in the correct location.

Obviously, this is a high-level overview, but the important thing to take away from this is that this is designd to be scripted. It is designed with zero interaction in mind.

The Restart Problem

Until the nginx plugin was stabilized, the common way to do this was to stop nginx, spin up a standalone server for the ACME challenge, then restart nginx. Obviously, this is not desirable in a production environment. Even a brief, less than 10 second outage it takes to do the ACME exchange is too long.

Fortunately, the letsencrypt client provides you another option --webroot.

Installing Certs without Restarts

Using a combination of some nginx config changes and the proper commands, we can do the challenge then tell nginx to just reload the configs, resulting in a zero downtime cert install.

First, you may need to make an nginx change. This is necessary if, for example, you are running nginx as a reverse proxy server and don’t have easy access to the remote end. Or if you just want to serve the requests from another location.

  location ^~ /.well-known {
    allow all;
    root /var/www/well-known/;
  }

Next, you just need to issue the right command to letsencrypt.

letsencrypt certonly --quiet -n --agree-tos --webroot -w /var/www/well-known/ --deploy-hook "systemctl reload nginx" -d example.com

Let’s take apart what we did here.

• certonly obtains a certificate, but does not install it. This is a bit of a misnomer. It downloads the cert, but does not install it into your config. You will have to handle that. Also useful for specific certs.
• -n Non interactive. We want to script this! :)
• --quiet again, we want to script this in cron, so we don’t want it making noise. You should probably remove this while you are testing.
• --agree-tos you agree to the terms of service.
• --webroot this is the magic sauce. You’re telling letsencrypt that you want to use the webroot auth, placing files in a location on your server.
• -w /var/www/well-known/ Tells webroot where to place the files. This is the same location as your config change above.
• --deploy-hook "systemctl reload nginx" on a successful certificate download it will run this command. Note that it will only run this if a new certificate is downloaded, making this safe to run daily!
• -d example.com is the domain you want the certificate for.

The systemctl reload nginx part there is important. Not only are we only running the command if we have new certs, we are telling nginx to reload, not restart. The server will continue to reply to existing requests, but new requests will be served with the new config, making this a zero downtime operation.

Finally, if you have not already done so, you will need to add the appropriate SSL configuration lines to your config:

  ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

Congratulations, you now have certs! And you can shove that command into a daily cronjob to be sure that you never have to deal with renewing an SSL certificate again.

• Letsencrypt
• Linux
• nginx