How to not brick Apache2 when using Let's Encrypt

  •  

Second time right…

A while ago I published an article on how to set up a free certificate using Let's Encrypt. This setup however, had a major flaw: Apache had to be stopped before renewal could start. Therefore, your services would be down during renewal, which is bad. Even worse if the process failed, because then Apache would stay down. Bad situation, all websites down, good bye great uptime score for this month…

In this article, I explain how I found a better method that's less prone to fail. Which is nice to have, if the certificates have to be renewed once every few weeks.

TL;DR

# 1. Create global alias to .well-known in Apache
echo "Alias /.well-known/acme-challenge/ /var/www/letsencrypt/.well-known/acme-challenge/" >> /etc/apache2/sites-available/100-letsencrypt.conf
echo "" >> /etc/apache2/sites-available/100-letsencrypt.conf
echo "# Bypass Auth" >> /etc/apache2/sites-available/100-letsencrypt.conf
echo "<IfModule mod_access_compat.c>" >> /etc/apache2/sites-available/100-letsencrypt.conf
echo "  <Directory /var/www/letsencrypt/.well-known/acme-challenge/>" >> /etc/apache2/sites-available/100-letsencrypt.conf
echo "    Satisfy any" >> /etc/apache2/sites-available/100-letsencrypt.conf
echo "  </Directory>" >> /etc/apache2/sites-available/100-letsencrypt.conf
echo "</IfModule>" >> /etc/apache2/sites-available/100-letsencrypt.conf
echo "" >> /etc/apache2/sites-available/100-letsencrypt.conf
echo "# Redirect before other rewrite rules" >> /etc/apache2/sites-available/100-letsencrypt.conf
echo "RewriteCond %{REQUEST_URI} /\.well\-known/acme\-challenge/" >> /etc/apache2/sites-available/100-letsencrypt.conf
echo "RewriteRule (.*) /.well-known/acme-challenge/$1 [L,QSA]" >> /etc/apache2/sites-available/100-letsencrypt.conf

# 2. Enable the alias
mkdir -p /var/www/letsencrypt
chown www-data: /var/www/letsencrypt
a2ensite 100-letsencrypt.conf
apachectl configtest && apachectl graceful

# 3. Generate a new certificate
certbot-auto certonly --agree-tos -m admin@mail.addr --rsa-key-size 4096 --renew-hook "apachectl graceful" \
  --webroot --webroot-path /var/www/letsencrypt/ \
  --cert-name optional_name --domains domain.tld,other.tld

Installing Certbot

The short and probably best answer is: Use certbot's official instructions for installing Certbot. Only follow the steps until the Install header, the rest of the steps can be followed from this article. Ensure to back-up the /etc/letsencrypt folder before uninstalling Certbot, to avoid existing config and renew information to be lost.

Update existing configurations

Your may skip this section if there are no active Letsencrypt certificates on your server. Because the new method uses the webroot authenticator, the renewal config of existing certificates need to be changed.

The following tasks need to be performed for every file in the /etc/letsencrypt/renewal directory:

  1. Change authenticator = apache to authenticator = webroot;
  2. Change or add renew_hook = apachectl graceful, remove any pre_hook and post_hook telling Apache to stop/start;
  3. Add webroot_path = /var/www/letsencrypt,, followed by a [[webroot_map]] section.

A proper configuration should look like this (note the comma after the webroot_path line):

[...]

# Options used in the renewal progress
[renewalparams]
authenticator = webroot
account = xxxyyyzzz
renew_hook = apachectl graceful
rsa_key_size = 4096
server = xxxyyyzzz
webroot_path = /var/www/letsencrypt,
[[webroot_map]]
www.domain.tld = /var/www/letsencrypt

In case you're lazy, you can force renewal with the proper options, to let the new configuration get generated automatically, see "Forcefully renew existing certificates" below. But you have to set up the Apache alias first.

Setting up the .well-known Apache alias

During the new renewal process, Certbot will put an authorisation file in the webroot path. Letsencrypt's servers will try to access the authorisation file remotely. For example, if you try to create a certificate for testdomain.tld, Letsencrypt tries to access http://tesdomain.tld/.well-known/acme-challenge/some-random-file. A global alias to .well-known will allow all domains to properly respond to these requests.

First of all, create /etc/apache2/sites-available/100-letsencrypt.conf with the following contents:

Alias /.well-known/acme-challenge/ /var/www/letsencrypt/.well-known/acme-challenge/

# Bypass Auth
<IfModule mod_access_compat.c>
  <Directory /var/www/letsencrypt/.well-known/acme-challenge/>
    Satisfy any
  </Directory>
</IfModule>

# Redirect before other rewrite rules
RewriteCond %{REQUEST_URI} /\.well\-known/acme\-challenge/
RewriteRule (.*) /.well-known/acme-challenge/$1 [L,QSA]

To enable the alias, the final directory has to be created and the configuration has to be loaded into Apache:

mkdir -p /var/www/letsencrypt
chown www-data: /var/www/letsencrypt
a2ensite 100-letsencrypt.conf
apachectl configtest && apachectl graceful

If the configuration works, the message "Syntax OK" will show, Apache will be reloaded and the alias will be active.

In all other cases, an error message shows up. Read the error and fix it before you continue.

Creating and renewing certificates

1. Creating new certificates

certbot-auto certonly --agree-tos -m admin@mail.addr --rsa-key-size 4096 --renew-hook "apachectl graceful" \
  --webroot --webroot-path /var/www/letsencrypt/ \
  --cert-name optional_name --domains domain.tld,other.tld

2. Forcefully renew existing certificates (only if you really have to)

Only use the command below if you have to, try to avoid any unnecessary stress on the Letsencrypt servers. Use certbot-auto certificates to get a list of available certificates and their names.

# Optionally list certificates
certbot-auto certificates

# Force renewal based on certificate name
certbot-auto certonly certificate.name --webroot --webroot-path /var/www/letsencrypt--force-renewal

Known issues

Redirect not working Check webserver config for missing '/' in redirect target

This usually occurs if there are other redirects active on the target domain, for example a 301 from http to https. Check redirects that might be active, and ensure your 301 from http to https is properly set up. This usually works for me:

RewriteEngine On
RewriteRule ^(.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301]

Sources

  1. Setting up a Global alias for webroot:
    https://serverfault.com/a/744252
  2. How to renew a certificate with webroot:
    https://community.letsencrypt.org/t/how-to-renew-with-webroot/74750/4