NGINX Proxy is deployed and used in many ways from bare metal install to a docker image running in a VM. Its primary purpose is to reverse proxy SSL web traffic and manage SSL certs for self hosted web apps behind of a firewall. Certainly there are many other uses, but for home labbers like me, this small microservice certainly does the job well.
Let’s Encrypt!
SSL Certificates are usually purchased for commercial websites. The cert duration can be from very short to almost infinite depended in the class of cert. Self hosting a web app does not require a purchased cert.
A limited free certificate is the choice. Many SSL Cert providers offer the free-of-charge cert, and I chose to use Let’s Encrypt. No need to look any further. Once installed, the cert expiry is 90 days from creation of a new cert. There after it is usually a simple matter of renewing this cert. Most people renwew by scripting, but depending on how your NGINX Proxy is deployed, updating a cert can fail. The system logs don’t tell the whole story of why the update failed in any detail. A quick look on GitHub will list out many issues posted about cert update failures. Since my proxy is behind the firewall, I need to do more manual work, and scripting is out of the question. I may change this over time. Renewing the cert manually is not a problem, takes little time and I am notified by email weeks in advance of expiration.
I will briefly describe my NGINX Proxy deployment. I had the same problems of refusal to update the certs on the first round renewal. After some research and review of my system, I came up with a solution, rather a process of updating the certs that meets my need. Your process will vary, but the settings I mention here could guide you to a fix for your certs not renewing properly.
NGINX Proxy Deployment
My deployment is probably typical for most home labs. NGINX Proxy running as a docker image in some type of VM inside the firewall of my network with a hard VM host IP address. My network is set up in such a manner that it is double-NAT in traffic flow. The ISP modem is NAT and pointing to my internal router. In turn, my router is in NAT mode to the internal network. There is nothing inherently wrong using a double-NAT. To keep the network flow unimpaired, port settings have to be tweaked on both devices. This discussion is not about the how-to of deployment, but of SSL Cert Renewal after deployment. So we will pick it up from the point of the first 90 day renewal.
DNS Proxy, Ports 443 and 80, SSL End-to-End Encryption
In my experience, there are four areas where parameters must be modified in order for an SSL Cert to renew on an NGINX Proxy after it has been in operation. If these parameters are not turned off, or proper ports opened, then the required traffic handshake cannot take place for the renewal. On the other hand, once the renewal is complete, locking down the following parameters add another layer of security and the NGINX Proxy functions as it was designed.
I use CloudFlare for my DNS needs and will begin here with the parameters that require modification.
- Proxied DNS
- SSL/TLS Full Encryption mode
Turn off the Proxied DNS. This allows the endpoint hard IP address of the home ISP to be exposed.
Set SSL/TLS from FULL to Flexible.
Modem/Router Changes
- Port 80 must be port forwarded on ISP and Router devices to allow unencrypted communications only during the renewal process.
NGINX Proxy Setting
- Turn OFF Force SSL and HSTS Enabled setting by editing the Proxy host.
Renew the SSL Certificate
Now that the proper ports are open, the DNS proxy is turned off, and the SSL/TSL settings are relaxed enough to promote the proper handshake traffic, its time to renew each host certificate in your network.
Then it is a simple matter of of logging into the NGINX Proxy Manager > SSL Certificates, and renewing each host by clicking the Renew Now button. Of course, this does not seem simple if the background information I described above was not implemented. The result would have been an error of : “Communication with the API failed, is NPM running correctly?” Obviously, there is no possible correlation between the error message and the traffic settings I outlined above. There is no NGINX documentation describing the renewal process. There are many people out there that run into this brick-wall error message and have no idea what to do next.
Summary
Nothing is ever easy in the open source world. Software changes happen, documentation often lags or is nonexistence, forum post can be misleading or are specific to a person’s configuration.
Through trial and error, skimming over forums, using common sense basic networking knowledge, I was able to find that the above settings worked for my network. The parameter changes opened up the required traffic flow enough to renew the host SSL Certs on the NGINX Proxy. As such, I have a proven, documented solution to updating host SSL Certs on the NGINX Proxy every 90 days. In less than 10 minutes, I am done with the process.
One last note. The settings above? Go back through the network and set them back to the original configuration for that added layer of security. I hope you found this article useful.
