Fix Certbot AuthorizationError and failed ACME challenges
Diagnose Certbot AuthorizationError and failed ACME challenges without retrying blindly. Separate public DNS, IPv6, HTTP-01 routing, redirects, CAA, DNSSEC, DNS-01 propagation, and multi-origin failures before issuing or renewing a certificate.
Restore reliable ACME authorization for an Apache endpoint by preserving the first failed challenge evidence, proving every public validation path, repairing only the responsible DNS or HTTP boundary, and rehearsing renewal without weakening domain-control checks.
- Ubuntu Server 24.04 LTS
- Apache HTTP Server 2.4.x
- Certbot current stable snap
- Console and recovery access Keep an independent console session and a copy of the current Apache configuration before changing listeners, modules, or access rules.
- DNS and network plan Resolve every production hostname to the intended public address and list the exact management and client networks that require access.
getent ahosts {{domain}} && ip route show - Port ownership Confirm that TCP port 443 is free or already belongs to the intended Apache instance.
sudo ss -lntp 'sport = :443' - Rollback artifact Snapshot the host or create an archive of the active configuration and web roots before the first material change.
- Authoritative DNS inventory Record the authoritative nameservers, every public A and AAAA answer, the effective CAA policy, and any CDN, load balancer, NAT, or split-DNS boundary before changing validation.
dig +short NS {{domain}} && dig +short A {{domain}} && dig +short AAAA {{domain}} && dig +short CAA {{domain}}
OneLiners never runs these steps or stores secrets. Review placeholders, versions, current state, and change-control requirements before using a command.
Full guide
What you will build
- A repeatable incident path for Certbot AuthorizationError and failed ACME challenges that preserves the first CA problem detail, tests each validation boundary, and avoids production retry loops.
- A publicly trusted TLS endpoint for an existing Apache name-based virtual host, issued only after authoritative DNS, IPv4 and IPv6 HTTP-01 routing, redirect behavior, CAA, time synchronization, and configuration syntax are proven.
- A renewable Certbot lineage whose exact names, paths, expiry, timer, Apache edits, challenge method, and staging dry-run result are recorded as operational evidence.
- A recoverable HTTPS deployment with an archived pre-change state, explicit stop conditions, and a clear distinction between Apache rollback, certificate deletion, and revocation.
- Operators can map AuthorizationError, Some challenges have failed, unauthorized, connection, dns, and caa details to the responsible DNS, transport, virtual-host, redirect, or policy boundary.
- Every authoritative nameserver and every advertised A or AAAA address returns one consistent validation result before another order is attempted.
- Requests to the selected HTTP hostname redirect to HTTPS after issuance, the HTTPS endpoint presents a valid chain for every requested name, and Apache still parses and maps virtual hosts correctly.
- Certbot's packaged unattended renewal mechanism exists and a staging dry run succeeds without replacing the production certificate.
Architecture
How the parts fit together
Public DNS and CAA select the names and certificate authorities allowed to validate them. For HTTP-01, every public A and AAAA route must carry TCP/80 to the same exact token response below /.well-known/acme-challenge/, through any CDN, load balancer, NAT, redirect, and Apache virtual-host boundary. For DNS-01, every authoritative server must expose the current TXT proof at the effective delegated owner. Certbot preserves order and authorization evidence in its log, installs an issued lineage below /etc/letsencrypt, and later rehearses the saved renewal path against staging.
- Certbot creates an order and one authorization per requested identifier; the CA returns a challenge and later a typed problem detail when validation fails.
- For HTTP-01, the CA resolves public DNS, prefers IPv6 when available, connects to port 80, follows only supported redirects, and retrieves the exact token from one or more validation locations.
- For DNS-01, the CA follows authoritative delegation to _acme-challenge, waits for the expected TXT proof, and applies the effective CAA and DNSSEC result.
- A successful authorization allows certificate issuance; Certbot stores the lineage and the Apache installer changes only the reviewed virtual host.
- The system timer later invokes certbot renew; staging dry runs and expiry monitoring reveal drift before production renewal becomes urgent.
Assumptions
- The named Apache HTTP virtual host already serves `/srv/www/{{domain}}/public`, is not an accidental default tenant, and passes `apache2ctl configtest` and `apache2ctl -S`.
- Every requested A and AAAA record resolves to this service. If an AAAA record exists, IPv6 clients and the CA must reach the same valid challenge endpoint.
- Inbound TCP/80 and TCP/443 are allowed through the provider edge, NAT, load balancer, and host firewall. Outbound DNS and HTTPS to the ACME service are permitted.
- The operations email is monitored, system time is synchronized, the hostname is eligible for a public certificate, and issuance rate limits have been reviewed.
- This guide uses the current stable classic Certbot snap and its Apache integration. Mixing apt, snap, pip, and legacy certbot-auto installations is outside scope.
- Any CDN, proxy, WAF, load balancer, multi-origin pool, or DNS-01 delegation in front of the service is documented and can be inspected without bypassing organizational ownership.
- The operator can test from an independent external network and can query every authoritative nameserver directly; a local or recursive resolver result alone is insufficient.
Key concepts
- ACME
- The automated protocol through which a client requests domain authorization, certificate issuance, renewal, and revocation from a certificate authority.
- HTTP-01
- A domain-control challenge retrieved over public TCP/80 at a token-specific path below `/.well-known/acme-challenge/`; it does not validate arbitrary HTTPS behavior.
- Lineage
- Certbot's managed history and renewal configuration for one certificate name, including stable live symlinks and numbered archived material.
- SNI
- The TLS client indication that tells a shared Apache listener which hostname and certificate to select before the HTTP request exists.
- Full chain
- The leaf certificate plus intermediate certificates Apache presents so clients can construct a path to a trusted root.
- Renewal dry run
- A staging-CA rehearsal of the saved renewal configuration; it validates automation without replacing the live production certificate.
- Revocation
- A CA-side statement that a certificate should no longer be trusted, normally used for compromise or deliberate invalidation, not as routine rollback.
- AuthorizationError
- Certbot's final exception when one or more ACME authorizations failed. The actionable cause is normally the earlier Domain, Type, and Detail problem record, not the exception name itself.
- ACME problem type
- A machine-readable failure class such as unauthorized, connection, dns, or caa that identifies which validation boundary rejected the order.
- Authoritative answer
- The DNS data served by the domain's authoritative nameservers. A recursive or split-DNS cache can look correct while the public authority still disagrees.
- CAA
- DNS policy that restricts which certificate authorities or validation methods may issue for a name. Let's Encrypt uses the identifier letsencrypt.org.
- DNS-01
- A challenge proven with a TXT record below _acme-challenge; it supports wildcards but requires safe recurring DNS automation and propagation evidence.
- Secondary validation
- Additional CA validation from independent network perspectives. Inconsistent geo-routing, anycast, CDN nodes, or multiple origins can pass from one location and fail from another.
Fill these once. Every matching command and configuration block updates immediately; values stay in this page only.
Security and production boundaries
- The private key under `/etc/letsencrypt/live` must remain root-controlled and readable only by the service path that needs it. Never print, copy into a ticket, or include it in a web root or general configuration export.
- A certificate proves control of names at issuance time; it does not make the application secure. Patch Apache and the application, use appropriate headers and cookies, restrict admin paths, and monitor authentication and authorization separately.
- Do not request names that are absent from the reviewed virtual-host map. A successful installer can still attach policy to the wrong site when ServerName and ServerAlias are ambiguous.
- Keep port 80 intentionally configured for redirect and HTTP-01 if that authenticator remains in use. Blocking it after issuance can create a renewal incident months later.
- Treat every installer edit as a privileged configuration change. Review the diff, parse before reload, test every name, and preserve a rollback artifact.
- Use DNS-01 instead when wildcard names, an unreachable port 80, or central DNS automation make it the correct design; scope DNS credentials to the smallest possible zone operation.
- Do not weaken CAA, DNSSEC, web authentication, WAF, or tenant isolation merely to make an order pass. Repair the exact validation path and preserve the intended security boundary.
- A retry creates a fresh authorization token. Do not keep retrying production while public answers disagree; use a deterministic preflight object and the staging CA first.
- For DNS-01, use narrowly scoped credentials or a delegated validation zone. Never store unrestricted DNS credentials in shell history, the webroot, a guide variable, or a public incident record.
Stop before continuing if
- Stop after Certbot AuthorizationError or Some challenges have failed until the first Domain, Type, Detail, challenge method, address, and validation URL have been preserved.
- Stop when any authoritative nameserver, public A or AAAA address, CDN node, load-balancer target, or origin returns a different challenge result.
- Stop before changing CAA, DNSSEC, delegation, or DNS API scope if ownership and intended policy are not documented.
- Stop before contacting the CA if any requested name resolves elsewhere, an unexpected AAAA record exists, public port 80 fails, or the deterministic challenge object reaches the wrong tenant.
- Do not install another Certbot distribution over an existing apt, snap, pip, or legacy installation until ownership, renewal jobs, account data, and rollback are understood.
- Do not accept an Apache installer diff that changes unrelated virtual hosts, weakens access policy, exposes a private key, or references an unexpected lineage.
- Stop after any configtest, SNI, chain, hostname, redirect, timer, or dry-run failure; keep the known-good HTTP site available while diagnosing.
- Do not revoke or delete a lineage merely to undo Apache edits. Revocation is deliberate trust invalidation, while rollback restores service configuration.
verification
Prove public prerequisites
Before contacting the CA, prove public A and AAAA answers, authoritative nameservers, the effective CAA response, port 80 ownership, the selected Apache host, synchronized time, and a deterministic challenge object. Run the HTTP request from outside the server network as well as locally.
Why this step matters
ACME production attempts are rate-limited, so DNS, time, public routing, vhost selection, and the exact HTTP-01 namespace must be proven with a harmless object before creating an order.
What to understand
Create the token-like preflight object under the established document root with non-secret content and mode inherited from the public path.
Resolve every A and AAAA record and compare them with the intended ingress. Test from an external network because a server-side curl may succeed through split DNS or NAT reflection.
Use the access log to prove which virtual host handled the object; a 200 from the wrong tenant is a security failure, not a pass.
System changes
- Creates one public, non-secret preflight file below the ACME challenge directory; all other commands are read-only.
Syntax explained
install -d -m 0755- Creates a traversable public challenge directory without granting write access to unprivileged users.
getent ahosts- Uses the host resolver path to display IPv4 and IPv6 answers.
timedatectl show -p NTPSynchronized- Reports whether systemd considers time synchronized.
curl --fail- Returns failure for HTTP error statuses so a missing token cannot look successful.
Values stay on this page and are never sent or saved.
sudo install -d -m 0755 /srv/www/{{domain}}/public/.well-known/acme-challenge && printf 'oneliners-acme-preflight\n' | sudo tee /srv/www/{{domain}}/public/.well-known/acme-challenge/oneliners-preflight >/dev/null && dig +short NS {{domain}} && dig +short A {{domain}} && dig +short AAAA {{domain}} && dig +short CAA {{domain}} && timedatectl show -p NTPSynchronized && sudo ss -lntp 'sport = :80' && sudo apache2ctl configtest && sudo apache2ctl -S && curl --fail http://{{domain}}/.well-known/acme-challenge/oneliners-preflightns1.example.net.
203.0.113.10
2001:db8:100::10
0 issue "letsencrypt.org"
NTPSynchronized=yes
LISTEN 0 511 0.0.0.0:80 users:(("apache2",pid=1842,fd=4))
Syntax OK
oneliners-acme-preflightCheckpoint: Checkpoint: preflight
Continue whenEvery requested address reaches the intended Apache vhost, the exact object returns `oneliners-acme-preflight`, time is synchronized, and syntax is valid.
Stop whenAny address, port, hostname, challenge path, clock, configtest, access-log vhost, or external response differs from the reviewed design.
If this step fails
The preflight URL returns the application page or a 404.
Likely causeA rewrite, proxy, document root, alias, or vhost selection intercepts the ACME path.
sudo apache2ctl -Scurl -v http://app.example.com/.well-known/acme-challenge/oneliners-preflightsudo tail -n 50 /var/log/apache2/app.example.com-access.log
ResolutionCorrect the intended vhost and exempt only the precise challenge path where necessary; repeat the external preflight before issuance.
Security notes
- Never place credentials or account data in the challenge directory; everything below it should be treated as publicly retrievable.
Alternatives
- Use DNS-01 with an official narrowly scoped plugin when public HTTP-01 cannot be made reliable.
Stop conditions
- Any address, port, hostname, challenge path, clock, configtest, access-log vhost, or external response differs from the reviewed design.
verification
Capture the first failed ACME authorization
Preserve the complete Certbot result before editing DNS, Apache, firewall, or renewal state. The first Domain, Type, Detail, challenge method, resolved address, and URL identify the failed boundary; the final AuthorizationError is only the summary. Inventory current lineages and read the recent log without exposing account keys or certificate private keys.
Why this step matters
A failed order can contain one useful causal detail and several wrapper messages. Preserving the first failed authorization prevents a destructive retry loop and lets the operator choose the right DNS, routing, virtual-host, or policy check.
What to understand
Record the exact domain, challenge type, ACME problem type, resolved address, URL, HTTP status, and response excerpt before the next attempt creates a different token.
Treat AuthorizationError and Some challenges have failed as summaries. Work from the preceding Domain, Type, and Detail fields.
Do not publish the complete log: it can contain account identifiers, paths, hostnames, and operational metadata even though Certbot does not print the private key.
System changes
- None. The commands inventory existing lineages and read the local Certbot log.
Syntax explained
certbot certificates- Lists managed certificate lineages, names, expiry, and stable paths without printing private-key contents.
grep -E -A8 -B4- Keeps nearby context around each recognized authorization failure so the domain and CA detail remain attached.
tail -n 160- Bounds displayed evidence to the most recent matching diagnostic window.
sudo certbot certificates && sudo grep -E -A8 -B4 'AuthorizationError|Failed authorization procedure|Some challenges have failed|Certbot failed to authenticate|urn:ietf:params:acme:error' /var/log/letsencrypt/letsencrypt.log | tail -n 160Certbot failed to authenticate some domains (authenticator: apache). Domain: app.example.com Type: unauthorized Detail: 203.0.113.10 returned 404 for the HTTP-01 token certbot.errors.AuthorizationError: Some challenges have failed.
Checkpoint: Checkpoint: failed boundary identified
Continue whenThe evidence identifies one requested name, challenge method, ACME problem type, address or DNS owner, and concrete failure detail.
Stop whenThe log is missing, truncated before the causal detail, belongs to another lineage, or a concurrent Certbot process is still changing authorization state.
If this step fails
Only AuthorizationError: Some challenges have failed is visible.
Likely causeThe terminal excerpt omitted the earlier CA problem document or a wrapper suppressed Certbot output.
sudo tail -n 240 /var/log/letsencrypt/letsencrypt.logsudo journalctl --since=-30min | grep -i certbot
ResolutionRecover the preceding Domain, Type, and Detail from the log before changing configuration or starting another order.
Security notes
- Share only the minimum redacted failure lines. Never attach /etc/letsencrypt/accounts, private keys, DNS API credentials, or an unrestricted complete log to a public ticket.
Alternatives
- Use certbot -v for one staging retry only after the current evidence has been preserved and the retry will not consume production issuance capacity.
Stop conditions
- A concurrent Certbot process, unknown lineage owner, missing causal detail, or evidence that the requested name is outside the approved change.
verification
Prove HTTP-01 from every public address
HTTP-01 always begins on public TCP port 80. Compare the token path over IPv4 and IPv6, inspect redirects without following them first, then follow the complete chain. A healthy local curl is not evidence when public DNS reaches a CDN, load balancer, second origin, stale AAAA address, or another Apache virtual host.
Why this step matters
Let us Encrypt can validate from multiple network locations and prefers IPv6 for the initial connection when AAAA exists. Every public origin must return the same token; one correct path cannot compensate for one wrong address.
What to understand
HTTP-01 validation fetches an exact token below /.well-known/acme-challenge/ over port 80. It does not validate an arbitrary port selected in Certbot.
Redirects are supported only to HTTP or HTTPS on ports 80 or 443 and only for a bounded chain. Prove each hop and make sure the challenge path never crosses into an unrelated tenant, login flow, storage bucket, or dead origin.
Test from an independent network. Split DNS, hairpin NAT, a provider edge, geo-routing, or a warm CDN cache can make a server-side request disagree with the CA.
System changes
- None. The commands query public DNS and retrieve the non-secret preflight object.
Syntax explained
curl -4- Forces the IPv4 path so its origin and response can be evaluated independently.
curl -6- Forces the IPv6 path used preferentially by Let us Encrypt when AAAA is published.
-D- -o-- Prints response headers and body so the selected vhost, redirect, status, and token can be compared.
-L --max-redirs 10- Follows the redirect chain with the same documented depth bound used for HTTP-01 diagnosis.
Values stay on this page and are never sent or saved.
dig +short A {{domain}}; dig +short AAAA {{domain}}; curl -4 -sS -D- -o- http://{{domain}}/.well-known/acme-challenge/oneliners-preflight; curl -6 -sS -D- -o- http://{{domain}}/.well-known/acme-challenge/oneliners-preflight; curl -sS -L --max-redirs 10 -D- -o- http://{{domain}}/.well-known/acme-challenge/oneliners-preflight203.0.113.10 2001:db8:100::10 HTTP/1.1 200 OK oneliners-acme-preflight HTTP/1.1 200 OK oneliners-acme-preflight
Checkpoint: Checkpoint: one public challenge response
Continue whenEvery A and AAAA route returns HTTP 200 and the exact preflight body, or follows only the reviewed HTTP/HTTPS redirect chain to that body.
Stop whenAny address times out, refuses the connection, selects another tenant, returns a different body, loops, crosses an unapproved domain, or uses an unsupported redirect port.
If this step fails
The ACME detail reports Invalid response and HTTP 404.
Likely causeThe CA reached a server, but the selected vhost, webroot, rewrite, reverse proxy, CDN, or secondary origin did not expose the token.
sudo apache2ctl -Scurl -4 -v http://app.example.com/.well-known/acme-challenge/oneliners-preflightcurl -6 -v http://app.example.com/.well-known/acme-challenge/oneliners-preflight
ResolutionCorrect the approved vhost or webroot and exempt only the exact challenge path. Synchronize the token to every intended origin, then repeat the external preflight.
The ACME detail reports connection refused or timeout.
Likely causePort 80 is closed or routed incorrectly at the provider edge, load balancer, NAT, host firewall, listener, or one advertised address.
sudo ss -lntp 'sport = :80'nc -vz app.example.com 80dig +short A app.example.com; dig +short AAAA app.example.com
ResolutionRestore the intended public port-80 route or choose an automated DNS-01 design. Do not redirect an arbitrary external port because HTTP-01 does not validate there.
Security notes
- Expose only the ACME challenge namespace and normal redirect behavior. Do not bypass authentication or proxy controls for broader paths to make validation pass.
Alternatives
- Use a scoped automated DNS-01 plugin when public port 80 cannot be provided reliably, or when wildcard issuance is required.
Stop conditions
- An unknown proxy or origin answers the name, public IPv6 is not owned, or the proposed repair broadens unauthenticated access beyond the challenge namespace.
verification
Prove authoritative DNS, DNS-01, and CAA policy
Ask each authoritative nameserver directly instead of trusting a recursive cache. For HTTP-01, confirm every A and AAAA address is intended. For DNS-01, confirm the current token at the effective _acme-challenge owner after CNAME or NS delegation and remove only stale tokens whose owning automation is known. Inspect CAA and DNSSEC failures before changing issuance policy.
Why this step matters
The CA follows the authoritative DNS graph, not the operator workstation cache. Inconsistent nameservers, a delegated challenge owner, stale TXT records, invalid DNSSEC, or a restrictive CAA record can make a locally plausible configuration fail globally.
What to understand
Query every authoritative nameserver with recursion disabled. Their A, AAAA, TXT, and CAA answers must agree for the current change and must not alternate between old and new infrastructure.
Follow CNAME or NS delegation for _acme-challenge to the effective owner. A TXT record published at the visible name is irrelevant when delegation moves the authoritative answer elsewhere.
Multiple TXT records are allowed during overlapping validations, but abandoned records increase ambiguity and response size. Remove a stale token only after identifying the automation, account, and active order that owns it.
CAA is inherited from the closest applicable owner and follows CNAMEs. For Let us Encrypt, the issuing identifier is letsencrypt.org; issuewild can apply differently to wildcard issuance.
SERVFAIL can indicate broken DNSSEC, unavailable authoritative servers, or incorrect handling of the CAA query type. Fix DNS integrity rather than deleting security policy blindly.
System changes
- None. The commands query DNS records and validation metadata without modifying the zone.
Syntax explained
dig +short NS- Finds the authoritative nameserver set that must agree before validation.
dig +norecurse @server- Queries one authoritative server directly and avoids hiding inconsistency behind a recursive cache.
_acme-challenge TXT- Reads DNS-01 proof data at the validation owner; follow any CNAME or NS delegation.
CAA- Shows whether the effective policy permits the selected CA and validation method.
+dnssec- Requests DNSSEC records and exposes validation-related response flags for deeper investigation.
Values stay on this page and are never sent or saved.
for ns in $(dig +short NS {{domain}}); do echo "== $ns =="; dig +norecurse @$ns {{domain}} A; dig +norecurse @$ns {{domain}} AAAA; dig +norecurse @$ns _acme-challenge.{{domain}} TXT; dig +norecurse @$ns {{domain}} CAA; done && dig +dnssec {{domain}} CAA== ns1.example.net. == app.example.com. 300 IN A 203.0.113.10 app.example.com. 300 IN AAAA 2001:db8:100::10 _acme-challenge.app.example.com. 60 IN TXT "expected-token-value" example.com. 300 IN CAA 0 issue "letsencrypt.org"
Checkpoint: Checkpoint: authoritative DNS agrees
Continue whenEvery authoritative server returns the intended addresses, current challenge TXT when DNS-01 is used, a CAA policy compatible with the request, and no unexplained SERVFAIL.
Stop whenNameservers disagree, DNSSEC is broken, delegation ownership is unknown, stale TXT data may belong to an active order, or CAA changes are not authorized.
If this step fails
The ACME detail reports DNS problem: NXDOMAIN or no valid A records.
Likely causeThe public name or delegated challenge owner does not exist at the authoritative DNS boundary, even if split DNS resolves internally.
dig +trace app.example.comdig +short NS example.comdig @ns1.example.net app.example.com A +norecurse
ResolutionPublish the approved public record at the effective authoritative zone, wait for authoritative agreement and relevant TTLs, then start a new staging authorization.
The ACME detail reports CAA forbids issuance or CAA SERVFAIL.
Likely causeThe closest CAA set does not authorize letsencrypt.org, a CNAME target has a conflicting policy, DNSSEC is invalid, or an authoritative server mishandles or drops CAA queries.
dig +trace app.example.com CAAdig +dnssec app.example.com CAAfor ns in $(dig +short NS example.com); do dig +norecurse @$ns app.example.com CAA; done
ResolutionRepair authoritative availability or DNSSEC first. Change CAA only with domain-owner approval and preserve the intended CA restriction.
Security notes
- Do not place unrestricted DNS API credentials on the web server. Use the narrowest record and zone scope, keep credentials outside served paths, and prefer delegated validation zones where supported.
Alternatives
- Delegate _acme-challenge by CNAME or NS to a validation-specific zone with tightly scoped automation when the primary DNS provider cannot support safe recurring updates.
Stop conditions
- Unknown DNS owner, inconsistent authoritative answers, broken DNSSEC, unapproved CAA relaxation, or credentials broader than the reviewed validation zone.
command
Archive Apache and existing certificate state
Preserve site definitions and any existing lineage metadata before allowing the installer plugin to edit configuration.
Why this step matters
The Apache installer can create or modify site files and the first issue creates account and lineage state, so a root-only pre-change archive is essential for service rollback and forensic comparison.
What to understand
Archive `/etc/apache2` and `/etc/letsencrypt` when present without treating a missing first-install certificate tree as an error.
Verify the archive table of contents, checksum, mode, owner, free space, and restore command before allowing an automated installer to edit configuration.
The archive contains private keys when a lineage already exists. Keep it root-only, encrypted according to backup policy, and outside every served or synchronized public path.
System changes
- Creates `/root/apache-before-certbot.tgz`, potentially containing sensitive existing TLS material, and a checksum record.
Syntax explained
tar -C /- Stores canonical relative paths suitable for a controlled restore.
2>/dev/null ||- Falls back to an Apache-only archive only when a first-install `/etc/letsencrypt` path is absent.
sha256sum- Records archive integrity evidence; it does not encrypt the contents.
sudo tar -C / -czf /root/apache-before-certbot.tgz etc/apache2 etc/letsencrypt 2>/dev/null || sudo tar -C / -czf /root/apache-before-certbot.tgz etc/apache2 && sudo sha256sum /root/apache-before-certbot.tgza5d8... /root/apache-before-certbot.tgz
Checkpoint: Checkpoint: backup
Continue whenA non-empty root-owned archive lists the intended configuration paths and its SHA-256 digest verifies.
Stop whenExisting configuration cannot be archived, disk space is insufficient, the archive is readable by unintended users, or a snapshot restore has not been rehearsed.
If this step fails
tar reports files changing or permission errors.
Likely causeAnother certificate job is active, storage is full, or files are being rotated during capture.
ps -ef | grep -E '[c]ertbot|[a]pache2'df -h /rootsudo tar -tzf /root/apache-before-certbot.tgz | head
ResolutionWait for the owning maintenance job, provide protected capacity, create a new archive, and verify it before continuing.
Security notes
- Treat this archive like a private-key backup and limit its retention, access, transfer, and incident exposure.
Alternatives
- Use a provider or filesystem snapshot only when it captures both Apache and Certbot state consistently and has a verified restore path.
Stop conditions
- Existing configuration cannot be archived, disk space is insufficient, the archive is readable by unintended users, or a snapshot restore has not been rehearsed.
command
Install Certbot's Apache integration
Install the current stable classic snap recommended by the Certbot project, expose its command at a predictable path, and record the exact version used to create the lineage.
Why this step matters
One project-supported installation method avoids competing binaries, renewal jobs, plugins, and account directories that make future renewal behavior ambiguous.
What to understand
Inventory `command -v certbot`, package ownership, snap state, and timers first. Remove or migrate an older method under its own change plan rather than overlaying it.
The classic snap is intentionally not strictly sandboxed because the Apache plugin must read and edit system configuration. That privilege makes provenance and path verification important.
Create the `/usr/local/bin` symlink only when absent and verify its resolved target; never overwrite an unknown administrative wrapper.
System changes
- Installs the current Certbot snap, its refresh lifecycle, and optionally one verified command symlink.
Syntax explained
snap install --classic- Installs the project-recommended snap with host-level access required by installer plugins.
test ! -e- Prevents replacement of an existing command path.
readlink -f- Verifies that the public command resolves to the managed snap target.
certbot --version- Records the actual release rather than relying on tutorial prose.
sudo snap install --classic certbot && if test ! -e /usr/local/bin/certbot; then sudo ln -s /snap/bin/certbot /usr/local/bin/certbot; fi && test "$(readlink /usr/local/bin/certbot)" = /snap/bin/certbot && certbot --versioncertbot 5.x (exact stable release may differ)
Checkpoint: Checkpoint: install
Continue whenExactly one intended Certbot binary runs from the stable snap, its exact version is recorded, and no competing renewal timer remains.
Stop whenAnother installation method, unexpected symlink, unapproved snap source, failed refresh, or duplicate timer exists.
If this step fails
The symlink command fails because `/usr/local/bin/certbot` exists.
Likely causeAn earlier package, wrapper, or manual installation already owns the path.
command -v certbotreadlink -f /usr/local/bin/certbotdpkg -S /usr/bin/certbot 2>/dev/null || truesnap list certbot
ResolutionIdentify and deliberately migrate the existing installation; do not force-overwrite the path.
Security notes
- Classic confinement grants broad host access. Install only the official signed snap and restrict who can invoke privileged issuance.
Alternatives
- Use a distribution package only when organizational policy owns its patch and renewal lifecycle; do not combine methods.
Stop conditions
- Another installation method, unexpected symlink, unapproved snap source, failed refresh, or duplicate timer exists.
warning
Issue and install the certificate
Request only names whose public validation paths have just been proven. Use the staging endpoint while diagnosing or changing challenge behavior; contact production only after one complete staging authorization succeeds. Supply a monitored operations address, accept the terms explicitly, and review every Apache installer change.
Why this step matters
The issuance command creates a public CA order, stores a private key, and lets a privileged installer modify Apache, so names, email, terms, redirect, and diff must be explicit.
What to understand
Request only DNS names already proven in the virtual-host map. `--domain` is repeated intentionally because every SAN must be authorized.
Use a monitored address and review the terms outside the command. `--redirect` changes HTTP behavior and must match application and ACME policy.
Immediately inspect the generated file diff, lineage, and Apache parser result. Success text alone does not prove the correct vhost or externally served certificate.
System changes
- Creates/updates an ACME account, private key and certificate lineage, Apache TLS configuration, redirect policy, and a reload.
Syntax explained
--apache- Uses Apache for challenge authentication and certificate installation.
--domain- Adds one requested certificate DNS name; repeat for each deliberate SAN.
--email- Registers the monitored ACME account contact.
--agree-tos- Records explicit acceptance of the CA subscriber agreement.
--no-eff-email- Declines unrelated email sharing.
--redirect- Configures HTTP-to-HTTPS redirection through the installer.
Values stay on this page and are never sent or saved.
sudo certbot --apache --domain {{domain}} --domain www.{{domain}} --email {{opsEmail}} --agree-tos --no-eff-email --redirectSuccessfully received certificate. Successfully deployed certificate for app.example.com. Congratulations! You have successfully enabled HTTPS.
Checkpoint: Checkpoint: issue
Continue whenAuthorization and issuance succeed for exactly the reviewed names, one expected lineage is created, and only intended Apache files change.
Stop whenCertbot proposes other names, selects another vhost, reports partial authorization, modifies unrelated sites, or Apache cannot parse the result.
If this step fails
Certbot cannot choose a virtual host for the requested name.
Likely causeServerName/ServerAlias is missing, duplicated, disabled, or defined only in an unexpected TLS site.
sudo apache2ctl -Ssudo grep -R -nE 'ServerName|ServerAlias' /etc/apache2/sites-enabled
ResolutionFix and parse the explicit name-based virtual-host map first, then retry the same reviewed order.
Security notes
- The command runs a root-capable installer and creates a private key. Protect terminal logs and do not add secret flags or DNS tokens to shell history.
Alternatives
- Use `certbot certonly --apache` when configuration installation must be performed manually after a separate review.
Stop conditions
- Certbot proposes other names, selects another vhost, reports partial authorization, modifies unrelated sites, or Apache cannot parse the result.
verification
Inspect the generated lineage and Apache files
Confirm certificate names, expiry, key paths, enabled SSL module, and the resulting virtual-host map without printing private key material.
Why this step matters
A lineage inventory and Apache configuration map connect Certbot's success message to the actual certificate files and enabled TLS host without exposing key contents.
What to understand
Record certificate name, SANs, expiry, authenticator, installer, and live paths. Confirm the SSL module and all address/name mappings.
Inspect only directive paths and public certificate metadata. Never cat or hash the private key into a broadly accessible ticket.
Compare changed Apache files with the archive and ensure no global default or other tenant acquired the certificate or redirect.
System changes
- No persistent change; reads public metadata and configuration paths.
Syntax explained
certbot certificates- Displays Certbot-managed lineages, names, expiry, and paths without printing private key bytes.
apache2ctl -M- Lists loaded modules and proves mod_ssl is active.
apache2ctl -S- Shows the complete HTTP/TLS virtual-host selection map.
grep SSLCertificate- Locates Apache certificate path references for review.
sudo certbot certificates && sudo apache2ctl -M | grep ssl && sudo apache2ctl -S && sudo grep -R 'SSLCertificate' /etc/apache2/sites-enabledCertificate Name: example.com Expiry Date: 2026-10-23 ssl_module (shared) SSLCertificateFile /etc/letsencrypt/live/example.com/fullchain.pem
Checkpoint: Checkpoint: inspect
Continue whenExactly one intended lineage covers every requested name, Apache references stable live paths, and the TLS vhost map has no unexpected default or overlap.
Stop whenNames, expiry, paths, file permissions, modules, vhost ordering, or changed-file scope is unexpected.
If this step fails
The lineage exists but no enabled Apache file references it.
Likely causeAuthentication succeeded but installation failed, the wrong installer was used, or a later rollback removed the generated site.
sudo certbot certificatessudo grep -R SSLCertificate /etc/apache2sudo apache2ctl -S
ResolutionKeep HTTP service stable, repair the explicit TLS vhost or rerun the installer only after reviewing the intended site.
Security notes
- Operational evidence should include public certificate paths and metadata, never private-key contents.
Alternatives
- Use `openssl x509` on `cert.pem` for machine-readable public metadata when building monitoring.
Stop conditions
- Names, expiry, paths, file permissions, modules, vhost ordering, or changed-file scope is unexpected.
verification
Validate redirect, hostname, and certificate chain
Check the HTTP redirect, HTTPS status, negotiated protocol, certificate subject, issuer, and expiry from the served endpoint.
Why this step matters
A successful local reload does not prove public DNS, redirect behavior, SNI selection, full-chain delivery, hostname coverage, or the response from the real ingress.
What to understand
Test HTTP and HTTPS separately and inspect Location rather than following it blindly; unexpected cross-domain redirects can expose users to another tenant.
Supply `-servername` to OpenSSL because shared listeners select certificates through SNI. Inspect subject alternative names in addition to the common name.
Repeat from an external IPv4 and IPv6 network and at any upstream TLS termination point. A local loopback test can bypass the actual edge.
System changes
- No persistent change; creates client connections and reads public certificate metadata.
Syntax explained
curl --head- Fetches response headers without downloading the body.
curl --fail- Treats HTTP 4xx/5xx as command failure.
openssl s_client -servername- Performs a TLS handshake with the explicit SNI hostname.
openssl x509 -noout- Prints selected public certificate metadata instead of encoded certificate content.
Values stay on this page and are never sent or saved.
curl --head http://{{domain}}/ && curl --fail --head https://{{domain}}/ && echo | openssl s_client -connect {{domain}}:443 -servername {{domain}} 2>/dev/null | openssl x509 -noout -subject -issuer -datesHTTP/1.1 301 Moved Permanently Location: https://example.com/ HTTP/2 200 subject=CN=example.com notAfter=...
Checkpoint: Checkpoint: protocol test
Continue whenHTTP redirects only to the same HTTPS hostname, HTTPS succeeds, and the served certificate has the expected issuer, validity, and SANs.
Stop whenAny address times out, redirect target differs, chain verification fails, certificate name/expiry is wrong, or the application response regresses.
If this step fails
OpenSSL shows the Apache default certificate rather than the new lineage.
Likely causeThe client omitted SNI, the TLS vhost map is ambiguous, or DNS reaches a different termination point.
sudo apache2ctl -Secho | openssl s_client -connect app.example.com:443 -servername app.example.com 2>/dev/null | openssl x509 -noout -subject -ext subjectAltName
ResolutionCorrect SNI/vhost mapping at the real TLS endpoint and retest every requested name externally.
Security notes
- Do not weaken certificate verification to make a probe green; investigate the hostname, trust path, and serving endpoint.
Alternatives
- Use an approved external synthetic monitor to exercise the same DNS and edge path continuously.
Stop conditions
- Any address times out, redirect target differs, chain verification fails, certificate name/expiry is wrong, or the application response regresses.
verification
Test unattended renewal
Run Certbot's dry-run against the staging CA and verify the packaged timer; a successful initial issue is incomplete until renewal is proven.
Why this step matters
Public HTTPS is temporary unless saved renewal settings, challenge reachability, plugin availability, timer execution, and deployment hooks work unattended.
What to understand
A dry run uses the staging CA and exercises saved lineage configuration. It can still create network traffic and temporary challenge state, so run it during a reviewed window.
Inspect both snap timer/service units and the latest logs. Record the next timer activation while remembering that Certbot may randomize execution.
Dry-run success is a checkpoint, not perpetual proof; DNS, firewalls, plugins, and vhost paths can drift before the next real renewal.
System changes
- Contacts the staging ACME service and may create temporary challenge/configuration state that Certbot removes; it should not replace the production lineage.
Syntax explained
renew- Processes managed lineages according to saved renewal configuration.
--dry-run- Exercises eligible renewals against staging without deploying a production certificate.
systemctl list-timers --all- Shows scheduled and previous timer execution state.
sudo certbot renew --dry-run && systemctl list-timers --all | grep -E 'certbot|snap.certbot.renew'Congratulations, all simulated renewals succeeded. certbot.timer loaded active waiting
Checkpoint: Checkpoint: renewal
Continue whenAll simulated renewals succeed and exactly one intended Certbot timer/service lifecycle is present with inspectable logs.
Stop whenAny authorization, plugin, parser, deployment, timer, or cleanup error occurs, or the dry run changes the production certificate.
If this step fails
Dry run fails with connection or authorization errors months after initial issuance.
Likely causeDNS, IPv6, firewall, challenge routing, plugin location, or renewal configuration drifted.
sudo certbot renew --dry-run -vsystemctl status snap.certbot.renew.timersudo journalctl -u snap.certbot.renew.service -n 100 --no-pager
ResolutionRepair the saved method and external path now, repeat staging rehearsal, and verify production expiry remains unchanged.
Security notes
- Do not place DNS credentials on a command line; official DNS plugins should read a root-only credential file with minimal API scope.
Alternatives
- For a deliberate method migration, amend one lineage and prove it with a dry run before changing all certificates.
Stop conditions
- Any authorization, plugin, parser, deployment, timer, or cleanup error occurs, or the dry run changes the production certificate.
instruction
Record expiry monitoring and incident commands
Capture a machine-readable expiry date, recent timer results, and the safe certificate inventory commands for the operations runbook.
Why this step matters
Renewal automation can fail silently until an outage unless expiry, service failures, and the public endpoint are observed independently with actionable lead time.
What to understand
Read the public cert expiry from the stable lineage and convert it to a machine-readable timestamp; alert well before operational response time becomes risky.
Monitor timer/service exit state and the externally served certificate, because a renewed file may not be the certificate delivered by an upstream edge.
Record inventory, logs, owners, escalation, and the safe manual dry-run command in the incident runbook without copying private key data.
System changes
- No persistent change in the supplied command; the resulting monitoring integration is an operational configuration change.
Syntax explained
openssl x509 -enddate -noout- Reads only the public certificate expiration field.
date -d- Parses the returned date and emits an ISO timestamp.
journalctl -u- Reads service-specific renewal events.
certbot certificates- Provides authoritative local lineage inventory.
Values stay on this page and are never sent or saved.
date -d "$(sudo openssl x509 -enddate -noout -in /etc/letsencrypt/live/{{domain}}/cert.pem | cut -d= -f2)" --iso-8601=seconds && sudo journalctl -u certbot.service -n 30 --no-pager && sudo certbot certificates2026-10-23T12:00:00+00:00 No renewals were attempted.
Checkpoint: Checkpoint: monitor
Continue whenExpiry is parseable, sufficiently distant, renewal logs are healthy, and an external probe observes the same certificate.
Stop whenExpiry cannot be read, monitoring sees another certificate, renewal failures exist, alert ownership is absent, or lead time violates policy.
If this step fails
The local lineage has a new expiry but clients still receive the old certificate.
Likely causeApache was not reloaded, another TLS terminator serves the connection, or monitoring targets a different address family.
sudo certbot certificatesecho | openssl s_client -connect app.example.com:443 -servername app.example.com 2>/dev/null | openssl x509 -noout -serial -enddatesudo apache2ctl -S
ResolutionFind the real termination point, deploy/reload there after syntax validation, and compare serial plus expiry externally.
Security notes
- Monitoring needs public certificate metadata and service status, not access to private-key bytes.
Alternatives
- Use an approved certificate transparency/endpoint monitor in addition to local timer alerts for independent evidence.
Stop conditions
- Expiry cannot be read, monitoring sees another certificate, renewal failures exist, alert ownership is absent, or lead time violates policy.
Finish line
Verification checklist
sudo grep -E -A8 -B4 'AuthorizationError|Some challenges have failed|urn:ietf:params:acme:error' /var/log/letsencrypt/letsencrypt.log | tail -n 120The last attempt either has no failed authorization or its typed cause is preserved and resolved.curl -4 --fail http://{{domain}}/.well-known/acme-challenge/oneliners-preflight && curl -6 --fail http://{{domain}}/.well-known/acme-challenge/oneliners-preflightEvery advertised address returns the exact preflight body; omit the IPv6 request only when no AAAA record is intentionally published.for ns in $(dig +short NS {{domain}}); do dig +norecurse @$ns {{domain}} A; dig +norecurse @$ns {{domain}} AAAA; dig +norecurse @$ns {{domain}} CAA; doneEvery authoritative server agrees on intended addresses and effective issuance policy.curl --fail --head https://{{domain}}/Returns a successful response with no certificate error.sudo certbot renew --dry-runAll simulated renewals succeed against staging.sudo apache2ctl configtestSyntax OK after Certbot edits.Recovery guidance
Common problems and safe checks
Certbot reports that authorization failed or the challenge returned 404.
Likely causeDNS points elsewhere, the request selects another vhost, the challenge path is rewritten/proxied, or public port 80 does not reach this Apache instance.
getent ahosts app.example.comsudo apache2ctl -Scurl -v http://app.example.com/.well-known/acme-challenge/oneliners-preflightsudo tail -n 100 /var/log/apache2/app.example.com-access.log
ResolutionCorrect DNS and vhost routing or exempt the exact challenge path from an application rewrite, then prove the preflight object from an external network before retrying.
Authorization succeeds for IPv4 but fails intermittently.
Likely causeA published AAAA record reaches a different or unconfigured IPv6 listener.
dig +short A app.example.comdig +short AAAA app.example.comcurl -6 -v http://app.example.com/.well-known/acme-challenge/oneliners-preflight
ResolutionConfigure the same site and firewall path over IPv6 or remove the inaccurate AAAA record according to change control before retrying.
Apache configtest fails after the installer change.
Likely causeA generated SSL vhost conflicts with existing directives, a referenced file is absent, or an earlier site configuration was already invalid.
sudo apache2ctl configtestsudo apache2ctl -Ssudo certbot --apache rollback --checkpoints 1
ResolutionDo not reload. Review the changed site files, use Certbot's installer rollback or restore the archived Apache tree, and parse again before activation.
HTTPS presents a valid certificate for the wrong hostname.
Likely causeSNI maps to the wrong TLS virtual host, ServerName/ServerAlias overlaps, or a load balancer terminates TLS before Apache.
sudo apache2ctl -Secho | openssl s_client -connect app.example.com:443 -servername app.example.com 2>/dev/null | openssl x509 -noout -subject -ext subjectAltName
ResolutionCorrect the explicit TLS vhost map and certificate name at the actual termination boundary; do not rely on the first enabled SSL site.
`certbot renew --dry-run` fails although production HTTPS works.
Likely causeThe saved authenticator path, firewall, DNS, plugin installation, account, or renewal configuration no longer matches current service state.
sudo certbot certificatessudo certbot renew --dry-run -vsudo journalctl -u snap.certbot.renew.service -n 100 --no-pager
ResolutionRepair the saved challenge path or plugin state and repeat the staging rehearsal; do not wait for the production certificate to approach expiry.
Clients report an incomplete chain while local browsers appear healthy.
Likely causeApache references a leaf certificate rather than `fullchain.pem`, an intermediary proxy serves stale material, or a client lacks a required trust path.
sudo grep -R SSLCertificateFile /etc/apache2/sites-enabledecho | openssl s_client -showcerts -connect app.example.com:443 -servername app.example.com
ResolutionReference the Certbot full chain at the actual TLS endpoint, reload only after configtest, and retest from an independent client.
Certbot reports AuthorizationError or Some challenges have failed with Type: unauthorized and Invalid response.
Likely causeThe CA reached an HTTP server but received a 404, an application page, a proxy error, or a token owned by a different order because the wrong virtual host, webroot, origin, or rewrite handled the path.
sudo grep -E -A8 -B4 'AuthorizationError|Some challenges have failed|Invalid response' /var/log/letsencrypt/letsencrypt.log | tail -n 120sudo apache2ctl -Scurl -4 -sS -D- -o- http://app.example.com/.well-known/acme-challenge/oneliners-preflightcurl -6 -sS -D- -o- http://app.example.com/.well-known/acme-challenge/oneliners-preflight
ResolutionRepair the approved vhost or webroot and make only the exact challenge namespace bypass an application rewrite when necessary. Prove the deterministic object from an external network and use staging before creating another production order.
Prove HTTP-01 from every public addressThe ACME problem type is connection, timeout, or connection refused on port 80.
Likely causeOne public address does not reach the intended listener because of provider firewall, security group, load balancer, NAT, host firewall, port ownership, or an obsolete DNS record.
dig +short A app.example.com; dig +short AAAA app.example.comsudo ss -lntp 'sport = :80'nc -vz app.example.com 80curl -4 -v http://app.example.com/.well-known/acme-challenge/oneliners-preflight
ResolutionRestore the intended port-80 path on every advertised address or adopt automated DNS-01. HTTP-01 cannot be redirected from an arbitrary external port.
Check whether the public port accepts connectionsHTTP-01 works over IPv4 but validation still returns the wrong response.
Likely causeAn AAAA record points to a different web server or one IPv6 listener returns an HTTP error. Let us Encrypt prefers IPv6 initially and does not fall back to IPv4 after receiving a definite wrong HTTP response.
dig +short AAAA app.example.comcurl -6 -sS -D- -o- http://app.example.com/.well-known/acme-challenge/oneliners-preflightsudo ss -lntp6 'sport = :80'
ResolutionConfigure IPv6 to serve the same challenge or remove the inaccurate AAAA record under change control. There is no switch that asks the CA to ignore published IPv6.
The challenge follows a redirect loop, reaches an unsupported port, or lands on another hostname.
Likely causeA global redirect, CDN rule, canonical-host rule, authentication gateway, or HTTP-to-HTTPS policy intercepts the ACME namespace.
curl -sS -D- -o/dev/null http://app.example.com/.well-known/acme-challenge/oneliners-preflightcurl -sS -L --max-redirs 10 -D- -o- http://app.example.com/.well-known/acme-challenge/oneliners-preflightsudo apache2ctl -S
ResolutionKeep the challenge path on the reviewed host and on ports supported by HTTP-01. Scope any exception to the challenge namespace and retain the normal redirect for other requests.
The ACME problem type is dns, NXDOMAIN, SERVFAIL, or no valid A records found.
Likely causePublic authoritative DNS differs from split DNS, nameservers disagree, delegation is lame, DNSSEC is invalid, or a recent record is not present at the effective zone.
dig +trace app.example.comfor ns in $(dig +short NS example.com); do dig +norecurse @$ns app.example.com A; dig +norecurse @$ns app.example.com AAAA; donedig +dnssec app.example.com A
ResolutionRepair authoritative delegation, record consistency, or DNSSEC. Wait for every authority to agree and relevant negative-cache TTLs to expire before another staging order.
DNS-01 reports an incorrect TXT record or no TXT record found.
Likely causeThe token was written in the wrong zone, a CNAME or NS delegates _acme-challenge elsewhere, one authority is stale, propagation wait is too short, or another automation owns overlapping TXT records.
dig +trace _acme-challenge.app.example.com TXTfor ns in $(dig +short NS example.com); do dig +norecurse @$ns _acme-challenge.app.example.com TXT; donedig _acme-challenge.app.example.com CNAME +short
ResolutionPublish the current token at the effective delegated owner and wait for all authoritative servers. Remove a stale token only after proving no active order or renewal automation owns it.
The ACME problem type is caa or the CAA lookup returns SERVFAIL.
Likely causeThe closest effective CAA set forbids Let us Encrypt, issuewild differs for a wildcard, a CNAME target has its own policy, DNSSEC is broken, or an authoritative server times out or mishandles CAA.
dig +trace app.example.com CAAdig +dnssec app.example.com CAAfor ns in $(dig +short NS example.com); do dig +norecurse @$ns app.example.com CAA; done
ResolutionRepair DNS availability or DNSSEC first. Change CAA only with domain-owner approval and keep the intended CA and validation-method restriction.
Validation is intermittent across retries or fails only from some locations.
Likely causeMultiple A or AAAA records, anycast, geo-DNS, CDN nodes, load-balancer targets, or replicated webroots do not serve the same token and redirect behavior.
dig +short A app.example.com; dig +short AAAA app.example.comsudo tail -n 200 /var/log/apache2/*access.log | grep '.well-known/acme-challenge'curl -sS -H 'Host: app.example.com' http://203.0.113.10/.well-known/acme-challenge/oneliners-preflight
ResolutionMake every advertised validation origin return the current token or temporarily remove only unready addresses under approved change control. Prove consistency before retrying.
A wildcard request keeps selecting or failing HTTP-01.
Likely causeHTTP-01 cannot authorize wildcard identifiers; the requested certificate requires DNS-01 automation and appropriate CAA issuewild policy.
sudo certbot certificatesdig +short CAA example.comdig +trace _acme-challenge.example.com TXT
ResolutionUse an official DNS plugin or delegated DNS validation zone with narrowly scoped credentials, prove propagation, and rehearse automated renewal against staging.
Repeated retries now produce rate-limit errors instead of the original challenge detail.
Likely causeProduction orders were retried before DNS or HTTP evidence converged, consuming issuance or failed-validation capacity.
sudo grep -E 'rateLimited|too many|retry after|AuthorizationError' /var/log/letsencrypt/letsencrypt.log | tail -n 100date -u
ResolutionStop production attempts, preserve Retry-After or limit details, repair with deterministic preflight checks, and use the staging CA until the complete path succeeds.
Reference
Frequently asked questions
What does certbot.errors.AuthorizationError: Some challenges have failed mean?
It means at least one requested ACME authorization did not pass. The exception is a summary, not a diagnosis. Read the earlier Domain, Type, and Detail fields to identify the name, challenge, ACME problem type, address, URL, status, or DNS response that failed.
Why does Certbot say Invalid response when the website works in my browser?
The browser may use a different address, cached DNS, a working CDN node, HTTPS only, or an internal split-DNS route. The CA validates the exact HTTP-01 token from public DNS, prefers IPv6 when AAAA exists, and may check from multiple perspectives. Test the exact path over every public A and AAAA route.
Can HTTP-01 use port 8080 or another external port?
No. HTTP-01 starts on public TCP port 80. Supported redirects are bounded and only target HTTP or HTTPS on ports 80 or 443. Use automated DNS-01 or a suitable TLS-ALPN-01 implementation when port 80 cannot be made reliably available.
Why does an old AAAA record break Certbot when IPv4 is correct?
When both A and AAAA exist, Let us Encrypt prefers IPv6 initially. A timeout can sometimes fall back to IPv4, but a wrong HTTP response over IPv6 is a definite validation failure. Fix IPv6 to serve the same challenge or remove the inaccurate AAAA record.
How long should I wait for a DNS-01 TXT record?
There is no universal delay. Query every authoritative nameserver directly and wait until each serves the current token at the effective delegated owner. Recursive resolver caches and a single provider dashboard are not sufficient evidence.
Can I delete old _acme-challenge TXT records?
Multiple TXT records are valid during overlapping orders. Remove a record only after proving which automation, ACME account, and active order owns it. Blind cleanup can break another renewal; abandoned records should be removed deliberately to reduce ambiguity and oversized responses.
What CAA value allows Let's Encrypt?
The CA identifier is letsencrypt.org. A basic non-wildcard policy can use CAA 0 issue letsencrypt.org, while issuewild, validationmethods, account restrictions, CNAME targets, and inherited records can change the effective result. Do not weaken CAA without owner approval.
Should I keep retrying with the production CA?
No. Preserve the first failure, repair and prove the affected boundary with a deterministic challenge object, then use the staging CA or certbot renew --dry-run. Return to production only after authoritative DNS and all public routes are consistent.
Why keep port 80 after enabling HTTPS?
It provides the normal redirect and remains the validation path for HTTP-01 renewal. Closing it without changing the saved authenticator turns a successful installation into a future renewal incident.
Does renew --dry-run shorten or replace the real certificate?
No. It uses the staging environment to rehearse saved renewal behavior and should leave the production lineage unchanged.
Should rollback revoke the certificate?
Usually not. Restore Apache service first. Revoke only when the key or authorization should no longer be trusted, then delete a lineage only after all consumers are retired.
Recovery
Rollback
Rollback restores Apache and challenge routing without deleting ACME evidence or revoking a certificate unnecessarily. Revoke only for compromised or intentionally invalid trust.
- Stop new production issuance attempts and preserve the latest Certbot log, Domain/Type/Detail fields, authoritative DNS answers, and current Apache virtual-host map.
- Use certbot --apache rollback for the most recent installer checkpoint when appropriate, then run apache2ctl configtest before reload.
- Restore /etc/apache2 from /root/apache-before-certbot.tgz if the installer rollback is insufficient; keep the existing /etc/letsencrypt lineage while diagnosing.
- Reverse temporary DNS, CAA, firewall, proxy, or challenge-path changes only to the documented previous values and verify public IPv4 and IPv6 behavior.
- Use certbot revoke only when the private key or authorization must no longer be trusted. Use certbot delete only after every consuming service has been retired.
Evidence