At 2 a.m. you do not want a novel. You want to know which request failed, which user, which action, and whether it is the database, the payment provider, or your null check. What you have is 40 million lines of debug from a health check, no request id, and a console.log(user) that once printed a token.
Logging is a product you build for your future self under stress. Too little and you guess. Too much and you cannot find the line. The skill is choosing what to emit, at which level, with which fields, and what never to print.
This is a practical logging guide for application developers, not a vendor bake-off.
Levels are a contract
error: something failed that you should look at. User-facing action did not complete. You will search for this at 2 a.m.
warn: unexpected but handled. Fallback used. Retry succeeded after a failure. If you page on warn, you will hate mornings.
info: lifecycle that is rare enough to be useful: server started, migration ran, payment captured (without the PAN). Not every HTTP 200.
debug: for local and temporary investigation. If debug is on in production for all routes, you do not have debug. You have noise.
Do not log at error because a user typed a bad password. That is info or debug, or a metric. Scanners will look like an outage.
Structure beats string poetry
User 18 failed checkout because card is hard to filter. { event: "checkout_failed", userId: "18", code: "card_declined", requestId: "..." } you can query.
Pick a field dictionary: requestId, userId, orgId, event, code, durationMs. Same names in every service. user_id vs userId vs uid is how you fail to join logs during an incident.
A request id must be created at the edge and passed to workers. Log it on every line for that request. Print it on 500 pages. Support will send it to you.
What to log on the hot path
Start and finish of a significant action, with duration: checkout_started, checkout_succeeded, checkout_failed. Failures include code. Successes include amount if you need it, not the full card.
Outbound HTTP: method, host, status, duration, a correlation id. Not the entire body.
Inbound: method, route template (/users/:id not /users/18 if the id is PII in a log retention sense—depends on your policy), status, duration. Logging raw URLs with tokens in query strings is how tokens leak into the log vendor.
What never to log
Passwords, cookies, Authorization headers, tokens, session ids, full card numbers, government ids, health data, the body of a login request.
If you log request bodies in development, put a redactor in front before production. Libraries exist. A homemade regex will miss a field. Prefer allowlists for production body logging, not denylists.
Stack traces on error, yes. Stack traces on every 404, no.
Cardinality and the bill
If you log a unique event per user per pixel, you will pay for a data warehouse you did not want. High cardinality in metric labels is worse than in logs, but logs cost too. Sample debug. Keep errors complete.
Do not put unfiltered user-agent strings into a metric label. Do put them in a log field if you need them for a bug, then drop them after the incident.
Correlation with traces and metrics
A log line should be able to open a trace. A trace should have the same requestId. Metrics should tell you “checkout_failed is up.” Logs tell you why for one example. If you only have metrics, you know you are dying. If you only have logs, you drown. You want both.
When you add a log, ask: would I want this line in a stream of 1,000 requests per second? If not, it is debug or a metric counter.
Local vs production
Pretty, colorized logs locally are fine. JSON in production is fine. Do not pretty-print JSON in production to a file you grep with tail. Use the vendor’s query language, or jq if you must.
console.log in a frontend app will not be in your server grep. Use an error reporter for client issues, with PII policy. Do not send every Redux action to the server.
A policy you can write in a wiki
- Every request has an id
- Errors have
event,code,requestId - PII fields listed and forbidden
- Health checks logged at debug or not at all
- No body logging in production without redaction
- Warn is not a page
Then review a random hour of logs once a quarter. You will find a forgotten print(user). Delete it.
2 a.m. query
Start with event=checkout_failed and a time window. Open one requestId. Read that id only. Follow it across services. If you cannot, the id was not propagated. That is the first fix after you restore the site.
Sampling and the error that happens once
If you sample info logs at 1%, you will miss the single weird payload. Do not sample errors. Sample debug and maybe info on ultra-hot paths. If a bug is rare, you need the full error line every time, with the request id, even if info is sampled.
When you add a temporary debug log for an incident, put a ticket number in the message and delete it after. Temporary logs that stay for a year become the noise you swore you would not create.
Frontend and backend together
A user report with only a screenshot of a toast needs a client event id that is also sent to the server, or a time plus user id. If the frontend error reporter and the backend logs cannot join, you will debug two half-stories. Pass the request id from the failed fetch into the toast. Ugly. Effective.
PII in URLs and referrers
If the user id is in the path, logs and access logs have PII. That may be acceptable under your policy. Tokens in query strings will leak via Referer to third-party scripts. Do not put session ids in GET query params. If a vendor requires it, treat it as a vendor defect and expire fast.
Multiline stack traces in JSON logs
JSON log lines that contain a stack as a single escaped string are searchable. Logs that split a stack across lines without a request id are not. Configure the logger to put stack in a field. Your 2 a.m. self will grep requestId and get the stack in the same object.
Clock skew in log timestamps
If app servers and the log vendor disagree on time, your window misses the event. Use UTC everywhere. If a host has NTP drift, fix NTP. Searching the wrong hour is a silent failure mode of 2 a.m. work.
Health checks as log spam
If every five seconds you log health ok at info, you will never find the error. Health at debug, or a metric, or silence. Keep errors for unhealth.
Artikals is for this kind of operational writing because it is how teams stay kind to their on-call. Logging is not extra print statements. It is a designed stream. Make it small, structured, and safe. Then at 2 a.m. you are reading a story about one request, not a dump of the entire process.
