Should I vibe code
Ingest bounded logs, search them, and retain a compact local event store
Your log store is an unaudited second copy of every secret your application has ever handled.
?
Their verdict, the Axiom Cloud price and the build-time estimate come from their entry, MIT-licensed. Checked 2026-08-04.
?
Our verdict, the regret score and everything below it. Editorial and unsponsored — nobody can pay to be moved.
The honest answer
why the verdict is what it is
A logs table with a JSONB column and a search box is an afternoon, and for a while it is better than the hosted thing because you can join against your own data. What nobody plans for is what a log store actually accumulates. Applications log request bodies, Authorization headers, session cookies, reset tokens, webhook payloads and whole customer records — not deliberately, but as a side effect of a debug line somebody added during an incident and never removed. Six months later the least-protected database you own contains a searchable copy of the most sensitive data your application has ever handled, with longer retention than anything else and no deletion story, and it is the one you grep casually and paste into Slack. That is the quiet failure. The loud one is simpler and lands on your users: a log store with no retention policy fills a disk, and if it shares a host or a volume with the thing it is watching, your observability tool takes down the service it observes — at which point you also have no logs to find out why.
What actually breaks
not "if". the specific failures.
- The disk, which is the failure that actually happens: no retention policy, growth that is linear until a bad deploy makes it exponential, and a full volume on the machine that was also running the application
- Secrets in the payload, because a log line is written by whoever added it — Authorization headers, session cookies, password-reset tokens and Stripe keys all end up in there, and the search box makes them findable
- Personal data, since IPs, user IDs, emails and request bodies are personal data with a retention obligation, and "delete this user" almost never reaches the log store
- Query performance, at a threshold you cross without noticing. Full-text or JSONB search with the wrong index is a sequential scan, and you find out during the incident it was supposed to help with
- The circular dependency, where the logging stack shares a host, a network or a cluster with the application, and therefore goes down in exactly the same event
- The ingest endpoint, which is usually an HTTP POST with one shared token — a write amplifier for anyone who finds it, and the reason someone else's junk is in your storage bill
- Retention as a business decision made by accident: whatever is on disk is what you have, which is both longer than your privacy policy claims and shorter than the auditor wants
- Alerting, or its absence. A log store that does not alert is a search box, and building the alerting — dedupe, flap suppression, escalation — is the actual product
- Cost, once ingest is billed. Turning on debug logging for an afternoon is a 100x volume event, and there is nothing between you and the bill unless you put it there
- Clock skew and ordering, which turn a distributed timeline into a guess right when you are trying to work out what happened first
The pager fired at 02:14 for the API, and the first thing you did was open the log search, which did not load. It took twenty minutes to work out why: the logging container and the application were on the same box, sharing the same volume, and a deploy the previous afternoon had left a request-body logger enabled on the busiest endpoint. Six weeks of normal growth had become nine hours of everything, the volume hit 100%, Postgres stopped accepting writes, and both the application and the thing watching the application died in the same second — with no log line explaining any of it, because there was nowhere to write one. You freed space by deleting the oldest partitions, which is fine until Tuesday, when someone points out that the request bodies you just spent the night grepping through contained session cookies and full card-holder names in plain text, retained for eight months, in a database whose only credential is in three developers' .env files.
Is that you?
the verdict is a default, not a law
- It is one application, one box, and a retention policy measured in days that you wrote before the first line was ingested
- The logs are structured, scrubbed at the source, and contain no request bodies or headers at all
- It runs somewhere the application does not — a different host, a different provider — so the two cannot fail together
- You want a searchable event store for product analytics rather than an incident tool, and being down during an incident costs you nothing
- It shares a disk, a host or a failure domain with the service it is supposed to observe
- There is no retention policy enforced by a job that actually runs, with an alert on disk headroom
- Anything is logged whole — request bodies, response bodies, headers — without a scrubber in front of it
- You cannot answer what a deletion request means for your logs, or how far back your retention actually goes
- The ingest endpoint accepts writes with one shared token and no rate limit
- You are going to be woken by it, because alerting is a separate product and you have not built it
If you build it anyway
the checklist, then the prompt that enforces it
- Write the scrubber before the ingester. An allowlist of fields, not a denylist of secrets — a denylist misses the header you have not thought of yet, and the redaction has to happen in the application before the line leaves the process.
- Set retention on day one and enforce it with time-partitioned tables and a drop job, not a DELETE. Then alert on disk headroom, not on disk full.
- Put the log store somewhere the application is not. Different host at minimum, different provider ideally. A monitoring system inside the failure domain it monitors is a diary.
- Bound the ingest: per-source tokens, rate limits, a maximum event size and a hard daily volume cap that drops rather than stores. Debug logging left on is a 100x event.
- Index deliberately. Timestamp plus service plus level covers most queries; a GIN index on a JSONB column is a different, much larger decision. Test search on a realistic volume, not on 10,000 rows.
- Treat the store as personal data: document the retention period, make sure it matches the privacy policy, and know in advance what a deletion request means for it.
- Restrict and log access. A log search box is an unaudited read of everything, and "who searched for what" is a question you will eventually be asked.
- Do not build alerting on top of it as an afterthought. Dedupe, flap suppression, escalation and delivery are a second system, and until it exists you have a search box rather than monitoring.
- Before writing any of it, try Grafana Loki or OpenObserve. Both self-host, both already solved the storage and index problem, and neither needs you to invent retention.
I am building a log store: ingest structured events over HTTP, keep them, search
them. Assume secrets will end up in the payload and that the disk will fill.
Order matters here — do not start with the search UI.
1. First ask whether Grafana Loki or OpenObserve solves this. Both self-host and
both already have retention, indexing and compaction. If either fits, say so.
2. If we build: the scrubber comes first, in the application, before a line leaves
the process. Allowlist the fields that may be logged rather than denylisting
secrets. Never log whole request bodies, response bodies or headers.
3. Then retention: time-partitioned storage with a drop job, a configured maximum
age, and an alert on disk headroom. Not a DELETE, and not "we will add it later".
4. Then bound the ingest — per-source tokens, per-source rate limits, a maximum
event size and a hard daily volume cap that drops on breach. Say plainly that
without the cap a debug flag left on is an outage and a bill.
5. Deploy the store somewhere the application is not. If I propose the same host or
volume, refuse and explain that a logger sharing a disk with its service takes
the service down with it.
6. Choose indexes deliberately: timestamp, service and level first. Explain what a
GIN index on a JSONB column costs before adding one, and generate a realistic
volume — tens of millions of rows — to test against.
7. Treat the store as personal data. Document the retention period, and implement
what a deletion request means for logs before I have to answer one.
8. Restrict who can query, and log the queries. A search box over logs is an
unaudited read of everything the application has ever handled.
9. Only then the UI, and only then alerting — and tell me honestly that alerting is
a separate system with dedupe, flap suppression and escalation in it.
10. Out of scope until I ask: tracing, metrics, dashboards and multi-tenant access.
And mention that Axiom's Personal tier is free with 500 GB a month, which is
more than this store will ever ingest.That one keeps you out of trouble. For the prompt that actually builds it, canivibecodeit.com has one.
their build prompt ↗Or don’t build it
the boring option, and the way back out
Sooner than the price suggests, because the price is $0. Axiom's Personal tier is free with 500 GB a month of ingest and 30-day retention, which is more than a side project or a small production service will produce, and it removes the disk, the retention job, the index tuning and the circular dependency in one go. The $25 Axiom Cloud tier is the one you reach when volume genuinely justifies it. The honest framing is that this is not a build-versus-buy decision at small scale at all — it is a decision about whether you want to own a database whose contents you cannot fully account for.
$25/mo is cheaper than your weekend.
Getting out is easy and getting the timing right is not, because the moment you want to migrate is usually the moment you are already over budget on disk. Keep events in a format you can ship somewhere else — newline-delimited JSON with a stable schema, partitioned by day — and prove the export by actually replaying a day of it into a scratch instance of whatever you are migrating to. Two things need deciding before you switch off: how much history you actually carry across, which is a cost question wearing a technical hat, and whether the old store gets destroyed properly. It does, because it holds the unscrubbed history — rotate every credential that has ever been in a log line rather than assuming none were, and delete the volume rather than the rows.
Horizontally scalable log aggregation system that indexes labels rather than full text, which is exactly the design decision a homemade store gets wrong.
Self-hostable observability platform for logs, metrics and traces with columnar storage and built-in retention.
Open-source observability stack over ClickHouse, worth reading for how it handles ingest volume and query cost.
Questions
Which Axiom is this?
axiom.co, the machine-data and log-analytics platform — ingest events, query them, retain them. There is an unrelated browser-automation product trading under the same name, and this entry is not about it. The domain in the record is the authority.
Why is dataSensitivity scored so high for logs?
Because nobody chooses what a log store contains. It is the union of every debug line anyone has ever added, which over time means Authorization headers, session cookies, reset tokens, webhook payloads and whole customer records. It has the loosest access control of any datastore you own, the longest retention, and a search box. Score it as what it accumulates, not as what you intended to put in it.
Isn't the disk-filling story just bad ops rather than an argument against building?
It is bad ops, and it is also the single most common way self-hosted logging causes an outage, which makes it the honest thing to lead with. The specific detail that turns it from annoying into serious is co-location: when the log store shares a volume or a host with the service, the failure is simultaneous and you have no telemetry about the event that just removed your telemetry.
What is the cheapest thing that removes most of this risk?
Scrub at the source with an allowlist and set a retention period you enforce with partition drops. Those two changes cost an afternoon and take out the secrets problem, the deletion-request problem and the disk problem together. Putting the store on a different host than the application is the third, and it costs a few dollars a month.
- GDPR Art. 5 — principles relating to processing of personal data
- GDPR Art. 32 — security of processing
Every week, someone ships something they shouldn’t have.
New verdicts, the worst thing that landed in the trap, and the occasional incident report. No other email, ever.
last reviewed 2026-08-04 · verdict is editorial and unsponsored · shared entry data from canivibecodeit under MIT · not legal advice