Someone asks why the revenue number looks wrong. You dig in, find that orders stopped loading on Tuesday, and think: I should have caught that automatically. So you write a check. Forty lines of SQL, a cron entry, an email if the count is zero. It takes an afternoon and it works.

That afternoon is real engineering value, and nobody should talk you out of it. The problem is that the afternoon is not the cost. The cost is what the check becomes by month three, once there are ten of them, and once the person who wrote them is not the person being paged.

When building is genuinely the right answer

If you can finish this sentence precisely — “this column must always satisfy X, and if it does not, the pipeline is broken” — then write the test. A known invariant on a table you fully own is exactly what a dbt test or a cron’d SQL assertion is for. It is cheap, it lives beside the transformation it guards, it fails loudly in CI before bad data reaches anyone, and it will never surprise you.

Good candidates look like this:

  • Referential integrity you control. Every order.customer_id exists in customers. That is a contract, not a trend.
  • Business rules with hard boundaries. discount_pct is between 0 and 100. status is one of five known values.
  • Uniqueness on a key you defined. order_id is the primary key because you made it one.
  • Accepted values that come from your own enum. If the application only writes four statuses, a fifth is a bug in the application.

These share one property: the correct answer is knowable in advance and does not drift. You are not guessing at a threshold, you are encoding a rule that already exists somewhere in your head or your schema. Writing that down is unambiguously worth doing, and no monitoring tool replaces it.

The costs that show up in month three

The trouble starts when checks stop being assertions and start being judgements. “Row count should be around 10,000” is not an invariant. It is a threshold, and thresholds rot.

Threshold maintenance. You set > 8000 in March. In June marketing runs a campaign and volume doubles, so the floor is meaningless. In September the company signs an enterprise customer whose backfill lands weekly instead of daily. Every one of those events is a pull request against a threshold nobody remembers choosing. Multiply by the number of tables that matter.

Seasonality. Almost every real table has a rhythm. Weekend dips, month-end spikes, a quiet week in August, a batch that lands at 03:00 on Mondays and not at all on Sundays. A static threshold either fires every Saturday or is set so loose it catches nothing. Handling this properly means storing history, computing a rolling baseline, and deciding what counts as a deviation from it. That is not an afternoon. That is a small system with its own storage and its own bugs.

-- The moment the afternoon ends: this is no longer a test, it is a model
with daily as (
  select date_trunc('day', created_at) as d, count(*) as rows_loaded
  from analytics.orders
  group by 1
),
baseline as (
  select d,
         rows_loaded,
         avg(rows_loaded) over (
           order by d rows between 28 preceding and 1 preceding
         ) as trailing_avg,
         stddev_samp(rows_loaded) over (
           order by d rows between 28 preceding and 1 preceding
         ) as trailing_sd
  from daily
)
select * from baseline
where rows_loaded < trailing_avg - 3 * trailing_sd;

That query is a starting point, not an answer. It has no concept of day-of-week, it is destabilised by a single outlier three weeks ago, and it silently returns nothing for a table’s first month of life.

The alerting and routing layer. A failing check that emails a shared inbox at 04:55 is not monitoring, it is a lottery. Real routing means severity (is this worth waking up for?), grouping (one email, not forty), deduplication (this has been failing for six days), suppression (yes, we know, the vendor is migrating), and an audit trail of who looked at what. Every observability team eventually builds this. It is a product surface, not a script.

The silence problem. This is the one that bites hardest. Your checks have not fired in three weeks. Is the data healthy, or did the cron job die, or did a credential expire, or did someone rename the column the check queried and the WHERE clause now matches nothing? Silence from a homegrown checker is ambiguous by default, and resolving that ambiguity means monitoring the monitor.

On-call for the checker itself. The check suite is now production software with a schedule, a database connection, a secret, and a failure mode. When the warehouse has a bad night and every query times out, someone owns the pager. That someone is the data person who built it.

Onboarding the next person. Ten checks written across nine months by one person encode a great deal of context that lives only in that person’s head. Why is this table’s floor 8,000 and that one’s 500? Why is the sessions check disabled? When they take a holiday, the answer to any alert is “I’ll ask when they’re back”, and when they leave, the suite quietly becomes untouchable and then gets switched off.

Count the cost in hours, not currency

The honest unit for this decision is not money, it is the attention of the one or two people who understand your pipelines.

Try the estimate on your own team. Per check, per year, roughly: the initial write, plus threshold revisions when the business changes, plus triage on false positives, plus the incidents where the checker itself was the thing that broke. For a suite of ten checks on a small team, that lands somewhere between one and two full engineering weeks a year, spent in interruption-shaped fragments rather than in blocks.

That is the real comparison. Not a subscription against zero, but a subscription against the highest-leverage hours you have. Every hour spent tuning a row-count threshold is an hour not spent on the model that the business actually asked for. If you want to weigh that against a tool, the numbers are on the pricing page; what matters here is that “build” is never the zero-cost column.

A decision rule

Ask one question: do I already know what the correct answer is?

Specific invariant you know Unknown unknowns across many tables
Example order_id is unique; status is one of five values A column’s null share triples; a table loads late on a Wednesday
Correct answer Known in advance, written down Learned from the table’s own history
Fails when The rule is genuinely violated Behaviour deviates from its own pattern
Cost of ownership Low and flat Grows with tables, seasonality and staff turnover
Best tool A test in your transformation layer Learned baselines and anomaly monitoring
Where it lives Beside the model, in version control Alongside the warehouse, running nightly

If the answer is yes, write the test. Put it in dbt or in CI, next to the code that produces the data, and let it block the pipeline. That is cheaper and more precise than any statistical method, because it does not have to infer what you already know.

If the answer is no, do not build a threshold and pretend it is a rule. What you actually want is something that learns each table’s normal behaviour, including its rhythms, and tells you when today does not look like the table’s own history.

For most teams the answer is both, and the split is roughly this: hand-written tests guard the handful of contracts you can state exactly; learned monitoring covers the long tail of tables where you would never think to write a check, which is precisely where the expensive surprises come from. The tables that break in ways you predicted are rarely the tables that ruin a quarter.

How Sentry handles it

Sentry is built for the second column of that table, and deliberately does not try to replace the first.

  • Baselines instead of thresholds. Sentry profiles each monitored table’s history to learn its normal behaviour, including weekend dips and weekly load schedules. There is nothing to tune when volume doubles.
  • Six checks across every monitored table. Freshness, volume, out-of-range, null-rate, format and duplicates run nightly, on tables you would never get around to writing checks for.
  • One digest, ordered by severity. A single morning email covering what was scanned, what is healthy, and what is not. Silence means the scan ran and found nothing, which is a different statement from a cron job that stopped.
  • Noise control that teaches. Mark a finding as expected and it folds into the baseline; mute one check on one table for a set period. You are curating a model, not editing conditionals.
  • Diagnoses only. Every finding carries a plain-English explanation and its evidence: the queries run, sample offending values, and the baseline it was judged against. Sentry never writes to your warehouse, so the decision stays with the person who understands the pipeline.

Keep your dbt tests. They are the cheapest data quality you will ever own. Hand over the part that would otherwise become a product you maintain.

Pick a plan and get started — read-only credentials, first digest tomorrow morning.