dbprint Configuration
Purpose: every key dbprint reads from
.dbprint.yamland~/.dbprint/connections.yaml. For the on-disk output these settings produce, see format/v1/SPEC.md (normative). For the command surface, see CLI.md. For theassertions:grammar, see ASSERTIONS.md.
Two files, with different lifetimes:
| File | Committed? | Holds |
|---|---|---|
.dbprint.yaml | Yes — it describes the project | Connections, rules, tuning |
~/.dbprint/connections.yaml | No — credentials | Host, user, password, keys |
dbprint init writes a starting pair. A worked .dbprint.yaml ships at
format/v1/examples/production/.dbprint.yaml
alongside the print it produces.
Unknown keys are ignored, not rejected. dbprint reads exactly the keys below; anything else is dropped silently, so a misspelled or mis-nested key leaves the default in place without a warning. When something appears to have no effect, check the spelling and the nesting depth first.
.dbprint.yaml
defaults: # OPTIONAL; each key cascades into every connection. A connection-level value overrides it, except for `rules` and `redact`, which append — see below output: prints include: ["<PATTERN>"] exclude: ["<PATTERN>"] max_age_days: 7 infer_relationships: true sketch_all_columns: false compute_timeline: true statistics: { ... } rules: [ ... ] redact: [ ... ] diff: { ... }
connections: # REQUIRED; at least one <name>: # free-form identifier; also the print subdirectory adapter: postgres | snowflake | mysql | duckdb | clickhouse | redshift | databricks | bigquery # REQUIRED auto: false output: prints include: ["<PATTERN>"] exclude: ["<PATTERN>"] max_age_days: 7 max_rows_scanned: <INT> # absent by default infer_relationships: true materialize_sample: true sketch_all_columns: false compute_timeline: true statistics: { ... } rules: [ ... ] redact: [ ... ] diff: { ... } assertions: { ... }defaults accepts every connection key except adapter, auto and assertions, which are
per-connection only. rules and redact are the two keys that do not override: a connection's
entries are appended to the ones from defaults, which is what makes a connection entry win.
Connection keys
| Key | Type | Default | Meaning |
|---|---|---|---|
adapter | enum | — (required) | postgres | snowflake | mysql | duckdb | clickhouse | redshift | databricks | bigquery |
auto | bool | false | Run this connection on a bare dbprint <command> with no CONN argument. Any number of connections may set it |
output | path | prints | Root directory, relative to .dbprint.yaml. Prints land in <output>/<name>/ — the connection name is appended, so do not include it |
include | list of glob | ["*"] | Tables to profile. Omitting this profiles everything the connection can see |
exclude | list of glob | [] | Removed from the include set |
max_age_days | int ≥ 0 | 7 | A print younger than this is left alone by generate, and passes check's freshness gate. A rules entry can override it per table, and can condition that override on the table's size. 0 re-profiles on every run — see below. A negative value is refused at load |
max_rows_scanned | int ≥ 1 | absent | A row-count ceiling covering every table this connection profiles; defaults and rules may also carry it. See below |
infer_relationships | bool | true | Derive the foreign keys the catalog does not declare, from column naming — see below |
materialize_sample | bool | true | Draw a sampled table's rows once into a temporary table, so every statistic for that table describes the same rows. Takes a temporary-table privilege on PostgreSQL and MySQL, none on Snowflake. On ClickHouse, Redshift and BigQuery the copy is not optional: a sample scope with the key set false, or a table the copy cannot be taken on, fails that table outright rather than reading it — see below |
sketch_all_columns | bool | false | Sketch every sketchable-type column, not only the smaller required set — see below |
compute_timeline | bool | true | Bucket one temporal column's activity over time — see below |
statistics | map | see below | What gets measured per column, and how much of it |
rules | list | [] | Per-table overrides of the keys above, plus the two that narrow what is read |
redact | list | [] | What to do with the cell values of the columns each entry covers |
diff | map | see below | Presentation thresholds for dbprint diff's human output |
assertions | map | {} | Data-quality checks dbprint check evaluates |
The five block-valued keys each have their own section below.
infer_relationships derives the foreign keys a schema never declared. Plenty of
warehouses declare none — Snowflake does not enforce them, and analytics schemas in
PostgreSQL routinely skip them — so a print of one carries an empty relationship graph and
a reader cannot tell that accession.collector_id points at collector.collector_id. With the key on, a column
named <stem>_id whose stem resolves to an in-scope table declaring a single-column
key of a compatible type becomes an edge marked detection: inferred; the rule and its
refusals are specified in SPEC 2.3.8. Turning it off removes every one
of those edges — the graph then carries only what the catalog declares — and skips the
catalog pre-pass that reads the columns and declared keys inference resolves against.
materialize_sample is the one setting that makes dbprint write to your database. A sampled
table is read by many statements, and a sampling construct re-evaluated per statement draws a
fresh set of rows each time — so a column's listed value counts and the non-null figure they are a
share of come from different reads, and the two disagree on a table nobody wrote to. With the key
on, the producer copies the draw into a temporary table once and every statement for that table
reads that instead. What the write costs you: a temporary-table privilege where the copy is made,
spelled differently per engine — TEMPORARY on the database for PostgreSQL, CREATE TEMPORARY TABLES on the database for MySQL, and nothing at all on Snowflake, which exempts temporary tables
from the schema's CREATE TABLE privilege. The copy lands in the session's own temporary space on
PostgreSQL and MySQL and in the profiled table's own schema on Snowflake. The object holds the
sampled fraction only rather than the whole table, and on every adapter but BigQuery its lifetime
is the session, so it is gone when the run ends whether the run succeeded or not — BigQuery has no
session-scoped table at all, so its own copy is a real dataset object with an expiration set
instead; see the per-adapter note below.
Where the privilege is absent, what happens next depends on the adapter. PostgreSQL's own
TABLESAMPLE BERNOULLI decides row membership per row from its seed, so it falls back to
sampling per statement with a warning on stderr, and the incoherence above is back. MySQL and
Snowflake document no such guarantee for their own constructs, so turning the key off (or a
refused write) fails that one table instead, naming materialize_sample in the message, rather
than publish a file whose fields describe different reads — see the per-adapter notes below for
the full split across all eight. A table that is not sampled never materializes — a full scan has
nothing to copy, and a filter is a predicate, so re-evaluating it selects the same rows every
time.
sketch_all_columns widens the second setting that changes what leaves the database.
SPEC 2.2.14 always sketches a column named by an edge, plus every
declared-unique column, every column at or below the sketch's own retained size, and every
column carrying a measured candidate key — a fixed-size summary of a column's distinct
values, from which a consumer computes set overlap against another column offline, no query
against either source. With the key on, every column whose type the sketch format covers is
sketched, whether or not it fits one of those four categories, at the cost of one extra query
per newly-sketched column. A KMV sketch is an unsalted hash of real cell values, so turning
this on widens that surface to every column redaction did not withhold, on top of whatever
the required set already carries.
The key has no effect on a table that carries a scope block, or on a plain view: neither is
sketched at all, whatever this is set to. A sketch answers set-overlap questions between two
columns, which needs a reproducible read of the whole column — a narrowed read cannot give
one, and a view is never queried. Since a large warehouse is also where sampling gets turned
on, expect the two settings to meet: a table narrowed by sample, filter or
max_rows_scanned publishes no sketches regardless.
compute_timeline picks one temporal column per table and buckets its non-null values by
day, week, or month. SPEC 2.2.16 names the anchor deterministically
— a temporal, non-redacted, calendar-typed column named in the table's own clustering or
partitioning key first, otherwise the eligible column with the lowest null rate — at the
cost of one extra grouped query per table that has one. Turning the key off skips that
query and the column entirely, the same way infer_relationships: false skips its own
pre-pass. Like sketch_all_columns, a table carrying a scope block or an empty table
publishes no timeline regardless of this setting.
max_age_days: 0 means the print is stale the moment it is written. generate
re-extracts it every run, and check's freshness gate cannot pass at 0 whatever order the
commands run in. Use it where check does not gate the pipeline, or pass check --max-age
explicitly — an explicit flag overrides every table's recorded threshold. A negative value
asks for the same thing and is refused at load, because it holds every table stale with no
way for the artifact to say so.
include and exclude decide which tables are profiled; rules decides how each one
is profiled. The two axes never mix: a rule cannot bring a table into scope. A rule selects
within the second axis — include / exclude / min_rows narrow which of the profiled
tables that rule governs, and none of them can widen the connection's scope.
Patterns are fnmatch globs over the lowercased fully-qualified name, so matching is
case-insensitive. * spans dot separators. The FQN shape is the adapter's:
database.schema.table (Snowflake, duckdb), schema.table (PostgreSQL, Redshift,
Databricks), database.table (MySQL, ClickHouse), dataset.table (BigQuery).
statistics
Tuning for SPEC 2.2. Every key is optional.
| Key | Type | Default | Meaning |
|---|---|---|---|
enumeration_threshold | int | 50 | Cardinality ≤ this makes a column categorical; above it the column classifies by type and may carry a range instead |
top_n_values | int | 20 | Cap on the values list. A column with at most this many distinct values is enumerated in full, whatever its classification |
top_n_null_patterns | int | 20 | Cap on the null_patterns list — how many distinct combinations of null columns a table publishes. null_patterns.coverage states what share of the rows the listed combinations account for |
looks_like_sample_size | int | 1000 | Distinct non-null values sampled for inferred.looks_like detection |
percentiles | list of int | [1, 25, 50, 75, 99] | Integer percents in 1..99. Fractions such as 0.25 are rejected at load |
Lowering enumeration_threshold is the cheapest way to cut cost on a wide table: the
values list is bounded by top_n_values, and values_coverage states how much of the column it covers.
rules
An ordered list. Each entry carries a matcher and the settings it overrides for the tables it matches, so one connection can sample a billion-row fact table without sampling anything else, and refresh dimensions daily while refreshing that fact table weekly.
rules: - include: ["seedbank.storage_reading*"] # OPTIONAL; defaults to ["*"] exclude: ["seedbank.storage_reading_v2"] # OPTIONAL; defaults to [] min_rows: 500000000 # OPTIONAL; only tables at least this large sample: 0.01 # OPTIONAL; fraction in (0, 1]. Excludes `filter` statistics: {top_n_values: 5} # OPTIONAL; merged key by key max_age_days: 30 # OPTIONAL - include: ["seedbank.germination_reading"] filter: "created_at >= current_date - interval '30 days'" # OPTIONAL. Excludes `sample` - include: ["seedbank.field_*"] max_rows_scanned: 1000000000 # OPTIONAL; also valid on a connection or in `defaults`| Key | Type | Default | Meaning |
|---|---|---|---|
include | list of glob | ["*"] | Tables this rule governs. Same glob grammar as the connection's include |
exclude | list of glob | [] | Removed from this rule's own match |
min_rows | int | absent | Rule applies only to tables of at least this many rows. Positive integers only |
sample | float | absent | Fraction of the table to read, per SPEC 2.2.8. Never alongside filter |
filter | string | absent | One SQL predicate, applied verbatim. Never alongside sample |
max_rows_scanned | int ≥ 1 | absent | A row-count ceiling, resolved into a sample fraction against the table's catalog estimate. See below |
statistics | map | {} | Any subset of the statistics keys, merged onto the connection's |
max_age_days | int ≥ 0 | absent | Freshness threshold for these tables. Same bound and same meaning of 0 as the connection key |
- Every matching rule applies, in declaration order, and later ones win. Rules from
defaultsare walked before the connection's own. Each ofsample,filterandmax_age_daysis last-wins against another rule setting the same key;statisticsmerges key by key, so two rules can set different keys and both hold.sampleandfilterare the one pair that does not resolve this way — see below. - A rule that matches a table but sets nothing is rejected at load rather than ignored — that shape is almost always a mis-nested key, which the ignore-unknown-keys policy would swallow.
- The mirror shape is rejected for the same reason: a rule whose
includeis an empty list matches no table and would never fire. Omittingincludeis the way to match everything;include: []matches nothing, so it is refused rather than silently ignored. An emptyexcludeis fine — it removes nothing. sample,filterandmin_rowsare read only inside a rule. Placing any of them directly on a connection or indefaultsis rejected by name rather than dropped, because it is the predictable way to mis-migrate a config and the ignore-unknown-keys policy would otherwise swallow it. That deny list is exactlyscope,sample,filterandmin_rows; every other unrecognized key is still ignored silently, andstatisticsandmax_age_daysare read at connection level as normal.- Two matching filters never combine: the later predicate replaces the earlier one, so no query runs a condition neither rule authored.
- A table is narrowed by a predicate or by a fraction, never both. A rule carrying
sampleandfiltertogether is rejected at load; rules that each carry one and match the same table are rejected when that table resolves, naming both of them. Neither key silently clears the other, because a config that reads as narrowing two ways must not quietly do one. To sample a slice, widen the predicate until it describes the rows you want. inferred.looks_likehonors bothfilterandsample: it must not describe rows outside the artifact, and its own draw composes with the sample fraction rather than replacing it - so honouringsamplecosts no extra rows. Composition is population-level only on MySQL and Snowflake, which take no seed on this sub-draw; Postgres and duckdb both cohere row for row.- A sampled table reads one row set within a run, and on PostgreSQL and duckdb the same one
next run. One table's profile issues many statements against the same narrowed source, so
an unseeded fraction would describe different rows per field on a table nobody wrote to.
What prevents that is the materialized copy, not the seed. MySQL, Snowflake and Databricks
also seed the fraction from the table's own name, but only PostgreSQL's and duckdb's
TABLESAMPLE ... REPEATABLEdocument a stable draw across runs (duckdb's measured directly, reproducing regardless of thread count): MySQL's seeded predicate holds only while scan order does, and Snowflake does not document two evaluations of one seeded expression reading the same rows — which is precisely why the copy exists, and why the engine refuses asamplescope with no materialized copy on both rather than publish a file whose fields disagree with each other (Adapter.SAMPLE_FALLBACK_COHERENT = False; see the per-adapter notes below). ClickHouse, Redshift and BigQuery take no seed on the fraction at all, so each refuses asamplescope with no materialized copy outright, for the same reason. The exception is the extra drawinferred.looks_liketakes on top of that row set, which takes no seed on MySQL or Snowflake — so on those two the shape claim agrees with the rest of the profile at the population level rather than row for row. duckdb seeds this draw too, the one axis where it reproduces more than Snowflake's own substrate can. - ClickHouse never reads a sampled table without the copy. A declared
SAMPLE BYkey does not meanSAMPLEnarrows anything — measured directly: a monotonic key reads the whole table at every requested fraction, and a table with no key at all raises rather than falling back to a full scan. Every other adapter here degrades to an unmaterialized, best-effort read when the copy is unavailable; ClickHouse fails that one table instead, namingmaterialize_samplein the message, because there is no unmaterialized reading of it that is honest. This is the one adapter wherematerialize_sample: falseis not a performance trade — it decides whether asample-scoped table is profiled at all. - Redshift never reads a sampled table without the copy either. There is no seeded sampling
clause at all:
WHERE RANDOM() < pis the only narrowing, and its result is not deterministic across a distributed cluster's compute slices, even within one run. Asamplescope with no materialized copy fails that table rather than mixing rows drawn by independent evaluations of the same predicate — the same refusal ClickHouse makes, for a different reason. - Databricks degrades instead of refusing.
TABLESAMPLE ... REPEATABLEis a genuinely reproducible draw on this engine, so a table the copy cannot be taken on still reads a stable fraction directly, with a warning, rather than failing outright. - BigQuery never reads a sampled table without the copy either.
TABLESAMPLE SYSTEMhas no seed clause in the grammar at all, and each execution processes an independently computed sample, so results are not guaranteed to match from one execution to the next. Asamplescope with no materialized copy fails that table rather than publish astatistics.yamlwhose fields describe different rows - the same refusal ClickHouse and Redshift make, for a third reason. The copy itself also differs from every other adapter's: BigQuery has no session-scoped temporary table, so the draw is copied into an ordinary table in the profiled dataset, created with its ownexpiration_timestamp(a few hours out) rather than the run's own cleanup being the only thing standing between it and an orphaned copy - a killed process or a revoked delete privilege leaves it to expire on its own instead of billing as storage indefinitely.CREATE OR REPLACEalso means an orphan from an earlier run never wedges a later one withAlready Exists. The scratch table's own name is excluded fromlist_tables, so it is never profiled as a table of its own. min_rowsselects by size, and both conditions must hold. A rule carrying it governs a table only when the name matchers admit it and it is at least that large, somin_rows: 500000000withsample: 0.01samples the tables that are too big to scan without naming them one by one. It gates whatever the rule sets —sample,filter,statisticsandmax_age_daysalike — because it sits on the matcher rather than on one key. It is a matcher, not a setting: a rule carryingmin_rowsand nothing else selects a set of tables and does nothing to them, and is rejected at load like any other rule that overrides nothing.- A size condition needs a database, so the offline commands cannot apply one.
checkandlistnever connect, so there is no row count formin_rowsto be tested against and a rule carrying it is left unapplied — the same answer the engine gives a table whose catalog holds no estimate. Where that matters ismax_age_days: a size-gated threshold governs whatgeneratedoes, and a print that records no threshold of its own is judged offline against the rules that match by name alone. Both commands say so on stderr, naming the tables, rather than leaving the number unexplained. - The size is a catalog estimate, so the bar is fuzzy near the boundary. Postgres reports
a planner statistic that lags writes and is unset until the table is
ANALYZEd; MySQL's InnoDBtable_rowsis approximate by design. A table sitting close to the threshold may fall either side of it between runs. When the catalog has no number at all the rule does not apply — sampling degrades the artifact, so an unknown size takes the un-narrowed path, and the run says so on stderr rather than deciding silently. - A config with neither
min_rowsnormax_rows_scannedanywhere costs nothing. No estimate is fetched, and the run issues no statement on a table's account beyond the ones it profiles it with. Either key, at any level, turns the pre-flight on for a table — a ceiling needs the estimate to derive its fraction. It has no effect on a plain view, whatever this is set to: a view is never queried, so no estimate is fetched for one and no size rule can govern it. - A narrowed run takes the table's
row_countfrom the catalog rather than counting it, sorow_count_methodreportsapproximate, and the emittedstatistics.yamlcarries thescopeblock namingrows_scannedwith every ratio computed against it. A table crossing the bar therefore changes the shape of its artifact with no schema change, anddiffshows that — there is no hysteresis. dbprint checkjudges each print against the threshold its own manifest entry records — the one the run that wrote it skipped it against. An entry recording none falls back to the rules, and offline that is the rules matching by name (see the size-condition note above). An explicit--max-ageoverrides every table's threshold directly and reads no rule to find one — but the rules are still read to catch the structural error below, since that is a property of the configuration independent of freshness.- A cascade that resolves one table to both a
filterand asampleis refused. Offline the refusal is contained to that one table:checkreports it as a check that did not run and still judges every other table in the connection;listreports the cause and skips that connection, since its output is aggregate counts and a table with no threshold has no bucket. Neither command aborts, and neither loses a connection it had already summarised. Undercheck's default (no--max-age), the refusal costs the connection its exit code —1. Under an explicit--max-agethe refusal is still reported, on stderr and in the machine envelope, but does not move the exit: the override already governs every table's freshness, so a scope error the override does not depend on cannot fail a run it decided.
The predicate is interpolated, not bound. It is your SQL, passed verbatim into every statistics query for that table; dbprint never parses, rewrites or validates it, because SPEC 2.2.8 requires it recorded as written. Treat
.dbprint.yamlas carrying the same trust as the credentials file — it already names the connection whose credentials a run uses.
max_rows_scanned
A row-count ceiling states the cost an operator can afford directly, in rows, rather than as a
fraction — the engine derives the fraction from a catalog estimate, fetched for this key as it is
for min_rows.
- A ceiling is a different policy from a fraction, not another way to spell one.
samplereads a fixed share regardless of table size; a ceiling caps the rows the draw returns, so a table under it is read whole and every table over it yields the same number of rows regardless of how far over. What a ceiling bounds is the downstream work — the rows aggregated,rows_scanned, and on Snowflake the warehouse time, sinceSAMPLE SYSTEMprunes at block level. It does not bound what leaves the disk on PostgreSQL or MySQL:TABLESAMPLE BERNOULLItests rows individually and aRAND() < ppredicate is unindexable, so both scan the whole table however small the fraction. Migrating amin_rows/sampleladder built to approximate a cost curve to onemax_rows_scannedvalue changes what gets read at the low end — a table just over the ceiling is now read whole rather than sampled — and that is the intended difference, not a bug. - Unlike
sample,filterandmin_rows, a ceiling is legal at connection anddefaultslevel as well as inside a rule. It cascades exactly likemax_age_days: the connection's own value wins overdefaults, and a rule's value — at whatever level it is declared — overrides both for the tables it names. It is deliberately absent from the deny list that rejects the other three outside a rule, because a project-wide budget is the point of the feature. - A resolved fraction of exactly
1.0is not a sample. When the ceiling is at or above a table's catalog estimate, the table is read whole: noscopeblock, norow_count_method: approximate, androw_countis counted rather than estimated.sample: 1.0never reaches the artifact through this path. - The resolved fraction snaps down to a geometric grid, 10% per step. A pure function of
the ceiling and the estimate alone — no run-to-run state — so a catalog estimate that drifted
by a few percent (
ANALYZEnoise, InnoDB's approximatetable_rows) resolves to the same fraction it did last run, anddiffreports nothing. A table that genuinely changed size by 10% or more crosses at least one grid step, anddiffshows the statistics move. Snapping down, never up, keeps the ceiling a true ceiling:rows_scannednever exceedsmax_rows_scannedbecause of the grid. - A ceiling and an explicit
samplecascade on the same timeline. Whichever was set later — by declaration order, connection value first, thendefaultsrules, then the connection's own — wins outright; the earlier one is discarded rather than blended, and a ceiling a latersampleoverrides is never converted to a fraction. One rule setting both prefers its ownsample. - A ceiling meeting a
filteryields to it, with a warning, rather than being refused.sampleandfilterare mutually exclusive and a cascade resolving to both is a load error (above) — a ceiling is not a third narrowing directive competing for that slot, since a connection-wide ceiling would otherwise collide with every filtered table on every run. A predicate already bounds cost, so the ceiling stands down andgeneratesays so on stderr, naming the table. - A ceiling gates nothing offline.
checkandlistnever connect, so a ceiling never resolves there — unlikemin_rows, it does not affect what an offline command reads, because it governs only whatgeneratescans.
redact
An ordered list. Each entry names the columns it covers and what to do with their cell values; everything measured about those columns is left alone.
connections: production: redact: - columns: ["*.collector.email", "*.curator.*_name"] # selector globs over <fqn>.<column> with: mask # OPTIONAL; mask | drop | hash - sensitivity: [personal_name, postal_address] # matches inferred.sensitivity with: drop - looks_like: [email] # matches inferred.looks_like with: hash| Key | Type | Default | Meaning |
|---|---|---|---|
columns | list of glob | [] | Globs over the qualified <fqn>.<column> |
sensitivity | list of enum | [] | Covers columns whose inferred.sensitivity is listed. One of personal_name, postal_address, geolocation, date_of_birth, national_id, financial_account, credential, health, demographic, employment, contact, online_identifier |
looks_like | list of enum | [] | Covers columns whose inferred.looks_like is listed. One of the SPEC 4.1 patterns |
with | enum | mask | mask | drop | hash |
- A rule covers a column matching ANY of its three criteria. Rules apply in declaration
order and the last matching one decides the primitive, the same resolution
rulesuses. A rule naming none of the three is rejected at load — it would cover everything, which is never what writing one means. sensitivityandlooks_likeare closed vocabularies, checked at load. A value outside the set is rejected by name rather than stored, because a rule targeting a category that does not exist covers nothing and produces a print that reads as redacted.columnsis an open glob and is not checked: a pattern matching no table today may match one tomorrow.redactcascades fromdefaultsand concatenates, the same wayrulesdoes: thedefaultsentries are walked first, then the connection's own, and the last matching entry decides the primitive. So a connection can change what a project-wide rule applies to a column —masktohash— but cannot lift the coverage, because no primitive means "not redacted". A connection that must stay unredacted is one whose rule does not belong indefaults.- Counts do not change; cell values and two derived day counts do.
null_count,null_rate,cardinality,cardinality_ratio, the value counts,values_coverageanddistributionare identical to an unredacted run.rangebounds andpercentilesare cell values and receive the same primitive; underdropthey are omitted entirely, along withunrepresentable. The two exceptions are derived rather than measured:range.span_daysandfreshness.max_age_daysare floored to the nearest 90 days under every primitive,dropincluded, since an exact age narrows the values it was computed from. - The column declares it with a
redactedmarker naming the primitive, so a consumer can tell a measurement from a substitution. Acheckpredicate overaccepted_values,range,percentilesorfreshness.max_age_dayson a redacted column is refused rather than evaluated against placeholders or against a coarsened figure. A column a rule covers but that publishes no cell value at all carries no marker, because nothing was withheld from it — whether that is because its classification never carries one (json,unsupported) or because atextcolumn detected as prose published no value list for an unrelated reason (SPEC 2.2.3's enumeration exemption). The absence of a marker means the emitted values are the real ones, which stays true either way. - A detected category with no rule covering it is reported, not silenced.
dbprint checkcarriesprivacy.unredacted-sensitive(a warning; SPEC §4.4.2) for a column that names its owninferred.sensitivityand still publishes a cell value nothing withheld. Writing the rule above is what clears it — the check reads the committed print, not this file. hashrequires a salt and is rejected without one. An unsalted digest of an email is reversible by dictionary attack in minutes. The salt lives with the credentials —redaction_saltin~/.dbprint/connections.yaml, orDBPRINT_<CONN>_REDACTION_SALT— never in.dbprint.yaml, which is committed. Keep it stable per project or every redacted column churns on every diff. A value that is empty or only whitespace is not a salt:DBPRINT_<CONN>_REDACTION_SALT=is what a shell produces when a secret did not resolve, and it is refused exactly as an absent one is.- Detection is unaffected.
looks_likeandsensitivityrun over sampled values that are never written, so a hashed email column still reportslooks_like: email. The shape claim describes the column, not the emitted literals.
diff
Presentation thresholds for dbprint diff's human output. Machine output (--format json
/ yaml) is always unfiltered.
diff: stat_change_threshold: cardinality_ratio: 0.02 percentile_pct: 0.05 values_coverage: 0.05 default: 0.01 # every statistic without its own entryEach threshold is a fraction in [0, 1] and is checked when the config loads: a value that is
not a number, or one outside that range, is refused with the file, the connection and the key.
The four keys above are the whole accepted set, and a key outside it is refused rather than
ignored — an unread key would leave default governing the statistic its author meant to
configure, which is indistinguishable from a working config.
--threshold overrides every per-stat value for one run and is parsed by the CLI, so it is not
subject to this check.
assertions
Data-quality checks evaluated by dbprint check. The block is stored unparsed by the config
loader and interpreted by the assertion layer; its grammar, severities and exit codes are
specified in ASSERTIONS.md.
--project locators
Every command except init accepts --project, pointing it at a project without a cd; init
scaffolds in the current directory only. Local by default: a
directory whose direct child is .dbprint.yaml, or that file itself - never an upward walk,
never a downward scan.
--project also accepts a git address, so a project committed to a repository can be read
without cloning it by hand first:
| Form | Resolves to |
|---|---|
https://github.com/<owner>/<repo> | .dbprint.yaml at the repository root, default branch |
git@github.com:<owner>/<repo>.git | Same, over SSH |
https://github.com/<owner>/<repo>/blob/<ref>/<path>/ | <path>/.dbprint.yaml at <ref> |
https://github.com/<owner>/<repo>/blob/<ref>/<path>/.dbprint.yaml | The same file, named directly |
<git-url>#<ref>:<subpath> | Explicit form - any git URL, any ref, any subpath |
GitLab (/-/blob/<ref>/<path>) and Bitbucket (/src/<ref>/<path>) web URLs parse the same way.
A bare remote always means the repository root at its default branch - a .dbprint.yaml nested
under one is never discovered from the bare form.
~/.dbprint/connections.yaml
Keyed by connection name, matching .dbprint.yaml. Never commit it.
production: host: db.internal port: 5432 database: arboretum user: dbprint_ro password: ...Required keys per adapter
| Adapter | Required | Optional |
|---|---|---|
| PostgreSQL | host, port, database, user, password | redaction_salt |
| MySQL | host, port, database, user, password | redaction_salt |
| Snowflake | account, user, warehouse, database, role | password, private_key_file, private_key_file_pwd, schema, redaction_salt |
| duckdb | database (a file path, or :memory:) | read_only, redaction_salt |
| ClickHouse | host, database | port (default 8123), user (default default), password, redaction_salt |
| Redshift | host, database, user, password | port (default 5439), redaction_salt |
| Databricks | server_hostname, http_path, access_token, catalog | redaction_salt |
| BigQuery | project, dataset | credentials_file, redaction_salt |
Snowflake takes exactly one of password or private_key_file; supplying both, or
neither, is an error. private_key_file_pwd decrypts an encrypted key.
BigQuery has no password key at all: credentials resolve through Application Default
Credentials, the google-cloud-bigquery client's own mechanism, unless credentials_file
names a service account key explicitly.
Every unresolved required key is collected and reported in one error rather than one at a time.
Resolution order
Per key, first hit wins:
DBPRINT_<CONN>_<KEY>environment variable — connection name and key upper-cased~/.dbprint/connections.yamlDBPRINT_<CONN>_<KEY>in the project's.env
So DBPRINT_PRODUCTION_PASSWORD overrides the file entry for production, and a .env
entry serves as the fallback a checkout can carry without a user-level file.
A variable carrying no value is skipped, not used. DBPRINT_PRODUCTION_HOST= — set to
empty, or to whitespace — is what a shell produces when a secret did not resolve, so
resolution continues to the next source rather than handing an empty credential to the
adapter. A .env entry with nothing after the = is skipped the same way, as is a bare key
with no = at all. An empty value in ~/.dbprint/connections.yaml (host: "") is
deliberate and is used as written.
password is the exception, because its empty value is a credential. A cluster
configured for trust authentication is reached with no password at all, and
DBPRINT_<CONN>_PASSWORD= is how a runner says so. An empty password is therefore taken as
given, from any source.