Verification

De-identification is not a setting you turn on once. Every claim below is a lower bound from planted or synthetic truth, never a proof about your un-planted real data, and each is labelled as such in the code and in the report it produces.

Per-batch tripwires

Tool	What it does	Depends on a model?
`run_canary(text, deps, config)`	plants known synthetic identifiers and their reformatted variants, runs the engine, asserts none survive into the artifact	the engine runs, but the check is a string oracle
`check_artifact(artifact, canaries)`	the same oracle over an artifact you already hold	no
`redetect_artifact(artifact, detector)`	re-runs the Stage-0 detector over the beat text; any hit is a regression	no (detector only)
`l_diversity(artifacts, min_distinct=2)`	each `{age band, sex}` class across a batch must carry at least `min_distinct` distinct presenting conditions	no
`export_defensibility_report(...)`	bundles the audit counts, the policy parameters, and the three results above into one JSON pack a compliance team can sign; each claim states its scope	no

The default canaries are obviously synthetic: Liam Whitaker, the NHI ZBX4417, 021 555 0182 with its collapsed form, 14 March 1986 with 14/03/1986, and Twizel. Re-detection reuses the gate’s detector, so it is detector liveness over the output, not an independent gate.

from tacit_deid import export_defensibility_report, l_diversity, redetect_artifact, run_canary

_, canary = run_canary(sample_text, deps, config)
redetects = [redetect_artifact(a, deps.detector) for a in artifacts]
report = export_defensibility_report(
    batch_id="2026-09-13-batch-01", artifacts=artifacts, config=config,
    canary=canary, redetects=redetects, l_div=l_diversity(artifacts),
)

Qualify a model first

qualify_model(deps, config) runs three checks against the model wired into deps and returns a verdict per check: a clean synthetic transcript must cross with at least one beat (a model that refuses everything is safe but useless); every planted canary must be stripped; and a planted singling phrase with no named entity (a sole role at a site) must be caught by the judge. Approve a model only when all three pass, and re-run when the model or its configuration changes. A mock model passes trivially, so run it with real in-boundary credentials.

The simulated client

Tacit runs the SDK the way a customer would, inside a boundary it can observe, and sends only artifacts into a real Tacit org. One run yields three layers of evidence: The egress proxy is mitmproxy with an addon that logs every request and refuses any host off the allowlist; the client is routed through it. The report asserts that the only hosts contacted are the in-boundary model and Tacit, and that no planted literal, raw or reformatted, appears in any body sent to Tacit. A run without the proxy reports egress: FAIL (not observed), because an unobserved wire is not evidence. The Docker variant puts the client on a network whose only exit is the proxy, so the boundary is enforced, not just observed. The corpus is planted truth, so every number below is a lower bound on an un-planted corpus.

What the measurements say today

Question	Answer	Source
Does anything planted cross?	0 of 105 canary values and 0 of 180 planted values survived across 21 synthetic consultation rows; on a 20-record synthetic health corpus, one patient ID crossed in 2 of 4 Presidio-only artifacts and 0 of 4 with GLiNER-PII stacked	simulated client runs of 9 and 10 September 2026
How much does the default model add?	Presidio alone found 77.4% of the planted identifiers in 294 synthetic health records and 44.9% / 43.9% in 20 synthetic doctor and patient dialogs (written / speech-recognized); with `gliner-pii-open`, the default, 97.0% and 65.2% / 64.2%	detector studies of 24 and 27 September 2026
What does the model get wrong?	across 27 transcripts it added 47 spans that match no planted value: about 20 look like real identifiers the corpus did not label (occupations, spoken ages) and about 8 change a clinical fact (“smoker”, a drug name); the guards removed clinician roles, departments, pronouns, and words tagged as numbers	fidelity gate of 27 September 2026
What does it cost?	about 0.8 s per record on a laptop CPU with `gliner-pii-open`, against 0.16 s for Presidio alone; a 0.7 GB download on first use	detector study of 24 September 2026
Why do cases refuse?	a mid-sized judge proposed strings the text does not hold (normalised numbers, a hyphen for a space) and the loop hit its cap; the matcher now accepts those forms, and refusals fell from 9 to 6 of 21 case runs for the digits fix and from 5 to 3 for the hyphen fix (overlapping per run, so direction only); a frontier judge crossed all seven corpus cases	simulated client runs of 9, 23, and 24 September 2026

Each row is an observation over a small number of runs.

Getting started

How it works

Operating it

Reference

Verification

Verification

Per-batch tripwires

Qualify a model first

The simulated client

What the measurements say today

​Verification

​Per-batch tripwires

​Qualify a model first

​The simulated client

​What the measurements say today

Verification

Per-batch tripwires

Qualify a model first

The simulated client

What the measurements say today