Feeding it transcripts
run, roundtrip, and inspect take a file, a batch file, a directory or .zip of files, or
- for standard input. The container format is chosen by extension, never guessed from content;
--format overrides it (and is required for stdin). Every resolution the SDK makes below is an
exact match against a declared list: the preflight prints what it resolved, and a flag overrides
it.
Containers
A directory takes the files directly inside it, or every file below it with
--recursive.
Files no format claims are not dropped quietly: inspect lists them and run warns and writes
them to the manifest under skipped.
PDF text comes out as the file stored it: two-column pages interleave, and hyphenated line ends
stay split. A scanned PDF holds no text and is refused as empty. Run
inspect first to see the
char count before a model call. .xlsx, HL7, FHIR, and audio are not read; save a sheet as CSV.JSON shapes
A.json file or a JSON Lines line is matched against the shape registry, richest first, and
the first shape whose paths resolve wins. inspect names the shape it used.
The vendor rows were written from each vendor’s public documentation and are exercised by a
synthetic fixture each. Until a real export of that vendor has been through a row,
inspect
prints (shape unverified: ...) beside it; a wrong guess about a vendor’s keys fails to resolve
rather than loading the wrong field.
Any shape can also carry ref (or caseId, case_id, id), your own identifier for the
record, and keep, a list of values that must survive. A number as a speaker renders as
Speaker 0; rename with --speaker-label 0=Doctor --speaker-label 1=Patient (a label that
matches no speaker is a warning). WebVTT voices (<v Doctor>) take the same labels. Turns
render one per line as Speaker: text.
A shape nobody registered. Name the keys as paths (a.b[0].c, [] for every item):
--text-key is the only required one; omit --records-key when the value holds one text.
--shape NAME forces a registered shape instead, and errors, naming the failing path, when it
does not fit. The two cannot be combined.
Files are read as UTF-8 with a byte order mark stripped. A UTF-16 or UTF-32 mark selects that
encoding, because a mark is a fact, not a guess. Any other encoding is an error that names the
byte and the fix: pass
--encoding cp1252 (or the export’s real encoding), or convert with
iconv -f cp1252 -t utf-8. An empty record, an utterance without text, and a JSON value that fits
no shape are refused at load, before any model is built.CSV and TSV columns
A header with exactly one oftext, transcript, utterance, content, message needs no
flag; the same holds for a ref column (ref, id, case_id, caseId) and a keep column
(keep, values separated by ;). Several candidates in one header is an error naming
--text-column, --ref-column, or --keep-column, because picking one would be a guess.
The keep list
Stage 0 detects entity types, not intent. It cannot tell the appointment date you booked against from the patient’s birthday, so by default it replaces both. Name the values that must survive:--keep repeats; --keep-file holds one value per line; a record’s own keep list merges with
both. Matching is case-insensitive and whole-token, every occurrence: keeping none does not
protect the none inside Noneman. A detection touching a protected value is dropped whole,
never trimmed. A blank value is an error. A value that matches nothing in the transcript is a
warning, because it is usually a typo.
The keep list reaches the two passes that redact (tokenization and beat neutralization) and not
the re-detection backstop, which must keep reporting every identifier it finds.
Check first
max_transcript_chars, names the
records that would be refused at load and the keep values that match nothing, reports the JSON
shape of each record and the speakers seen, lists the files it skipped, and with --stage0
counts entity types from the local detectors. --survey lists what those detectors left in place
that looks like an identifier, by shape and never by value (see
Your own identifier formats). Both build the detector stack from the
config, the default [presidio, gliner-pii-open] when it names none, so the first --stage0 can
download the model. It constructs no language-model client and sends nothing; --json returns the
same report as data. Exit 2 when anything would be refused.
Batches
ref
or its position, and an owner-only manifest.json mapping each ref to its source, artifact,
token, refusal reason, error, and run folder, plus the files that were skipped. Exit 0 when every
record crossed, 2 when any refused, 1 when any errored.
Run folders under runs/ are named by a hash of the source, not its filename, so a file named
after a patient never reaches stderr or a shell log. The manifest maps the hash back.
Standard input works for one transcript at the end of a pipe: