HyperPKPD version 0.0.0.12

Features at a Glance

A high-level tour. Every feature has a full description in README.md and CONFIGS.md.

1. Four masking levels

You choose how much to hide; each level includes the one before it.

Level Name What it masks
1 light Identifiers: ID, subject (USUBJID), study, site and user columns are recoded.
2 clinical (default) Level 1, plus treatment, sex, race and country labels, dates shifted by a constant, and free text redacted.
3 covariate Level 2, plus covariates (WT, AGE, HT, CRCL, ...) multiplied or divided by a hidden random factor per column.
4 maximum Level 3, but every value gets its own factor (the column's factor plus or minus 15 %), free-text columns are dropped, and subjects are re-ordered.

2. The PK/PD structure is protected

TIME, TAD, DV, AMT, RATE, EVID, MDV, CMT, ADDL, II, SS, DVID, BLQ and similar columns are never altered, and rows of a subject keep their time order. Dosing events and observations stay byte-for-byte as they were, unless you explicitly drop those columns.

3. Identifiers and labels

  • Recoded consistently: the same subject always gets the same new code.
  • Three code styles: sequential (1, 2, 3), padded (0001, 0002, ... for IDs; prefixed labels such as SUBJ001 for the other roles) and hashed.
  • Several columns per role are supported (for example STU, STUDYID and STUDY); columns that pair one to one share the same masked label.

4. Dates

All dates move by one constant number of days, so intervals and order are preserved. The default shift is a random decimal between 1 and 100 days; you can fix it. Dates in many formats are recognised, including YYYYMMDD, ISO date-times and DD-MMM-YYYY. Dates that cannot be read are left alone and reported, never silently wrong.

5. Numeric scaling (levels 3 and 4)

Covariates and any columns you list are multiplied or divided by a random factor (1 to 20 by default). At level 3 the factor is one per column, so ratios and ranks within the column are preserved; at level 4 it varies from value to value, which also blurs ratios.

6. Column renaming

Optionally replace column names with neutral ones (COL07) or names of your choice, for columns that themselves give something away.

7. Prompt pack for the LLM: llm.txt

A ready-to-paste file with an analysis prompt, a NONMEM control-stream prompt and a QC prompt, each written for the dataset you just masked (its columns, endpoints, which columns were rescaled and why that is not a data error).

8. Token estimates: tokens.txt

How many tokens the masked file will use in GPT, Claude, Gemini, Llama, Qwen and Mistral families, so you know whether it fits a model's context window.

9. The private key and reverse

Every run writes a key (keymap.txt, plus keymap_columns.txt at level 4) with the original-to-masked mapping. The reverse command uses it to restore a masked dataset even when its rows or columns have been re-ordered and extra columns were added. It undoes renaming, scaling (exactly), the date shift and the recoding, and reports what it could not restore (redacted free text and dropped columns).

10. run2: several datasets, one master dataset

For CDISC ADaM work: mask ADSL, ADEX, ADPC, ADLB and others together. They share identifier codes, one date shift and one factor per scaled column, so an LLM can join them into a NONMEM-style master dataset. reverse then puts the real values back into that master dataset, column by column, with optional --map rules for columns the LLM renamed or merged (for example --map ID=USUBJID --map DV=AVAL).

11. Reproducible and safe by design

  • Same input, options and seed give a byte-identical masked file; its SHA-256 is recorded.
  • A run never writes into a folder that already has files unless you add --overwrite.
  • Rows with the wrong number of fields are not silently repaired; the run warns.
  • A flushed log (obfuscator.log) is kept even if the program crashes.
  • Warnings for things that could leak or break the data: unshifted dates, e-mail-like text left in the output, removed core NONMEM columns, missing TIME/DV/ID, unknown column names.

12. Usable your way

  • Plain command line, or an options file (key = value text) for repeatable runs; flags override the file.
  • CSV, TSV, semicolon-delimited and whitespace-delimited NONMEM files, with or without a header.
  • Coloured terminal messages (green success, orange warning, red error); --no-color turns them off.
  • Ready-made example configs and example datasets, including QC datasets with planted defects and ADaM-style files.