HyperPKPD version 0.0.0.12

IDs and Seeds

How identifiers are recoded (--id-style) and what the random seed (--seed) controls. Two options, one page, because they interact: the seed changes some codes and not others.

1. Identifier recoding

Every identifier column (ID, subject, study, site, user) and every label column (treatment, sex, race, country) is recoded with a code book: each distinct original value gets one code, and the same value always gets the same code within a run. The book is written to keymap.txt so that reverse can turn the codes back into the originals.

data-obfuscator run --input pk_study.csv --output ./masked --level 1 --seed 2026 --id-style padded

2. The three ID styles

The same five subjects under sequential, padded and hashed ID styles
Figure 2.6.1. --id-style sequential, padded and hashed, on the same subjects (level 1, seed 2026).
Style ID column Other recoded columns Use it when
sequential (default) 1, 2, 3, ... in order of first appearance SUBJ001, STU001, SITE001, USER001; treatments ARM_A, ARM_B, ...; races CAT001; countries C001 you want short, readable codes
padded 0001, 0002, ... (same width) same as sequential a fixed-width ID is needed (a NONMEM $INPUT that expects it, for example)
hashed a 5-digit number from a seeded SHA-256 of the value (10000 to 99999) prefix plus 8 hexadecimal characters: SUBJEE873275, SITE42FC24A4 the order of codes must not reveal the order of the file

Two consequences worth knowing:

  • Sequential and padded codes reveal the order in which subjects first appear in the file. If the file is sorted by enrolment date, subject 1 is the first enrolled. Use hashed when that matters.
  • Hashed codes depend on the seed, so they differ from run to run unless you fix it. Two subjects that share a site get the same site code (look at SITE42FC24A4 twice in the figure).

The same rules apply to every dataset of a run2, with one shared table per identifier kind (page Masking Several Datasets: run2).

3. What the seed controls

--seed N (default 42) is the only source of randomness in a run.

What the seed controls: date shift, scaling factors, hashed codes; not the sequential and padded codes
Figure 2.6.2. What the seed controls, and what it does not.
Part of the masking Depends on the seed?
the date shift (when drawn from a window) yes
the scaling factors (level 3 and 4) yes
hashed codes (--id-style hashed) yes
sequential and padded codes no: order of first appearance
a fixed date shift (--date-shift-days 23.6) no: you gave the value

The same data under three seeds

Level 3 with --id-style hashed, seeds 1, 99 and 2026:

A table of the date shift, the WT and AGE factors and the hashed codes for seeds 1, 99 and 2026
Figure 2.6.3. Every seed gives another shift, other factors and other hashed codes.

Read across a row: with seed 1 the dates move by 62.3 days, WT is multiplied by 3.907 and AGE divided by 9.188; with seed 2026 the shift is 50.8 days, WT is multiplied by 12.909 and AGE divided by 5.671. The hashed ID of the first subject changes as well (17175, 44574, 13102).

4. Reproducible runs

Same input file + same options + same seed + same program version = a byte-identical dataset. The proof is the SHA-256 fingerprint printed at the end of each run and written in summary.txt.

Three runs: seed 1, seed 99, and seed 99 again; the last two have the same SHA-256
Figure 2.6.4. Compare the SHA-256 of two runs to check that they are identical.

Try it: run the same command twice into two new folders and compare the files.

data-obfuscator run --input pk_study.csv --output ./run_a --level 2 --seed 99
data-obfuscator run --input pk_study.csv --output ./run_b --level 2 --seed 99

The two obfuscated.csv files are identical (and so are the SHA-256 lines in the two logs). With --seed 1 instead of 99 the date shift is 62.3 days and the file is different.

5. Choosing a seed

  • Choose your own number (--seed 20261012). The default 42 is known to everybody, which makes the date shift and the factors guessable for anyone who knows the data were masked with defaults.
  • Write it down with the key: the seed is recorded in summary.txt and config.used.txt, and the key (keymap.txt) holds the actual shift and factors, so a lost seed does not stop a restore.
  • A different masking on every run: in an options file write seed = auto; the program draws a seed and records it in summary.txt. Keep that file (privately) if you ever need to repeat the run.
  • Changing the program version can change the result for the same seed (the version is part of the reproducibility statement), so keep the archive of the version you used for work you must repeat.

Next

Dates and Free Text describes the date shift in detail, including the formats it understands.