When is the column transformed: on read, on write, or in a separate job?

On the way to the Delta write. The rule runs in memory during the read path of the insert, so the value written to the Parquet file is already protected.

Which transforms are available and when do you pick each?

Five transforms: keyed hashing for identifiers, deterministic authenticated encryption for reversible protection, redaction for structured fields, generalisation for k-anonymity, and tokenisation for stable surrogate keys.

Where do the cryptographic keys live?

In your key store. DeltaForge supports cloud secret managers (AWS KMS, Azure Key Vault, Google Cloud KMS) and on-prem vaults.

Is this GDPR pseudonymisation under Article 4(5)?

Yes. The transforms align with GDPR Article 4(5) and Recital 75 guidance, using industry-standard hashing, authenticated encryption, and key derivation.

Are the rules auditable and reversible?

Yes. Rules are SQL DDL, stored in the catalog, and versioned with the rest of the schema. Every change is captured in the audit log.

Data Protection

Pseudonymisation before the write

Configure a column once. DeltaForge transforms it in memory during the read path, so the value written to the Delta table is already protected. No separate masking pipeline. No exposure window.

Hashing, encryption, redaction, generalisation, and tokenisation

Configure per column with SQL, no separate scrubbing job

Standard cryptography with your own key store

Five ways to transform a value

Choose by what the column means and who needs to see it

One-way hash

For identifiers that must never be recovered. Same input always lands on the same hash, so joins still work.

Reversible encryption

For values you may need to recover under a key custody process. Deterministic, so encrypted columns remain joinable.

Redaction

Display-style masking for emails, phones, cards, and other structured fields. Keeps the shape, hides the value.

Generalisation

Reduces precision for k-anonymity: dates to coarser buckets, ages to ranges, postal codes to prefixes.

Tokenisation

Replaces values with stable tokens within a configurable scope. Linkability is a knob, not a default.

Configured in SQL, not a sidecar

A single command attaches a transform to a column

Declarative

Rules are SQL DDL, stored in the catalog, versioned with the rest of your schema. No external config file to drift.

Enforced on write

The raw value never reaches the Delta table. The transform runs in memory before the write commits.

Audited and reversible

Enable, disable, and inspect rules at any time. Every change is captured in the audit log.

Standard cryptography, your key store

Battle-tested primitives, no bespoke schemes

Algorithms you already trust

Industry-standard hashing, authenticated encryption, and key derivation. No custom or proprietary cryptography.

Keys live where you want

Bring your own key store. Cloud secret managers and on-prem vaults are supported. Keys never have to leave them.