Supported Data TypesToken Types

Token Types

How token types control the shape and structure of masked output — choosing the right token type for your data and downstream requirements.

Token types control how masked values are generated, independent of entity or format. The same input can produce different output shapes depending on which token_name you specify.

Available token types

Token TypeIntended UseExample InputExample Output
Default TokenGeneral textGeorgeFreTdf
Text TokenEmails, usernameskumar@ss.fss.comLEFQS@cTdqGVB2Ay
Numeric TokenNumeric-like values(408)-426-9989(07432)-29915-56713
Special TokenRandom-length outputjohn.doe@example.comxuLDKEM4bAXh8yIEY1L9
Person Token *NamesWilliamsCynthia
Date Token *Dates6/12/19670001-06-22
State Token *US state codesTNIL

* Available upon customer request.

Choosing a token type

ScenarioRecommended token type
Masking a namePerson Token (output is a realistic name)
Masking a phone or account numberNumeric Token (output preserves numeric shape)
Masking an emailText Token (output preserves email-like structure)
Masking a dateDate Token (output is a valid date)
Masking a US stateState Token (output is a valid state code)
General string with no format requirementsDefault Token
Variable-length outputSpecial Token

Token determinism

Deterministic tokenization applies within a specific execution context. The namespace is the outer boundary, so the same input can produce different tokens in different namespaces.

Within one namespace, consistency follows the entity category and policy:

ContextConsistency guarantee
Pseudonymizable entity, same input and token typeThe token remains consistent across policies within the namespace.
Anonymizable entity, same input and token typeThe token may differ under different policies within the namespace. Do not assume cross-policy consistency.
Same namespace, policy, category, input, and token typeThe token is expected to remain consistent across sync and async execution, including retries.

Compare results only when the namespace, entity category treatment, and token type are the same. Changing any of these inputs or context can change the result, especially changing the policy for an anonymizable entity.

Sync and async processing are not separate token namespaces. For the same namespace, policy, entity category, input, and token type, improved consistency across retries and async jobs is expected. Results may change across namespaces or across policies for anonymizable entities.

Using the same input with different token types

Different token types produce different outputs for the same input:

{
  "mask": [
    {
      "value": "Williams",
      "format": "Person Name",
      "token_name": "Person Token"
    },
    {
      "value": "Williams",
      "format": "Person Name",
      "token_name": "Default Token"
    }
  ]
}

The first produces a realistic name. The second produces a random string. Both are deterministic for their respective token types.