Token Types
How token types control the shape and structure of masked output — choosing the right token type for your data and downstream requirements.
Token types control how masked values are generated, independent of entity or format. The same input can produce different output shapes depending on which token_name you specify.
Available token types
| Token Type | Intended Use | Example Input | Example Output |
|---|---|---|---|
Default Token | General text | George | FreTdf |
Text Token | Emails, usernames | kumar@ss.fss.com | LEFQS@cTdqGVB2Ay |
Numeric Token | Numeric-like values | (408)-426-9989 | (07432)-29915-56713 |
Special Token | Random-length output | john.doe@example.com | xuLDKEM4bAXh8yIEY1L9 |
Person Token * | Names | Williams | Cynthia |
Date Token * | Dates | 6/12/1967 | 0001-06-22 |
State Token * | US state codes | TN | IL |
* Available upon customer request.
Choosing a token type
| Scenario | Recommended token type |
|---|---|
| Masking a name | Person Token (output is a realistic name) |
| Masking a phone or account number | Numeric Token (output preserves numeric shape) |
| Masking an email | Text Token (output preserves email-like structure) |
| Masking a date | Date Token (output is a valid date) |
| Masking a US state | State Token (output is a valid state code) |
| General string with no format requirements | Default Token |
| Variable-length output | Special Token |
Token determinism
Deterministic tokenization applies within a specific execution context. The namespace is the outer boundary, so the same input can produce different tokens in different namespaces.
Within one namespace, consistency follows the entity category and policy:
| Context | Consistency guarantee |
|---|---|
| Pseudonymizable entity, same input and token type | The token remains consistent across policies within the namespace. |
| Anonymizable entity, same input and token type | The token may differ under different policies within the namespace. Do not assume cross-policy consistency. |
| Same namespace, policy, category, input, and token type | The token is expected to remain consistent across sync and async execution, including retries. |
Compare results only when the namespace, entity category treatment, and token type are the same. Changing any of these inputs or context can change the result, especially changing the policy for an anonymizable entity.
Sync and async processing are not separate token namespaces. For the same namespace, policy, entity category, input, and token type, improved consistency across retries and async jobs is expected. Results may change across namespaces or across policies for anonymizable entities.
Using the same input with different token types
Different token types produce different outputs for the same input:
{
"mask": [
{
"value": "Williams",
"format": "Person Name",
"token_name": "Person Token"
},
{
"value": "Williams",
"format": "Person Name",
"token_name": "Default Token"
}
]
}
The first produces a realistic name. The second produces a random string. Both are deterministic for their respective token types.