Data Masking & Privacy

Overview

Secure60 Collector implements comprehensive data masking and privacy protection capabilities that allow organisations to protect sensitive data before it reaches the Secure60 platform. These solutions are applied within your environment, ensuring sensitive information is never transmitted or stored in its original form.

Key Benefits

Data Protection Strategies

Secure60 Collector offers multiple complementary data protection strategies:

  1. Content Redaction - Regex-based redaction of sensitive patterns within field content
  2. Replacement Masking - Replace sensitive field values with X characters
  3. Cryptographic Hashing - Transform sensitive data using secure hash algorithms
  4. Field Removal - Remove entire fields containing sensitive information
  5. Event Filtering - Drop entire events based on content criteria

Masking and Privacy Summary

Below is a quick-reference summary of each feature with an example transformation. Click a feature to jump to details and configuration.

Feature Scope Example input Output
Content Redaction Partial field (e.g., message_text) <content>hello</content><address>123 street</address> <content>hello</content><address>REDACTED</address>
Targeted Content Redaction Partial field with structure preserved <MessageType><AppCode DEFAULT=6555>97000027365</AppCode></MessageType> <MessageType><AppCode DEFAULT=6555>XXXXXX</AppCode></MessageType>
Replacement Masking Whole field (e.g., address) 123 street XXXXXX
Replacement Masking (Partial) Whole field (partial reveal) 394858397 3XXXXXX7
Cryptographic Hashing Whole field (e.g., email) john.doe@company.com b1946ac92492d2347c6235b4d2611184
Field Removal Whole field { \"credit_card\": \"1111111111111111\" } { }
Event Filtering Entire event { \"message_text\": \"error: confidential\" } — dropped —

Content Redaction

Overview

Content redaction protects sensitive information by applying regex-based filters to field content while preserving the overall data structure and context.

Features

Configuration

Via Portal UI

  1. Navigate to Integrations → Secure60 Collector
  2. Click “Advanced Config”
  3. Find “Content Redaction” section
  4. Configure redaction blocks with target fields and patterns

Via Environment Variables

# Credit card number redaction
REDACT_CONTENT_FIELD_NAME=transaction_log
REDACT_CONTENT_REGEX=r'\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b'

# Password redaction in URLs
REDACT_CONTENT_FIELD_NAME_2=request_url
REDACT_CONTENT_REGEX_2=r'password=[^&\s]+'

# Social security number redaction
REDACT_CONTENT_FIELD_NAME_3=message
REDACT_CONTENT_REGEX_3=r'\b\d{3}-\d{2}-\d{4}\b'

# Email address redaction
REDACT_CONTENT_FIELD_NAME_4=user_data
REDACT_CONTENT_REGEX_4=r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b'

# Phone number redaction
REDACT_CONTENT_FIELD_NAME_5=contact_info
REDACT_CONTENT_REGEX_5=r'\b\d{3}[-.]?\d{3}[-.]?\d{4}\b'

Regex Syntax

The Secure60 Collector uses the Rust regex engine for all pattern matching operations:

For complete syntax reference, see the Rust regex documentation

Common Redaction Patterns

Credit Card Numbers

# Matches various credit card formats
REDACT_CONTENT_REGEX=r'\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b'

# Examples:
# "4532 1234 5678 9012" → "****************"
# "4532-1234-5678-9012" → "****************"
# "4532123456789012" → "****************"

Social Security Numbers

# Matches SSN formats
REDACT_CONTENT_REGEX=r'\b\d{3}-\d{2}-\d{4}\b'

# Example:
# "SSN: 123-45-6789" → "SSN: ***-**-****"

Email Addresses

# Comprehensive email pattern
REDACT_CONTENT_REGEX=r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b'

# Example:
# "Contact: john.doe@company.com" → "Contact: *********************"

API Keys and Tokens

# Generic API key pattern
REDACT_CONTENT_REGEX=r'\b[A-Za-z0-9]{32,}\b'

# Bearer token pattern
REDACT_CONTENT_REGEX=r'Bearer\s+[A-Za-z0-9+/=]+'

# Example:
# "Authorization: Bearer abc123xyz789" → "Authorization: Bearer ***********"

URL Parameters

# Password parameters
REDACT_CONTENT_REGEX=r'password=[^&\s]+'

# Token parameters
REDACT_CONTENT_REGEX=r'[?&](token|key|secret)=[^&\s]*'

# Example:
# "?user=john&password=secret123&action=login" → "?user=john&password=***&action=login"

Targeted Content Redaction

Overview

Targeted content redaction masks only the sensitive portion of a match while preserving surrounding context or structure by using named capture groups in both the regex and the replacement string.

Features

Configuration

Via Portal UI

  1. Navigate to Integrations → Secure60 Collector
  2. Click “Advanced Config”
  3. Find “Targeted Content Redaction”
  4. Configure target field, regex (with named groups), and replacement

Via Environment Variables

# Basic targeted redaction with structure preservation
REDACT_TARGETED_CONTENT_FIELD_NAME=message_text
REDACT_TARGETED_CONTENT_REGEX=r'(?P<pre><AppCode[^>]*>)([^<]+)(?P<post></AppCode>)'
REDACT_TARGETED_CONTENT_REPLACEMENT='${pre}XXXXXX${post}'

# Partial reveal: keep last 3 characters of the value via a named group
REDACT_TARGETED_CONTENT_FIELD_NAME_2=message_text
REDACT_TARGETED_CONTENT_REGEX_2=r'(?P<pre><AppCode[^>]*>)(?P<drop>[^<]+?)(?P<show>[^<]{3})(?P<post></AppCode>)'
REDACT_TARGETED_CONTENT_REPLACEMENT_2='${pre}XXXXXX${show}${post}'

# Add additional blocks as needed (up to 20)
REDACT_TARGETED_CONTENT_FIELD_NAME_3=...
REDACT_TARGETED_CONTENT_REGEX_3=...
REDACT_TARGETED_CONTENT_REPLACEMENT_3=...

Examples

Preserve XML tag, redact only value

# Input
message_text="<MessageType><AppCode DEFAULT=6555>97000027365</AppCode><TransactionAmt>0000000000000000</TransactionAmt>"

# Config
REDACT_TARGETED_CONTENT_FIELD_NAME=message_text
REDACT_TARGETED_CONTENT_REGEX=r'(?P<pre><AppCode[^>]*>)([^<]+)(?P<post></AppCode>)'
REDACT_TARGETED_CONTENT_REPLACEMENT='${pre}XXXXXX${post}'

# Output
message_text="<MessageType><AppCode DEFAULT=6555>XXXXXX</AppCode><TransactionAmt>0000000000000000</TransactionAmt>"

Reveal last 3 characters for validation

# Input
message_text="<MessageType><AppCode DEFAULT=6555>97000027365</AppCode><TransactionAmt>0000000000000000</TransactionAmt>"

# Config
REDACT_TARGETED_CONTENT_FIELD_NAME=message_text
REDACT_TARGETED_CONTENT_REGEX=r'(?P<pre><AppCode[^>]*>)(?P<drop>[^<]+?)(?P<show>[^<]{3})(?P<post></AppCode>)'
REDACT_TARGETED_CONTENT_REPLACEMENT='${pre}XXXXXX${show}${post}'

# Output
message_text="<MessageType><AppCode DEFAULT=6555>XXXXXX365</AppCode><TransactionAmt>0000000000000000</TransactionAmt>"

Notes:

Replacement Masking

Overview

Replacement masking transforms sensitive field values by replacing them with X characters, either fully or partially.

Features

Configuration

Basic Configuration

# Enable replacement masking
ENABLE_DATA_MASKING_X=true

# Specify fields to mask
DATA_MASKING_ARRAY=["password", "credit_card", "ssn", "api_key"]

# Enable partial redaction (preserve first/last characters)
ENABLE_DATA_MASKING_PARTIAL_REDACT=true

Advanced Configuration

# Complete masking configuration
DATA_MASKING_ARRAY=["password", "credit_card_number", "social_security", "api_token", "user_password"]
ENABLE_DATA_MASKING_X=true
ENABLE_DATA_MASKING_PARTIAL_REDACT=true

Examples

Full Replacement Masking

# Original Event
{
  "username": "john_doe",
  "password": "mySecret123",
  "credit_card": "4532123456789012"
}

# After Full Masking
{
  "username": "john_doe", 
  "password": "XXXXXXXXXXX",
  "credit_card": "XXXXXXXXXXXXXXXX"
}

Partial Redaction

# Original Event
{
  "username": "john_doe",
  "password": "mySecret123",
  "credit_card": "4532123456789012"
}

# After Partial Redaction
{
  "username": "john_doe",
  "password": "mXXXXXXXX3", 
  "credit_card": "4XXXXXXXXXXXXXX2"
}

URL Parameter Masking

# Original Event
{
  "request_url": "https://api.example.com/login?user=john&password=secret123&token=abc123xyz"
}

# After URL Parameter Masking
{
  "request_url": "https://api.example.com/login?user=john&password=XXXXXXXX&token=XXX123"
}

Cryptographic Hashing

Overview

Cryptographic hashing transforms sensitive data into irreversible hash values, providing strong protection while maintaining some analytical value through consistency.

Supported Algorithms

Configuration

Basic Hashing Setup

# Enable cryptographic hashing
ENABLE_DATA_MASKING_HASH=true

# Specify fields to hash
DATA_MASKING_ARRAY=["user_id", "email_address", "phone_number"]

# Select hashing algorithm
DATA_MASKING_ENCRYPTION_ALGORITHM=SHA3  # SHA3-256 default

Algorithm Selection

# MD5 (fastest, least secure)
DATA_MASKING_ENCRYPTION_ALGORITHM=MD5

# SHA1 (moderate security/speed)
DATA_MASKING_ENCRYPTION_ALGORITHM=SHA1

# SHA2 (good balance)
DATA_MASKING_ENCRYPTION_ALGORITHM=SHA2

# SHA3-256 (default, high security)
DATA_MASKING_ENCRYPTION_ALGORITHM=SHA3

# SHA3-512 (highest security, slower)
DATA_MASKING_ENCRYPTION_ALGORITHM=SHA3-512

Examples

SHA3-256 Hashing (Default)

# Original Event
{
  "user_id": "john.doe@company.com",
  "username": "john_doe",
  "session_id": "abc123xyz789"
}

# After SHA3-256 Hashing
{
  "user_id": "a665a45920422f9d417e4867efdc4fb8a04a1f3fff1fa07e998e86f7f7a27ae3",
  "username": "john_doe",
  "session_id": "b1946ac92492d2347c6235b4d2611184"
}

MD5 Hashing (Legacy Support)

# Original Event
{
  "user_email": "john.doe@company.com"
}

# After MD5 Hashing
{
  "user_email": "5d41402abc4b2a76b9719d911017c592"
}

Field Removal

Overview

Remove entire fields that contain sensitive data when they are present.

Configuration

# Drop fields by exact name (comma-separated)
DROP_FIELD_NAMED=credit_card,ssn,secret_token

Example

# Input
{ "credit_card": "1111111111111111", "name": "John" }

# Output
{ "name": "John" }

Event Filtering

Overview

Drop entire events that contain certain string matches in specified fields.

Configuration

# Format: field=value,field=value
DROP_EVENT_CONTAINING=message_text=confidential

Example

# Input
{ "message_text": "error: confidential", "status": "500" }

# Output
# event is dropped (no output)

Combined Protection Strategies

Layered Protection

Combine multiple protection methods for comprehensive coverage:

# Layer 1: Content redaction for patterns
REDACT_CONTENT_FIELD_NAME=raw_log
REDACT_CONTENT_REGEX=r'\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b'

# Layer 2: Field-level masking
DATA_MASKING_ARRAY=["password", "api_key", "access_token"]
ENABLE_DATA_MASKING_X=true

# Layer 3: Hashing for consistent identifiers
ENABLE_DATA_MASKING_HASH=true
DATA_MASKING_ENCRYPTION_ALGORITHM=SHA3

Field-Specific Strategies

User Identity Protection

# Hash email addresses for consistency
DATA_MASKING_ARRAY=["email_address", "user_email"]
ENABLE_DATA_MASKING_HASH=true

# Redact email patterns in message fields
REDACT_CONTENT_FIELD_NAME=message
REDACT_CONTENT_REGEX=r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b'

Financial Data Protection

# Redact credit card patterns
REDACT_CONTENT_FIELD_NAME=transaction_log
REDACT_CONTENT_REGEX=r'\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b'

# Mask account number fields
DATA_MASKING_ARRAY=["account_number", "routing_number"]
ENABLE_DATA_MASKING_X=true
ENABLE_DATA_MASKING_PARTIAL_REDACT=true

API Security Protection

# Redact bearer tokens in URLs
REDACT_CONTENT_FIELD_NAME=request_url
REDACT_CONTENT_REGEX=r'Bearer\s+[A-Za-z0-9+/=]+'

# Mask API key fields
DATA_MASKING_ARRAY=["api_key", "access_token", "secret_key"]
ENABLE_DATA_MASKING_X=true

Implementation Best Practices

Security Considerations

Algorithm Selection

  1. SHA3-256: Recommended for new implementations (default)
  2. SHA2: Good balance for performance-sensitive environments
  3. SHA1: Use only for legacy compatibility
  4. MD5: Avoid for new implementations due to security vulnerabilities

Field Classification

# High-security fields (use hashing)
HIGH_SECURITY=["social_security", "passport_number", "driver_license"]

# Medium-security fields (use partial masking)  
MEDIUM_SECURITY=["phone_number", "account_number"]

# Low-security fields (use full masking)
LOW_SECURITY=["password", "api_key", "session_token"]

Performance Optimization

Processing Order

  1. Apply content redaction first (most specific)
  2. Apply field-level masking second
  3. Apply hashing last (most computationally intensive)

Resource Management

# Optimize for high-volume environments
DATA_MASKING_ENCRYPTION_ALGORITHM=SHA2  # Faster than SHA3
ENABLE_DATA_MASKING_PARTIAL_REDACT=false  # Simpler processing

Compliance Alignment

GDPR Compliance

# Protect personal identifiers
DATA_MASKING_ARRAY=["email", "phone", "name", "address"]
ENABLE_DATA_MASKING_HASH=true

# Redact personal data in content
REDACT_CONTENT_FIELD_NAME=message
REDACT_CONTENT_REGEX=r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b'

HIPAA Compliance

# Protect healthcare identifiers
DATA_MASKING_ARRAY=["patient_id", "medical_record_number", "insurance_id"]
ENABLE_DATA_MASKING_HASH=true

# Redact healthcare patterns
REDACT_CONTENT_FIELD_NAME=medical_notes
REDACT_CONTENT_REGEX=r'\b\d{3}-\d{2}-\d{4}\b'  # SSN pattern
Back to top