HyperSec XDR Data Engine

Environment Setup

The XDR Data Engine uses a layered configuration system to manage environment variables and settings. This document explains the configuration hierarchy and available options.

Configuration Hierarchy

Settings are applied in the following order (highest to lowest priority):

  1. Environment Variables
  2. CLI Parameters
  3. xdr_package.yaml settings
  4. xdr_targets.yaml settings

Environment Variables

Variable Name Description Example Value
XDR_CH_HOST ClickHouse endpoint 'localhost' or 'sdfe234df.ap-southeast-2.aws.clickhouse.cloud'
XDR_CH_PORT ClickHouse port 8123, 8443
XDR_CH_USERNAME ClickHouse authentication username 'user1'
XDR_CH_PASSWORD ClickHouse authentication password 'password1'
XDR_IP_CONFIG_BUCKET_NAME S3 bucket for configuration 'dev_config_bucket_afterburner'
XDR_IP_CONFIG_BUCKET_REGION AWS region for S3 bucket 'ap-southeast-2'
XDR_IP_CSV_PATH Path for standard enrichment files 'vector_templates/standard_enrichment_files'
XDR_IP_RECEIVER_PATH Path for vector receiver files 'vector_templates/vector_receiver'
XDR_IP_TEMPLATES_PATH Path for vector templates 'vector_templates'
XDR_IP_GEO_IP_PATH Path for geo IP files 'vector_templates/geoip'
XDR_HUNT_CONFIG_PATH Path for hunt configuration files 'xdrcli/stable/hunts'
XDR_HUNT_RULES_PATH Path for hunt rules 'xdrcli/stable/rules'

Target Configuration File (xdr_targets.yaml)

The target configuration file defines environment-specific settings. Example structure:

targets:
  production:
    ch_host: "clickhouse.production.example.com"
    ch_port: 8443
    ch_username: "prod_user"
    ch_password: "prod_password"
    ip_config_bucket_name: "prod-config-bucket"
    ip_config_bucket_region: "ap-southeast-2"

  development:
    ch_host: "localhost"
    ch_port: 8123
    ch_username: "dev_user"
    ch_password: "dev_password"
    ip_config_bucket_name: "dev-config-bucket"
    ip_config_bucket_region: "ap-southeast-2"

Managing Targets

Initialize or Update Target

xdrcli init-target --target production --target_file_path /pathto/.xdr/xdr_targets.yaml

List Available Targets

xdrcli list-targets --xdr_root_log_path /path/to/logs --target_file_path /pathto/.xdr/xdr_targets.yaml

Display Default Target

xdrcli print-default-target --xdr_root_log_path /path/to/logs --target_file_path /pathto/.xdr/xdr_targets.yaml

Best Practices

  1. Environment Variables * Use environment variables for sensitive information * Consider using .env files for local development * Never commit sensitive values to version control
  2. Target Configuration * Maintain separate target configurations for different environments * Use descriptive target names * Document any special requirements for each target
  3. Security * Rotate credentials regularly * Use IAM roles where possible * Limit permissions to minimum required