Environment Setup

The XDR Data Engine uses a layered configuration system to manage environment variables and settings. This document explains the configuration hierarchy and available options.

Configuration Hierarchy

Settings are applied in the following order (highest to lowest priority):

1. Environment Variables

2. CLI Parameters

3. xdr_package.yaml settings

4. xdr_targets.yaml settings

Environment Variables

Variable Name	Description	Example Value
XDR_CH_HOST	ClickHouse endpoint	'localhost' or 'sdfe234df.ap-southeast-2.aws.clickhouse.cloud'
XDR_CH_PORT	ClickHouse port	8123, 8443
XDR_CH_USERNAME	ClickHouse authentication username	'user1'
XDR_CH_PASSWORD	ClickHouse authentication password	'password1'
XDR_IP_CONFIG_BUCKET_NAME	S3 bucket for configuration	'dev_config_bucket_afterburner'
XDR_IP_CONFIG_BUCKET_REGION	AWS region for S3 bucket	'ap-southeast-2'
XDR_IP_CSV_PATH	Path for standard enrichment files	'vector_templates/standard_enrichment_files'
XDR_IP_RECEIVER_PATH	Path for vector receiver files	'vector_templates/vector_receiver'
XDR_IP_TEMPLATES_PATH	Path for vector templates	'vector_templates'
XDR_IP_GEO_IP_PATH	Path for geo IP files	'vector_templates/geoip'
XDR_HUNT_CONFIG_PATH	Path for hunt configuration files	'xdrcli/stable/hunts'
XDR_HUNT_RULES_PATH	Path for hunt rules	'xdrcli/stable/rules'

Target Configuration File (xdr_targets.yaml)

The target configuration file defines environment-specific settings. Example structure:

targets:
  production:
    ch_host: "clickhouse.production.example.com"
    ch_port: 8443
    ch_username: "prod_user"
    ch_password: "prod_password"
    ip_config_bucket_name: "prod-config-bucket"
    ip_config_bucket_region: "ap-southeast-2"
    
  development:
    ch_host: "localhost"
    ch_port: 8123
    ch_username: "dev_user"
    ch_password: "dev_password"
    ip_config_bucket_name: "dev-config-bucket"
    ip_config_bucket_region: "ap-southeast-2"

Managing Targets

Initialize or Update Target

xdrcli init-target --target production --target_file_path /pathto/.xdr/xdr_targets.yaml

List Available Targets

xdrcli list-targets --xdr_root_log_path /path/to/logs --target_file_path /pathto/.xdr/xdr_targets.yaml

Display Default Target

xdrcli print-default-target --xdr_root_log_path /path/to/logs --target_file_path /pathto/.xdr/xdr_targets.yaml

Best Practices

1. Environment Variables

   • Use environment variables for sensitive information

   • Consider using .env files for local development

   • Never commit sensitive values to version control

2. Target Configuration

   • Maintain separate target configurations for different environments

   • Use descriptive target names

   • Document any special requirements for each target

3. Security

   • Rotate credentials regularly

   • Use IAM roles where possible

   • Limit permissions to minimum required