Commands Reference

apply-schemas

• Description: Apply and update ClickHouse schemas based on the given configuration.
• Options:
• --xdr_package_file_path: Path to the configuration file.
• --schema_directory: Name of the schema config directory in the .xdr_schema/ folder.
• --do_add_roles: If True, apply HyperSec core roles for RBAC.
• --do_add_columns: If True, add columns to the schemas.
• --xdr_root_log_path: Path to common directory.
• --target: Target name for specific environment we are building with.
• --target_file_path: Path to the targets configuration file.
• --schema_update_flag: Run additional schema updates if set to YES. This will force tables to be re-built, consuming substantial compute and memory. It is recommended to plan this carefully and execute one org or schema at a time to avoid system overload.
• --drop_replacement_table_flag: Flag to determine if replacement table can be deleted or not.
• --use_replicated_merge_tree: Use Replicated Merge Trees. Turn this off when using single node cluster.
• --schema_filter_list: Comma separated list to filter schemas to apply.
• --derived_schema_filter_list: Comma separated list to filter sub-schemas to apply.
• --schema_filter_wildchar: Filter schemas using wildcards.
• --derived_schema_filter_wildchar: Filter sub-schemas using wildcards.
• --org_filter_list: Comma separated list to filter schemas to plan.
• --max_insert_threads: Max threads for INSERT SELECT query.
• --min_insert_block_size_rows: Minimum block size for insertion.
• --verbose: Enable verbose logging.
• Example:


    $ xdrcli apply-schemas --xdr_package_file_path /path/to/xdr_package.yaml --schema_filter_list logs_alerts --sub_schema_filter_list logs_nxlog_windows,logs_nxlog_windows_dns --schema_filter_wildchar logs_alerts* --sub_schema_filter_wildchar logs_alerts* --schema_update_flag YES --drop_replacement_table_flag YES --verbose

build-ingestion-pipelines

• Description: Build ingestion pipelines based on the configuration.
• Options:
• --xdr_package_file_path: Path to the configuration file.
• --ingestion_pipeline_output_path: Path to ingestion output.
• --xdr_root_log_path: Path to common directory.
• --target: Target name for specific environment we are building with.
• --target_file_path: Path to the targets configuration file.
• Example:


    $ xdrcli build-ingestion-pipelines --xdr_package_file_path /path/to/xdr_package.yaml --ingestion_pipeline_output_path /path/to/output --xdr_root_log_path /path/to/logs --target production --target_file_path /pathto/.xdr/xdr_targets.yaml

build-schemas

• Description: Builds ClickHouse schemas based on given configuration.
• Options:
• --xdr_package_file_path: Path to the configuration file.
• --schema_directory: Path to custom schema directory.
• --schema_filter_list: Comma seperated list to filter the schemas that get build.
• --derived_schema_filter_list: Comma seperated list to filter the sub-schemas that get build.
• --schema_filter_wildchar: Filter the schemas that get build using wildchar.
• --derived_schema_filter_wildchar: Filter the sub-schemas that get build using wildchar.
• --no_cluster_declarations_needed: Remove references to the cluster as the SaaS offering abstracts the concept of clusters.
• --use_replicated_merge_tree: Use Replicated Merge Trees. Turn this off when using single node cluster.
• --xdr_root_log_path: Path to common directory.
• --target: Target name for specific environment we are building with.
• --target_file_path: Path to the targets configuration file.
• --all: Process all schemas.
• --only-beats: Scan for all beats schemas.
• --verbose: Enable verbose logging.
• --max_workers: Maximum Number of Workers.
• --opensearch_flag: For OpenSeach Templates
• Example:


    $ xdrcli build-schemas --xdr_package_file_path /path/to/xdr_package.yaml --schema_directory /path/to/custom_schema --schema_filter_list logs_beats_filebeat,logs_alerts --derived_schema_filter_list logs_nxlog_windows,logs_nxlog_windows_dns --schema_filter_wildchar logs_alerts* --derived_schema_filter_wildchar logs_alerts* --no_cluster_declarations_needed --use_replicated_merge_tree --xdr_root_log_path /path/to/logs --target production --target_file_path /pathto/xdr_targets.yaml --verbose --opensearch_flag True

build-sigma

• Description: Generate Sigma rules from YAML files in the specified input directory and save the converted rules to the output directory.
• Options:
• --xdr_package_file_path: Path to the XDR package configuration file.
• -i: Input directory containing YAML rules.
• -o: Output directory for converted rules.
• --xdr_root_log_path: Path to common directory.
• Example:


    $ xdrcli build-sigma --input_directory /path/to/yaml --output_directory /path/to/output --xdr_root_log_path /path/to/logs

describe-xdr-clickhouse-schema-types

• Description: Lists available meta schemas .
• Options:
• --xdr_root_log_path: Path to common directory.
• --xdr_package_file_path: Path to the configuration file.
• Example:


    $ xdrcli describe_xdr_clickhouse_schema_types --xdr_root_log_path /path/to/logs -- xdr_package_file_path /path/to/logs

download-ingestion-pipeline-templates

• Description: Downloads only vector templates that are referenced in xdr_package.yaml and saves them as a ZIP file.
• Options:
• --xdr_package_file_path: Path to the configuration file.
• --xdr_root_log_path: Path to common directory.
• --output_zip: Output ZIP file name.
• Example:


    $ xdrcli download-ingestion-pipeline-templates --xdr_root_log_path /path/to/logs --output_zip vector_templates.zip

download-ingestion-pipeline-templates-all

• Description: Downloads ALL vector templates and saves them as a ZIP file.
• Options:
• --xdr_package_file_path: Path to the configuration file.
• --xdr_root_log_path: Path to common directory.
• --output_zip: Output ZIP file name.
• Example:


    $ xdrcli download-ingestion-pipeline-templates-all --xdr_root_log_path /path/to/logs --output_zip vector_templates.zip

download-meta-schemas

• Description: Downloads meta schemas and saves them as a ZIP file.
• Options:
• --xdr_package_file_path: Path to the configuration file.
• --xdr_root_log_path: Path to common directory.
• --output_zip: Output ZIP file name.
• Example:


    $ xdrcli download-meta-schemas --xdr_root_log_path /path/to/logs --output_zip meta_schemas.zip

init-target

• Description: Initialize or update configuration.
• Options:
• --target: Target name for specific environment we are building with.
• --target_file_path: Path to the targets configuration file.
• Example:


    $ xdrcli init-target --target production --target_file_path /pathto/.xdr/xdr_targets.yaml

is-hunt-scheduler-healthy

• Description: Health check command to verify if the application is running.
• Example:

xdrcli is-hunt-scheduler-healthy

kill-all-hunts

• Description: Terminate all activities based on PIDs in the thread_tracking.log and delete the file.
• Options:
• --xdr_root_log_path: Path to common directory.
• --hunt_log_path: Path to hunt logs directory.
• Example:


    $ xdrcli kill-all-hunts --xdr_root_log_path /path/to/logs --hunt_log_path /path/to/hunt_logs

kill-hunt

• Description: Kill Hunt
• Options:
• --xdr_package_file_path: Path to the XDR package configuration file.
• --kill_pid: PID of the process to stop.
• --xdr_root_log_path: Path to common directory.
• Example:


    $ xdrcli kill-hunt --xdr_package_file_path /path/to/xdr_package.yaml --kill_pid 12345 --xdr_root_log_path /path/to/logs

list-hunts

• Description: List Hunts
• Options:
• --look_back_hours: Number of hours to look back for hunts.
• --xdr_root_log_path: Path to log path directory.
• --hunt_log_path: Path to hunt logs directory.
• Example:


    $ xdrcli list-hunts --look_back_hours 24 --xdr_root_log_path /path/to/logs --hunt_log_path /path/to/hunt_logs

list-ingestion-templates

• Description: List available ingestion templates.
• Options:
• --xdr_package_file_path: Path to the configuration file.
• --xdr_root_log_path: Path to common directory.
• Example:


    $ xdrcli list-ingestion-templates --xdr_root_log_path /path/to/logs

list-meta-schemas

• Description: Lists available meta schemas .
• Options:
• --xdr_package_file_path: Path to the configuration file.
• --xdr_root_log_path: Path to common directory.
• Example:


    $ xdrcli list-meta-schemas --xdr_root_log_path /path/to/logs

list-schema-fields

• Description: Lists fields for a given schema and template version.
• Options:
• --xdr_package_file_path: Path to the configuration file.
• --schema_name: Schema name.
• --template_version: Template version.
• --xdr_root_log_path: Path to common directory.
• Example:


    $ xdrcli list-schema-fields --schema_name logs_alerts --template_version v100_000_000 --xdr_root_log_path /path/to/logs

list-targets

• Description: List all profiles in the configuration.
• Options:
• --xdr_root_log_path: Path to common directory.
• --target_file_path: Path to the targets configuration file.
• Example:


    $ xdrcli list-targets --xdr_root_log_path /path/to/logs --target_file_path /pathto/.xdr/xdr_targets.yaml

plan-schemas

• Description: Run plan for schema changes
• Options:
• --xdr_package_file_path: Path to the configuration file.
• --xdr_root_log_path: Path to common directory.
• --target: Target name for specific environment we are building with.
• --target_file_path: Path to the targets configuration file.
• --schema_filter_list: Comma seperated list to filter the schemas that get planned.
• --derived_schema_filter_list: Comma seperated list to filter the sub-schemas that get planned.
• --schema_filter_wildchar: Filter the schemas that get planned using wildchar.
• --derived_schema_filter_wildchar: Filter the sub-schemas that get planned using wildchar.
• --org_filter_list: Comma seperated list to filter the schemas that get planned.
• --verbose: Enable verbose logging.
• Example:


    $ xdrcli plan-schemas --xdr_package_file_path /path/to/xdr_package.yaml --xdr_root_log_path /path/to/logs --target production --target_file_path /pathto/xdr_targets.yaml --schema_filter_list logs_alerts --derived_schema_filter_list logs_nxlog_windows* --schema_filter_wildchar logs_alerts* --derived_schema_filter_wildchar logs_nxlog_windows* --org_filter_list org321 --verbose

print-default-target

• Description: List all profiles in the configuration.
• Options:
• --xdr_root_log_path: Path to common directory.
• --target_file_path: Path to the targets configuration file.
• Example:


    $ xdrcli print-default-target --xdr_root_log_path /path/to/logs --target_file_path /pathto/.xdr/xdr_targets.yaml

print-hunt-parameters

• Description: Print XDR hunt parameters.
• Options:
• --xdr_package_file_path: Path to configuration file
• --xdr_root_log_path: Path to log path directory
• --hunt_log_path: Path to hunt logs directory
• --target: Target name for specific environment we are building with.
• --target_file_path: Path to the targets configuration file.
• Example:


    $ xdrcli print-hunt-parameters --xdr_package_file_path /path/to/xdr_package.yaml --xdr_root_log_path /path/to/logs --hunt_log_path /path/to/hunt_logs --target production --target_file_path /pathto/xdr_targets.yaml

run-hunt

• Description: Run XDR hunt.
• Options:
• --xdr_package_file_path: Path to the configuration file.
• --hunt_dir: Directory containing hunt configuration describing the hunt actions against specific customers.
• --hunt_rule_repo_dir: Directory containing target YAML files.
• --checkpoint_destination: Directory containing hunt configuration describing the hunt actions against specific customers.
• --checkpoint_timestamp_field: Timestamp field used by each hunt config to checkpoint its position in the logs.
• --hunt_timeout: Timeout setting for individual Cron tasks.
• --hunt_num_threads: Number of threads.
• --xdr_root_log_path: Path to the common directory.
• --hunt_log_path: Path to hunt logs directory.
• --target: Target name for specific environment we are building with.
• --target_file_path: Path to the targets configuration file.
• Example:


    $ xdrcli run-hunt --xdr_package_file_path /path/to/xdr_package.yaml --hunt_dir /path/to/hunt_dir --hunt_rule_repo_dir /path/to/hunt_rules --hunt_timeout 60 --hunt_num_threads 4 --xdr_root_log_path /path/to/logs --hunt_log_path /path/to/hunt_logs --target production --target_file_path /path/to/xdr_targets.yaml

upload-ingestion-pipeline-core-config

• Description: Uploads vector core enrichment configuration files to S3 based on given YAML configuration.
• Options:
• --xdr_package_file_path: Path to the configuration file.
• --aws_profile: AWS profile to use for credentials.
• --xdr_root_log_path: Path to common directory.
• --target: Target name for specific environment we are building with.
• --target_file_path: Path to the targets configuration file.
• --upload_file_target: File target for the upload. Choose either "local" or "S3".
• Example:


    $ xdrcli upload-ingestion-pipeline-core-config --xdr_package_file_path /path/to/config.yaml --aws_profile my-aws-profile --xdr_root_log_path /path/to/logs --target production --target_file_path /pathto/.xdr/xdr_targets.yaml

upload-ingestion-pipeline-maxmind-mmdb

• Description: Uploads MaxMind MMDB files to S3 based on configuration.
• Options:
• --xdr_package_file_path: Path to the configuration file.
• --aws_profile: AWS profile to use for credentials.
• --xdr_root_log_path: Path to common directory.
• --target: Target name for specific environment we are building with.
• --target_file_path: Path to the targets configuration file.
• --upload_file_target: File target for the upload. Choose either "local" or "S3".
• Example:


    $ xdrcli upload-ingestion-pipeline-maxmind-mmdb --aws_profile my-aws-profile --xdr_root_log_path /path/to/logs --target production --target_file_path /pathto/.xdr/xdr_targets.yaml ----upload_file_target local

upload-ingestion-pipeline-templates

• Description: Uploads vector templates to S3 based on configuration.
• Options:
• --xdr_package_file_path: Path to the configuration file.
• --aws_profile: AWS profile to use for credentials.
• --xdr_root_log_path: Path to common directory.
• --target: Target name for specific environment we are building with.
• --target_file_path: Path to the targets configuration file.
• --upload_file_target: File target for the upload. Choose either "local" or "S3".
• Example:


    $ xdrcli upload-ingestion-pipeline-templates --aws_profile my-aws-profile --xdr_root_log_path /path/to/logs --target production --target_file_path /pathto/.xdr/xdr_targets.yaml

view-hunts

• Description: View Hunts
• Options:
• --view_pid: Process ID (PID) or list of PIDs to fetch hunts for.
• --xdr_root_log_path: Path to common directory.
• --hunt_log_path: Path to hunt logs directory.
• Example:


    $ xdrcli view-hunts --view_pid 12345 --xdr_root_log_path /path/to/logs --hunt_log_path /path/to/hunt_logs