HyperSec XDR Data Engine

Quick Start Guide

Introduction

The XDR Data Engine is a comprehensive data processing and analysis platform for HyperSec XDR. This guide will help you get started with the core components.

System Requirements

  1. Python 3.11 or higher
  2. Poetry for dependency management
  3. Access to required services: * ClickHouse database * OpenSearch (optional) * AWS S3 (optional)

Installation

  1. Clone the repository:

    bash git clone https://github.com/your-org/xdr-data-engine.git cd xdr-data-engine 2. Install dependencies:

    bash poetry install 3. Set up environment:

    ```bash cp .env.example .env

    Edit .env with your configuration

    ```

Core Components

The XDR Data Engine consists of several key components:

  1. Schema Management * Define and manage database schemas * Handle schema updates and migrations * See Schema Quick Start for details
  2. Ingestion Pipelines * Process incoming data streams * Transform and enrich data * Load data into databases
  3. Hunt Framework * Define and execute data queries * Analyze security events * Generate alerts and reports

Basic Configuration

  1. Create configuration directory:

    bash mkdir -p ~/.xdr/config 2. Set up targets:

    ```bash cp xdr_targets.example.yaml ~/.xdr/config/xdr_targets.yaml

    Edit with your environment details

    ``` 3. Configure packages:

    ```bash cp xdr_package.example.yaml ./xdr_package.yaml

    Edit with your package configuration

    ```

First Steps

  1. Verify installation:

    bash xdrcli --version 2. Check configuration:

    bash xdrcli check-config 3. Initialize environment:

    bash xdrcli init

Common Workflows

Data Ingestion

  1. Set up receivers:

    bash xdrcli setup-receivers 2. Start ingestion:

    bash xdrcli start-ingestion

Data Analysis

  1. Run a hunt:

    bash xdrcli run-hunt --hunt-name example_hunt 2. View results:

    bash xdrcli view-results --hunt-id <hunt_id>

Next Steps

Troubleshooting

  1. Check logs:

    bash tail -f ~/.xdr/logs/xdr.log 2. Verify services:

    bash xdrcli check-services 3. Common issues: * Database connection errors: Check credentials and network * Permission issues: Verify file permissions * Configuration errors: Validate YAML syntax

Getting Help