Deployment Guide

Deployment Stages

The XDR Data Engine deployment follows a two-stage process, ensuring a robust and scalable security data processing platform.

Stage 1: Core Infrastructure

The foundation layer establishes the essential infrastructure components:

• Kubernetes Environment

  • Cluster setup and configuration

  • Resource quotas and limits

  • Service accounts and RBAC

• Kafka Infrastructure

  • Broker setup

  • Topic management

  • Replication configuration

• Security Controls

  • Network policies

  • Access controls

  • Encryption settings

• Networking

  • Load balancers

  • DMZ configuration for HyperCollectors

  • Internal routing

Stage 2: XDE Components

The XDE build system manages the data processing components:

Moving Artifacts

1. Meta Schemas

   • Core schema definitions

   • Base data structures

   • Schema relationships

   • Version control metadata

2. Derived Schemas

   • Extended schema definitions

   • Custom field mappings

   • Schema transformations

   • Business logic implementations

3. Ingestion Pipelines

   • Helm charts for deployment

   • Vector configurations

   • Transformation rules

   • Data routing definitions

4. Receivers

   • Input configurations

   • Protocol handlers

   • Data validation rules

   • Initial processing logic

5. Hunts and Rules

   • Detection logic

   • Search patterns

   • Alert configurations

   • Response actions

Unified Repository Approach

Benefits

The XDE platform adopts a unified repository approach, keeping all components in a single repository. This strategy provides several advantages:

1. Reduced Mean Time to Recovery (MTTR)

   • Quick access to all components

   • Simplified troubleshooting

   • Consistent versioning

   • Faster deployment rollbacks

2. Improved Change Management

   • Reduced change failure rates

   • Better change tracking

   • Coordinated updates

   • Integrated testing

3. Enhanced Developer Velocity

   • Single source of truth

   • Streamlined workflows

   • Integrated development environment

   • Reduced context switching

4. Tighter Integration

   • Component synchronization

   • Consistent interfaces

   • Reduced version conflicts

   • Simplified dependency management

Best Practices

1. Version Control

   • Use semantic versioning

   • Maintain a clear branching strategy

   • Document all changes

   • Tag releases consistently

2. Component Integration

   • Test components together

   • Maintain interface compatibility

   • Document dependencies

   • Version components as a unit

3. Deployment Process

   • Use automated deployment pipelines

   • Implement staged rollouts

   • Maintain rollback procedures

   • Monitor deployment health

4. Testing Strategy

   • Integration tests across components

   • End-to-end testing

   • Performance validation

   • Security testing

Deployment Flow

1. Infrastructure Preparation

   graph TD
   A[Core Infrastructure] --> B[Kubernetes Setup]
   A --> C[Kafka Setup]
   A --> D[Security Controls]
   A --> E[Network Configuration]

2. XDE Component Deployment

   graph TD
   A[XDE Components] --> B[Meta Schemas]
   A --> C[Derived Schemas]
   A --> D[Ingestion Pipelines]
   A --> E[Receivers]
   A --> F[Hunts and Rules]

Monitoring and Maintenance

1. Health Checks

   • Component status monitoring

   • Performance metrics

   • Error tracking

   • Resource utilization

2. Updates and Upgrades

   • Coordinated component updates

   • Dependency management

   • Version compatibility

   • Update validation

3. Backup and Recovery

   • Component state backup

   • Configuration backup

   • Recovery procedures

   • Data preservation

Security Considerations

1. Access Control

   • Role-based access

   • Authentication mechanisms

   • Authorization policies

   • Audit logging

2. Network Security

   • Segmentation

   • Encryption

   • Firewall rules

   • Traffic monitoring

3. Data Protection

   • Data encryption

   • Access logging

   • Data retention

   • Privacy controls

Troubleshooting

1. Common Issues

   • Component connectivity

   • Resource constraints

   • Configuration conflicts

   • Performance bottlenecks

2. Resolution Steps

   • Diagnostic procedures

   • Log analysis

   • Component isolation

   • Recovery actions

Next Steps

• Environment Configuration

• Schema Management

• Hunt Framework

• Release Notes