Deployment Guide

Deployment Stages

The XDR Data Engine deployment follows a two-stage process, ensuring a robust and scalable security data processing platform.

Stage 1: Core Infrastructure

The foundation layer establishes the essential infrastructure components:

• Kubernetes Environment
• Cluster setup and configuration
• Resource quotas and limits
• Service accounts and RBAC
• Kafka Infrastructure
• Broker setup
• Topic management
• Replication configuration
• Security Controls
• Network policies
• Access controls
• Encryption settings
• Networking
• Load balancers
• DMZ configuration for HyperCollectors
• Internal routing

Stage 2: XDE Components

The XDE build system manages the data processing components:

Moving Artifacts

1. Meta Schemas * Core schema definitions * Base data structures * Schema relationships * Version control metadata
2. Derived Schemas * Extended schema definitions * Custom field mappings * Schema transformations * Business logic implementations
3. Ingestion Pipelines * Helm charts for deployment * Vector configurations * Transformation rules * Data routing definitions
4. Receivers * Input configurations * Protocol handlers * Data validation rules * Initial processing logic
5. Hunts and Rules * Detection logic * Search patterns * Alert configurations * Response actions

Unified Repository Approach

Benefits

The XDE platform adopts a unified repository approach, keeping all components in a single repository. This strategy provides several advantages:

1. Reduced Mean Time to Recovery (MTTR) * Quick access to all components * Simplified troubleshooting * Consistent versioning * Faster deployment rollbacks
2. Improved Change Management * Reduced change failure rates * Better change tracking * Coordinated updates * Integrated testing
3. Enhanced Developer Velocity * Single source of truth * Streamlined workflows * Integrated development environment * Reduced context switching
4. Tighter Integration * Component synchronization * Consistent interfaces * Reduced version conflicts * Simplified dependency management

Best Practices

1. Version Control * Use semantic versioning * Maintain a clear branching strategy * Document all changes * Tag releases consistently
2. Component Integration * Test components together * Maintain interface compatibility * Document dependencies * Version components as a unit
3. Deployment Process * Use automated deployment pipelines * Implement staged rollouts * Maintain rollback procedures * Monitor deployment health
4. Testing Strategy * Integration tests across components * End-to-end testing * Performance validation * Security testing

Deployment Flow

1. Infrastructure Preparation

   mermaid graph TD A[Core Infrastructure] --> B[Kubernetes Setup] A --> C[Kafka Setup] A --> D[Security Controls] A --> E[Network Configuration] 2. XDE Component Deployment

   mermaid graph TD A[XDE Components] --> B[Meta Schemas] A --> C[Derived Schemas] A --> D[Ingestion Pipelines] A --> E[Receivers] A --> F[Hunts and Rules]

Monitoring and Maintenance

1. Health Checks * Component status monitoring * Performance metrics * Error tracking * Resource utilization
2. Updates and Upgrades * Coordinated component updates * Dependency management * Version compatibility * Update validation
3. Backup and Recovery * Component state backup * Configuration backup * Recovery procedures * Data preservation

Security Considerations

1. Access Control * Role-based access * Authentication mechanisms * Authorization policies * Audit logging
2. Network Security * Segmentation * Encryption * Firewall rules * Traffic monitoring
3. Data Protection * Data encryption * Access logging * Data retention * Privacy controls

Troubleshooting

1. Common Issues * Component connectivity * Resource constraints * Configuration conflicts * Performance bottlenecks
2. Resolution Steps * Diagnostic procedures * Log analysis * Component isolation * Recovery actions

Next Steps

• Environment Configuration
• Schema Management
• Hunt Framework
• Release Notes