Meta Schemas

Introduction

XDR Meta Schemas define the core structure and validation rules for various data types in the XDR Data Engine. These schemas serve as the foundation for derived schemas and ensure data consistency across the platform.

Directory Structure

The meta schemas follow a standardized directory structure:

post_build_artefacts/xdr_meta_schemas/
└── schemas/
    ├── logs_alerts
    |   └── v{major}_{minor}_{patch}/
    |       └── logs_alerts.csv
    ├── logs_beats_filebeat
    |   └── v{major}_{minor}_{patch}/
    |       └── logs_beats_filebeat.csv
    ├── logs_beats_winlogbeat
    |   └── v{major}_{minor}_{patch}/
    |       └── logs_beats_winlogbeat.csv
    ├── logs_base
    |   └── v{major}_{minor}_{patch}/
    |       └── logs_base.csv
    ├── logs_hypercol_internal
    |   └── v{major}_{minor}_{patch}/
    |       └── logs_hypercol_internal.csv
    ├── logs_hypercol_metric
    |   └── v{major}_{minor}_{patch}/
    |       └── logs_hypercol_metric.csv
    ├── logs_nxlog_windows
    |   └── v{major}_{minor}_{patch}/
    |       └── logs_nxlog_windows.csv
    ├── logs_syslog
    |   └── v{major}_{minor}_{patch}/
    |      └── logs_syslog.csv
    ├── logs_syslog_linux
    |   └── v{major}_{minor}_{patch}/
    |      └── logs_syslog_linux.csv
    └── logs_syslog_linux_audited
        └── v{major}_{minor}_{patch}/
            └── logs_syslog_linux_audited.csv

Schema Types

Core Meta Schemas

1. logs_base.csv

   • Base schema for all log types

   • Defines common fields like timestamp, message, host

Specialized Meta Schemas

1. logs_beats_filebeat.csv (v001.000.003)

   • Schema for Filebeat log data

   • Used by multiple derived schemas (activemq, aws, zookeeper)

2. logs_beats_winlogbeat.csv (v001.000.000)

   • Windows event log schema

   • Specialized fields for Windows events

3. logs_nxlog_windows.csv (v001.000.003)

   • NXLog Windows data schema

   • Supports Windows event collection

4. logs_syslog.csv (v001.000.000)

   • Syslog format schema

   • Standard syslog fields and parsing

5. logs_hypercol_metric.csv (v001.000.001)

   • Metric data schema

   • Time-series and measurement fields

Integration with Schema Management

Configuration in xdr_package.yaml

meta_schema_paths: ../post_build_artefacts/xdr_meta_schemas_package

schemas:
  logs_alerts:
    meta_schema: logs_alerts.csv
    meta_schema_version: v001.000.000
  logs_beats_filebeat_activemq:
    meta_schema: logs_beats_filebeat.csv
    meta_schema_version: v001.000.003

Key Configuration Points:

• meta_schema_paths: Points to the meta schemas directory

• meta_schema: Specifies which meta schema to use

• meta_schema_version: Controls schema versioning

Version Control

• Version format: v{major}{minor}{patch}

• Each schema versioned independently

• Changes documented in release notes

• Backward compatibility maintained

Related Documentation

• Schema Builder Guide

• Schema Parameters

• Schema Updates

• Release Notes