Mapping files

Mapping file usage and management.

The following rule is used as example to show how mappping files work.

title: Zeek HTTP and HTTPS data leak rule
id: b8d5604b98c7d4e121a431b56c0bb7c8
status: experimental
description: Sum the number of destination.bytes for a unique destination.address and trigger an alert when threshold is exceeded.
tags:
- attack.exfiltration
- attack.t1567
author: Sebastien Lehuede
level: high
logsource:
category: network_monitoring
product: zeek
detection:
selection:
dest_port:
- 80
- 443
event_dataset: "zeek.connection"
timeframe: 5m
condition: selection|sum(bytes_out) by dest_addr > 20000
fields:
- dest_addr
- src_host
- src_address

This rule contains generic field names which must be change to match field names from events. That's where the mapping files come in.

Here is the field names that must be mapped:

  • dest_port (from detection section)

  • event_dataset (from detection section)

  • bytes_out (from condition section)

  • dest_addr (from condition section)

  • src_host (from fields section)

  • src_address (from fields section)

Global Sigma map

The Global Sigma map is used to automatically create the mapping file associated with a specific rules.

It's recommended to define mapping destination value with most commonly used value for the associated initial rule fields name, so less changes to be made in the specific rule mapping file.

The Global Sigma map is a JSON formatted file containing 2 elements:

Key

Value

Type

Condition

Description

id

global

K/V

required

Map identification

map

K/V List

required

Fields name mapping

Global map example:

{
"id": "global",
"map": {
"CommandLine":"process.args",
"Image":"process.executable",
"EventID":"event.code",
"ServiceName":"winlog.event_data.ServiceName",
"ParentImage":"process.parent.executable",
"dst_ip":"destination.ip",
"dest_ip":"destination.ip",
"src_ip":"source.ip",
"TargetObject":"winlog.event_data.TargetObject",
...
}
}

Each Key in the map element is a field name from Sigma rule and each corresponding value is the destination field name that will be used to match events against rules.

Nybble package contains a Global map for Sigma Rules to ECS format mapping. For other format a new Global file need to be created with proper values.

Sigma rules map

A Sigma rule map is automatically created at start, for each separated Sigma rules if not already existing.

Sigma rule maps are JSON formatted files containing the following elements:

Key

Value

Type

Condition

Description

id

$RuleID

K/V

required

Map identification, corresponding to rule ID.

map

K/V List

required

Fields name mapping for "detection" and "condition" sections.

fields

K/V List

Optional

Fields name mapping for "fields" rule section.

"fields" key in Sigma rule map file is optional, but if missing the important fields will not be added to generated alerts.

Sigma map rule example:

{
"id" : "b8d5604b98c7d4e121a431b56c0bb7c8",
"map" : {
"detection" : {
"dest_port" : "destination.port",
"event_dataset" : "event.dataset",
"bytes_out" : "destination.bytes",
"dest_addr" : "destination.address"
},
"fields" : {
"dest_addr" : "destination.address",
"src_host" : "host.hostname",
"src_addr" : "source.address"
}
}
}

Zeek log example, created by filebeat and using ECS format:

{
"@timestamp": "2020-08-06T00:08:03.038Z",
"@metadata": {
"beat": "filebeat",
"type": "_doc",
"version": "7.6.2",
"pipeline": "filebeat-7.6.2-zeek-connection-pipeline"
},
"log": {
"offset": 88514,
"file": {
"path": "/opt/zeek/logs/current/conn.log"
}
},
"service": {
"type": "zeek"
},
"ecs": {
"version": "1.4.0"
},
"event": {
"module": "zeek",
"timezone": "-07:00",
"created": "2020-08-06T00:08:03.038Z",
"dataset": "zeek.connection"
},
"destination": {
"address": "bad:beef::0",
"port": 443,
"bytes": 0,
"packets": 0,
"ip_public": true
},
"agent": {
"version": "7.6.2",
"type": "filebeat",
"ephemeral_id": "e8e7a11d-fd73-4e11-bcd6-70757e324cab",
"hostname": "nybble.nybble.local",
"id": "aaf47372-0380-4fc3-abba-dbe348f4070a"
},
"fileset": {
"name": "connection"
},
"network": {
"transport": "tcp",
"community_id": "1:XdUm8EAEgiQy7H4xFDGbWnIalCc="
},
"host": {
"os": {
"platform": "centos",
"version": "8 (Core)",
"family": "redhat",
"name": "CentOS Linux",
"kernel": "4.18.0-193.6.3.el8_2.x86_64",
"codename": "Core"
},
"id": "b82a3d16e6eb487b8ad722ba302a72c3",
"containerized": false,
"name": "nybble.nybble.local",
"hostname": "nybble.nybble.local",
"architecture": "x86_64"
},
"logsource": {
"category": "network_monitoring",
"product": "zeek"
},
"zeek": {
"connection": {
"history": "DTAF",
"state": "SH",
"local_orig": false,
"local_resp": false,
"ts": 1596672363.208055,
"missed_bytes": 0
},
"session_id": "CDyfdu2KCguxH60r9k"
},
"tags": [
"zeek.connection"
],
"input": {
"type": "log"
},
"temp": {
"duration": 113.21837496757507
},
"source": {
"packets": 10,
"ip_public": true,
"address": "1ce:1ce::babe",
"port": 52983,
"bytes": 1012
}
}

The previous examples show mapping between initial field names from Sigma rules and field names from events created by Filebeat in ECS schema.

MISP map

The MISP map is used to map MISP attribute names and event field names for enrichment.

Event field names are first mapped to MISP tags which may contain attributes related to them. Then each MISP tags are arrays which contain parameters to enrich events though MISP API.

MISP map is a JSON formatted file containing the following elements:

Key

Value

Type

Condition

Description

$field_To_Enrich

K/V List

required

Event field names to enrich.

$MISP_Tags

Array

required

MISP Tags which may be related to event fields.

mispAttribute

K/V

required

MISP attribute type.

type

K/V

required

Type used internally by Nybble

enrichPublicOnly

K/V

required

Public/Private IP enrichment flag.

resolveName

K/V

required

DNS resolution flag.

MISP map example:

{
"destination.address" : {
"C2-IP" : [{
"mispAttribute" : "ip-dst",
"type" : "ip",
"enrichPublicOnly" : true
}],
"TOR-node" : [{
"mispAttribute" : "ip-dst",
"type" : "ip",
"enrichPublicOnly" : true
}]
},
"destination.registered_domain" : {
"C2-Domain" : [{
"mispAttribute" : "domain",
"type" : "domain",
"resolveName" : false
}]
},
...
}

Zeek log example, created by filebeat and using ECS format:

{
...
"destination": {
"address": "bad:beef::0",
"port": 443,
"bytes": 0,
"packets": 0,
"ip_public": true
},
...
}

In this example, Nybble will request MISP API and check if there is any MISP attributes type "ip-dst" with value "bad:beef::0" and tagged "C2-IP" or "TOR-node". Event will be enriched with MISP attributes values if there is a match.