Mimecast Pack

Edge Delta Pipeline Pack for Microsoft DNS

Overview

The Edge Delta Mimecast pack processes JSON SIEM logs using the time provided by the log as the timestamp and then routing the separate SIEM logs down separate paths.

Pack Description

1. Data Ingestion

The data flow starts with the compound_input node as the entry point into the pack.

- name: compound_input
  type: compound_input

2. Parse JSON

Parses out the JSON from the body

  - name: parse_json_attributes_c8c6
    type: parse_json_attributes
    user_description: Parse JSON Attributes

3. Log Transformation

Logs are processed by the transform node, which is an OTTL Transform node.

- name: transform
  type: ottl_transform
  statements: |-
    set(timestamp, UnixMilli(Time(body["datetime"], "2006-01-02T15:04:05-0700")))    

This allows the timestamp in the log to be used at the timestamp within our system.

4. Routing

After transformation, the three different type of SIEM logs go down their respective paths based off the presence of certain fields.

  - name: route_8aef
    type: route
    user_description: Route
    expression_type: ottl
    paths:
    - path: Mimecast SIEM Receipt Logs
        condition: attributes["headerFrom"] != nil
        exit_if_matched: true
    - path: Mimecast Process Logs
        condition: attributes["AttNames"] != nil
        exit_if_matched: true
    - path: MImecast Delivery Logs
        condition: attributes["Delivered"] != nil
        exit_if_matched: true

Sample Input

{ "datetime": "2017-05-26T16:47:41+0100", "aCode": "7O7I7MvGP1mj8plHRDuHEA", "acc": "C0A0", "SpamLimit": 0, "IP": "123.123.123.123", "Dir": "Internal", "MsgId": "<messageId@messageId>", "Subject": "message subject", "headerFrom": "from@mimecast.com", "Sender": "from@mimecast.com", "Rcpt": "auser@mimecast.com", "SpamInfo": "[]", "Act": "Acc", "TlsVer": "TLSv1", "Cphr": "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA", "SpamProcessingDetail": { "spf": { "info": "SPF_FAIL", "allow": true }, "dkim": { "info": "DKIM_UNKNOWN", "allow": true } }, "SpamScore": 1 }

{
  "datetime": "2017-05-26T19:36:48+0100",
  "aCode": "BY81J52RPjSmp7MrubnlZg",
  "acc": "C0A0",
  "AttSize": 1267,
  "Act": "Acc",
  "AttCnt": 2,
  "AttNames": ["filename.docx", "filename2.xlsx"],
  "MsgSize": 2116,
  "MsgId": "messageId@messageId"
}

{
  "datetime": "2017-05-26T19:40:33+0100",
  "aCode": "9q_HeIHHPYejZTBsnipWmQ",
  "acc": "C0A0",
  "Delivered": true,
  "IP": "123.123.123.123",
  "AttCnt": 0,
  "Dir": "Inbound",
  "ReceiptAck": "250 2.6.0 messageId@messageId [InternalId=25473608] Queued mail for delivery",
  "MsgId": "messageId@messageId",
  "Subject": "Auto Reply",
  "Latency": 5618,
  "Sender": "from@domain.com",
  "Rcpt": "auser@mimecast.com",
  "AttSize": 0,
  "Attempt": 1,
  "TlsVer": "TLSv1",
  "Cphr": "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA",
  "Snt": 28237,
  "UseTls": "Yes",
  "Route": "Mimecast Exchange Route"
}