[Kubernetes][audit_logs] Add mapping; map some fields to ECS fields #10138

tetianakravchenko · 2024-06-12T13:29:16Z

Proposed commit message

WHAT: Add mapping, map some fields to ECS fields, replace processor with the ingest pipeline
WHY: add more fields mapping and map some fields to ECS fields according to [Kubernetes audit logs] Additional Fields mapping for detection #9046; replace processor with the ingest pipeline - map some fields to ECS fields require adding more processors (like set), It is better to follow the best practices to use ingest pipeline instead

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

[ ]

How to test this PR locally

kind create cluster --config kind-config.yaml

kind-config.yaml

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: ClusterConfiguration
    apiServer:
        # enable auditing flags on the API server
        extraArgs:
          audit-log-path: /var/log/kubernetes/kube-apiserver-audit.log
          audit-policy-file: /etc/kubernetes/policies/audit-policy.yaml
        # mount new files / directories on the control plane
        extraVolumes:
          - name: audit-policies
            hostPath: /etc/kubernetes/policies
            mountPath: /etc/kubernetes/policies
            readOnly: true
            pathType: "DirectoryOrCreate"
          - name: "audit-logs"
            hostPath: "/var/log/kubernetes"
            mountPath: "/var/log/kubernetes"
            readOnly: false
            pathType: DirectoryOrCreate
  # mount the local file on the control plane
  extraMounts:
  - hostPath: ./audit-policy.yaml
    containerPath: /etc/kubernetes/policies/audit-policy.yaml
    readOnly: true

audit-policy.yaml

apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata
- level: Request
- level: RequestResponse

Related issues

Closes [Kubernetes audit logs] Additional Fields mapping for detection #9046

Screenshots

covers mapping to ecs fields:
kubernetes.audit.annotations.authorization_k8s_io/decision -> event.outcome
kubernetes.audit.verb -> event.action
kubernetes.audit.sourceIPs -> client.ip & source.ip

added: kubernetes.audit.requestObject.roleRef.name

kubernetes.audit.requestObject.spec.containers.image
kubernetes.audit.requestObject.spec.containers.command
kubernetes.audit.requestObject.spec.containers.name
kubernetes.audit.requestObject.spec.containers.securityContext.seccompProfile.type

kubernetes.audit.user.uid -> user.id
kubernetes.audit.user.username -> user.name
kubernetes.audit.userAgent -> user_agent processor

… ingest pipeline Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

MichaelKatsoulis · 2024-06-13T07:25:42Z

packages/kubernetes/data_stream/audit_logs/elasticsearch/ingest_pipeline/default.yml

+      copy_from: kubernetes.audit.user.uid
+      ignore_empty_value: true
+  - set:
+      field: user_agent.original


Is it the kubernetes.audit.userAgent the unparsed user_agent in the form of "Mozilla/5.0 (iPhone; CPU iPhone OS 12_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1" or just the name?

I've added few examples in this comment - #9046 (comment): kindnetd/v0.0.0 (linux/amd64) kubernetes/$Format, kube-controller-manager/v1.29.1 (linux/amd64) kubernetes/bc401b9/leader-election, kube-probe/1.29

MichaelKatsoulis · 2024-06-13T07:26:40Z

packages/kubernetes/data_stream/audit_logs/elasticsearch/ingest_pipeline/default.yml

+      ignore_empty_value: true
+  - convert:
+      field: kubernetes.audit.sourceIPs
+      target_field: source.ip


According to #9046 (comment) kubernetes.audit.sourceIPs should be also mapped to client.ip

I though that it might be better to have kubernetes.audit.sourceIPs as ip type and simply copy value with set processor instead of convert twice - e1e6a26
but it introduce the type conflict
I've added set processor to copy source.ip - https://github.com/elastic/integrations/pull/10138/files#diff-ba27e35b988f4118d3605895eccfefce09e39b13df670ba4f1415d0c8f25a94dR59-R62

MichaelKatsoulis · 2024-06-13T07:27:45Z

packages/kubernetes/data_stream/audit_logs/elasticsearch/ingest_pipeline/default.yml

+  - set:
+      field: event.action
+      copy_from: kubernetes.audit.verb
+      ignore_empty_value: true


What about the following mapping?

kubernetes.audit.annotations.authorization_k8s_io/decision -> event.outcome

I've added it here e1e6a26

note: field value must be one of the following: failure, success, unknown - https://www.elastic.co/guide/en/ecs/current/ecs-event.html#field-event-outcome

packages/kubernetes/changelog.yml

…tead of text Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

Co-authored-by: Michael Katsoulis <michaelkatsoulis88@gmail.com>

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

tetianakravchenko · 2024-06-13T14:25:55Z

packages/kubernetes/_dev/build/build.yml

 dependencies:
  ecs:
-    reference: git@v8.0.0
+    reference: git@v8.11.0


FYI: reason for this change - container.security_context.privileged field is not available in < 8.10.0

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

tetianakravchenko · 2024-06-13T14:46:13Z

packages/kubernetes/data_stream/audit_logs/sample_event.json

        "type": "logs",
        "dataset": "kubernetes.audit_logs"
    },
-    "host": {


reason for this: in ecs 8.3 was added pattern attribute to .mac fields - elastic/ecs#1871
after upgrading the ecs from 8.0 to 8.11 static tests are not working - I've removed host object to fix. Note: for some data_streams host object was already omitted:

╭────────────┬─────────────────────────────┬───────────┬──────────────────────────┬────────────────────────────────────────────┬──────────────╮ │ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │ ├────────────┼─────────────────────────────┼───────────┼──────────────────────────┼────────────────────────────────────────────┼──────────────┤ │ kubernetes │ apiserver │ static │ Verify sample_event.json │ PASS │ 68.397358ms │ │ kubernetes │ audit_logs │ static │ Verify sample_event.json │ PASS │ 63.505139ms │ │ kubernetes │ container │ static │ Verify sample_event.json │ FAIL: one or more errors found in document │ 67.984205ms │ │ kubernetes │ container_logs │ static │ Verify sample_event.json │ FAIL: one or more errors found in document │ 62.75686ms │ │ kubernetes │ controllermanager │ static │ Verify sample_event.json │ PASS │ 63.080172ms │ │ kubernetes │ event │ static │ Verify sample_event.json │ FAIL: one or more errors found in document │ 58.794828ms │ │ kubernetes │ node │ static │ Verify sample_event.json │ FAIL: one or more errors found in document │ 57.321807ms │ │ kubernetes │ pod │ static │ Verify sample_event.json │ FAIL: one or more errors found in document │ 60.585872ms │ │ kubernetes │ proxy │ static │ Verify sample_event.json │ PASS │ 70.774477ms │ │ kubernetes │ scheduler │ static │ Verify sample_event.json │ PASS │ 60.451825ms │ │ kubernetes │ state_container │ static │ Verify sample_event.json │ FAIL: one or more errors found in document │ 60.242947ms │ │ kubernetes │ state_cronjob │ static │ Verify sample_event.json │ PASS │ 63.243224ms │ │ kubernetes │ state_daemonset │ static │ Verify sample_event.json │ PASS │ 62.284566ms │ │ kubernetes │ state_deployment │ static │ Verify sample_event.json │ PASS │ 61.316046ms │ │ kubernetes │ state_job │ static │ Verify sample_event.json │ PASS │ 66.531294ms │ │ kubernetes │ state_namespace │ static │ Verify sample_event.json │ PASS │ 65.444342ms │ │ kubernetes │ state_node │ static │ Verify sample_event.json │ PASS │ 63.073798ms │ │ kubernetes │ state_persistentvolume │ static │ Verify sample_event.json │ FAIL: one or more errors found in document │ 58.022577ms │ │ kubernetes │ state_persistentvolumeclaim │ static │ Verify sample_event.json │ PASS │ 60.903057ms │ │ kubernetes │ state_pod │ static │ Verify sample_event.json │ FAIL: one or more errors found in document │ 61.11333ms │ │ kubernetes │ state_replicaset │ static │ Verify sample_event.json │ PASS │ 59.230285ms │ │ kubernetes │ state_resourcequota │ static │ Verify sample_event.json │ FAIL: one or more errors found in document │ 61.368662ms │ │ kubernetes │ state_service │ static │ Verify sample_event.json │ FAIL: one or more errors found in document │ 59.198362ms │ │ kubernetes │ state_statefulset │ static │ Verify sample_event.json │ PASS │ 58.916231ms │ │ kubernetes │ state_storageclass │ static │ Verify sample_event.json │ FAIL: one or more errors found in document │ 59.207278ms │ │ kubernetes │ system │ static │ Verify sample_event.json │ FAIL: one or more errors found in document │ 58.320341ms │ │ kubernetes │ volume │ static │ Verify sample_event.json │ FAIL: one or more errors found in document │ 60.318885ms │ ╰────────────┴─────────────────────────────┴───────────┴──────────────────────────┴────────────────────────────────────────────┴──────────────╯

You mean that because we have ELASTIC_NETINFO:false and we dont have mac addresses at all, this is why you removed it?

Should not this be then the fact then, just to remove only mac addresses?

You mean that because we have ELASTIC_NETINFO:false and we dont have mac addresses at all, this is why you removed it?

ELASTIC_NETINFO:false - hm, not sure what this field is about and if it is related somehow, but yes - we do not have mac field at all, I can remove host.mac field only - I think it should work, but I thought that host object in general is not relevant to the kubernetes integration itself (it is coming from the add_host_metadata processor as I understand)

Also some datastreams don't have host object (like state_deployment, see the table above - all datastreams with the test result PASS don't have host object in sample_event), so I decided to align all datastreams to follow similar approach. WDYT?

Indeed host comes from the add_host_metadata processor. Neglected that.
So dont worry remove all host

tetianakravchenko · 2024-06-13T14:50:52Z

packages/kubernetes/data_stream/audit_logs/agent/stream/stream.yml.hbs

-            throw "expected kubernetes.audit.annotations.authorization_k8s_io/decision === allow";
-        }
-      }
+    target: "kubernetes.audit"


FYI: there was a bug - elastic/beats#29395 , so the workaroud used before was to use target: "kubernetes_audit" and later rename to "kubernetes.audit" - it was fixed in 8.10 and since the current Kibana restriction is 8.14 - I removed the workaround

tetianakravchenko · 2024-06-25T12:06:35Z

/test

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

elasticmachine · 2024-06-25T13:10:50Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

gizas · 2024-06-25T14:00:45Z

packages/kubernetes/data_stream/audit_logs/fields/fields.yml

        - name: rules
          dynamic: true
          type: nested
+        - name: roleRef


This should be grouped with following

integrations/packages/kubernetes/data_stream/audit_logs/fields/fields.yml

Line 214 in 7cf501d

- name: roleRef

Those 2 are different fields:
kubernetes.audit.requestObject.roleRef.name requested in #9046
kubernetes.audit.responseObject.roleRef.kind requested in #3259

gizas · 2024-06-25T14:09:46Z

Tested this locally and works fine!

I would say to add some screenshots for the proof of the fields you added in the PR description:

For me the kubernetes.audit.requestObject.spec.containers.securityContext.seccompProfile.type is always empty

Also kubernetes.audit.userAgent is keyword. Should be type text rather than keyword?

gizas · 2024-06-27T07:19:15Z

Besides the latest comment about updating with some screenshots and specific questions for seccompProfile.type and kubernetes.audit.userAgent, LGTM!

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

tetianakravchenko · 2024-06-27T10:50:28Z

Besides the latest comment about updating with some screenshots and specific questions for seccompProfile.type and kubernetes.audit.userAgent, LGTM!

@gizas

seccompProfile.type:

pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: security-context-demo2
spec:
  securityContext:
    runAsUser: 1000
    runAsGroup: 3000
    fsGroup: 2000
  volumes:
  - name: sec-ctx-vol
    emptyDir: {}
  containers:
  - name: sec-ctx-demo
    image: busybox:1.28
    command: [ "sh", "-c", "sleep 1h" ]
    volumeMounts:
    - name: sec-ctx-vol
      mountPath: /data/demo
    securityContext:
      privileged: true
  - name: nginx
    image: nginx:1.14.2
    ports:
    - containerPort: 80
    securityContext:
      privileged: true
      seccompProfile:
        type: RuntimeDefault

kubernetes.audit.userAgent - changed in f2aa344

I've also added screenshots in description

@gizas @MichaelKatsoulis FYI: I also needed to remove mapping to the container.* fields, because kubernetes.audit.requestObject.spec.containers.image is an array, and this processor:

- set:
    field: container.image.name
    copy_from: kubernetes.audit.requestObject.spec.containers.image
    ignore_empty_value: true

cause this error:

elasticmachine · 2024-06-27T11:44:58Z

💚 Build Succeeded

Buildkite Build
Commit: 0195c8f

History

💔 Build #12955 failed f2aa344
💚 Build #12827 succeeded c227e22
💔 Build #12826 failed 7cf501d
💔 Build #12528 failed 7cf501d
💔 Build #12527 failed 869ef53

elastic-sonarqube · 2024-06-27T11:45:01Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

elasticmachine · 2024-06-28T13:11:08Z

Package kubernetes - 1.62.0 containing this change is available at https://epr.elastic.co/search?package=kubernetes

tetianakravchenko added 2 commits June 12, 2024 14:33

additional fields mapping; add some fields to the ecs fields; move to…

6516fcd

… ingest pipeline Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

add audit logs ingest pipeline

9165ded

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

tetianakravchenko requested a review from a team as a code owner June 12, 2024 13:29

tetianakravchenko requested review from MichaelKatsoulis and gizas June 12, 2024 13:29

tetianakravchenko mentioned this pull request Jun 12, 2024

[Kubernetes audit logs] Additional Fields mapping for detection #9046

Closed

MichaelKatsoulis reviewed Jun 13, 2024

View reviewed changes

packages/kubernetes/changelog.yml Outdated Show resolved Hide resolved

MichaelKatsoulis reviewed Jun 13, 2024

View reviewed changes

packages/kubernetes/changelog.yml Outdated Show resolved Hide resolved

tetianakravchenko and others added 4 commits June 13, 2024 15:28

add event.outcome field; convert kubernetes.audit.sourceIPs to ip ins…

e1e6a26

…tead of text Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

Update packages/kubernetes/changelog.yml

d3c2c7b

Co-authored-by: Michael Katsoulis <michaelkatsoulis88@gmail.com>

Update packages/kubernetes/changelog.yml

eca2b41

Co-authored-by: Michael Katsoulis <michaelkatsoulis88@gmail.com>

revert changes of the sourceIPs field type

869ef53

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

tetianakravchenko commented Jun 13, 2024

View reviewed changes

remove host object from the sample_event.json to fix static tests

7cf501d

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

tetianakravchenko commented Jun 13, 2024

View reviewed changes

commit readme changes

c227e22

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

MichaelKatsoulis approved these changes Jun 25, 2024

View reviewed changes

gizas reviewed Jun 25, 2024

View reviewed changes

gizas approved these changes Jun 27, 2024

View reviewed changes

remove mapping to the container fields; change userAgent type to text

f2aa344

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

Merge branch 'main' into k8s_audit_logs_mapping

0195c8f

tetianakravchenko merged commit 79fb552 into elastic:main Jun 28, 2024

tetianakravchenko deleted the k8s_audit_logs_mapping branch June 28, 2024 12:53

ltflb-bgdi mentioned this pull request Jul 16, 2024

[Kubernetes Audit Logs] [AWS Cloudwatch] Create a new datastream for Ingesting EKS Audit Logs #5799

Closed

andrewkroh added the Integration:kubernetes Kubernetes label Jul 22, 2024

[Kubernetes][audit_logs] Add mapping; map some fields to ECS fields #10138

[Kubernetes][audit_logs] Add mapping; map some fields to ECS fields #10138

Uh oh!

Conversation

tetianakravchenko commented Jun 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

Author's Checklist

How to test this PR locally

Related issues

Screenshots

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tetianakravchenko Jun 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tetianakravchenko Jun 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tetianakravchenko Jun 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tetianakravchenko commented Jun 25, 2024

Uh oh!

elasticmachine commented Jun 25, 2024

🚀 Benchmarks report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gizas commented Jun 25, 2024

Uh oh!

gizas commented Jun 27, 2024

Uh oh!

tetianakravchenko commented Jun 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticmachine commented Jun 27, 2024

💚 Build Succeeded

History

Uh oh!

elastic-sonarqube bot commented Jun 27, 2024

Quality Gate passed

Uh oh!

elasticmachine commented Jun 28, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

tetianakravchenko commented Jun 12, 2024 •

edited

Loading

tetianakravchenko Jun 13, 2024 •

edited

Loading

tetianakravchenko Jun 13, 2024 •

edited

Loading

tetianakravchenko Jun 13, 2024 •

edited

Loading

tetianakravchenko commented Jun 27, 2024 •

edited

Loading