Docker

Kubernetes

You can also run LimaCharlie at the host level in a container cluster system like Kubernetes in order to monitor all running containers on the host with a single sensor. In fact, this is the prefered method as it reduces the overhead of running LC within every single container.

This privileged container has visibility into everything happening in the cluster and sends telemetry directly back to the LimaCharlie, in real-time, for processing.

This is accomplished by a combination of a few techniques:

1. A privilged container running LC.
2. LC runs with HOST_FS environment variable pointing to the host's root filesystem mounted within the container.
3. LC runs with the NET_NS environment variable pointing to the host's directory listing network namespaces.
4. Running the container with the required flags to make sure it can have proper access.

The first step is straight forward. For example, set the environment like ENV HOST_FS=/rootfs and ENV NET_NS=/netns as part of your Dockerfile. This will let the LC sensor know where it can expect host-level information.

The second step is to run the container like: docker run --privileged --net=host -v /:/rootfs:ro --env HOST_FS=/rootfs -v /var/run/docker/netns:/netns:ro --env NET_NS=/netns --env LC_INSTALLATION_KEY=your_key your-lc-container-name.

Remember to pick the approriate LC sensor architecture installer for the container that will be running LC (not the host).

So if your privileged container runs Alpine Linux, use the alpine64 version of LC.

A public version of the container described below is available from dockerhub as: refractionpoint/limacharlie_sensor:latest.

##### Sample Configurations
This is a sample Dockerfile you may use to run LC within a privileged container as described above:

# Requires an LC_INSTALLATION_KEY environment variable
# specifying the installation key value.
# Requires a HOST_FS environment variable that specifies where
# the host's root filesystem is mounted within the container
# like "/rootfs".
# Requires a NET_NS environment variable that specific where
# the host's namespaces directory is mounted within the container
# like "/netns".
# Example:
# export ENV HOST_FS=/rootfs
# docker run --privileged --net=host -v /:/rootfs:ro -v /var/run/docker/netns:/netns:ro --env HOST_FS=/rootfs --env NET_NS=/netns --env LC_INSTALLATION_KEY=your_key refractionpoint/limacharlie_sensor

FROM alpine

RUN mkdir lc
WORKDIR /lc

RUN wget https://downloads.limacharlie.io/sensor/linux/alpine64 -O lc_sensor
RUN chmod 500 ./lc_sensor

CMD ./lc_sensor -d -

And this is a sample Kubernetes deployment on

a cluster supporting eBPF (kernel > 5.7):

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: lc-sensor
  namespace: lc-monitoring
  labels:
    app: lc-monitoring
spec:
  minReadySeconds: 30
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
  selector:
    matchLabels:
      app: lc-monitoring
  template:
    metadata:
      namespace: lc-monitoring
      labels:
        app: lc-monitoring
    spec:
      hostNetwork: true
      hostPID: true
      containers:
        - name: lc-sensor
          image: refractionpoint/limacharlie_sensor:latest
          imagePullPolicy: IfNotPresent
          securityContext:
            allowPrivilegeEscalation: true
            privileged: true
            capabilities:
              add: ['CAP_SYS_ADMIN']
          resources:
            requests:
              memory: 128M
              cpu: 0.01
            limits:
              memory: 256M
              cpu: 0.9
          volumeMounts:
            - mountPath: /rootfs
              name: all-host
            - mountPath: /netns
              name: all-host-ns
            - mountPath: /sys/kernel/debug
              name: all-host-krnl
            - mountPath: /sys/kernel/btf
              name: btf
            - mountPath: /lib/modules
              name: libmodules
          env:
            - name: HOST_FS
              value: /rootfs
            - name: NET_NS
              value: /netns
            - name: LC_INSTALLATION_KEY
              value: <<<< YOUR INSTALLATION KEY GOES HERE >>>>
      volumes:
        - name: all-host
          hostPath:
            path: /
        - name: all-host-ns
          hostPath:
            path: /var/run/docker/netns
        - name: all-host-krnl
          hostPath:
            path: /sys/kernel/debug
        - name: btf
          hostPath:
            path: /sys/kernel/btf
        - name: libmodules
          hostPath:
            path: /lib/modules

a cluster not supporting eBPF (kernel < 5.7):

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: lc-sensor
  namespace: lc-monitoring
  labels:
    app: lc-monitoring
spec:
  minReadySeconds: 30
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
  selector:
    matchLabels:
      app: lc-monitoring
  template:
    metadata:
      namespace: lc-monitoring
      labels:
        app: lc-monitoring
    spec:
      containers:
        - name: lc-sensor
          image: refractionpoint/limacharlie_sensor:latest
          imagePullPolicy: IfNotPresent
          securityContext:
            allowPrivilegeEscalation: true
            privileged: true
          resources:
            requests:
              memory: 128M
              cpu: 0.01
            limits:
              memory: 256M
              cpu: 0.9
          volumeMounts:
            - mountPath: /rootfs
              name: all-host-fs
            - mountPath: /netns
              name: all-host-ns
          env:
            - name: HOST_FS
              value: /rootfs
            - name: NET_NS
              value: /netns
            - name: LC_INSTALLATION_KEY
              value: <<<< YOUR INSTALLATION KEY GOES HERE >>>>
      volumes:
        - name: all-host-fs
          hostPath:
            path: /
        - name: all-host-ns
          hostPath:
            path: /var/run/docker/netns
      hostNetwork: true