Monitoring Kubernetes Pods with kwatch.

Monitoring your Kubernetes cluster effectively is crucial for maintaining the health and performance of your applications. Kwatch is a lightweight and easy-to-use tool that helps you monitor Kubernetes events and send alerts when specific issues arise, such as pod evictions, crashes, or other critical events. This guide will walk you through the installation and configuration of Kwatch to monitor your Kubernetes pods and send alerts based on specific criteria.

Step 1: Prerequisites

Before installing Kwatch, make sure you have the following prerequisites:

• A running Kubernetes cluster.

• kubectl installed.

• Helm installed (optional, but makes deployment easier).

Step 2: Install Kwatch

Kwatch can be installed via Helm and this is what I will be using for this article.

helm repo add kwatch https://kwatch.dev/charts
helm repo update
helm install kwatch kwatch/kwatch -n kwatch --create-namespace

OpenShift Users If you are using OpenShift you will get some PodSecurity warning and the deployment will not be able to start. You can run the command below to do the install.

W0904 16:58:09.644608   18521 warnings.go:70] would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "kwatch" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "kwatch" must set securityContext.capabilities.drop=["ALL"]), seccompProfile (pod or container "kwatch" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

Since this is my staging cluster, I assigned the anyuid SCC to the kwatch service account, which allows me to run containers with any user ID, removing restrictions on the user that the container must run as.

oc adm policy add-scc-to-user anyuid -z kwatch -n kwatch

Step 3: Configure Kwatch

Kwatch configuration is done through a ConfigMap, where you can specify what to monitor and how to send alerts.

Configure Kwatch ConfigMap:

You can create a new ConfigMap or edit the existing one created by Helm. Here is an example ConfigMap configuration I'm using.

apiVersion: v1
kind: ConfigMap
metadata:
  name: kwatch-config
  namespace: kwatch
data:
  config.yaml: |
    namespaceSelector:
      matchLabels:
        tenantType: "production"
    alert:
      slack:
        webhook: "https://hooks.slack.com/services/your-webhook-url" 

Apply the config map:

kubectl apply -f kwatch-config.yaml