Skip to main content
Learn how to monitor FluxCD controllers using Prometheus and visualize key metrics with Grafana. This guide covers:
  • Exposing FluxCD metrics via /metrics
  • Leveraging the Prometheus Operator for auto-discovery
  • Configuring a PodMonitor for Flux controllers
  • Inspecting generated Prometheus scrape configs
  • Building a Grafana dashboard for FluxCD health and performance

​
1. FluxCD Metrics Overview

Flux controllers expose Prometheus metrics on port 8080 at the /metrics endpoint. By deploying the Prometheus Operator, you can manage Prometheus, Alertmanager, and Grafana components using Kubernetes custom resources.

​
2. Prometheus Operator and Auto-Discovery

The Prometheus Operator discovers scrape targets via the ServiceMonitor and PodMonitor CRDs. To include Flux controllers:
  • Deploy a PodMonitor that targets Flux pods in the flux-system namespace
  • Ensure each controller pod is annotated for Prometheus scraping

​
3. Inspecting Prometheus Configuration

After installing the Prometheus Operator, you can verify the scrape configuration that gets generated: 
kubectl exec -it prometheus-0 -c config-reloader -n monitoring -- /bin/sh
cat /etc/prometheus/config_out/prometheus.env.yaml
Look for entries under scrape_configs: 
scrape_configs:
  - job_name: serviceMonitor/monitoring/kube-etcd/0
  - job_name: serviceMonitor/monitoring/apiserver/0
  # ...
By default, Prometheus scrapes Kubernetes components like the API server, etcd, CoreDNS, and kubelet.

​
4. Enabling Flux Controller Metrics

​
4.1 Annotate Flux Pods

Verify that each Flux controller pod is annotated for Prometheus. For example, inspect the source-controller pod: 
kubectl get pod source-controller-5495bf4d7d-nw57 -n flux-system -o yaml
Ensure you see these annotations: 
metadata:
  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/port: "8080"

​
4.2 Create a PodMonitor

Make sure the targetPort matches the container port (default is 8080) and the path is /metrics.
Apply the following PodMonitor to enable Prometheus scraping for all Flux controllers: 
kubectl apply -f - <<EOF
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: flux-system
spec:
  namespaceSelector:
    matchNames:
      - flux-system
  selector:
    matchExpressions:
      - key: app
        operator: In
        values:
          - helm-controller
          - source-controller
          - kustomize-controller
          - notification-controller
          - image-automation-controller
          - image-reflector-controller
  podMetricsEndpoints:
    - targetPort: 8080
      path: /metrics
EOF
After applying, the Prometheus Operator will regenerate the scrape config. You should see a new job entry: 
- job_name: podMonitor/monitoring/flux-system/0

​
5. Visualizing Metrics in Grafana

Import or build a Grafana dashboard to monitor:
Metric CategoryDescription
Health ChecksNumber of successful vs. failed reconciliations
Reconciliation LatencyTime taken per reconciliation cycle
Event CountsTotal events processed by each controller
Errors & WarningsCount and rate of errors in FluxCD controllers
  1. Open your Grafana UI.
  2. Create a new dashboard or import a JSON template.
  3. Add panels for FluxCD metrics, e.g.,
    • flux_reconcile_duration_seconds
    • flux_reconcile_errors_total
    • flux_reconcile_success_total

Watch Video