Observability (Prometheus + Grafana + Loki + Tempo)

Sulu ships an optional bundled observability stack — Prometheus for metrics, Loki for logs, Promtail for log shipping, and Grafana for dashboards. Two deployment surfaces:

• Docker Compose — apply the docker-compose.observability.yml overlay
• Kubernetes — enable the observability.kubePrometheusStack and observability.lokiStack toggles in the Helm chart

Both surfaces produce the same end result: a dashboard at the URL of your choice that shows backend availability, HTTP request rate + latency, JVM runtime, Hikari DB pool, ingest pipeline throughput, and a tail of backend ERROR/WARN logs.

Compose

Add the overlay to your existing compose invocation:

docker compose \
  -f docker-compose.yml \
  -f docker-compose.prod.yml \
  -f docker-compose.observability.yml \
  up -d

The overlay binds Prometheus to 127.0.0.1:9090, Grafana to 127.0.0.1:3000, and Loki to 127.0.0.1:3100. Front them with Caddy / nginx and a basic-auth-protected subdomain. The Grafana admin password is the value of SULU_GRAFANA_ADMIN_PASSWORD (must be set in .env).

The Sulu Overview dashboard is auto-provisioned. Add your own JSON exports under infra/observability/grafana/dashboards/.

Kubernetes (Helm)

In your overrides file:

observability:
  kubePrometheusStack:
    enabled: true
  lokiStack:
    enabled: true

kube-prometheus-stack:
  grafana:
    adminPassword: "<openssl rand -base64 24>"

loki-stack:
  loki:
    persistence:
      enabled: true
      size: 20Gi

Then helm upgrade --install sulu .... Both subcharts deploy into the same namespace; Grafana is published at http://sulu-grafana.sulu.svc.cluster.local. Expose via an Ingress entry if you want browser access (Sulu's default Ingress doesn't include it — add it to ingress.annotations and a custom rule).

Required: grafana.adminPassword

The chart will abort helm install when kubePrometheusStack.enabled=true but kube-prometheus-stack.grafana.adminPassword is unset. Without it, the upstream subchart silently falls back to compiled-in admin/admin — which is unsafe for any pod that can reach the in-cluster Grafana Service.

How Prometheus discovers the backend

With observability.kubePrometheusStack.enabled: true, the chart renders a ServiceMonitor matching the backend Service. kube-prometheus-stack's Prometheus Operator auto-picks it up by the release: <name> label and starts scraping /actuator/prometheus every 15 seconds.

To opt out (e.g. you manage scrape config via additionalScrapeConfigs in your own ConfigMap):

observability:
  kubePrometheusStack:
    enabled: true
    createServiceMonitor: false

Bring your own

Disable both subcharts and point the backend at your existing Prometheus / Loki by leaving observability.*.enabled: false. The backend already exposes /actuator/prometheus unauthenticated — scrape it from your external stack.

Customizing dashboards

The bundled dashboard is a starting point. Fork the JSON under infra/observability/grafana/dashboards/sulu-overview.json and add panels for your use case. Reload by restarting the Grafana container (Compose) or rolling the Grafana StatefulSet (Helm).

Related

• Distributed tracing — OpenTelemetry → Tempo, opt-in companion to the metrics + logs stack
• Backup and restore