Auto-monitor a service

Use kix.monitor when a package exposes Prometheus metrics through a Service. Kix creates a ServiceMonitor whose selector, namespace, and dependency edges come from that Service.

This guide assumes:

• The package returns a Service named service with a port named metrics.
• The cluster has a Prometheus server configured to select ServiceMonitors.

Add monitoring options to the package

Add the standard monitoring options:

how-to/platform/monitoring-app.nix

  options.metrics = kix.options.monitor;

View source on GitHub ↗

These options let a cluster author enable or disable monitoring and set a scrape interval for each instance.

Name the option metrics. kix.monitor reads config.metrics directly, so a different name such as monitoring or prometheus is ignored. The cluster author would then have no way to change the defaults.

Generate the ServiceMonitor

Add kix.monitor to the package’s build list:

how-to/platform/monitoring-app.nix

    (kix.monitor {
      service = "service";
      port = "metrics";
    })

View source on GitHub ↗

Set service to the returned Service part and port to one of that Service’s named ports. The default metrics path is /metrics.

If the program serves metrics only over HTTPS, set scheme and the endpoint’s tlsConfig. A program that serves metrics with a self-signed certificate and no client authentication needs:

(kix.monitor {
  service = "metricsService";
  port = "metrics";
  scheme = "https";
  tlsConfig.insecureSkipVerify = true;
})

scheme and tlsConfig set only the endpoint for port. Each entry in extraEndpoints is a ServiceMonitor endpoint of its own, which defaults to scheme = "http", path = "/metrics", and the instance’s scrape interval.

Provide the ServiceMonitor API

The prometheus dependency must expose the ServiceMonitor builder used by kix.monitor. Add Prometheus Operator when the cluster does not already have a compatible provider:

how-to/platform/monitoring-cluster.nix

      instances.monitoring-system.prometheus = {
        package = packages."prometheus-operator";
      };

View source on GitHub ↗

This instance installs Prometheus Operator and its CRDs. It does not create a Prometheus server. Use the cluster’s existing server or install a monitoring stack that includes one.

kix.monitor treats prometheus as an optional dependency. If no cluster instance provides it, the builder produces no ServiceMonitor. Evaluation and kix check still succeed. After adding monitoring to a package, inspect the rendered output to confirm that the ServiceMonitor exists.

Configure the application instance

Set the scrape interval on the application instance:

how-to/platform/monitoring-cluster.nix

      instances.monitor-example.metrics = {
        package = monitoringApp;
        config.metrics.interval = "30s";
      };

View source on GitHub ↗

Omit config.metrics.interval to let Prometheus use its configured default. Set config.metrics.enabled = false to suppress the ServiceMonitor for one instance.

Check the result

Evaluate the cluster:

kix-examples/

❱ kix check how-to-platform-monitoring
 TOOL       RESULT  DETAILS                       
 eval       pass    29 manifests evaluated        
 scorecard  pass    0 errors, 10 warnings, 1 info

Inspect the generated ServiceMonitor:

kix-examples/ output excerpt

❱ kix build how-to-platform-monitoring --output json
{
  "apiVersion": "monitoring.coreos.com/v1",
  "kind": "ServiceMonitor",
  "metadata": {
    "name": "metrics",
    "namespace": "monitor-example"
  },
  "spec": {
    "endpoints": [
      {
        "interval": "30s",
        "path": "/metrics",
        "port": "metrics",
        "scheme": "http"
      }
    ],
    "namespaceSelector": {
      "matchNames": [
        "monitor-example"
      ]
    },
    "selector": {
      "matchLabels": {
        "app.kubernetes.io/instance": "metrics",
        "app.kubernetes.io/managed-by": "kix"
      }
    }
  }
}

The monitor selects the metrics Service in monitor-example and scrapes its named metrics port at /metrics every 30 seconds.