Skip to main content

Observability

Cosmonic Control includes a suite of observability tooling:

  • Prometheus emits metrics signals at http://cosmonic-prometheus:9090, leveraging data generated by nats-surveyor to create dashboards for NATS activity.
  • Grafana provides dashboards as svc/cosmonic-grafana on port 3000.

Accessing Grafana

Use your namespace

All kubectl commands on this page must be performed in the Kubernetes namespace where Cosmonic Control is deployed. It can be helpful to create an environment variable for the Kubernetes namespace where you have deployed Cosmonic Control. The example below assumes Cosmonic Control is installed in the default namespace.

export COSMONIC_CONTROL_NAMESPACE=default

To find where you can access Grafana:

kubectl get svc cosmonic-grafana --namespace=$COSMONIC_CONTROL_NAMESPACE
NAME               TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
cosmonic-grafana   ClusterIP   10.96.17.177   <none>        3000/TCP   3h35m

If the output contains an EXTERNAL-IP value, navigate to that address in your browser to find the Grafana sign-in page.

If you don't see an EXTERNAL-IP value (as in the example above), you can port-forward the cosmonic-grafana service using the port value:

kubectl port-forward svc/cosmonic-grafana 3000:3000 --namespace=$COSMONIC_CONTROL_NAMESPACE

Navigate to localhost:3000 in your browser to find the Grafana sign-in page.

Log in as administrator

In the terminal, you can find the password for administrator access to Grafana as a Kubernetes secret named cosmonic-grafana:

kubectl get secret --namespace=$COSMONIC_CONTROL_NAMESPACE cosmonic-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

Log in to Grafana using admin as the username and the returned secret as the password.

Grafana sign-in page

Click Log in.

NATS dashboards

NATS dashboards are available under "Dashboards" in the Grafana navigation sidebar. In a Cosmonic Control deployment, Grafana includes dashboards for NATS activity:

  • Clients: Subscriptions and connections.
  • Clusters: NATS cluster messages, bandwidth, and state.
  • JetStream State and Metrics: Filestore and memstore capacity and usage, HA asset distribution by server, HA asset count by server, and more.
  • NATS Overview: Summary view of metrics such as subscriptions, connections, route connections.
  • Network Usage: Bandwidth, messages, and network conditions.
  • Node Resource Usage: CPU and memory.
  • Service Observations: Service duration percentiles, System/Requestor/Responder RTT times and percentiles, and more.

Further reading