Observability
Cosmonic Control includes a suite of observability tooling:
- Prometheus emits metrics signals at
http://cosmonic-prometheus:9090
, leveraging data generated bynats-surveyor
to create dashboards for NATS activity. - Grafana provides dashboards as
svc/cosmonic-grafana
on port3000
.
Accessing Grafana
All kubectl
commands on this page must be performed in the Kubernetes namespace where Cosmonic Control is deployed. It can be helpful to create an environment variable for the Kubernetes namespace where you have deployed Cosmonic Control. The example below assumes Cosmonic Control is installed in the default
namespace.
export COSMONIC_CONTROL_NAMESPACE=default
To find where you can access Grafana:
kubectl get svc cosmonic-grafana --namespace=$COSMONIC_CONTROL_NAMESPACE
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
cosmonic-grafana ClusterIP 10.96.17.177 <none> 3000/TCP 3h35m
If the output contains an EXTERNAL-IP
value, navigate to that address in your browser to find the Grafana sign-in page.
If you don't see an EXTERNAL-IP
value (as in the example above), you can port-forward the cosmonic-grafana
service using the port value:
kubectl port-forward svc/cosmonic-grafana 3000:3000 --namespace=$COSMONIC_CONTROL_NAMESPACE
Navigate to localhost:3000
in your browser to find the Grafana sign-in page.
Log in as administrator
In the terminal, you can find the password for administrator access to Grafana as a Kubernetes secret named cosmonic-grafana
:
kubectl get secret --namespace=$COSMONIC_CONTROL_NAMESPACE cosmonic-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
Log in to Grafana using admin
as the username and the returned secret as the password.
Click Log in.
NATS dashboards
NATS dashboards are available under "Dashboards" in the Grafana navigation sidebar. In a Cosmonic Control deployment, Grafana includes dashboards for NATS activity:
- Clients: Subscriptions and connections.
- Clusters: NATS cluster messages, bandwidth, and state.
- JetStream State and Metrics: Filestore and memstore capacity and usage, HA asset distribution by server, HA asset count by server, and more.
- NATS Overview: Summary view of metrics such as subscriptions, connections, route connections.
- Network Usage: Bandwidth, messages, and network conditions.
- Node Resource Usage: CPU and memory.
- Service Observations: Service duration percentiles, System/Requestor/Responder RTT times and percentiles, and more.