Metrics¶

Collected metrics are thousands. Instead of listing a full list of metrics, this documentation will focus on main exporters and their most important metrics.

To get an exhaustive list of metrics collected, you can connect to the exporters with

kubectl -n <namespace> port-forward <exporter pod name> <port>:<port>

and then visit

http://localhost:<port>/metrics

with curl or with your favourite web browser.

Below is a list of main exporters and some of the main metrics it exposes. When you know the prefix of a metric, you can use Grafana (Explore tab), type the prefix and let the auto-completion suggest you all possible metrics.

kubelet and cadvisor¶

The exporter is integrated into the Kubelet. There is no documentation about the exposed metrics.

Metrics start with container_ or kubelet_.

container_cpu_*
container_fs_*
container_memory_*
container_network_*
kubelet_certificate_manager_*
kubelet_cgroup_manager_*
kubelet_containers_*
kubelet_docker_operations_*
kubelet_http_*
kubelet_network_*
kubelet_node_config_error
kubelet_node_name
kubelet_pleg_relist_*
kubelet_pod_*
kubelet_running_container_count
kubelet_running_pod_count
kubelet_runtime_operations_*
kubelet_volume_stats_*

Note

There are other container_* and kubelet_* metrics. Explore by yourself !

Kube-state-metrics¶

The metrics of Kube-state-metrics are described in this directory.

Kube-state-metrics exposes metrics about Kubernetes® objects :

kube_configmap_*
kube_daemonset_*
kube_deployment_*
kube_endpoint_*
kube_ingress_*
kube_namespace_*
kube_node_*
kube_persistentvolume_*
kube_persistentvolumeclaim_*
kube_pod_container_*
kube_pod_*
kube_replicaset_*
kube_secret_*
kube_service_*
kube_statefulset_*
kube_storageclass_*

Node Exporter¶

The metrics of Node-exporter are described as a list of collectors.

Node exporter retrives metrics from the Operating System of the machine that runs the kubelet.

Metrics are prefixed with node_*.

node_cpu_*
node_disk_*
node_filefd_*
node_filesystem_*
node_load1
node_load15
node_load5
node_memory_*
node_netstat_*
node_network_*
node_network_up
node_uname_info
node_vmstat_*

Note

There are other node_*metrics. Explore by yourself !

Blackbox Exporter¶

The metrics of Blackbox Exporter are not described in any official documentation. Their definition can be found in the source code

Blackbox exporter metrics are prefixed with probe_*.

probe_dns_lookup_time_seconds
probe_duration_seconds
probe_failed_due_to_regex
probe_http_content_length
probe_http_duration_seconds
probe_http_redirects
probe_http_ssl
probe_http_status_code
probe_http_uncompressed_body_length
probe_http_version
probe_ip_addr_hash
probe_ip_protocol
prober_probe_total
probe_ssl_earliest_cert_expiry
probe_ssl_last_chain_expiry_timestamp_seconds
probe_success
probe_tls_version_info

Prometheus metrics¶

Prometheus explains some of its internal metrics here

The most important metrics is up. Its syntax is

up{job="<job-name>", instance="<instance-id>"}

and returns 1 if the Prometheus target is up.

This metric can be used to see if a target (an exporter) is scrappable.

S3 metrics¶

S3 on Flexible Engine¶

S3 metrics on Flexible Engine come from an exporter internally developed by Caascad Team. Here is the exhaustive list of exported metrics :

s3_bucket_size : size of the S3 buckets needed by Caascad. The name of the bucket is given by the s3_bucket_name Prometheus tag.
s3_bucket_object_number : number of objects in the S3 buckets needed by Caascad. The name of the bucket is given by the s3_bucket_name Prometheus tag.

S3 on AWS¶

We do not collect those metrics yet. This should change soon.

Most exporters coded with GO language¶

Most exporters coded with GO language also export some internal metrics on how they work.

go_gc_* : metrics about the Garbage Collector
go_goroutines : metrics about the goroutines
go_info : metrics with labels about the GO compiler that compiled the exporter
go_memstats_*: memory metrics about the exporter
go_threads : metrics about the threads.