Arnike/watcher
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Query by fqdn_label instead of instance for host metrics

Browse Source 
Currently we are using `instance` label to query about host metrics to
prometheus. This label is assigned to the url of each endpoint being
scrapped.

While this work fine in one-exporter-per-compute cases as the driver is
mapping the fqdn_label value to the `instance` label value, it fails
when there are more that one target with the same value for the fqdn
label. This is a valid case, to be able to query by fqdn and do not
care about what exporter in the host is providing the metric.

This patch is changing the queries we use for hosts to be based on the
fqdn_label instead of the instance one. To implement it, we are also
simplifying the way we check the metric exist for the host by converting
prometheus_fqdn_instance_map into a prometheus_fqdn_labels set
which stores the list of fqdn found in  prometheus.

Closes-Bug: #2103451
Change-Id: I3bcc317441b73da5c876e53edd4622370c6d575e
This commit is contained in:
Alfredo Moralejo
2025-03-17 19:06:28 +01:00
parent 52bba70fec
commit a65e7e9b59
4 changed files with 173 additions and 90 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
doc/source/datasources/prometheus.rst
View File

@@ -29,19 +29,19 @@ This default can be overridden when a deployer uses a different label to
identify the exporter host (for example ``hostname`` or ``host``, or any other
label, as long as it identifies the host).
Internally this label is used in creating a ``fqdn_instance_map``, mapping
the fqdn with the Prometheus instance label associated with each exporter.
The keys of the resulting fqdn_instance_map are expected to match the
Internally this label is used in creating ``fqdn_instance_labels``, containing
the list of values assigned to the the label in the Prometheus targets.
The elements of the resulting fqdn_instance_labels are expected to match the
``ComputeNode.hostname`` used in the Watcher decision engine cluster model.
An example ``fqdn_instance_map`` is the following:
An example ``fqdn_instance_labels`` is the following:
.. code-block::
{
'ena.controlplane.domain': '10.1.2.1:9100',
'dio.controlplane.domain': '10.1.2.2:9100',
'tria.controlplane.domain': '10.1.2.3:9100'
}
[
'ena.controlplane.domain',
'dio.controlplane.domain',
'tria.controlplane.domain',
]
For instance metrics, it is required that Prometheus contains a label
with the uuid of the OpenStack instance in each relevant metric. By default,