Implement the spec for multi-tenancy support for metrics. This adds
a new 'Aetos' datasource very similar to the current Prometheus
datasource. Because of that, the original PrometheusHelper class
was split into two classes and the base class is used for
PrometheusHelper and for AetosHelper. Except for the split, there
is one more change to the original PrometheusHelper class code, which
is the addition and use of the _get_fqdn_label() and
_get_instance_uuid_label() methods.
As part of the change, I refactored the current prometheus datasource
unit tests. Most of them are now used to test the PrometheusBase class
with minimal changes. Changes I've made to the original tests:
- the ones that can be be used to test the base class are moved into the
TestPrometheusBase class
- the _setup_prometheus_client, _get_instance_uuid_label and
_get_fqdn_label functions are mocked in the base class tests.
Their concrete implementations are tested in each datasource tests
separately.
- a self._create_helper() is used to instantiate the helper class with
correct mocking.
- all config value modification is the original tests got moved out and
instead of modifying the config values, the _get_* methods are mocked
to return the wanted values
- to keep similar test coverage, config retrieval is tested for each
concrete class by testing the _get_* methods.
New watcher-aetos-integration and watcher-aetos-integration-realdata
zuul jobs are added to test the new datasource. These use the same set
of tempest tests as the current watcher-prometheus-integration jobs.
The only difference is the environment setup and the Watcher config,
so that the job deploys Aetos and Watcher uses it instead of accessing
Prometheus directly.
At first this was generated by asking cursor to implement the linked spec
with some additional prompts for some smaller changes. Afterwards I manually
went through the code doing some cleanups, ensuring it complies with
PEP8 and hacking and so on. Later on I manually adjusted the code to use
the latest observabilityclient changes.
The zuul job was also mostly generated by cursor.
Implements: https://blueprints.launchpad.net/watcher/+spec/prometheus-multitenancy-support
Generated-By: Cursor with claude-4-sonnet model
Change-Id: I72c2171f72819bbde6c9cbbf565ee895e5d2bd53
Signed-off-by: Jaromir Wysoglad <jwysogla@redhat.com>
With the events of eventlet removal, Watcher will need
to be adapted to support both modes, eventlet and threading, for
a couple of releases before removing all eventlet code.
This patch adds methods and classes that allow decision engine
modules to create futurist thread pools instead of green thread pools,
based on a environment variable that can be enabled by service.
It moves continuous audit handler instance to decison engine service,
so it can be started together with the main decision engine service.
Adds an environment variable that allows the user to disable
eventlet monkey patching and to use oslo.service threading backend.
Change-Id: I8a8be0a7cebdc44005fd77ec960543828c7da318
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This adds a new data source for the Watcher decision engine that
implements the watcher.decision_engine.datasources.DataSourceBase.
related spec was merged at [1].
Implements: blueprint prometheus-datasource
[1] https://review.opendev.org/c/openstack/watcher-specs/+/933300
Change-Id: I6a70c4acc70a864c418cf347f5f6951cb92ec906
olso.policy 4.5.0[1] changed the config options policy_file
default value to 'policy.yaml', which means it is changed
for all the OpenStack services and they do not need to
override the default anymore.
NOTE: There is no change in behaviour here, oslo.policy provides
the same configuration that services have overridden till now.
[1] https://review.opendev.org/c/openstack/releases/+/934012
[2] https://review.opendev.org/c/openstack/requirements/+/934295
Change-Id: I46cc9e05fbc8f6c95c0b2d50093ecfb070a4170f
The datetime.utcnow() is deprecated in Python 3.12.
Replace datetime.utcnow() with oslo_utils.timeutils.utcnow().
This bumps oslo.utils to 7.0.0.
Change-Id: Icccbb0549add686a744a72b354932471cbf91c92
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
This code worked around a bug in eventlet[1] that has been fixed in
115103d5608cbe8f15df10e27eba1644f5364e95. The fix has been available in
every eventlet release since v0.27.0.
[1] https://github.com/eventlet/eventlet/issues/592
Co-Authored-By: Cyril Roelandt <cyril@redhat.com>
Change-Id: Ifc0b9c1d7f022db54c34c48c903a1719f9404d04
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:
1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.
2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.
Also replace policy.json to policy.yaml ref from doc and tests.
[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html
Change-Id: I207c02ba71fe60635fd3406c9c9364c11f259bae
Current requirements-check job is failing with
below error:
ERROR: Requirement for package PrettyTable excludes a version not excluded in the global list.
Local settings : {'<0.8'}
Global settings: set()
Unexpected : set()
Validating test-requirements.txt
Keeping PrettyTable same as what we have in openstack/requirements repo
Change-Id: I63633d2932757ca23bcea69fd655a2499a5b6d31
The fields(vcpus, memory and disk_capacity) in the Watcher ComputeNode
do not take allocation ratios used for overcommit into account so there
may be disparity between this and the used count.
This patch added some new fields to solve this problem.
Partially Implements: blueprint improve-compute-data-model
Change-Id: Id33496f368fb23cb8e744c7e8451e1cd1397866b
The nova CDM builder code and notification handling
code had some inefficiencies when it came to looking
up a hypevisor to get details. The general pattern
used before was:
1. get the minimal hypervisor information by hypervisor_hostname
2. make another query to get the hypervisor details by id
In the notifications case, it was actually three calls because
the first is listing hyprvisors to filter client-side by service
host.
This change collapses 1 and 2 above into a single API call
to get the hypervisor by hypervisor_hostname with details
which will include the service (compute) host information
which is what get_compute_node_by_id() was being used for.
Now that nothing is using get_compute_node_by_id it is removed.
There is more work we could do in get_compute_node_by_hostname
if the compute API allowed filtering hypervisors by service
host so a TODO is left for that.
One final thing: the TODO in get_compute_node_by_hostname about
there being more than one hypervisor per compute service host
for vmware vcenter is not accurate - nova's vcenter driver
hasn't supported a host:node 1:M topology like that since the
Liberty release [1]. The only in-tree driver in nova that supports
1:M is the ironic baremetal driver, so the comment is updated.
[1] Ifc17c5049e3ed29c8dd130339207907b00433960
Depends-On: https://review.opendev.org/661785/
Change-Id: I5e0e88d7b2dd1a69117ab03e0e66851c687606da
The -x option for bandit changed in 1.6.0 and now
supports glob patterns so use that to correctly
exclude test code from bandit scans.
Since this change requires bandit>=1.6.0, we have
to also fix the networkx requirement to pass the
requirements-check job so that the networkx requirement
matches what is in global-requirements from change
I0a9700926c9a0db93e782c853c33f1aaee3d4876.
Change-Id: I4fc1166daee5d8739296419216d11d684be27c0a
Closes-Bug: #1828419
When the lower-constraints tox target was added, it was assumed the
install_command was just running the install and that the dependencies
and constraints were being set using "deps = ".
This fixed the install_command and deps to follow the expected pattern
so the lower-constraints job actual does install the lower constraints.
This also raises the oslo.context minimum as
Ic96c1f1e1a80099d9dafa95a014fc47f05b88e42 added a dependency on a newer
versions kwarg.
Depends-On: https://review.openstack.org/#/c/647726/
Change-Id: I4cc2c3ac158a607b22295c50f83896969a4007ee
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
When use tox-elower-constraint, we will meet these errors:
1. line 417, in test_clients_ironic:
b"AssertionError: Expected call: Client('1', endpoint_override...
b"Actual call: Client('1', 'http://localhost:6385/'...
2. line 39, in test_wrong_major_version:
b"KeyError: 'HTTP_ACCEPT'"
3. RUN END RESULT_TIMED_OUT:
[untrusted : git.openstack.org/openstack-infra/
zuul-jobs/playbooks/tox/run.yaml@master]
For the first error, The reason is that the unittest for the
ironicclient is too strict and must be adapted to the latest code.
In fact, the watcher can use the previous ironicclient version.
Therefore, we modified the unittest so that the watcher does not
have to rely on the latest ironicclient version.
For the second error, The reason is that we need to update the minimum
version of pecan and webOb.
For the third error, the reason is that the version of the oslo_messaging
is too low.
Change-Id: Icb3eda3d27fa4452e13e2dcd3c016cc76fc2c7c7
This patch set adds API microversion support along
with the first API microversion: start/end time for
CONTINUOUS audits.
APIImpact
Implements: blueprint api-microversioning
Depends-On: I6bb838d777b2c7aa799a70485980e5dc87838456
Change-Id: I17309d80b637f02bc5e6d33294472e02add88f86
This commit adds the functionality of watcher-status CLI for performing
upgrade checks as part of the Stein cycle upgrade-checkers goal.
It only includes a sample check which must be replaced by real checks in
future.
Change-Id: Ic3d066af439797d6f705e805334f729b52ce3aac
Story: 2003657
Task: 26164
We have replaced voluptuous with jsonschema in [1].
Now voluptuous can be removed.
[1]: https://review.openstack.org/#/c/561182/
Change-Id: I99c65ed79ef166839838559a808ee7607389e07a
