Commit Graph

2556 Commits

Author SHA1 Message Date
jgilaber
c6302edeca Handle missing dst_node parameter in zone_migration
For compute nodes, nova works fine if a destination node is not
specified, so this change makes sure we're not passing None when the
user does not set one to avoid an error.

Partial-Bug: 2108988

Change-Id: Ida1f18b97697c041819e29f935aa5e232848226a
2025-05-16 13:51:47 +02:00
jgilaber
63626d6fc3 Add test for missing destination in zone migration
Add some tests to show that the zone migration strategy generates
problematic input parameters for actions in some cases when destination
parameters are not passed for instances or volumes.

Change-Id: Idc3af0e6d9d2d5388ff3d152d81e63364758607b
2025-05-15 13:00:39 +02:00
Zuul
59607f616a Merge "Drop nova command reference from the code" 2025-05-13 12:39:25 +00:00
Chandan Kumar (raukadah)
3f6c7e406a Drop nova command reference from the code
In DevStack environment, nova service-list command does not
exist. Distro suggests to install python-novaclient from package.

In Strategies documentation, we generate the docs from following
code.[1]
```
       * - ``migration``
         - .. watcher-term:: watcher.applier.actions.migration.Migrate
       * - ``change_nova_service_state``
         - .. watcher-term:: watcher.applier.actions.change_nova_service_state.ChangeNovaServiceState
```
and with in code, we use nova python binding to get list services[2]
and we are not calling openstack cli reference with in the code.

Documenting the equivalent openstack command does not seems to be useful
in the help text as we are using python binding.

Links:
[1]. c4acce91d6/doc/source/strategies/host_maintenance.rst (L45)
[2]. c4acce91d6/watcher/common/nova_helper.py (L150-L152)

Change-Id: I0c663c9741fae94bdb9c30f46d3d396325a33948
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
2025-05-13 08:49:54 +05:30
Zuul
fd3d8b67ff Merge "Set number of decimal digits in efficacy indicator" 2025-05-13 00:06:07 +00:00
Zuul
c73f126b15 Merge "Deprecated Noisy Neighbor strategy" 2025-05-12 23:50:12 +00:00
Douglas Viroel
17d1cf535a Deprecated Noisy Neighbor strategy
Noisy neighbor strategy is a proof of concept strategy that was
built based on LLC metric, which is not available in Nova since
Victoria release[1].
This patch marks this strategy as deprecated, to be removed in
future releases.

[1] https://docs.openstack.org/releasenotes/nova/victoria.html#relnotes-22-0-0-unmaintained-victoria-upgrade-notes

Change-Id: I940b88555007312c76a86706bd44a38fbcf7701e
2025-05-12 15:44:39 -03:00
jgilaber
0ed3d4de83 Set number of decimal digits in efficacy indicator
Configure the numeric type of the EfficacyIndicator value to use Float.
Add a new column named data and deprecate the existing value columen.
With the current model, value will use the default scale of the
Decimal type of mysql, which in some enviornments is 0.

This change also adds a test with mysql as backend to reproduce the
issue, since the existing tests using sqlite do not reproduce the
problem, as well as some simple migration tests.

Closes-Bug: #2103458
Change-Id: Ib281fa32e902d2181449091f493d6506b5199094
2025-05-07 16:20:31 +02:00
jgilaber
6c5845721b Add test for EfficacyIndicator value in mysql
Add a test with mysql as backend to show that the current
EfficacyIndicator model does not store any decimal digit for the value.

Change-Id: I0cdbd7d87cd6869a10b48eda3d59558831c8dd36
2025-05-07 16:20:03 +02:00
Sean Mooney
77e7e4ef7b drop jammy jobs
ubuntu jammy is nolonger part of the required
testing runtime so this change simply removes
the jammy jobs.

Change-Id: I1e3bbb14cea5b856e8146f3a32d60c3a4ffdcfcc
2025-05-02 17:41:06 +00:00
Sean Mooney
f38ab70ba4 drop suse supprot in the devstack plugin
suse has not been a testing runtime for a few releases
and we have no jobs currently validating it still work.

this change just removes the suse specific logic

Change-Id: I357fa71704af7aa6239054ede29d0fdcdc3fb8b5
2025-05-02 17:41:00 +00:00
Sean Mooney
7aabd6dd5a update pre-commit hook versions
This updates all hooks to there latest verions
notable this adds python 3.13 support to autopep8

Change-Id: Ia67ed74c9942ff26bb1f8c1d72bf57aedfcd3846
2025-05-02 17:40:50 +00:00
Zuul
1b12e80882 Merge "Make prometheus the default devstack example" 2025-05-02 13:50:50 +00:00
Zuul
9f685a8cf1 Merge "[host_maintenance] Pass des hostname in add_action solution" 2025-05-02 13:45:57 +00:00
Sean Mooney
57b248f9fe Add support for pyproject.toml and wsgi module paths
pip 23.1 removed the "setup.py install" fallback for projects that do
not have pyproject.toml and now uses a pyproject.toml which is vendored
in pip [1][2]. pip 24.2 has now deprecated a similar fallback to
"setup.py develop" and plans to fully remove this in pip 25.0 [3][4][5].
pbr supports editable installs since 6.0.0

pip 25.1 has now been released and the removal is complete.
by adding our own minimal pyproject.toml to ensure we are using the
correct build system.

This change also requires that we adapt how we generate our wsgi
entry point. when pyproject.toml is used the wsgi console script is
not generated in an editbale install such as is used in devstck

To adress this we need to refactor our usage of our wsgi applciation
to use a module path instead. This change does not remove
the declaration of our wsgi_scrtip entry point but it shoudl
be considered deprecated and it will be removed in the future.

To unblock the gate the devstack plugin is modifed to to deploy
using the wsgi module instead of the console script.

Finally supprot for the mod_wsgi wsgi mode is removed.
that was deprecated in devstack a few cycle ago and
support was removed in I8823e98809ed6b66c27dbcf21a00eea68ef403e8

[1] https://pip.pypa.io/en/stable/news/#v23-1
[2] https://github.com/pypa/pip/issues/8368
[3] https://pip.pypa.io/en/stable/news/#v24-2
[4] https://github.com/pypa/pip/issues/11457
[5] https://ichard26.github.io/blog/2024/08/whats-new-in-pip-24.2/
Closes-Bug: #2109608

Depends-on: https://review.opendev.org/c/openstack/watcher/+/948502
Change-Id: Iad77939ab0403c5720c549f96edfc77d2b7d90ee
2025-05-01 00:19:59 +00:00
Chandan Kumar (raukadah)
278cb7e98c [host_maintenance] Pass des hostname in add_action solution
Currently we are passing src_node and des_node uuid when we try to run
migrate action.

In the watcher-applier log, migration fails with following exception
```
Nova client exception occurred while live migrating instance <uuid>Exception: Compute host <uuid> could not be found
```
Based on 57f55190ff/watcher/applier/actions/migration.py (L122)
and
57f55190ff/watcher/common/nova_helper.py (L322),
live_migrate_instance expects destination hostname not uuid.

This cr replaces dest_node uuid to hostname.

Closes-Bug: #2109309

Change-Id: I3911ff24ea612f69dddae5eab15fabb4891f938d
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
2025-04-25 15:51:20 +05:30
jgilaber
2c76da2868 Make prometheus the default devstack example
Change the devstack local.conf samples and devstack multinode
contributor doc to demonstrate deploying watcher with prometheus as
datasource instead of gnocchi. Keep the gnocchi as an alternative
deployment example.

Depends-On: https://review.opendev.org/c/openstack/watcher/+/946230
Depends-On: https://review.opendev.org/c/openstack/devstack-plugin-prometheus/+/946254

Change-Id: I721b550a03f9e5350a3f1ab10292faa1c50049a7
2025-04-24 16:06:50 +02:00
Alfredo Moralejo
c4acce91d6 Add real-data based tests to experimental and weekly pipelines
This job is adding a new job using prometheus datastore and real
workload data into the experimental pipeline so that we can run it
on-demand.

Also, it is adding it to the weekly periodic pipeline as agreed on
Watcher meeting.

Also I am excluding strategies execution with annotation `real_load` in
non-real-load jobs.

Finally, I'm moving the project configuration to the end of the file
as requested in the comments, as it's the usual location by convention.

Change-Id: Id41efda2f0dd8b1521df3f6179c3504f298e0e59
2025-04-15 16:11:21 +02:00
Zuul
adbcac9319 Merge "Replace watcherclient functional job with python-watcherclient-functional" 2025-04-15 13:32:34 +00:00
Zuul
c9a1d06e7c Merge "Aggregate by fqdn label instead instance in host cpu metrics" 2025-04-08 17:37:10 +00:00
Zuul
25c1a8207f Merge "Drop sg_core prometheus related vars" 2025-04-08 11:55:39 +00:00
Chandan Kumar (raukadah)
0702cb3869 Drop sg_core prometheus related vars
The depends-on pr removes the installation of promotheus[1] and node
exporter[2] from sg_core. We no longer need to define those vars in
the devstack config.

Links:
[1]. https://github.com/openstack-k8s-operators/sg-core/pull/21
[2]. https://github.com/openstack-k8s-operators/sg-core/pull/23

Note: We do not need to enable sg_core service on compute node,
so removing it's plugin call.

Change-Id: Ie8645813a360605635de4dff9e8d1ba0d7a0cdc3
Signed-off-by: Chandan Kumar (raukadah) <raukadah@gmail.com>
2025-04-04 19:36:54 +05:30
Zuul
03c107a4ce Merge "Imported Translations from Zanata" 2025-04-03 18:49:08 +00:00
Alfredo Moralejo
c7158b08d1 Aggregate by fqdn label instead instance in host cpu metrics
While in a regular case a specific metric for a specific host will be
provider by a single instance (exporter) so aggregating by label and by
intances should be the same, it is more correct to aggregate by the same
label that the one we use to filter the metrics.

This is follow up of https://review.opendev.org/c/openstack/watcher/+/944795

Related-Bug: #2103451

Change-Id: Ia61f051547ddc51e0d1ccd5a56485ab49ce84c2e
2025-04-02 15:36:17 +02:00
Zuul
035e6584c7 Merge "Query by fqdn_label instead of instance for host metrics" 2025-03-20 12:50:28 +00:00
Chandan Kumar (raukadah)
253e97678c Replace watcherclient functional job with python-watcherclient-functional
https://review.opendev.org/c/openstack/python-watcherclient/+/943132
Move functional tests from watcher_tempest_plugin to watcherclient and
adds new zuul job based on devstack-tox-functional to run functional tests.

This pr replaces the existing zuul job using tempest regex with
devstack tox functional job. The new job will run only watcher/api
changes.

Closes-Bug: #2100741

Depends-On: https://review.opendev.org/c/openstack/python-watcherclient/+/943132

Change-Id: Ic2371745fe8aaf6f283151111fec4f92ea6bdf69
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
2025-03-20 10:04:13 +00:00
OpenStack Proposal Bot
c7bb1fe52d Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: Ifbc199a502734046ea57f5190c328447d66013ce
2025-03-20 03:49:09 +00:00
Alfredo Moralejo
a65e7e9b59 Query by fqdn_label instead of instance for host metrics
Currently we are using `instance` label to query about host metrics to
prometheus. This label is assigned to the url of each endpoint being
scrapped.

While this work fine in one-exporter-per-compute cases as the driver is
mapping the fqdn_label value to the `instance` label value, it fails
when there are more that one target with the same value for the fqdn
label. This is a valid case, to be able to query by fqdn and do not
care about what exporter in the host is providing the metric.

This patch is changing the queries we use for hosts to be based on the
fqdn_label instead of the instance one. To implement it, we are also
simplifying the way we check the metric exist for the host by converting
prometheus_fqdn_instance_map into a prometheus_fqdn_labels set
which stores the list of fqdn found in  prometheus.

Closes-Bug: #2103451
Change-Id: I3bcc317441b73da5c876e53edd4622370c6d575e
2025-03-19 15:25:24 +01:00
OpenStack Release Bot
b671550c91 Update master for stable/2025.1
Add file to the reno documentation build to show release notes for
stable/2025.1.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2025.1.

Sem-Ver: feature
Change-Id: Ie7a1845d7b02b852e776ed8ec73598caab2fb5c6
2025-03-13 13:51:55 +00:00
Zuul
52bba70fec Merge "Do not collect node_exporter cpu metrics in prometheus job" 2025-03-12 14:38:24 +00:00
Zuul
f2ee231f14 Merge "pre-commit: Integrate bandit" 14.0.0.0rc1 14.0.0 2025-03-11 09:58:29 +00:00
Zuul
3861701f4a Merge "Replace deprecated abc.abstractproperty" 2025-03-11 09:47:31 +00:00
Zuul
d167134265 Merge "Drop implicit test dependency on iso8601" 2025-03-11 09:47:30 +00:00
Douglas Viroel
539be503f0 Do not collect node_exporter cpu metrics in prometheus job
Prometheus job already injects fake metrics for hosts and
instances. This patch disables node_exporter cpu metric
collector to avoid mixing both real and fake values in
test execution.

Depends-On: https://review.opendev.org/c/openstack/devstack-plugin-prometheus/+/942181
Depends-On: https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/943825

Change-Id: Ie2b7269ab15af6190ce72ba2b149e84375f4419d
2025-03-08 10:57:31 -03:00
Sean Mooney
bbf5c41cab Add epoxy prelude
This change added the prelude for the 2025.1 Expoxy release cycle.

Change-Id: I8223842a57491a91c565e47bd1819db4d142e628
2025-03-05 17:57:55 +00:00
Takashi Kajinami
df3d67a4ed Replace deprecated abc.abstractproperty
It was deprecated in Python 3.3 [1].

[1] https://docs.python.org/3.13/whatsnew/3.3.html#abc

Change-Id: Ibd98cb93f697a6da6a6bc5a5030640a262c7a66b
2025-03-02 15:36:48 +09:00
Takashi Kajinami
82f1c720dd Drop implicit test dependency on iso8601
The library has been missing from the test requirements although it is
directly used. Replace it by the built-in datetime module to get rid
of the unmaintained direct dependency.

Change-Id: I1d08b38862b54fee4c7c26161f59264fb3f2ce51
2025-03-01 15:23:15 +09:00
Zuul
77a30ef281 Merge "Enable prometheus datasource in watcher-prometheus-integration job" 2025-02-28 13:26:10 +00:00
Zuul
383751904c Merge "Further database refactoring" 2025-02-27 11:52:59 +00:00
Zuul
6a1f19d314 Merge "Deprecate Monasca data source" 2025-02-27 11:45:15 +00:00
Douglas Viroel
342fe8882a Enable prometheus datasource in watcher-prometheus-integration job
Enable prometheus as datasouce in tempest configuration,
to enable metric generation needed to run some scenario
tests. It is enabled on the watcher-prometheus-integration
job

Depends-On: https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/942141
Depends-On: https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/942308

Change-Id: I2b657782aedf61d89766fcd18bb453b62c0b0e3b
2025-02-22 10:46:01 -03:00
Chandan Kumar (raukadah)
7fcca0cc46 Enable prometheus and node_exporter from devstack-plugin-prometheus
https://opendev.org/openstack/devstack-plugin-prometheus is the new
devstack plugin providing functionality to install/configure
prometheus/node_exporter.

It will replace sg_core devstack plugin in future.

Depends-On: https://review.opendev.org/c/openstack/watcher/+/938893
Depends-On: https://review.opendev.org/c/openstack/devstack-plugin-prometheus/+/940426

Change-Id: Ia75e6597275b36c04cde653c16f7d45ed23bc261
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
2025-02-19 08:49:53 -03:00
Takashi Kajinami
977f014cba Deprecate Monasca data source
The Monasca project was marked inactive during 2023.1. Although we have
seen multiple people showing interest to keep the project, we haven't
seen any real progress.

Because the project is likely retired soon, let's deprecate the feature
dependent on Monasca so that we can remove it in a future release.

Change-Id: Ifd64f5ba59bbac238ff62302ec36a3e36954d6d0
2025-02-16 18:45:31 +09:00
James Page
753c44b0c4 Further database refactoring
More refactoring of the SQLAlchemy database layer to improve
compatility with eventlet on newer Pythons.

Inspired by 0ce2c41404

Related-Bug: 2067815
Change-Id: Ib5e9aa288232cc1b766bbf2a8ce2113d5a8e2f7d
2025-02-14 11:42:47 +00:00
Takashi Kajinami
dd0082c343 pre-commit: Integrate bandit
Run bandit check from per-commit so that the check is executed in pep8
job.

Also remove requirements installed automatically by pre-commit from
test-requirements.

Change-Id: I45af8c47afb262882ebbee74ae52446fed741e26
2025-02-10 22:50:34 +09:00
Takashi Kajinami
5f6fbaea56 Remove unused os-api-ref from test requirements
It is used when building API reference but is not used in any testing.

Change-Id: I6af7c7b110b338acad10eccf42344a338afbc915
2025-02-09 08:14:17 +09:00
Takashi Kajinami
6b81b34b27 Drop import fallback for Python 2
cPickle no longer exists in Python 3 and pickle should be used always.

Change-Id: I5ddedb3e996d9a0679bab38ea94263886274ece4
2025-02-09 08:04:36 +09:00
Zuul
961bbb9460 Merge "Update master for stable/2024.2" 2025-02-06 08:07:22 +00:00
Zuul
d56e8ee65a Merge "X-Project-Name key in test code was duplicated" 2025-02-03 18:29:23 +00:00
Zuul
4527f89d8d Merge "Add support for instance metrics to prometheus datasource" 2025-02-03 13:22:28 +00:00