Compare commits

...

23 Commits

Author SHA1 Message Date
jgilaber
03073a1b0d Remove outdated zone migration documentation note
In a recent patch [1], a bug in the zone migration strategy was fixed,
which prevented audits using this strategy to create action plans
with both instance and volume migrations. We documented this limitation,
but forgot to remove the note when fixing this bug.

[1] https://review.opendev.org/c/openstack/watcher/+/952115

Change-Id: I2074f2b911dfcbf44716ff30d8ea35a5046b8520
Signed-off-by: jgilaber <jgilaber@redhat.com>
2025-10-01 17:02:40 +02:00
Ronelle Landy
ced0d58d23 Remove last column from strategies table
Removed the "Can Be Triggered from Horizon (UI)"
column and adjusted remaining column widths to
be equal.

Assisted-By: claude-sonnet-4 (Claude Code)
Signed-off-by: Ronelle Landy <rlandy@redhat.com>
Change-Id: I50eef1dee9071eeb532378bd5abcd1d994d299b5
2025-09-26 15:17:56 -04:00
Chandan Kumar (raukadah)
3c8bc6be62 Add user guide for continuous audits
Introduce a new user guide describing how to run continuous audits using
the dummy strategy. The guide covers:
- Overview and state machine
- Creating audits with interval and cron expressions
- Time window constraints (start/end time)
- Monitoring executions and action plan lifecycle
- Managing audits (stop/modify)
- Configuration reference and links to related specs

Closes-Bug: #2120437

Assisted-By: GPT-5 (Cursor)
Assisted-By: claude-sonnet-4 (Claude Code)
Change-Id: I842139271752cedb138e422027020488f22fe248
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
2025-09-23 18:42:30 +05:30
Zuul
635be7a009 Merge "Enable Continuous Audit tests in CI" 2025-09-22 13:30:10 +00:00
Zuul
fe50d270c3 Merge "Resolve deprecation warning from pecan" 2025-09-19 10:50:57 +00:00
Zuul
27961d8574 Merge "Add missing 1.6 API doc in rest version history" 2025-09-17 20:37:00 +00:00
Douglas Viroel
408abaee49 Enable Continuous Audit tests in CI
Scenario continuous audit tests is being added
but will not run by default, since not all stable
branches have the zone_migration fixes needed to
make tests stable.

Depends-On: https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/954264

Change-Id: I5c49b251a49ee439bad024a1cf2569fcbeb2eaf1
Signed-off-by: Douglas Viroel <viroel@gmail.com>
2025-09-17 15:22:16 -03:00
Douglas Viroel
ed0f7457fb Add missing 1.6 API doc in rest version history
The version history was not updated in the patch that
bumped the API to 1.6[1]. This patch adds the missing doc
and also sets 1.6 to the maximun API for the latest release.

[1] https://review.opendev.org/c/openstack/watcher/+/955827

Closes-Bug: #2124938

Change-Id: I62473e84415896387fda8ca6d0982f78d2a1a9f1
Signed-off-by: Douglas Viroel <viroel@gmail.com>
2025-09-17 11:53:31 -03:00
Douglas Viroel
680518ad6d Fix zone migration instance not found issue
When retrieving the list of instances and volumes to propose a
solution, the zone migration strategy can raise an exception for
instance or volume not found, which will make the audit goes to a
failure state. This fix maintains the logic of listing all elements
directly from the client (nova) but now checks if the instance
is already in the model. The storage model check was already fixed
in another patch[1].

[1] cb6fb16097

Closes-Bug: #2098984
Assisted-By: Cursor (claude-3.5-sonnet)

Change-Id: I4c8993f051b797104172047eaae1fe1523eaf7eb
Signed-off-by: Douglas Viroel <viroel@gmail.com>
2025-09-16 16:12:35 -03:00
Douglas Viroel
cada9acced Add unit tests for instance and volume not found in model
The Zone Migration strategy was implemented to list all
instances and volumes from clients (nova and cinder) and
check if they exist in the models. But the code is not
properly treating model exceptions, taking audit to a failure
state when the model doesn't have the requested element.
This patch adds unit tests to validate this scenario, which
should be fixed in a follow up change.
The additional check for volumes in the model was recently
added in [1]

[1] cb6fb16097

Related-Bug: #2098984

Assisted-By: Cursor (claude-3.5-sonnet)

Change-Id: Icf1e5d4c83862c848d11dae994842ad0ee62ba12
Signed-off-by: Douglas Viroel <viroel@gmail.com>
2025-09-16 15:56:13 -03:00
jgilaber
8211475478 Improve unit tests for zone migration strategy
The unit tests were mocking part of the Zone Migration strategy class,
which could hide possible bugs. This patch removes this mocking, leaving
mocked only other classes that are used by the zone migration one.

Additionally, it includes improved suggestions as follow-up from the
review of previous patches, like more explicit comments and additional
asserts of mocked functions.

Assisted-By: Cursor (Claude-4-sonnet)
Change-Id: Ie1894311b0e384ab52b1b3dfe0eb50618eef6c9f
Signed-off-by: jgilaber <jgilaber@redhat.com>
2025-09-16 12:57:54 +02:00
jgilaber
c2ad4b28da Support zone migration audit without compute_nodes
When only running volume migrations, a zone migration
strategy audit without setting compute_nodes should work.

Before this change, an audit with defined storage_pools,
no compute_nodes parameters, and with_attached_volume is set to True
would trigger the migration of the instances attached to the volumes
being migrated.

This patch decouples instance and volume migrations unless the user
explicitely asks for both. When migrating attached volumes, the zone
migration strategy will check for which instances should be migrated
according to the audit parameters, and if the instance the volume is
attached to can be migrated, it will be just after the volume.

On the other hand, when the attached instances should not be migrated
according to user input, only the volumes will be migrated.

In an audit that migrates instnaces but not volumes, the
with_attached_volume parameter will continue doing nothing.

Closes-Bug: 2111429
Change-Id: If641af77ba368946398f9860c537a639d1053f69
Signed-off-by: jgilaber <jgilaber@redhat.com>
2025-09-16 12:20:18 +02:00
Alfredo Moralejo
296856101f Allow volume and vm migrations in zone_migration
Currently, when an audit with strategy zone_migration has added at least
one volume_migration action, it will not process the instances
migrations according to the definition of the `compute_nodes` parameter.
This behavior is unexpected according to the documentation of the
strategy.

This patch is fixing that behavior and making sure that not duplicated
actions are added to the solution, to handle the case where instances
migration actions are created when analyzing the volumes if the
`with_attached_volume` parameter is enabled. The patch is also removing
the method `instances_no_attached` which is not longer used.

Finally, it's adding some unit tests for the new method and fixing the
ones to cover the mixed instances and volumes migration situation.

Closes-Bug: #2109722
Change-Id: Ief7386ab448c2711d0d8a94a77fa9ba189c8b7d2
Signed-off-by: jgilaber <jgilaber@redhat.com>
2025-09-16 12:20:18 +02:00
Alfredo Moralejo
2f2134fc7a Add test for zone_migration with instances and volumes
Currently, unit tests for zone_migration strategy do not include any
test for instances and volumes mixed, which is currently not working as
expected.

This patch is adding two new tests which include both compute_nodes and
storage_pools in audit configuration. One of them is also setting
with_attached_volume option.

These tests will be fixed to validate the expected behavior of the
strategy in the fixing patch.

Related-Bug: #2109722
Change-Id: I496ce3e1f21b7a4165aa47d5862cf0497be79487
Signed-off-by: jgilaber <jgilaber@redhat.com>
2025-09-16 12:20:18 +02:00
jgilaber
cb6fb16097 Use src_type to filter volumes in zone migration
Despite having the src_type paremeter for the storage_pool dictionary as
a mandatory parameter, the value is not being used to filter the volumes
to migrate, using only 'src_pool'.

This change makes 'src_type' optional, since it was ignored until this
point, making it optional keeps the same behaviour by default. If
'src_type' is in the audit parameters, the strategy uses both 'src_pool' and
'src_type' to filter the volumes to migrate.

Closes-Bug: 2111507
Change-Id: Id83a96de85ada1ae6c0e25f8b7fcf54034604911
Signed-off-by: jgilaber <jgilaber@redhat.com>
2025-09-16 12:20:18 +02:00
Zuul
cee72d2bda Merge "Fix missing CORS middleware" 2025-09-15 17:00:56 +00:00
Zuul
cd1154d09c Merge "Add capability to parse forward headers" 2025-09-15 16:23:46 +00:00
Zuul
90f6552c74 Merge "Fix missing X-OpenStack-Request-ID header" 2025-09-15 15:56:03 +00:00
OpenStack Release Bot
0368cea4c1 Update master for stable/2025.2
Add file to the reno documentation build to show release notes for
stable/2025.2.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2025.2.

Sem-Ver: feature
Change-Id: I21fd5f9a613e5e2ee81ae4fe34165f3f4a6ae479
Signed-off-by: OpenStack Release Bot <infra-root@openstack.org>
Generated-By: openstack/project-config:roles/copy-release-tools-scripts/files/release-tools/add_release_note_page.sh
2025-09-15 10:13:29 +00:00
Takashi Kajinami
e1c8961a7c Fix missing CORS middleware
CORS middleware needs to be added to api pipeline to support
Cross-Origin Resource Sharing(CORS). CORS is supported globally by
multiple OpenStack services but is not by watcher, due to lack of
CORS middleware and no mechanism to inject it into api pipeline.

Closes-Bug: #2122347
Change-Id: I6b47abe4f08dc257e9156b254fa60005b82898d7
Signed-off-by: Takashi Kajinami <kajinamit@oss.nttdata.com>
2025-09-13 11:49:11 +09:00
Takashi Kajinami
17a4c96c66 Add capability to parse forward headers
In case standalone watcher-api runs behind forwarders (like load
balancers), it should parse specific request headers to determine
the endpoint url clients actually use.

Add http_proxy_to_wsgi middleware to api pipeline to handle this.

Closes-Bug: #2122353
Change-Id: I27ade17f7ce1649295f92f3ea1af620df63ba1bc
Signed-off-by: Takashi Kajinami <kajinamit@oss.nttdata.com>
2025-09-11 15:50:04 +00:00
Takashi Kajinami
a562880b1c Fix missing X-OpenStack-Request-ID header
Request ID is essential in operating OpenStack services, especially
when troubleshooting some API problems. It allows us to find out
the log lines actually related to a specific request.

However watcher api hasn't returned it properly, so operators had no
way to determine the exact ID they should search.

Add RequestID middleware to return the id in X-OpenStack-Request-Id
header, which is globally used.

Closes-Bug: #2122350
Change-Id: Ie4a8307e8e7e981cedbeaf5fe731dbd47a50bade
Signed-off-by: Takashi Kajinami <kajinamit@oss.nttdata.com>
2025-09-11 15:46:21 +00:00
Takashi Kajinami
7b5243fa8d Resolve deprecation warning from pecan
Resolve the following warning raised from pecan.

```
DeprecationWarning: The function signature for
watcher.api.controllers.root.RootController._route is changing in
the next version of pecan.
Please update to: `def _route(self, args, request)`.
```

Change-Id: I7081cf956a8baa05cd70ced0496ca8192fff979e
Signed-off-by: Takashi Kajinami <kajinamit@oss.nttdata.com>
2025-08-18 22:06:20 +09:00
26 changed files with 1420 additions and 175 deletions

View File

@@ -85,6 +85,8 @@
min_microversion: 1.29
telemetry:
ceilometer_polling_interval: 15
optimize:
run_continuous_audit_tests: true
devstack_plugins:
ceilometer: https://opendev.org/openstack/ceilometer
@@ -233,6 +235,7 @@
datasource: prometheus
extended_attributes_nova_microversion: "2.96"
data_model_collectors_period: 120
run_continuous_audit_tests: true
tempest_plugins:
- watcher-tempest-plugin
# All tests inside watcher_tempest_plugin.tests.scenario with tag "strategy"

View File

@@ -11,62 +11,48 @@ Strategies status matrix
------------------------
.. list-table::
:widths: 20 20 20 20
:widths: 33 33 34
:header-rows: 1
* - Strategy Name
- Status
- Testing
- Can Be Triggered from Horizon (UI)
* - :doc:`actuation`
- Experimental
- Unit, Integration
- No
* - :doc:`basic-server-consolidation`
- Experimental
- Missing
- Yes, with default values
* - :doc:`host_maintenance`
- Supported
- Unit, Integration
- No (requires parameters)
* - :doc:`node_resource_consolidation`
- Supported
- Unit, Integration
- Yes, with default values
* - :doc:`noisy_neighbor`
- Deprecated
- Unit
- N/A
* - :doc:`outlet_temp_control`
- Experimental
- Unit
- Yes, with default values
* - :doc:`saving_energy`
- Experimental
- Unit
- Yes, with default values
* - :doc:`storage_capacity_balance`
- Experimental
- Unit
- Yes, with default values
* - :doc:`uniform_airflow`
- Experimental
- Unit
- Yes, with default values
* - :doc:`vm_workload_consolidation`
- Supported
- Unit, Integration
- Yes, with default values
* - :doc:`workload-stabilization`
- Experimental
- Missing
- Yes, with default values
* - :doc:`workload_balance`
- Supported
- Unit, Integration
- Yes, with default values
* - :doc:`zone_migration`
- Supported (Instance migrations), Experimental (Volume migration)
- Unit, Some Integration
- No

View File

@@ -126,7 +126,7 @@ parameter type default required description
volumes migrate.
``dst_pool`` string None Optional Storage pool to which
volumes migrate.
``src_type`` string None Required Source volume type.
``src_type`` string None Optional Source volume type.
``dst_type`` string None Required Destination volume type
============= ======= ======== ========= ========================
@@ -188,9 +188,6 @@ How to use it ?
-p compute_nodes='[{"src_node": "s01", "dst_node": "d01"}]'
.. note::
* Currently, the strategy will not generate both volume migration and
instance migrations in the same audit. If both are requested,
only volume migrations will be included in the action plan.
* The Cinder model collector is not enabled by default.
If the Cinder model collector is not enabled while deploying Watcher,
the model will become outdated and cause errors eventually.

View File

@@ -0,0 +1,430 @@
..
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
=======================
Using Continuous Audit
=======================
Continuous audits allow Watcher to continuously monitor and optimize your
OpenStack infrastructure based on predefined schedules or intervals. This guide
demonstrates how to set up and use continuous audits with the dummy strategy,
which is useful for testing, development, and understanding the continuous
audit workflow. However, this doc is valid for any other combination of
strategy and goal.
Overview
========
A continuous audit differs from a oneshot audit in that it runs repeatedly
at specified intervals. It supports both time-based intervals
(in seconds) and cron-like expressions for more complex scheduling patterns.
The dummy strategy is a test strategy that doesn't perform actual optimization
but creates sample actions (nop and sleep) to demonstrate the complete audit
workflow. It's ideal for:
- Testing continuous audit functionality
- Development and debugging
- Learning how Watcher works
Prerequisites
=============
Before setting up continuous audits, ensure:
1. Watcher services are running and configured properly
2. You have administrator access to OpenStack
You can verify the services are running:
.. code-block:: bash
$ openstack optimize service list
+----+-------------------------+------------+--------+
| ID | Name | Host | Status |
+----+-------------------------+------------+--------+
| 1 | watcher-decision-engine | controller | ACTIVE |
| 2 | watcher-applier | controller | ACTIVE |
+----+-------------------------+------------+--------+
Continuous Audit State Machine
==============================
You can view the Audit state machine diagram in the Watcher documentation:
`Audit State Machine`_
.. _Audit State Machine: https://docs.openstack.org/watcher/latest/architecture.html#audit-state-machine
Transitions:
- An audit is created and enters the **PENDING** state.
- When the scheduled time arrives, a **PENDING** audit becomes **ONGOING**.
- A continuous audit remains in the **ONGOING** state across executions.
It does not switch to **SUCCEEDED** after each run.
- If an execution fails, the audit transitions to **FAILED** and is no longer
executed.
- Each execution produces a new action plan. When a new action plan is created
by the same continuous audit, previous **RECOMMENDED** action plans are moved
to **CANCELLED**. Only the latest action plan remains in **RECOMMENDED**.
- An administrator can **CANCEL** an audit that is **PENDING** or **ONGOING**.
- An administrator can **SUSPEND** an **ONGOING** audit.
- A **SUSPENDED** audit can be resumed by an administrator, at which point it
becomes **ONGOING** again.
- An administrator can **DELETE** an audit only when its state is
**SUCCEEDED**, **FAILED**, or **CANCELLED**.
.. note::
You can enable the auto-trigger option if you want to automatically apply action
plans generated by continuous audits as soon as they are created.
Depending on the environment, continuous audits are often good candidates for
auto-trigger.
Create a Continuous Audit
--------------------------
Create a continuous audit that will run at regular intervals. You can specify
the interval in seconds or use cron-like expressions.
Using Time Interval (seconds)
------------------------------
This example creates a continuous audit that runs every 5 minutes indefinitely
(300 seconds):
.. code-block:: bash
$ openstack optimize audit create \
--goal dummy \
--strategy dummy \
--audit_type CONTINUOUS \
--interval 300 \
--name "continuous-dummy-5min"
+---------------+--------------------------------------+
| Field | Value |
+---------------+--------------------------------------+
| UUID | 7607cf57-ea05-4e1a-b8d7-34e570f95132 |
| Name | continuous-dummy-5min |
| Created At | 2025-08-12T07:26:18.496536+00:00 |
| Updated At | None |
| Deleted At | None |
| State | PENDING |
| Audit Type | CONTINUOUS |
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
| Interval | 300 |
| Goal | dummy |
| Strategy | dummy |
| Audit Scope | [] |
| Auto Trigger | False |
| Next Run Time | None |
| Hostname | None |
| Start Time | None |
| End Time | None |
| Force | False |
+---------------+--------------------------------------+
Using Cron Expression
----------------------
For more complex scheduling, you can use cron-like expressions. This example
runs the audit every hour at the 15-minute mark:
.. code-block:: bash
$ openstack optimize audit create \
--goal dummy \
--strategy dummy \
--audit_type CONTINUOUS \
--interval "15 * * * *" \
--name "continuous-dummy-hourly"
+---------------+--------------------------------------+
| Field | Value |
+---------------+--------------------------------------+
| UUID | 9cbce4f1-eb75-405a-8f4e-108eb08fdd0a |
| Name | continuous-dummy-hourly |
| Created At | 2025-08-12T07:32:31.469309+00:00 |
| Updated At | None |
| Deleted At | None |
| State | PENDING |
| Audit Type | CONTINUOUS |
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
| Interval | 15 * * * * |
| Goal | dummy |
| Strategy | dummy |
| Audit Scope | [] |
| Auto Trigger | False |
| Next Run Time | None |
| Hostname | None |
| Start Time | None |
| End Time | None |
| Force | False |
+---------------+--------------------------------------+
Time Constraints via start_time and end_time
--------------------------------------------
We can limit when the continuous audit runs by setting start and end times
in a time-interval schedule. The interval can passed in seconds or cron expression.
.. note::
Start and End Time are interpreted in the timezone configured on the host where the
Watcher Decision Engine service is running. We can provide ``start_time`` and
``end_time`` in ISO 8601 format, for example ``'2025-08-13T14:30:00'``.
The example below creates a continuous audit that runs from 12:00 to 13:00
with a 5 minute interval.
.. code-block:: bash
$ openstack optimize audit create \
--goal dummy \
--strategy dummy \
--audit_type CONTINUOUS \
--interval 300 \
--start-time "$(date -d 'today 12:00' +%Y-%m-%dT%H:%M:%S)" \
--end-time "$(date -d 'today 13:00' +%Y-%m-%dT%H:%M:%S)" \
--name "continuous-dummy-5min"
+---------------+--------------------------------------+
| Field | Value |
+---------------+--------------------------------------+
| UUID | dadd279b-1e3d-4c38-aba6-4a730a78589b |
| Name | continuous-dummy-5min |
| Created At | 2025-08-12T08:36:42.924460+00:00 |
| Updated At | None |
| Deleted At | None |
| State | PENDING |
| Audit Type | CONTINUOUS |
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
| Interval | 300 |
| Goal | dummy |
| Strategy | dummy |
| Audit Scope | [] |
| Auto Trigger | False |
| Next Run Time | None |
| Hostname | None |
| Start Time | 2025-08-12T12:00:00 |
| End Time | 2025-08-12T13:00:00 |
| Force | False |
+---------------+--------------------------------------+
Monitoring Continuous Audit Execution
======================================
Create a continuous audit
--------------------------
Create a continuous audit with 5 second interval:
.. code-block:: bash
$ openstack optimize audit create \
--goal dummy \
--strategy dummy \
--audit_type CONTINUOUS \
--interval 5 \
--name "continuous-dummy-5sec"
+---------------+--------------------------------------+
| Field | Value |
+---------------+--------------------------------------+
| UUID | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd |
| Name | continuous-dummy-5sec |
| Created At | 2025-08-12T09:27:33.592575+00:00 |
| Updated At | None |
| Deleted At | None |
| State | PENDING |
| Audit Type | CONTINUOUS |
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
| Interval | 5 |
| Goal | dummy |
| Strategy | dummy |
| Audit Scope | [] |
| Auto Trigger | False |
| Next Run Time | None |
| Hostname | None |
| Start Time | None |
| End Time | None |
| Force | False |
+---------------+--------------------------------------+
Once created, the continuous audit will be automatically scheduled and executed
by the Watcher Decision Engine. You can monitor its progress:
Check Audit Status
------------------
.. code-block:: bash
$ openstack optimize audit show 7d1f1961-41a6-47ae-a94a-cf5e43174fbd
+---------------+--------------------------------------+
| Field | Value |
+---------------+--------------------------------------+
| UUID | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd |
| Name | continuous-dummy-5sec |
| Created At | 2025-08-12T09:27:34+00:00 |
| Updated At | 2025-08-12T09:28:28+00:00 |
| Deleted At | None |
| State | ONGOING |
| Audit Type | CONTINUOUS |
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
| Interval | 5 |
| Goal | dummy |
| Strategy | dummy |
| Audit Scope | [] |
| Auto Trigger | False |
| Next Run Time | 2025-08-12T09:28:33 |
| Hostname | chkumar-devstack-1 |
| Start Time | None |
| End Time | None |
| Force | False |
+---------------+--------------------------------------+
.. note::
The *Next Run Time* is the next time the audit will run. It is calculated based on the
interval and the start and end times.
List Generated Action Plans
---------------------------
Each execution of the continuous audit generates a new action plan:
.. code-block:: bash
$ openstack optimize actionplan list --audit 7d1f1961-41a6-47ae-a94a-cf5e43174fbd
+--------------------------------------+--------------------------------------+-------------+
| UUID | Audit | State |
+--------------------------------------+--------------------------------------+-------------+
| b301dd17-a139-4a45-ade2-b2c2ddf006ef | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd | CANCELLED |
| 22a5bc60-adef-447a-aa27-731b4f5f7ee3 | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd | RECOMMENDED |
+--------------------------------------+--------------------------------------+-------------+
.. note::
In continuous audits, when a new action plan is generated, previous
RECOMMENDED action plans are automatically set to CANCELLED state to
avoid conflicts.
Manage Continuous Audits
========================
Stop a Continuous Audit
------------------------
To stop a continuous audit, update its state:
.. code-block:: bash
$ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace state=CANCELLED
Modify Audit Interval
---------------------
You can change the interval of a running continuous audit:
.. code-block:: bash
$ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace interval=900
The Decision Engine will automatically reschedule the audit with the new
interval.
Modify End Time
---------------
You can change the end time of a running continuous audit:
.. code-block:: bash
$ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace end_time=2025-08-12T14:00:00
Delete a Continuous Audit
--------------------------
In order to delete a continuous audit, the audit state must be
SUCCEEDED, FAILED, or CANCELLED.
An audit with PENDING or ONGOING state cannot be deleted.
To delete an ongoing or pending continuous audit, update its state to
CANCELLED:
.. code-block:: bash
$ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace state=CANCELLED
Then, delete the audit:
.. code-block:: bash
$ openstack optimize audit delete 550e8400-e29b-41d4-a716-446655440000
Configuration Reference
========================
Continuous Audit Intervals
---------------------------
**Numeric Intervals (seconds):**
- Minimum recommended: 60 seconds
- Common values: 300 (5 min), 600 (10 min), 1800 (30 min), 3600 (1 hour)
**Cron Expressions (5 format fields):**
See the `POSIX crontab specification <https://pubs.opengroup.org/onlinepubs/9799919799/utilities/crontab.html>`_.
- ``0 * * * *``: Every hour at minute 0
- ``*/15 * * * *``: Every 15 minutes
- ``0 9-17 * * 1-5``: Every hour during business hours (9 AM - 5 PM, Mon-Fri)
- ``30 2 * * *``: Daily at 2:30 AM
Decision Engine Configuration
-----------------------------
The continuous audit polling interval is configured in ``watcher.conf``:
.. code-block:: ini
[watcher_decision_engine]
# Interval for checking continuous audits (seconds)
continuous_audit_interval = 30
Spec Linked with Continuous Audit
=================================
- `Watcher Continuous Optimization <https://specs.openstack.org/openstack/watcher-specs/specs/newton/implemented/continuously-optimization.html>`_
- `Cron-based continuous audits <https://specs.openstack.org/openstack/watcher-specs/specs/pike/implemented/cron-based-continuous-audits.html>`_
- `Add the start and end time for CONTINUOUS audit <https://specs.openstack.org/openstack/watcher-specs/specs/stein/implemented/add-start-end-time-for-continuous-audit.html>`_

View File

@@ -8,3 +8,4 @@ User Guide
ways-to-install
user-guide
event_type_audit
continuous_type_audit

View File

@@ -9,6 +9,8 @@ namespace = oslo.concurrency
namespace = oslo.db
namespace = oslo.log
namespace = oslo.messaging
namespace = oslo.middleware.cors
namespace = oslo.middleware.http_proxy_to_wsgi
namespace = oslo.policy
namespace = oslo.reports
namespace = oslo.service.periodic_task

View File

@@ -0,0 +1,9 @@
---
fixes:
- |
Fixes a bug in the zone migration strategy where audits would fail due to
an unhandled exception when trying to plan instance migration that exist
in Nova but not in Watcher's compute model.
The strategy now filters out the elements that are not found in the model,
allowing the audit to complete successfully. For more details, please see
`Bug #2098984 <https://bugs.launchpad.net/watcher/+bug/2098984>`_.

View File

@@ -0,0 +1,13 @@
---
fixes:
- |
Previously, when an audit was created with zone_migration strategy and
both storage_pools and compute_nodes parameters are passed, the audit
did not created the required instances migration actions if any volume
migration action was created.
Now, in that situation the audit will create both instance and volume
migrations according to the expected behavior and the limits defined
by the parallelization parameters.
For more information: https://bugs.launchpad.net/watcher/+bug/2109722

View File

@@ -0,0 +1,6 @@
---
fixes:
- |
The `CORS middleware
<https://docs.openstack.org/oslo.middleware/2025.1/admin/cross-project-cors.html>`__
has been added to api pipeline, to support Cross-Origin Resource Sharing.

View File

@@ -0,0 +1,8 @@
---
fixes:
- |
The ``http_proxy_to_wsgi`` middleware has been added to the api pipeline.
Now setting the ``[oslo_middleware] enable_proxy_headers_parsing`` option
to true enables parsing the HTTP headers set by forwarders, to detect
endpoint urls clients actually use.

View File

@@ -0,0 +1,5 @@
---
fixes:
- |
Now request id is returned by Watcher API in the `X-OpenStack-Request-ID`
response header.

View File

@@ -0,0 +1,11 @@
---
fixes:
- |
The zone migration strategy no longer fails when when an audit is created
with defined storage_pools, compute_nodes is not provided, and
with_attached_volume is set to True.
The strategy now creates the required volume migrations, but no instance
migrations. Both volumes and instances will only be migrated if the audit parameters
have both compute_nodes and storage_pools.
See: https://bugs.launchpad.net/watcher/+bug/2111429 for more details.

View File

@@ -0,0 +1,10 @@
---
fixes:
- |
Currently, the zone migration strategy has a ``src_type`` parameter in the ``storage_pools``
input parameter which is ignored, even though it's required when storage_pools is defined.
This patch makes the src_type parameter optional in the zone migration strategy, and when
passed by the user, will use its values to filter the volumes which can be migrated.
For more details: https://launchpad.net/bugs/2111507

View File

@@ -0,0 +1,6 @@
===========================
2025.2 Series Release Notes
===========================
.. release-notes::
:branch: stable/2025.2

View File

@@ -21,6 +21,7 @@ Contents:
:maxdepth: 1
unreleased
2025.2
2025.1
2024.2
2024.1

View File

@@ -19,6 +19,7 @@ oslo.db>=4.44.0 # Apache-2.0
oslo.i18n>=3.20.0 # Apache-2.0
oslo.log>=3.37.0 # Apache-2.0
oslo.messaging>=14.1.0 # Apache-2.0
oslo.middleware>=3.31.0 # Apache-2.0
oslo.policy>=4.5.0 # Apache-2.0
oslo.reports>=1.27.0 # Apache-2.0
oslo.serialization>=2.25.0 # Apache-2.0

View File

@@ -35,6 +35,9 @@ monasca =
oslo.config.opts =
watcher = watcher.conf.opts:list_opts
oslo.config.opts.defaults =
watcher = watcher.common.config:set_lib_defaults
oslo.policy.policies =
watcher = watcher.common.policies:list_rules

View File

@@ -17,6 +17,9 @@
# under the License.
from oslo_middleware import cors
from oslo_middleware import http_proxy_to_wsgi
from oslo_middleware import request_id
import pecan
from watcher.api import acl
@@ -42,13 +45,31 @@ def setup_app(config=None):
app_conf.pop('root'),
logging=getattr(config, 'logging', {}),
debug=CONF.debug,
wrap_app=parsable_error.ParsableErrorMiddleware,
wrap_app=_wrap_app,
**app_conf
)
return acl.install(app, CONF, config.app.acl_public_routes)
def _wrap_app(app):
"""Wraps wsgi app with additional middlewares."""
app = parsable_error.ParsableErrorMiddleware(app)
app = request_id.RequestId(app)
app = http_proxy_to_wsgi.HTTPProxyToWSGI(app)
# This should be the last middleware in the list (which results in
# it being the first in the middleware chain). This is to ensure
# that any errors thrown by other middleware, such as an auth
# middleware - are annotated with CORS headers, and thus accessible
# by the browser.
app = cors.CORS(app, CONF)
return app
class VersionSelectorApplication(object):
def __init__(self):
pc = get_pecan_config()

View File

@@ -48,3 +48,11 @@ and action plans. The ``status_message`` field can be set when transitioning
an action to SKIPPED state, and can also be updated for actions that are
already in SKIPPED state, allowing administrators to fix typos, provide more
detailed explanations, or expand on reasons that were initially omitted.
1.6 (Maximum in 2025.2 Flamingo)
---
Added new server attributes, ``server_flavor_extra_specs`` and
``server_pinned_az``, to the response of ``GET /v1/data_model`` API when
selecting ``compute`` as the ``data_model_type`` parameter. The collection of
these extended attributes is controlled by
``[compute_model] enable_extended_attributes`` configuration option.

View File

@@ -109,7 +109,7 @@ class RootController(rest.RestController):
return Root.convert()
@pecan.expose()
def _route(self, args):
def _route(self, args, request=None):
"""Overrides the default routing behavior.
It redirects the request to the default version of the watcher API
@@ -118,4 +118,4 @@ class RootController(rest.RestController):
if args[0] and args[0] not in self._versions:
args = [self._default_version] + args
return super(RootController, self)._route(args)
return super(RootController, self)._route(args, request)

View File

@@ -16,11 +16,33 @@
# under the License.
from oslo_config import cfg
from oslo_middleware import cors
from watcher.common import rpc
from watcher import version
def set_lib_defaults():
cors.set_defaults(
allow_headers=['X-Auth-Token',
'X-Identity-Status',
'X-Roles',
'X-Service-Catalog',
'X-User-Id',
'X-Tenant-Id',
'X-OpenStack-Request-ID'],
expose_headers=['X-Auth-Token',
'X-Subject-Token',
'X-Service-Token',
'X-OpenStack-Request-ID'],
allow_methods=['GET',
'PUT',
'POST',
'DELETE',
'PATCH']
)
def parse_args(argv, default_config_files=None, default_config_dirs=None):
default_config_files = (default_config_files or
cfg.find_config_files(project='watcher'))

View File

@@ -281,6 +281,7 @@ def prepare_service(argv=(), conf=cfg.CONF):
config.parse_args(argv)
cfg.set_defaults(_options.log_opts,
default_log_levels=_DEFAULT_LOG_LEVELS)
config.set_lib_defaults()
log.setup(conf, 'python-watcher')
conf.log_opt_values(LOG, log.DEBUG)
objects.register_all()

View File

@@ -132,7 +132,7 @@ class ZoneMigration(base.ZoneMigrationBaseStrategy):
"type": "string"
}
},
"required": ["src_pool", "src_type", "dst_type"],
"required": ["src_pool", "dst_type"],
"additionalProperties": False
}
},
@@ -291,19 +291,13 @@ class ZoneMigration(base.ZoneMigrationBaseStrategy):
action_counter = ActionCounter(total_limit,
per_pool_limit, per_node_limit)
for k, targets in iter(filtered_targets.items()):
if k == VOLUME:
self.volumes_migration(targets, action_counter)
elif k == INSTANCE:
if self.volume_count == 0 and self.volume_update_count == 0:
# if with_attached_volume is true,
# instance having attached volumes already migrated,
# migrate instances which does not have attached volumes
if self.with_attached_volume:
targets = self.instances_no_attached(targets)
self.instances_migration(targets, action_counter)
else:
self.instances_migration(targets, action_counter)
instance_targets = filtered_targets.get(INSTANCE, [])
if VOLUME in filtered_targets:
self.volumes_migration(filtered_targets[VOLUME], action_counter,
instance_targets)
if INSTANCE in filtered_targets:
self.instances_migration(instance_targets,
action_counter)
LOG.debug("action total: %s, pools: %s, nodes %s ",
action_counter.total_count,
@@ -357,10 +351,6 @@ class ZoneMigration(base.ZoneMigrationBaseStrategy):
def is_in_use(self, volume):
return getattr(volume, 'status') == IN_USE
def instances_no_attached(self, instances):
return [i for i in instances
if not getattr(i, "os-extended-volumes:volumes_attached")]
def get_host_by_pool(self, pool):
"""Get host name from pool name
@@ -382,11 +372,10 @@ class ZoneMigration(base.ZoneMigrationBaseStrategy):
if node.get("src_node") == src_node:
return node.get("dst_node")
def get_dst_pool_and_type(self, src_pool, src_type):
def get_dst_pool_and_type(self, src_pool):
"""Get destination pool and type from self.migration_storage_pools
:param src_pool: storage pool name
:param src_type: storage volume type
:returns: set of storage pool name and volume type name
"""
for pool in self.migrate_storage_pools:
@@ -394,7 +383,8 @@ class ZoneMigration(base.ZoneMigrationBaseStrategy):
return (pool.get("dst_pool", None),
pool.get("dst_type"))
def volumes_migration(self, volumes, action_counter):
def volumes_migration(self, volumes, action_counter, instance_targets=[]):
instance_target_ids = {instance.id for instance in instance_targets}
for volume in volumes:
if action_counter.is_total_max():
LOG.debug('total reached limit')
@@ -407,7 +397,7 @@ class ZoneMigration(base.ZoneMigrationBaseStrategy):
continue
src_type = volume.volume_type
dst_pool, dst_type = self.get_dst_pool_and_type(pool, src_type)
dst_pool, dst_type = self.get_dst_pool_and_type(pool)
LOG.debug(src_type)
LOG.debug("%s %s", dst_pool, dst_type)
@@ -423,10 +413,14 @@ class ZoneMigration(base.ZoneMigrationBaseStrategy):
else:
self._volume_retype(volume, dst_type)
# if with_attached_volume is True, migrate attaching instances
# if with_attached_volume is True, migrate instances attached
# to the volumes, only if they are possible migration targets
if self.with_attached_volume:
instances = [self.nova.find_instance(dic.get('server_id'))
for dic in volume.attachments]
instances = [
self.nova.find_instance(dic.get('server_id'))
for dic in volume.attachments
if dic.get('server_id') in instance_target_ids
]
self.instances_migration(instances, action_counter)
action_counter.add_pool(pool)
@@ -434,6 +428,10 @@ class ZoneMigration(base.ZoneMigrationBaseStrategy):
def instances_migration(self, instances, action_counter):
for instance in instances:
if self._instance_migration_exists(instance.id):
LOG.debug("A migration action for instance %s already exist",
instance.id)
continue
src_node = getattr(instance, 'OS-EXT-SRV-ATTR:host')
if action_counter.is_total_max():
@@ -481,6 +479,14 @@ class ZoneMigration(base.ZoneMigrationBaseStrategy):
input_parameters=parameters)
self.planned_cold_count += 1
def _instance_migration_exists(self, instance_id):
for action in self.solution.actions:
resource_id = action['input_parameters'].get('resource_id', None)
if (action['action_type'] == 'migrate' and
resource_id == instance_id):
return True
return False
def _volume_migrate(self, volume, dst_pool):
parameters = {"migration_type": "migrate",
"destination_node": dst_pool,
@@ -534,19 +540,34 @@ class ZoneMigration(base.ZoneMigrationBaseStrategy):
return [i for i in self.nova.get_instance_list()
if getattr(i, 'OS-EXT-SRV-ATTR:host') in src_node_list and
self.compute_model.get_instance_by_uuid(i.id)]
self.compute_model.has_node(i.id)]
def get_volumes(self):
"""Get migrate target volumes
:returns: volume list on src pools and storage scope
"""
def _is_src_type(volume, src_type):
return (src_type is None or
(src_type is not None and volume.volume_type == src_type))
src_pool_list = self.get_src_pool_list()
target_volumes = []
for volume in self.cinder.get_volume_list():
if not self.storage_model.has_node(volume.id):
# skip volumes that are not in the storage model to satisfy
# scope constraints
continue
for migrate_input in self.migrate_storage_pools:
src_pool = migrate_input["src_pool"]
src_type = migrate_input.get("src_type")
if (getattr(volume, 'os-vol-host-attr:host') == src_pool and
_is_src_type(volume, src_type)):
target_volumes.append(volume)
# once the volume satisfies one the storage_pools
# inputs, we don't need to check the rest
break
return [i for i in self.cinder.get_volume_list()
if getattr(i, 'os-vol-host-attr:host') in src_pool_list and
self.storage_model.get_volume_by_uuid(i.id)]
return target_volumes
def filtered_targets(self):
"""Filter targets

View File

@@ -24,6 +24,12 @@ class TestRoot(base.FunctionalTest):
# Check fields are not empty
[self.assertNotIn(f, ['', []]) for f in data.keys()]
# NOTE(tkajinam): Request ID should be present in any request so we verify
# it only at the root path.
def test_request_id(self):
resp = self.get_json('/', path_prefix='', return_json=False)
self.assertIn('X-OpenStack-Request-Id', resp.headers)
class TestV1Root(base.FunctionalTest):

View File

@@ -1332,11 +1332,6 @@ class TestAuditZoneMigration(TestPostBase):
audit_input_dict = self._prepare_audit_params(zm_params)
response = self.post_json('/audits', audit_input_dict,
expect_errors=True)
response = self.post_json('/audits', audit_input_dict)
self.assertEqual("application/json", response.content_type)
self.assertEqual(HTTPStatus.BAD_REQUEST, response.status_int)
expected_error_msg = ("'src_type' is a required property")
self.assertTrue(response.json['error_message'])
self.assertIn(expected_error_msg, response.json['error_message'])
assert not mock_trigger_audit.called
self.assertEqual(HTTPStatus.CREATED, response.status_int)