Add user guide for continuous audits
Introduce a new user guide describing how to run continuous audits using the dummy strategy. The guide covers: - Overview and state machine - Creating audits with interval and cron expressions - Time window constraints (start/end time) - Monitoring executions and action plan lifecycle - Managing audits (stop/modify) - Configuration reference and links to related specs Closes-Bug: #2120437 Assisted-By: GPT-5 (Cursor) Assisted-By: claude-sonnet-4 (Claude Code) Change-Id: I842139271752cedb138e422027020488f22fe248 Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
This commit is contained in:
430
doc/source/user/continuous_type_audit.rst
Normal file
430
doc/source/user/continuous_type_audit.rst
Normal file
@@ -0,0 +1,430 @@
|
|||||||
|
..
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||||
|
not use this file except in compliance with the License. You may obtain
|
||||||
|
a copy of the License at
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software
|
||||||
|
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||||
|
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||||
|
License for the specific language governing permissions and limitations
|
||||||
|
under the License.
|
||||||
|
|
||||||
|
|
||||||
|
=======================
|
||||||
|
Using Continuous Audit
|
||||||
|
=======================
|
||||||
|
|
||||||
|
Continuous audits allow Watcher to continuously monitor and optimize your
|
||||||
|
OpenStack infrastructure based on predefined schedules or intervals. This guide
|
||||||
|
demonstrates how to set up and use continuous audits with the dummy strategy,
|
||||||
|
which is useful for testing, development, and understanding the continuous
|
||||||
|
audit workflow. However, this doc is valid for any other combination of
|
||||||
|
strategy and goal.
|
||||||
|
|
||||||
|
Overview
|
||||||
|
========
|
||||||
|
|
||||||
|
A continuous audit differs from a oneshot audit in that it runs repeatedly
|
||||||
|
at specified intervals. It supports both time-based intervals
|
||||||
|
(in seconds) and cron-like expressions for more complex scheduling patterns.
|
||||||
|
|
||||||
|
The dummy strategy is a test strategy that doesn't perform actual optimization
|
||||||
|
but creates sample actions (nop and sleep) to demonstrate the complete audit
|
||||||
|
workflow. It's ideal for:
|
||||||
|
|
||||||
|
- Testing continuous audit functionality
|
||||||
|
- Development and debugging
|
||||||
|
- Learning how Watcher works
|
||||||
|
|
||||||
|
Prerequisites
|
||||||
|
=============
|
||||||
|
|
||||||
|
Before setting up continuous audits, ensure:
|
||||||
|
|
||||||
|
1. Watcher services are running and configured properly
|
||||||
|
2. You have administrator access to OpenStack
|
||||||
|
|
||||||
|
|
||||||
|
You can verify the services are running:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ openstack optimize service list
|
||||||
|
+----+-------------------------+------------+--------+
|
||||||
|
| ID | Name | Host | Status |
|
||||||
|
+----+-------------------------+------------+--------+
|
||||||
|
| 1 | watcher-decision-engine | controller | ACTIVE |
|
||||||
|
| 2 | watcher-applier | controller | ACTIVE |
|
||||||
|
+----+-------------------------+------------+--------+
|
||||||
|
|
||||||
|
|
||||||
|
Continuous Audit State Machine
|
||||||
|
==============================
|
||||||
|
|
||||||
|
You can view the Audit state machine diagram in the Watcher documentation:
|
||||||
|
`Audit State Machine`_
|
||||||
|
|
||||||
|
.. _Audit State Machine: https://docs.openstack.org/watcher/latest/architecture.html#audit-state-machine
|
||||||
|
|
||||||
|
|
||||||
|
Transitions:
|
||||||
|
|
||||||
|
- An audit is created and enters the **PENDING** state.
|
||||||
|
|
||||||
|
- When the scheduled time arrives, a **PENDING** audit becomes **ONGOING**.
|
||||||
|
|
||||||
|
- A continuous audit remains in the **ONGOING** state across executions.
|
||||||
|
It does not switch to **SUCCEEDED** after each run.
|
||||||
|
|
||||||
|
- If an execution fails, the audit transitions to **FAILED** and is no longer
|
||||||
|
executed.
|
||||||
|
|
||||||
|
- Each execution produces a new action plan. When a new action plan is created
|
||||||
|
by the same continuous audit, previous **RECOMMENDED** action plans are moved
|
||||||
|
to **CANCELLED**. Only the latest action plan remains in **RECOMMENDED**.
|
||||||
|
|
||||||
|
- An administrator can **CANCEL** an audit that is **PENDING** or **ONGOING**.
|
||||||
|
|
||||||
|
- An administrator can **SUSPEND** an **ONGOING** audit.
|
||||||
|
|
||||||
|
- A **SUSPENDED** audit can be resumed by an administrator, at which point it
|
||||||
|
becomes **ONGOING** again.
|
||||||
|
|
||||||
|
- An administrator can **DELETE** an audit only when its state is
|
||||||
|
**SUCCEEDED**, **FAILED**, or **CANCELLED**.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
You can enable the auto-trigger option if you want to automatically apply action
|
||||||
|
plans generated by continuous audits as soon as they are created.
|
||||||
|
Depending on the environment, continuous audits are often good candidates for
|
||||||
|
auto-trigger.
|
||||||
|
|
||||||
|
|
||||||
|
Create a Continuous Audit
|
||||||
|
--------------------------
|
||||||
|
|
||||||
|
Create a continuous audit that will run at regular intervals. You can specify
|
||||||
|
the interval in seconds or use cron-like expressions.
|
||||||
|
|
||||||
|
Using Time Interval (seconds)
|
||||||
|
------------------------------
|
||||||
|
|
||||||
|
This example creates a continuous audit that runs every 5 minutes indefinitely
|
||||||
|
(300 seconds):
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ openstack optimize audit create \
|
||||||
|
--goal dummy \
|
||||||
|
--strategy dummy \
|
||||||
|
--audit_type CONTINUOUS \
|
||||||
|
--interval 300 \
|
||||||
|
--name "continuous-dummy-5min"
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
| Field | Value |
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
| UUID | 7607cf57-ea05-4e1a-b8d7-34e570f95132 |
|
||||||
|
| Name | continuous-dummy-5min |
|
||||||
|
| Created At | 2025-08-12T07:26:18.496536+00:00 |
|
||||||
|
| Updated At | None |
|
||||||
|
| Deleted At | None |
|
||||||
|
| State | PENDING |
|
||||||
|
| Audit Type | CONTINUOUS |
|
||||||
|
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
|
||||||
|
| Interval | 300 |
|
||||||
|
| Goal | dummy |
|
||||||
|
| Strategy | dummy |
|
||||||
|
| Audit Scope | [] |
|
||||||
|
| Auto Trigger | False |
|
||||||
|
| Next Run Time | None |
|
||||||
|
| Hostname | None |
|
||||||
|
| Start Time | None |
|
||||||
|
| End Time | None |
|
||||||
|
| Force | False |
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
|
||||||
|
|
||||||
|
Using Cron Expression
|
||||||
|
----------------------
|
||||||
|
|
||||||
|
For more complex scheduling, you can use cron-like expressions. This example
|
||||||
|
runs the audit every hour at the 15-minute mark:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ openstack optimize audit create \
|
||||||
|
--goal dummy \
|
||||||
|
--strategy dummy \
|
||||||
|
--audit_type CONTINUOUS \
|
||||||
|
--interval "15 * * * *" \
|
||||||
|
--name "continuous-dummy-hourly"
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
| Field | Value |
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
| UUID | 9cbce4f1-eb75-405a-8f4e-108eb08fdd0a |
|
||||||
|
| Name | continuous-dummy-hourly |
|
||||||
|
| Created At | 2025-08-12T07:32:31.469309+00:00 |
|
||||||
|
| Updated At | None |
|
||||||
|
| Deleted At | None |
|
||||||
|
| State | PENDING |
|
||||||
|
| Audit Type | CONTINUOUS |
|
||||||
|
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
|
||||||
|
| Interval | 15 * * * * |
|
||||||
|
| Goal | dummy |
|
||||||
|
| Strategy | dummy |
|
||||||
|
| Audit Scope | [] |
|
||||||
|
| Auto Trigger | False |
|
||||||
|
| Next Run Time | None |
|
||||||
|
| Hostname | None |
|
||||||
|
| Start Time | None |
|
||||||
|
| End Time | None |
|
||||||
|
| Force | False |
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
|
||||||
|
Time Constraints via start_time and end_time
|
||||||
|
--------------------------------------------
|
||||||
|
|
||||||
|
We can limit when the continuous audit runs by setting start and end times
|
||||||
|
in a time-interval schedule. The interval can passed in seconds or cron expression.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
Start and End Time are interpreted in the timezone configured on the host where the
|
||||||
|
Watcher Decision Engine service is running. We can provide ``start_time`` and
|
||||||
|
``end_time`` in ISO 8601 format, for example ``'2025-08-13T14:30:00'``.
|
||||||
|
|
||||||
|
|
||||||
|
The example below creates a continuous audit that runs from 12:00 to 13:00
|
||||||
|
with a 5 minute interval.
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ openstack optimize audit create \
|
||||||
|
--goal dummy \
|
||||||
|
--strategy dummy \
|
||||||
|
--audit_type CONTINUOUS \
|
||||||
|
--interval 300 \
|
||||||
|
--start-time "$(date -d 'today 12:00' +%Y-%m-%dT%H:%M:%S)" \
|
||||||
|
--end-time "$(date -d 'today 13:00' +%Y-%m-%dT%H:%M:%S)" \
|
||||||
|
--name "continuous-dummy-5min"
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
| Field | Value |
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
| UUID | dadd279b-1e3d-4c38-aba6-4a730a78589b |
|
||||||
|
| Name | continuous-dummy-5min |
|
||||||
|
| Created At | 2025-08-12T08:36:42.924460+00:00 |
|
||||||
|
| Updated At | None |
|
||||||
|
| Deleted At | None |
|
||||||
|
| State | PENDING |
|
||||||
|
| Audit Type | CONTINUOUS |
|
||||||
|
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
|
||||||
|
| Interval | 300 |
|
||||||
|
| Goal | dummy |
|
||||||
|
| Strategy | dummy |
|
||||||
|
| Audit Scope | [] |
|
||||||
|
| Auto Trigger | False |
|
||||||
|
| Next Run Time | None |
|
||||||
|
| Hostname | None |
|
||||||
|
| Start Time | 2025-08-12T12:00:00 |
|
||||||
|
| End Time | 2025-08-12T13:00:00 |
|
||||||
|
| Force | False |
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
|
||||||
|
Monitoring Continuous Audit Execution
|
||||||
|
======================================
|
||||||
|
|
||||||
|
Create a continuous audit
|
||||||
|
--------------------------
|
||||||
|
|
||||||
|
Create a continuous audit with 5 second interval:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ openstack optimize audit create \
|
||||||
|
--goal dummy \
|
||||||
|
--strategy dummy \
|
||||||
|
--audit_type CONTINUOUS \
|
||||||
|
--interval 5 \
|
||||||
|
--name "continuous-dummy-5sec"
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
| Field | Value |
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
| UUID | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd |
|
||||||
|
| Name | continuous-dummy-5sec |
|
||||||
|
| Created At | 2025-08-12T09:27:33.592575+00:00 |
|
||||||
|
| Updated At | None |
|
||||||
|
| Deleted At | None |
|
||||||
|
| State | PENDING |
|
||||||
|
| Audit Type | CONTINUOUS |
|
||||||
|
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
|
||||||
|
| Interval | 5 |
|
||||||
|
| Goal | dummy |
|
||||||
|
| Strategy | dummy |
|
||||||
|
| Audit Scope | [] |
|
||||||
|
| Auto Trigger | False |
|
||||||
|
| Next Run Time | None |
|
||||||
|
| Hostname | None |
|
||||||
|
| Start Time | None |
|
||||||
|
| End Time | None |
|
||||||
|
| Force | False |
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
|
||||||
|
Once created, the continuous audit will be automatically scheduled and executed
|
||||||
|
by the Watcher Decision Engine. You can monitor its progress:
|
||||||
|
|
||||||
|
Check Audit Status
|
||||||
|
------------------
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ openstack optimize audit show 7d1f1961-41a6-47ae-a94a-cf5e43174fbd
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
| Field | Value |
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
| UUID | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd |
|
||||||
|
| Name | continuous-dummy-5sec |
|
||||||
|
| Created At | 2025-08-12T09:27:34+00:00 |
|
||||||
|
| Updated At | 2025-08-12T09:28:28+00:00 |
|
||||||
|
| Deleted At | None |
|
||||||
|
| State | ONGOING |
|
||||||
|
| Audit Type | CONTINUOUS |
|
||||||
|
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
|
||||||
|
| Interval | 5 |
|
||||||
|
| Goal | dummy |
|
||||||
|
| Strategy | dummy |
|
||||||
|
| Audit Scope | [] |
|
||||||
|
| Auto Trigger | False |
|
||||||
|
| Next Run Time | 2025-08-12T09:28:33 |
|
||||||
|
| Hostname | chkumar-devstack-1 |
|
||||||
|
| Start Time | None |
|
||||||
|
| End Time | None |
|
||||||
|
| Force | False |
|
||||||
|
+---------------+--------------------------------------+
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
The *Next Run Time* is the next time the audit will run. It is calculated based on the
|
||||||
|
interval and the start and end times.
|
||||||
|
|
||||||
|
|
||||||
|
List Generated Action Plans
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
Each execution of the continuous audit generates a new action plan:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ openstack optimize actionplan list --audit 7d1f1961-41a6-47ae-a94a-cf5e43174fbd
|
||||||
|
+--------------------------------------+--------------------------------------+-------------+
|
||||||
|
| UUID | Audit | State |
|
||||||
|
+--------------------------------------+--------------------------------------+-------------+
|
||||||
|
| b301dd17-a139-4a45-ade2-b2c2ddf006ef | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd | CANCELLED |
|
||||||
|
| 22a5bc60-adef-447a-aa27-731b4f5f7ee3 | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd | RECOMMENDED |
|
||||||
|
+--------------------------------------+--------------------------------------+-------------+
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
In continuous audits, when a new action plan is generated, previous
|
||||||
|
RECOMMENDED action plans are automatically set to CANCELLED state to
|
||||||
|
avoid conflicts.
|
||||||
|
|
||||||
|
|
||||||
|
Manage Continuous Audits
|
||||||
|
========================
|
||||||
|
|
||||||
|
Stop a Continuous Audit
|
||||||
|
------------------------
|
||||||
|
|
||||||
|
To stop a continuous audit, update its state:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace state=CANCELLED
|
||||||
|
|
||||||
|
|
||||||
|
Modify Audit Interval
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
You can change the interval of a running continuous audit:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace interval=900
|
||||||
|
|
||||||
|
The Decision Engine will automatically reschedule the audit with the new
|
||||||
|
interval.
|
||||||
|
|
||||||
|
Modify End Time
|
||||||
|
---------------
|
||||||
|
|
||||||
|
You can change the end time of a running continuous audit:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace end_time=2025-08-12T14:00:00
|
||||||
|
|
||||||
|
Delete a Continuous Audit
|
||||||
|
--------------------------
|
||||||
|
|
||||||
|
In order to delete a continuous audit, the audit state must be
|
||||||
|
SUCCEEDED, FAILED, or CANCELLED.
|
||||||
|
An audit with PENDING or ONGOING state cannot be deleted.
|
||||||
|
|
||||||
|
To delete an ongoing or pending continuous audit, update its state to
|
||||||
|
CANCELLED:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace state=CANCELLED
|
||||||
|
|
||||||
|
|
||||||
|
Then, delete the audit:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
$ openstack optimize audit delete 550e8400-e29b-41d4-a716-446655440000
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Configuration Reference
|
||||||
|
========================
|
||||||
|
|
||||||
|
Continuous Audit Intervals
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
**Numeric Intervals (seconds):**
|
||||||
|
|
||||||
|
- Minimum recommended: 60 seconds
|
||||||
|
- Common values: 300 (5 min), 600 (10 min), 1800 (30 min), 3600 (1 hour)
|
||||||
|
|
||||||
|
**Cron Expressions (5 format fields):**
|
||||||
|
|
||||||
|
See the `POSIX crontab specification <https://pubs.opengroup.org/onlinepubs/9799919799/utilities/crontab.html>`_.
|
||||||
|
|
||||||
|
- ``0 * * * *``: Every hour at minute 0
|
||||||
|
- ``*/15 * * * *``: Every 15 minutes
|
||||||
|
- ``0 9-17 * * 1-5``: Every hour during business hours (9 AM - 5 PM, Mon-Fri)
|
||||||
|
- ``30 2 * * *``: Daily at 2:30 AM
|
||||||
|
|
||||||
|
|
||||||
|
Decision Engine Configuration
|
||||||
|
-----------------------------
|
||||||
|
|
||||||
|
The continuous audit polling interval is configured in ``watcher.conf``:
|
||||||
|
|
||||||
|
.. code-block:: ini
|
||||||
|
|
||||||
|
[watcher_decision_engine]
|
||||||
|
# Interval for checking continuous audits (seconds)
|
||||||
|
continuous_audit_interval = 30
|
||||||
|
|
||||||
|
Spec Linked with Continuous Audit
|
||||||
|
=================================
|
||||||
|
|
||||||
|
- `Watcher Continuous Optimization <https://specs.openstack.org/openstack/watcher-specs/specs/newton/implemented/continuously-optimization.html>`_
|
||||||
|
- `Cron-based continuous audits <https://specs.openstack.org/openstack/watcher-specs/specs/pike/implemented/cron-based-continuous-audits.html>`_
|
||||||
|
- `Add the start and end time for CONTINUOUS audit <https://specs.openstack.org/openstack/watcher-specs/specs/stein/implemented/add-start-end-time-for-continuous-audit.html>`_
|
||||||
|
|
||||||
@@ -8,3 +8,4 @@ User Guide
|
|||||||
ways-to-install
|
ways-to-install
|
||||||
user-guide
|
user-guide
|
||||||
event_type_audit
|
event_type_audit
|
||||||
|
continuous_type_audit
|
||||||
|
|||||||
Reference in New Issue
Block a user