New audit state SUSPENDED is added in this patch set. If audit
state with continuous mode is changed from ONGOING to SUSPENDED,
audit's job is removed and the audit is not executed.
If audit state changed from SUSPENDED to ONGOING in reverse,
audit is executed again periodically.
Change-Id: I32257f56a40c0352a7c24f3fb80ad95ec28dc614
Implements: blueprint suspended-audit-state
This patch set adds support of auto-triggering of action plans.
Change-Id: I36b7dff8eab5f6ebb18f6f4e752cf4b263456293
Partially-Implements: blueprint automatic-triggering-audit
In this changeset, I implemented the audit.create notification.
Change-Id: Ia092ca3a3dc951e3313a07f15a98aec5818e9ab0
Partially-Implements: blueprint audit-versioned-notifications-api
In this changeset, I implemented the sending of update notifications
whenever an audit is modified.
Change-Id: I5ccc2516ce896ae7d4ef542b133e8f052eaed602
Partially-Implements: blueprint audit-versioned-notifications-api
In this changeset, I added the "action_plan" ObjectField which can
either be loaded by setting the new "eager" parameter as True
or not loaded (as before) by setting it to False. The advantage of
introducing this eager parameter is that this way,
we can reduce to a minimum the overhead of DB queries whenever the
related object fields is not actually needed.
Change-Id: Iae255d9a0a919e2b6db710be24dbf9033b72166e
Partially-Implements: blueprint watcher-versioned-objects
In this changeset, I added the "audit" and the "strategy "ObjectField
which can either be loaded by setting the new "eager" parameter as True
or not loaded (as before) by setting it to False.
The advantage of introducing this eager parameter is that this way,
we can reduce to a minimum the overhead of DB queries whenever the
related object fields is not actually needed.
Change-Id: Ieca6fa438cbf2267f5ae2ea0d059b2953e65076b
Partially-Implements: blueprint watcher-versioned-objects
In this changeset, I added the "goal" and the "strategy "ObjectField
which can either be loaded by setting the new "eager" parameter as True
or not loaded (as before) by setting it to False.
The advantage of introducing this eager parameter is that this way,
we can reduce to a minimum the overhead of DB queries whenever the
related goal is not actually needed.
Change-Id: I5a1b25e395e3d904dae954952f8e0862c3556210
Partially-Implements: blueprint watcher-versioned-objects
In this changeset, I added the "goal" and the "strategy "ObjectField
which can either be loaded by setting the new "eager" parameter as True
or not loaded (as before) by setting it to False.
The advantage of introducing this eager parameter is that this way,
we can reduce to a minimum the overhead of DB queries whenever the
related goal is not actually needed.
Change-Id: I97418163f68aea2f1cda80be94e4035f0b3700ae
Partially-Implements: blueprint watcher-versioned-objects
In this changeset, I added the "goal" ObjectField which can
either be loaded by setting the new "eager" parameter as True or
not loaded (as before) by setting it to False.
The advantage of introducing this eager parameter is that this way,
we can reduce to a minimum the overhead of DB queries whenever the
related goal is not actually needed.
Partially-Implements: blueprint watcher-versioned-objects
Change-Id: I103c9ed161d2cedf7b43c55f9e095ef66bf44dea
In this changeset, I modified all existing Watcher objects to now
rely on oslo.versionedobjects as a base.
Change-Id: I3c9b1ca6da529d128743b99020350f28926ea1a2
Partially-Implements: blueprint watcher-versioned-objects
As we are about to version the Watcher objects, we need to make sure
that upcoming model/object modifications are additive in order to
avoid having to bump the major version of the API. Therefore,
this changeset removes 4 unused DB fields that were exposed in their
associated Watcher objects (i.e. AuditTemplate and Audit).
Change-Id: Ifb0783f21cd66db16b31e3c8e376fc9d6c07dea3
Partially-Implements: blueprint watcher-versioned-objects
In this changeset, I added ORM relationships to the DB models
concerning the already-declared foreign keys.
I also modified the DB query building to now handle a new 'eager'
parameter that, if True, is responsible to also fetch the data
relative to these 'parent' DB entities (no cascading).
Change-Id: Ieea181af9a4b173c54621dcc6c549161f5a35aeb
Partially-Implements: blueprint watcher-versioned-objects
This patch set adds audit scope mechanism.
It also removes host_aggregate field.
Change-Id: Ia98ed180a93fc8c19599735e2b41471d322bae9a
Partially-Implements: blueprint define-the-audit-scope
This patch set adds supervisor mechanism for Watcher services
to get ability to track states.
Partially-Implements: blueprint watcher-service-list
Change-Id: Iab1cefb971c79ed27b22b6a5d1bed8698e35f9a4
Config option sqlite_db is deprecated in
0a1bae9859079fb21a03716be947c5f1da6db0a2, and deprecate argumentsqlite_db in method set_defaults in
https://review.openstack.org/#/c/350945/, should use config option
connection instead. For watcher, we test database with sqlite in memory
mode [1] as below:
cfg.CONF.set_override("connection", "sqlite://",
group="database", enforce_type=True)
and don't use config option sqlite_db, so remove unused code.
[1]http://docs.sqlalchemy.org/en/rel_1_0/core/engines.html#sqlite
Change-Id: I9b1f995e1b7004bcfe6c5a854c2f83b24e5bfb56
In this changeset, I implemented the logic which cancels
any audit or action plan whose goal has been re-synced
(upon restarting the Decision Engine).
Partially Implements: blueprint efficacy-indicator
Change-Id: I95d2739eb552d4a7a02c822b11844591008f648e
A Scoring Module needs to expose a list of available
scoring engines through API and Watcher CLI. This list
is stored in database and synchronized by Decision Engine.
Partially-Implements: blueprint scoring-module
Change-Id: I32168adeaf34fd12a731204c5b58fe68434ad087
APIImpact
Modifying the api controller for audit objects to allow
creation of audit objects by specifying either an
audit_template uuid/id and/or a goal_id.
strategy_id is optional.
Partially Implements: blueprint persistent-audit-parameters
Change-Id: I7b3eae4d0752a11208f5f92ee13ab1018d8521ad
This patch set adds implementation for CONTINUOUS type
of audit.
Change-Id: I5f4ec97b2082c8a6b3ccebe36b2a343fa4a67d19
Implements: blueprint continuously-optimization
There's a number of places where a field of an object is named "type",
which isn't good thing. Actually type is keyword in python, so wherever
I found type used in models/api/object/test, I have fixed it to
audit_type in the audit files
Change-Id: Iea6bd3acb0b2af2a833b3916701aad88f6064bba
Closes-Bug: #1533392
Strategy provides parameters to customize algorithm behavior. End user
could query then specify parameters for their requirements.
Change-Id: Id097db5f6e79c94b57674c8e5d55b06098abf18c
Implements-bp: optimization-threshold
In this changeset, I refactored the DB filter system to support
comparison operators using a django-like syntax.
A filter can take 2 forms:
- "<FIELDNAME>" which is a syntactic sugar for "<FIELDNAME>__eq"
- "<FIELDNAME>__<OPERATOR>" where <OPERATOR> is the comparison operator
to be used.
Here is the list of the supported operators:
- 'eq' (==)
- 'neq' (!=)
- 'gt' (>)
- 'gte' (>=)
- 'lt' (<)
- 'lte' (<=)
- 'in' (in)
- 'notin' (not in)
Change-Id: I53a61d50a3253342a40f0ff87cb5612ed57a3bd1
I this changeset, I added the efficacy indicators both at the DB
and at the API level alongside the associated logic.
Partially Implements: blueprint efficacy-indicator
Change-Id: I824553637621da67966103c1b0c01348b09bd836
In this changeset, I added the new Efficacy, EfficacySpecification
and IndicatorSpecification classes which are the main components for
computing an efficacy.
Partially Implements: blueprint efficacy-indicator
Change-Id: I3a1d62569de2dd6bb6f9a52f6058313fa2b886ce
In this changeset, I created a new model named EfficacyIndicator
which is responsible for storing information regarding the efficacy
indicators that were computed by the strategy in its solution. Every
efficacy indicator should relate to a single Action Plan.
Partially Implements: blueprint efficacy-indicator
Change-Id: Ifc14ea5e16e92f032d7912c9b3fdc270af79cab6
In this changeset, I updated the purge script to now take into
account the registered goals and strategies.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: I2f1d58bb812fa45bc4bc6467760a071d8612e6a4
In this changeset, I updated the 'goal_id' field into the AuditTemplate
to now become a mandatory foreign key towards the Goal model. I also
added the 'strategy_id' field into the AuditTemplate model to be an
optional foreign key onto the Strategy model.
This changeset also includes an update of the /audit_template
Watcher API endpoint to reflect the previous changes.
As this changeset changes the API, this should be merged alongside the
related changeset from python-watcherclient.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: Ic0573d036d1bbd7820f8eb963e47912d6b3ed1a9
In this changeset, I changed the Strategy base class to add new
abstract class methods. I also added an abstract strategy class
per Goal type (dummy, server consolidation, thermal optimization).
This changeset also includes an update of the /goals Watcher API
endpoint to now use the new Goal model (DB entries) instead of
reading from the configuration file.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: Iecfed58c72f3f9df4e9d27e50a3a274a1fc0a75f
In this changeset, I add the Strategy model as well as the DB
functionalities we need to manipulate strategies.
This changeset implies a DB schema update.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: I438a8788844fbc514edfe1e9e3136f46ba5a82f2
In this changeset, I added the Goal object into Watcher along with
a sync module that is responsible for syncing the goals with the
Watcher DB.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: Ia3a2032dd9023d668c6f32ebbce44f8c1d77b0a3
In this changeset, I added the Goal model into Watcher.
This implies a change into the Watcher DB schema
Partially Implements: blueprint get-goal-from-strategy
Change-Id: I5b5b0ffc7cff8affb59f17743e1af0e1277c2878
This patch set removes the possibility of using UUID field
in POST methods of Watcher API.
Closes-Bug: #1572625
Change-Id: I88a8aa5346e937e3e9409b55da3316cbe1ed832a
We need to update sqlalchemy/api and sqlalchemy/models (and appropriate tests)
to support deleting audit templates and recreating them with the same names.
Change-Id: Icf54cf1ed989a3f2ad689e25be4474b16a3a3eb2
Related-Bug: #1510179
This patchset implements the purge script as specified in its
related blueprint:
- The '--age-in-days' option allows to specify the number of
days before expiry
- The '--max-number' option allows us to specify a limit on the number
of objects to delete
- The '--audit-template' option allows you to only delete objects
related to the specified audit template UUID or name
- The '--dry-run' option to go through the purge procedure without
actually deleting anything
- The '--exclude-orphans' option which allows you to exclude from the
purge any object that does not have a parent (i.e. and audit without
a related audit template)
A prompt has been added to also propose to narrow down the number of
deletions to be below the specified limit.
Change-Id: I3ce83ab95277c109df67a6b5b920a878f6e59d3f
Implements: blueprint db-purge-engine
As a pre-requisite for being able to query the database for objects
that are expired, I need a way to express date comparison on the
'deleted_at' field which is common for every Watcher object. As they
are coming from mixins, I decided to implement these filters with a
syntax borrowed from the Django ORM where the field is suffixed by the
comparison operator you want to apply:
- The '__lt' suffix stands for 'less than'
- The '__lte' suffix stands for 'less than or equal to'
- The '__gt' suffix stands for 'greater than'
- The '__gte' suffix stands for 'greater than or equal to'
- The '__eq' suffix stands for 'equal to'
I also added a 'uuid' filter to later on be able to filter by uuid.
Partially Implements: blueprint db-purge-engine
Change-Id: I763f330c1b8ea8395990d2276b71e87f5b3f3ddc
We want a simplest way to validate the input parameters of an
Action through a schema.
APIImpact
DocImpact
Partially implements: blueprint watcher-add-actions-via-conf
Change-Id: I139775f467fe7778c7354b0cfacf796fc27ffcb2
Change I6c43eba941022a88851a199b56a6c20f017b9e71 seemed to have remove
most references to the SERVERS_CONSOLIDATION goal. Since this goal does
not currently exist in the actual code and all usages of it are for
samples or for tests, it is replaced with the DUMMY goal to avoid
confusion.
Change-Id: I4d2240d3b22c42ebf4e6120e2cd7677ec49d8e98
Closes-Bug: #1538388
The InvalidParameterValue exception does not define a meaningful
msg_fmt. It is currently _("%(err)s"), which is the equivalent of
nothing and does not help with translation.
Replace InvalidParameterValue with Invalid exceptions.
Change-Id: If8b064e446cbc97e380127f360f262be9e8877a1
Closes-Bug: #1538398
Each config option has limitation for type and value.
We make enforce_type=True to check whether we pass wrong type.
Also fixes a type error issue in test_db_manager.py.
Change-Id: I6e111e21588525d32b05eeba75b06583d4e605ed
Related-Bug: #1517839
In watcher, an audit generates a set of actions which
aims at achieving a given goal (lower energy consumption, ...).
It is possible to configure different strategies in order to achieve
each goal. Each strategy is written as a Python class which produces
a set of actions. Today, the set of possible actions is fixed for a
given version of Watcher and enables optimization algorithms to
include actions such as instance migration, changing hypervisor state,
changing power state (ACPI level, ...).
This patchset propose a generic and extensible way to describe
the actions and his parameters that we want to add to Action Plan.
It also remove the static actions because they are now deprecated.
The documentation regarding strategy plugin need to be
updated (plugins.rst).
DocImpact
Partially implements: blueprint watcher-add-actions-via-conf
Change-Id: I3d641080e8ad89786abca79a942c8deb2d53355b
In watcher, an audit generates a set of actions which
aims at achieving a given goal (lower energy consumption, ...).
It is possible to configure different strategies in order to achieve
each goal. Each strategy is written as a Python class which produces
a set of actions. Today, the set of possible actions is fixed for a
given version of Watcher and enables optimization algorithms to
include actions such as instance migration, changing hypervisor state,
changing power state (ACPI level, ...).
This patchset add the possibility to store several parameters
for an Action.
The parameters store info that the custom Action needs.
The parameters provided as tuples with the following fields:
(parameter_name, parameter_type).
It remove also the deprecated attributes
(src,dst,description)
APIImpact
Partially implements: blueprint watcher-add-actions-via-conf
Change-Id: Ic6727341822f8ac62f212d337814b2dca76044e3
Some of the modules still utilized string concatenation
instead of using formatting. In order to align with other
modules in the project, I refactored these modules to use string
formatting instead.
Change-Id: I708392e1d03b6331a134419aa0ae9dc02a05c31b
Closes-Bug: 1522738
