This patch adds a new load consolidation strategy based on a heuristic
algorithm which focuses on measured CPU utilization and tries to
minimize hosts which have too much or too little load.
A new goal "vm_workload_consolidation" was added which executes
the strategy "VM_WORKLOAD_CONSOLIDATION".
This work depends on the implemetation of the bug:
https://bugs.launchpad.net/watcher/+bug/1553124
Change-Id: Ide05bddb5c85a3df05b94658ee5bd98f32e554b0
Implements: blueprint basic-cloud-consolidation-integration
In watcher/metrics_engine/cluster_history/api.py,
we can find the BaseClusterHistory abstract base class.
To follow the same naming convention observed throughout the rest
of the project, I renamed watcher/metrics_engine/cluster_history/api.py
to watcher/metrics_engine/cluster_history/base.py
Change-Id: If18f8db7f0982e47c1998a469c54952670c262f5
Closes-Bug: #1548398
In several places, assertEqual is used the following way:
assertEqual(observed, expected)
However, the correct way to use assertEqual is:
assertEqual(expected, observed)
Change-Id: I5a7442f4adf98bf7bc73cef1d17d20da39d9a7f8
Closes-Bug: #1551861
We want a simplest way to validate the input parameters of an
Action through a schema.
APIImpact
DocImpact
Partially implements: blueprint watcher-add-actions-via-conf
Change-Id: I139775f467fe7778c7354b0cfacf796fc27ffcb2
The OpenStackClients class provides a convenient way to create and
cache client instances. The idea behind this code comes from Magnum
[0].
The OpenStackClients class will act as the manager of other project's
clients, providing an easy way to fetch instances of said clients. This
will allow the clients to be cached.
An instance of OpenStackClients is created for every call that comes
into the decision engine and the applier, using the request context to
pass needed (domain id) parameters to get a Keystone session. This
instance should be shared as much as possible to avoid additional
unneccessary connections to the other services.
This class will also allow for the version of each client to be
configurable via the watcher.conf file.
The method by which a Keystone session is also changed to use the
keystoneauth1.loading library. In order to avoid DuplicateOptErrors
with the keystone_authtoken group used for the keystonemiddleware in the
API code, a new conf group named "watcher_clients_auth" is created. A
typical configuration using a password authentication scheme will look
like:
[watcher_clients_auth]
auth_type = password
auth_url = http://<server-ip>:<port>
username = <username>
password = <password>
project_domain_id = default
user_domain_id = default
[0]: https://github.com/openstack/magnum/blob/master/magnum/common/clients.py
DocImpact
Change-Id: Iab9d0b304099686da2e9e2b19e8b1de4332ff378
Implements: blueprint external-api-versioning
Closes-Bug: #1530790
Closes-Bug: #1539670
Closes-Bug: #1522774
Currently self.client is referenced within MessagingCore,
but no definition is made in its constructor. Additionally
self.client is defined in children classes of MessagingCore.
This patchset defines self.client in the constructor of
MessagingCore and removes the redefinition in its children.
-self.client lazily loaded
Co-Authored-By: v-francoise <Vincent.FRANCOISE@b-com.com>
Change-Id: I14525a175bf1ebde3d2636024ad2f2219c79d6e1
Closes-Bug: #1521636
This patchset fixes the lack of field validation that are provided
by an API user.
Via a PATCH on /action_plans, the only field that can be modified
is now the 'state'. This field can only perform to the following
state transitions:
- RECOMMENDED --> TRIGGERED
- RECOMMENDED --> CANCELLED
- ONGOING --> CANCELLED
- TRIGGERED --> CANCELLED
The DELETED state can only be set using a DELETE request.
Closes-Bug: #1531106
Change-Id: I6669cbe63407f0bbb792fb2e2ce6b1e8a7365238
As we can see in the codebase, we have 3 "Status" enums which are
located at:
- watcher/objects/action.py
- watcher/objects/action_plan.py
- watcher/objects/audit.py
So I renamed them from "Status" to "State" to be consistent with
the DB schema.
Change-Id: If3d180c9daba6ae9083775ad6813aa4c21763dbf
Closes-Bug: #1522733
The aim of this patchset is to integrate taskflow in
the Watcher Applier. Taskflow will help us a lot to make
Action Plan execution easy, consistent, scalable and reliable.
DocImpact
Partially implements: blueprint use-taskflow
Change-Id: I903d6509d74a61ad64e1506b8a7156e6e91abcfb
Closes-Bug: #1535326
Closes-Bug: #1531912
In watcher, an audit generates a set of actions which
aims at achieving a given goal (lower energy consumption, ...).
It is possible to configure different strategies in order to achieve
each goal. Each strategy is written as a Python class which produces
a set of actions. Today, the set of possible actions is fixed for a
given version of Watcher and enables optimization algorithms to
include actions such as instance migration, changing hypervisor state,
changing power state (ACPI level, ...).
The objective of this patchset is to give the ability to load
the actions dynamically in order to apply the Action Plan.
DocImpact
Partially implements: blueprint watcher-add-actions-via-conf
Change-Id: Idf295b94dca549ac65d4636e8889c8ab2ecc0df6
Renamed diskInfo.py to disk_info.py and the associated test to
test_disk_info. Also changed the usage in the test to reflect
the name change.
Closes-bug: #1533189
Change-Id: Ice63cf8ea6cd4fcc770f88952cf784e5d46cca5c
Each config option has limitation for type and value.
We make enforce_type=True to check whether we pass wrong type.
Also fixes a type error issue in test_db_manager.py.
Change-Id: I6e111e21588525d32b05eeba75b06583d4e605ed
Related-Bug: #1517839
In watcher, an audit generates a set of actions which
aims at achieving a given goal (lower energy consumption, ...).
It is possible to configure different strategies in order to achieve
each goal. Each strategy is written as a Python class which produces
a set of actions. Today, the set of possible actions is fixed for a
given version of Watcher and enables optimization algorithms to
include actions such as instance migration, changing hypervisor state,
changing power state (ACPI level, ...).
This patchset propose a generic and extensible way to describe
the actions and his parameters that we want to add to Action Plan.
It also remove the static actions because they are now deprecated.
The documentation regarding strategy plugin need to be
updated (plugins.rst).
DocImpact
Partially implements: blueprint watcher-add-actions-via-conf
Change-Id: I3d641080e8ad89786abca79a942c8deb2d53355b
In watcher, an audit generates a set of actions which
aims at achieving a given goal (lower energy consumption, ...).
It is possible to configure different strategies in order to achieve
each goal. Each strategy is written as a Python class which produces
a set of actions. Today, the set of possible actions is fixed for a
given version of Watcher and enables optimization algorithms to
include actions such as instance migration, changing hypervisor state,
changing power state (ACPI level, ...).
The objective of this patchset is to give the ability to extend the
default set of planner algorithms currently available in Watcher
using Stevedore.
The doc need to explain how create a new planner.
DocImpact
Partially implements: blueprint watcher-add-actions-via-conf
Change-Id: I2fd73f8c4a457ee391d764a7a3f494deecd2634f
Some Python class and packages need to be renamed
for a better compliance with the shared terminology
which provides a better understanding of Watcher objects
and components by every contributor.
Partially implements: blueprint glossary-related-refactoring
Change-Id: Ie0e33562f5e990c264a50ab3f533cfa62eac1d19
It implements one of the algorithm of Intel thermal POC.
It extends the BaseStrategy class, getting the Outlet
metrics of servers via Ceilometer, and generates solutions
when the outlet Temperature is greater than threshold.
current threshold is hard-coded, will make it configurable
in the next patches.
Implements: blueprint outlet-temperature-based-strategy
Change-Id: I248147329d34eddf408652205a077895be572010
Co-Authored-By: Zhenzan Zhou <zhenzan.zhou@intel.com>
This aim of this patchset is to move the management of the
Audit-Template into StrategyContext
in order to prepare to pass parameters to strategies
but also to prepare to add more dynamic Actions management
Partially implements: blueprint glossary-related-refactoring
Change-Id: I13ee063da947113ce349855aa331a22f40567051
Some of the modules still utilized string concatenation
instead of using formatting. In order to align with other
modules in the project, I refactored these modules to use string
formatting instead.
Change-Id: I708392e1d03b6331a134419aa0ae9dc02a05c31b
Closes-Bug: 1522738
This patchset aim to remove useless code in StrategyContext
and AuditEndPoint.
This patchset also add a parameter for strategy context to define the
numbers of thread of execute the strategies.
DocImpact
Change-Id: I83e87165b03b42fe6b863921502a300bd94d2982
This patchset is there to change the code structure.
Some Python class and packages need to be renamed
for a better compliance with the shared terminology
which provides a better understanding of Watcher
objects and components by every contributor.
This patchset is there to rename the folder command to audit
Partially implements: blueprint glossary-related-refactoring
Change-Id: I76616fb58d5e79a7dc209b80e882d216850d18a4
Some Python class and packages need to be renamed
for a better compliance with the shared terminology
which provides a better understanding of Watcher
objects and components by every contributor.
This patchset is there to change the code structure by adding the folder
"strategies" and "loading".
Partially implements: blueprint glossary-related-refactoring
Change-Id: I56fb24ee6762b3186eccde5983233e17bb227cc1
Some Python class and packages need to be renamed
for a better compliance with the shared terminology
which provides a better understanding of Watcher
objects and components by every contributor.
This patchset add missing Base in class name
Partially implements: blueprint glossary-related-refactoring
Change-Id: I95a3e41fbd5fcd90a99d81c9cf278940f50c7732
Even though Watcher was mentioning python 3.4 as supported, it
really wasn't the case as all the unit tests were not passing in this
version of Python.
This patchset fixes all the failing tests in Python 3.4 while
keeping Watcher Python 2.7 compatible.
DocImpact
BugImpact
Change-Id: Ie74acc08ef0a2899349a4b419728c89e416a18cb
Some Python class and packages need to be renamed
for a better compliance with the shared terminology
which provides a better understanding of Watcher
objects and components by every contributor.
This patchset is there to change Meta-Action to Action
Partially implements: blueprint glossary-related-refactoring
Change-Id: Ie67b800332179be93718030ddd7a36ddc76a544c
Some Python class and packages need to be renamed
for a better compliance with the shared Terminology
which provides a better understanding of Watcher
objects and components by every contributor.
This patchset is there to change efficiency to efficacy
Partially implements: blueprint glossary-related-refactoring
Change-Id: I4c84192d49a147e0fd406da35e2805143b902331
Querying compute.node.* with Ceilometer requires some additional
information in the cluster data model in order to build the
resource_id. This patchset add the attribute hostname and change
the query.
Change-Id: Ifcefcd70a6d0f5967ab4f638ce077e38ef214f64
Closes-Bug: #1521559
The old 'watcher_messaging' section of the Watcher configuration file
has now been replaced by the more standard oslo.configuration one.
DocImpact
Change-Id: Ie027df023e6133f3188e57b42846083f28c282bd
This should make our 'dummy' strategy available for use.
I also removed all previously defined default goals, which means
that they will have to define it themselves from now on.
This can also be useful for future integration tests.
Change-Id: I6c43eba941022a88851a199b56a6c20f017b9e71
DocImpact
Closes-Bug: #1520178
In several places in Watcher, there are statuses which corresponds
to a status in the respective life cycle. There were used past tense
everywhere, except "SUCCESS" which was written in present tense.
So I searched for all the occurences of this state, and replaced them
with their past tense form.
Change-Id: Ic41def96d1a608da06a1e394e24704d4f06ab32a
Closes-Bug: #1520276
This patchset is there to change the code structure.
The objective is to flatten the project file tree by merging
'api/' and 'framework/' into a single package. This also contains
some tidy ups in package naming (like using only singular nouns).
This should only affect file/folder names and their subsequent
import paths wherever they were used.
Change-Id: Ie903ba20ca5cf03b0b42efa60131d1b919b0c2c9
Update requirements.txt and test-requirements.txt from Global Requirements
defined into openstack/requirements project (stable/liberty).
Change-Id: I9ecbf5a7fb2c38624cc4f52b13515f8af75a4889
Closes-Bug: #1513118
The objective for this is to give the ability to extend the default
set of placement algorithms (i.e. strategies) currently available
in Watcher using Stevedore.
Now, you can add your new strategy as an entry point under the
'[watcher_strategies]' section.
Change-Id: I4aecf629015e41b0389d07e47220333e50bbbe1a
In current implementation is not easy to use ceilometer.
Watcher must query metrics from the Telemetry v2 API to allow an easiest integration with OpenStack components (especially devstack).
blueprint telemetry-integration
Change-Id: Ide515472f1d160925d9f4aabf48c96dea4f6bc05
