This is one of the algorithm of Intel thermal POC.
It's based on the VM workloads of hypervisors.
Change-Id: I45ab0cf0f05786e6f68025bdd315f38381900a68
blueprint: workload-balance-migration-strategy
In this changeset, I added the possibility for all plugins to define
configuration parameters for themselves.
Partially Implements: blueprint plugins-parameters
Change-Id: I676b2583b3b4841c64c862b2b0c234b4eb5fd0fd
The main purpose of this strategy is to choose the pair VM:dest_host that
minimizes the standard deviation in a cluster best.
Change-Id: I95a31b7bcab83411ef6b6e1e01818ca21ef96883
Implements: blueprint watcher-overload-sd
In this changeset, I remove the now unused [watcher_goals] section.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: I91e4e1ac3a58bb6f3e30b11449cf1a6eb18cd0ca
In this changeset, I updated the Watcher documentation to reflect
the changes that are introduced by this blueprint.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: I40be39624097365220bf7d94cbe177bbf5bbe0ed
In this changeset, I updated the 'goal_id' field into the AuditTemplate
to now become a mandatory foreign key towards the Goal model. I also
added the 'strategy_id' field into the AuditTemplate model to be an
optional foreign key onto the Strategy model.
This changeset also includes an update of the /audit_template
Watcher API endpoint to reflect the previous changes.
As this changeset changes the API, this should be merged alongside the
related changeset from python-watcherclient.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: Ic0573d036d1bbd7820f8eb963e47912d6b3ed1a9
In this changeset, I refactored the strategy selector to now
look into the Watcher DB instead of looking into the configuration
file.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: I2bcb63542f6237f26796a3e5a781c8b62820cf6f
In this changeset, I added the /strategies endpoint to the Watcher
API service.
This also includes the related Tempest tests.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: I1b70836e0df2082ab0016ecc207e89fdcb0fc8b9
In this changeset, I changed the Strategy base class to add new
abstract class methods. I also added an abstract strategy class
per Goal type (dummy, server consolidation, thermal optimization).
This changeset also includes an update of the /goals Watcher API
endpoint to now use the new Goal model (DB entries) instead of
reading from the configuration file.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: Iecfed58c72f3f9df4e9d27e50a3a274a1fc0a75f
In this changeset, I added the Goal object into Watcher along with
a sync module that is responsible for syncing the goals with the
Watcher DB.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: Ia3a2032dd9023d668c6f32ebbce44f8c1d77b0a3
In some modules the global LOG is not used any more. And the import
of logging is not used. This patch removes the unused logging import
and LOG vars.
Change-Id: I794ee719d76f04e70154cf67f726152fbb1ba15a
Unicode type resource state is now handled in the same fashion as resource state specified by general string.
Change-Id: I35ffa09015283b51c935515436735aecbe83a9d6
Closes-Bug: #1565764
Added extra information regarding the plugin mechanism for:
action, strategy, and Watcher planner.
Change-Id: I9a7523282e229b83c16b06e3806ff795a0699c78
Closes-Bug: #1558470
This patch adds a new load consolidation strategy based on a heuristic
algorithm which focuses on measured CPU utilization and tries to
minimize hosts which have too much or too little load.
A new goal "vm_workload_consolidation" was added which executes
the strategy "VM_WORKLOAD_CONSOLIDATION".
This work depends on the implemetation of the bug:
https://bugs.launchpad.net/watcher/+bug/1553124
Change-Id: Ide05bddb5c85a3df05b94658ee5bd98f32e554b0
Implements: blueprint basic-cloud-consolidation-integration
The variable vm_avg_cpu_util was renamed to host_avg_cpu_util for
clarity, as it was really referring to the host average cpu util.
Change-Id: I7aaef9eb2c8421d01715c86afa36ab67f2fd5f30
Closes-Bug: #1559113
This patchset introduces a new custom directive called 'drivers-doc'
which loads all available drivers under a given namespace and import
their respective docstring into the .rst document.
This patchset also contains some modification/addition to the
docstring of these drivers to make the final document complete.
Change-Id: Ib3df59fa45cea9d11d20fb73a5f0f1d564135bca
Closes-Bug: #1536218
Closes-Bug: #1536735
The primitive ChangeNovaServiceState allows us to change the state of
the nova-compute by calling nova api. The state of a nova-compute can
be ENABLED or DISABLED, however in the current implementation we use
OFFLINE and ONLINE.
Update the code to use ENABLED or DISABLED.
Change-Id: If3d9726bc5ae980b66c7fd4c5b7986f89d8bc690
Closes-Bug: #1523891
Renamed many of the variables and method parameters
in the strategy implementations to make the names
more meaningful. Also changed the abstract method
signature in base.py to reflect these changes.
Closes-Bug: #1541615
Change-Id: Ibeba6c6ef6d5b70482930f387b05d5d650812355
In some part in the code we import objects.
In the Openstack style guidelines they recommand
to import only modules.
We need to fix that.
Change-Id: I4bfee2b94d101940d615f78f9bebb83310ed90ba
Partial-Bug:1543101
We want a simplest way to validate the input parameters of an
Action through a schema.
APIImpact
DocImpact
Partially implements: blueprint watcher-add-actions-via-conf
Change-Id: I139775f467fe7778c7354b0cfacf796fc27ffcb2
The OpenStackClients class provides a convenient way to create and
cache client instances. The idea behind this code comes from Magnum
[0].
The OpenStackClients class will act as the manager of other project's
clients, providing an easy way to fetch instances of said clients. This
will allow the clients to be cached.
An instance of OpenStackClients is created for every call that comes
into the decision engine and the applier, using the request context to
pass needed (domain id) parameters to get a Keystone session. This
instance should be shared as much as possible to avoid additional
unneccessary connections to the other services.
This class will also allow for the version of each client to be
configurable via the watcher.conf file.
The method by which a Keystone session is also changed to use the
keystoneauth1.loading library. In order to avoid DuplicateOptErrors
with the keystone_authtoken group used for the keystonemiddleware in the
API code, a new conf group named "watcher_clients_auth" is created. A
typical configuration using a password authentication scheme will look
like:
[watcher_clients_auth]
auth_type = password
auth_url = http://<server-ip>:<port>
username = <username>
password = <password>
project_domain_id = default
user_domain_id = default
[0]: https://github.com/openstack/magnum/blob/master/magnum/common/clients.py
DocImpact
Change-Id: Iab9d0b304099686da2e9e2b19e8b1de4332ff378
Implements: blueprint external-api-versioning
Closes-Bug: #1530790
Closes-Bug: #1539670
Closes-Bug: #1522774
The aim of this patchset is to integrate taskflow in
the Watcher Applier. Taskflow will help us a lot to make
Action Plan execution easy, consistent, scalable and reliable.
DocImpact
Partially implements: blueprint use-taskflow
Change-Id: I903d6509d74a61ad64e1506b8a7156e6e91abcfb
Closes-Bug: #1535326
Closes-Bug: #1531912
The execute() method is very large - almost 150 lines,
and McCabe's cyclomatic complexity 22.
(watcher/decision_engine/strategy/strategies/basic_consolidation.py)
I extracted some of the functionalities into helper functions
to reduce the length and complexity of execute().
http://openqa.sed.hu/dashboard/index/544838?did=1
Additionally it became more readable as well, without
changing its functionality.
Change-Id: I760929f56e258b87d7f1d4586bcc90665f1e0d8f
Closes-Bug: 1535729
In watcher, an audit generates a set of actions which
aims at achieving a given goal (lower energy consumption, ...).
It is possible to configure different strategies in order to achieve
each goal. Each strategy is written as a Python class which produces
a set of actions. Today, the set of possible actions is fixed for a
given version of Watcher and enables optimization algorithms to
include actions such as instance migration, changing hypervisor state,
changing power state (ACPI level, ...).
The objective of this patchset is to give the ability to load
the actions dynamically in order to apply the Action Plan.
DocImpact
Partially implements: blueprint watcher-add-actions-via-conf
Change-Id: Idf295b94dca549ac65d4636e8889c8ab2ecc0df6
As of now, the glossary defined in our documentation reflects the
current state of the codebase. In order to avoid any discrepancy
between the codebase and each definition, the objective here is to
gather both in a single place and link it into the rst documentation
via a custom directive.
Also re-aligned the requirements with liberty for doc.
Change-Id: I3089fd9f948b948115672f10937b1500b96ce180
Partial-Bug: #1526671
In watcher, an audit generates a set of actions which
aims at achieving a given goal (lower energy consumption, ...).
It is possible to configure different strategies in order to achieve
each goal. Each strategy is written as a Python class which produces
a set of actions. Today, the set of possible actions is fixed for a
given version of Watcher and enables optimization algorithms to
include actions such as instance migration, changing hypervisor state,
changing power state (ACPI level, ...).
This patchset propose a generic and extensible way to describe
the actions and his parameters that we want to add to Action Plan.
It also remove the static actions because they are now deprecated.
The documentation regarding strategy plugin need to be
updated (plugins.rst).
DocImpact
Partially implements: blueprint watcher-add-actions-via-conf
Change-Id: I3d641080e8ad89786abca79a942c8deb2d53355b
In watcher, an audit generates a set of actions which
aims at achieving a given goal (lower energy consumption, ...).
It is possible to configure different strategies in order to achieve
each goal. Each strategy is written as a Python class which produces
a set of actions. Today, the set of possible actions is fixed for a
given version of Watcher and enables optimization algorithms to
include actions such as instance migration, changing hypervisor state,
changing power state (ACPI level, ...).
This patchset add a common generic dynamic loader for plugins,
such as for custom Actions, Strategies, Planners, etc.
Partially implements: blueprint watcher-add-actions-via-conf
Change-Id: I59d031b93865fff2540e3973921e1bdafa95f88e
In the section "WATCHER_GOALS_OPTS" the dict option "goals"
is not mandatory. However it should be.
Change-Id: I2e0770cf7787fed449c012bc45462e3138992ebf
Closes-Bug: #1531116
It implements one of the algorithm of Intel thermal POC.
It extends the BaseStrategy class, getting the Outlet
metrics of servers via Ceilometer, and generates solutions
when the outlet Temperature is greater than threshold.
current threshold is hard-coded, will make it configurable
in the next patches.
Implements: blueprint outlet-temperature-based-strategy
Change-Id: I248147329d34eddf408652205a077895be572010
Co-Authored-By: Zhenzan Zhou <zhenzan.zhou@intel.com>
This aim of this patchset is to move the management of the
Audit-Template into StrategyContext
in order to prepare to pass parameters to strategies
but also to prepare to add more dynamic Actions management
Partially implements: blueprint glossary-related-refactoring
Change-Id: I13ee063da947113ce349855aa331a22f40567051
Since internationalization should be enabled in Watcher, this
patchset refactors the Watcher codebase to wrap previously
untranslatable strings with i18n translation functions so we can
import them for translation into the .pot template file.
Partially Implements: blueprint support-translation
Change-Id: I425967a60b5a7957f753894e5d2ba0d2c5009d1d
This patchset aim to remove useless code in StrategyContext
and AuditEndPoint.
This patchset also add a parameter for strategy context to define the
numbers of thread of execute the strategies.
DocImpact
Change-Id: I83e87165b03b42fe6b863921502a300bd94d2982
Some Python class and packages need to be renamed
for a better compliance with the shared terminology
which provides a better understanding of Watcher
objects and components by every contributor.
This patchset is there to change the code structure by adding the folder
"strategies" and "loading".
Partially implements: blueprint glossary-related-refactoring
Change-Id: I56fb24ee6762b3186eccde5983233e17bb227cc1
Some Python class and packages need to be renamed
for a better compliance with the shared terminology
which provides a better understanding of Watcher
objects and components by every contributor.
This patchset add missing Base in class name
Partially implements: blueprint glossary-related-refactoring
Change-Id: I95a3e41fbd5fcd90a99d81c9cf278940f50c7732
Even though Watcher was mentioning python 3.4 as supported, it
really wasn't the case as all the unit tests were not passing in this
version of Python.
This patchset fixes all the failing tests in Python 3.4 while
keeping Watcher Python 2.7 compatible.
DocImpact
BugImpact
Change-Id: Ie74acc08ef0a2899349a4b419728c89e416a18cb
