Commit Graph

130 Commits

Author SHA1 Message Date
Drew Thorstensen
7d704dbeec Add hacking checks to watcher
The hacking checks enforce during the pep8 run functional validations of
the code to ensure deeper filters and code consistency.  This change set
adds the hacking checks to the wathcer project.  These checks were
seeded from the neutron project, which had a good set of base defaults.

This change set also updates the watcher project to be compliant with
these new hacking checks.

Change-Id: I6f4566d384a7400bddf228aa127a53e6ecc82c2e
2016-07-25 07:52:45 -04:00
Jenkins
6727d53072 Merge "Fix dict.keys() PY3 compatible" 2016-07-21 13:02:16 +00:00
Jenkins
8ab80894c3 Merge "Fix typos and messages in strategies" 2016-07-20 23:58:26 +00:00
Joe Cropper
6730202151 Fix typos and messages in strategies
This patch fixes various typos and other nits in the strategies.  It
also updates some of the log messages to be a little more operator
friendly.

Change-Id: Ic9268c6d7376dad215a1a40798485b1d836ba7ae
Closes-Bug: #1604248
2016-07-20 14:19:45 -07:00
weiweigu
cd60336e20 Fix dict.keys() PY3 compatible
The dict.keys()[0] will raise a TypeError in PY3,
as dict.keys() doesn't return a list any more in PY3
but a view of list.

Change-Id: If15a153c9db9b654e761f8ad50d5d66a427efa4e
Closes-Bug: #1583419
2016-07-14 13:47:29 +08:00
zhangyanxian
e7a1ba7e74 There are some spelling errors in the code.
Change-Id: Ifb88a13783fc12258489abb4caabca1b1038a77d
2016-07-13 02:36:47 +00:00
David TARDIVEL
517f1d083e Bad goal and strategy name for Airflow Optimization
The goal name 'airflow optimization' must be synched with the
entry point name.
The strategy name 'uniform airflow' must be synched with the
entry point name.

Partial-Bug: #1602262

Change-Id: Ibc6f0a7c5047e57549910818200c323b878152b0
2016-07-12 15:45:59 +02:00
Jenkins
e78610e7d1 Merge "Make default Planner generic to handle new action" 2016-07-12 07:35:21 +00:00
Jenkins
405cde2ff9 Merge "Uniform Airflow migration strategy implementation" 2016-07-11 07:56:26 +00:00
Jenkins
b25ecde0db Merge "Add continuously optimization" 2016-07-08 14:07:51 +00:00
Alexander Chadin
1de00086f5 Add continuously optimization
This patch set adds implementation for CONTINUOUS type
of audit.

Change-Id: I5f4ec97b2082c8a6b3ccebe36b2a343fa4a67d19
Implements: blueprint continuously-optimization
2016-07-07 19:17:17 +03:00
Edwin Zhai
1a1d9d09f4 Fix watcher doc build error
Updates the module name to reflect the right defination of efficacy
specification, and adds doc string to some class.

Change-Id: If54e65a5e98521d74d8e19a21dd900ecd37edfe5
Closes-Bug: #1599368
2016-07-06 06:06:00 +00:00
Edwin Zhai
0b29a8394b Enable strategy parameters
Strategy provides parameters to customize algorithm behavior. End user
could query then specify parameters for their requirements.

Change-Id: Id097db5f6e79c94b57674c8e5d55b06098abf18c
Implements-bp: optimization-threshold
2016-07-01 03:01:41 +00:00
jinquanni
b4c5e2bb81 Make default Planner generic to handle new action
Currently we have to amend the code of the Planner
for each new added Watcher Action. This patch set will
modifying the DefaultPlanner class and  leverage the
plugins-parameters blueprint so that we can configure
weights via the configuration file.

Documentation and unittest also be update in this patch.

Co-Authored-By: Vincent Françoise <vincent.francoise@b-com.com>

Change-Id: Ib794488fcafd1e153a7d7b1f7253686136501872
blueprint: configurable-weights-default-planner
2016-06-28 07:15:36 +08:00
junjie huang
2381a677b6 Uniform Airflow migration strategy implementation
This is one of the algorithm of Intel thermal POC.
It's based on the Airflow of hypervisors.

Change-Id: I48bbab75b20129d85b1fb15ede823474546fe399
blueprint: uniform-airflow-migration-strategy
2016-06-27 04:46:18 +00:00
Jenkins
d7a7b3ea54 Merge "Add importing modules instead of classes" 2016-06-23 15:39:08 +00:00
Alexandr Stavitskiy
cb693e4093 Add importing modules instead of classes
This patch removes import of classes and replaces them with import of modules.

Change-Id: Id2502cf96ac7227cf1036cd54a74f3d7acd83479
Closes-Bug: #1594434
2016-06-22 13:32:34 +03:00
Tin Lam
9918f59227 Centralize plugin loaders in decision engine
Refactor planner loader in watcher/decision_engine/planner/loading/default.py
to watcher/decision_engine/loading/default.py.

Change-Id: I3d4f3668d2269b5a77a35f4470a4d1c96c3128dd
Closes-Bug: #1591092
2016-06-21 23:37:38 -05:00
jinquanni
80abcd6fd9 Use disabled/enabled to change service state
The VM workload consolidation strategy sends 'down' instead
of 'disabled' to change nova-compute service state. This patch
will correct it. The same applies for enabling nova-compute service.

Change-Id: I257411ef711b215bd9b56d0bf0479c79a9ef61d8
Closes-Bug: #1591901
2016-06-21 18:25:14 +08:00
Bruno Grazioli
b3c2d3af1f Check if nova-service is already disabled
This patch updates VM workload consolidation strategy adding a new
condition to check whether nova-compute service is disabled before
creating the action.

Change-Id: I1accbc7bbd62684dce14ca3b35c92121e923a73a
Closes-Bug: #1591927
2016-06-21 09:54:36 +02:00
Jean-Emile DARTOIS
7b403c0d3b Fix StrategyContext to use the strategy_id in the Audit Template
This patch fixes the StrategyContext to use the optional
attribute strategy_id.

Change-Id: Ib78581f564282de6cfc7f07495c846615ec1866a
Closed-bug: #1590357
2016-06-15 13:08:28 +02:00
Vincent Françoise
45801cf9c5 Documentation on goal and efficacy
In this changeset, I wrote a documentation detailing how one can
implement a new goal plugin. I also mention to define the efficacy
specification for a given goal.

Partially Implements: blueprint efficacy-indicator

Change-Id: Iba267ae312f248b49d4600504f11678cdc225622
2016-06-10 17:08:28 +02:00
Vincent Françoise
442512cd71 Added efficacy indicators to /action_plans
I this changeset, I added the efficacy indicators both at the DB
and at the API level alongside the associated logic.

Partially Implements: blueprint efficacy-indicator

Change-Id: I824553637621da67966103c1b0c01348b09bd836
2016-06-10 09:37:14 +02:00
Vincent Françoise
2b95a4cbc4 Added pre/post execution methods to strategies
In this changeset, I broke down the execute() method to sequentially
call 3 methods:

- pre_execute()
- do_execute()
- post_execute()

This changeset also removes the cluster model parameter from the
execute() method to now become a `model` property of a strategy which
is lazy loaded whenever needed.

Partially Implements: blueprint efficacy-indicator

Change-Id: I2f697938db693acfa95b2c2fbecfdc1b733c93fd
2016-06-10 09:37:11 +02:00
Vincent Françoise
f665d83657 Added efficacy specification to /goals
In this changeset, I added the new Efficacy, EfficacySpecification
and IndicatorSpecification classes which are the main components for
computing an efficacy.

Partially Implements: blueprint efficacy-indicator

Change-Id: I3a1d62569de2dd6bb6f9a52f6058313fa2b886ce
2016-06-10 09:36:59 +02:00
Vincent Françoise
2544327979 Decoupled Goal from Strategy
In this changeset, I decoupled the notion of Goal from the Strategy
by making it a distinct object. Goals are plugins that can be loaded
just like for the strategies.

Partially Implements: blueprint efficacy-indicator

Change-Id: I4378dccd508170b305aa968843228bbc8af78895
2016-06-08 14:00:43 +02:00
Alexander Chadin
ca37358cac Add fix for hardware.cpu.util meter in sd-strategy
This patch set removes normalizing for hardware.cpu.util meter
since the values that comes from ceilometer hardware.cpu.util
are already normalized.

Closes-Bug: #1588257

Change-Id: I9494f2cc9bbaa6dfd168fb515f679eb6d7f2398a
2016-06-02 13:12:48 +03:00
Alexander Chadin
5d66f66050 Add fix for __init__() error
This patch set fixes __init__() got multiple values
for keyword argument osc error.

Closes-Bug: #1587824

Change-Id: Ieaa1250774ec0fdab5450fe9c3962bb3d1f4136b
2016-06-02 10:48:28 +03:00
Jenkins
e9f237dc80 Merge "Enabled config parameters to plugins" 2016-05-30 09:40:30 +00:00
junjie huang
30bdf29002 Workload balance migration strategy implementation
This is one of the algorithm of Intel thermal POC.
It's based on the VM workloads of hypervisors.

Change-Id: I45ab0cf0f05786e6f68025bdd315f38381900a68
blueprint: workload-balance-migration-strategy
2016-05-30 07:52:05 +00:00
Vincent Françoise
5aa6b16238 Enabled config parameters to plugins
In this changeset, I added the possibility for all plugins to define
configuration parameters for themselves.

Partially Implements: blueprint plugins-parameters

Change-Id: I676b2583b3b4841c64c862b2b0c234b4eb5fd0fd
2016-05-30 09:48:34 +02:00
Alexander Chadin
9d3671af37 Add Overload standard deviation strategy
The main purpose of this strategy is to choose the pair VM:dest_host that
minimizes the standard deviation in a cluster best.

Change-Id: I95a31b7bcab83411ef6b6e1e01818ca21ef96883
Implements: blueprint watcher-overload-sd
2016-05-27 16:16:36 +03:00
Vincent Françoise
5f6a97148f Remove [watcher_goals] config section
In this changeset, I remove the now unused [watcher_goals] section.

Partially Implements: blueprint get-goal-from-strategy

Change-Id: I91e4e1ac3a58bb6f3e30b11449cf1a6eb18cd0ca
2016-05-11 15:48:09 +02:00
Vincent Françoise
f9a1b9d3ce Documentation update for get-goal-from-strategy
In this changeset, I updated the Watcher documentation to reflect
the changes that are introduced by this blueprint.

Partially Implements: blueprint get-goal-from-strategy

Change-Id: I40be39624097365220bf7d94cbe177bbf5bbe0ed
2016-05-11 15:48:02 +02:00
Vincent Françoise
18e5c7d844 Syncer now syncs stale audit templates
In this changeset, I introduce the syncing of audit templates.

Partially Implements: blueprint get-goal-from-strategy

Change-Id: Ie394c12fe51f73eff95465fd5140d82ebd212599
2016-05-11 15:31:02 +02:00
Vincent Françoise
2966b93777 Add strategy_id & goal_id fields in audit template
In this changeset, I updated the 'goal_id' field into the AuditTemplate
to now become a mandatory foreign key towards the Goal model. I also
added the 'strategy_id' field into the AuditTemplate model to be an
optional foreign key onto the Strategy model.

This changeset also includes an update of the /audit_template
Watcher API endpoint to reflect the previous changes.

As this changeset changes the API, this should be merged alongside the
related changeset from python-watcherclient.

Partially Implements: blueprint get-goal-from-strategy

Change-Id: Ic0573d036d1bbd7820f8eb963e47912d6b3ed1a9
2016-05-11 15:31:02 +02:00
Vincent Françoise
e67b532110 Refactored Strategy selector to select from DB
In this changeset, I refactored the strategy selector to now
look into the Watcher DB instead of looking into the configuration
file.

Partially Implements: blueprint get-goal-from-strategy

Change-Id: I2bcb63542f6237f26796a3e5a781c8b62820cf6f
2016-05-11 15:31:01 +02:00
Vincent Françoise
81765b9aa5 Added /strategies endpoint in Watcher API
In this changeset, I added the /strategies endpoint to the Watcher
API service.
This also includes the related Tempest tests.

Partially Implements: blueprint get-goal-from-strategy

Change-Id: I1b70836e0df2082ab0016ecc207e89fdcb0fc8b9
2016-05-11 15:31:01 +02:00
Vincent Françoise
673642e436 Add Goal in BaseStrategy + Goal API reads from DB
In this changeset, I changed the Strategy base class to add new
abstract class methods. I also added an abstract strategy class
per Goal type (dummy, server consolidation, thermal optimization).

This changeset also includes an update of the /goals Watcher API
endpoint to now use the new Goal model (DB entries) instead of
reading from the configuration file.

Partially Implements: blueprint get-goal-from-strategy
Change-Id: Iecfed58c72f3f9df4e9d27e50a3a274a1fc0a75f
2016-05-11 15:31:00 +02:00
Vincent Françoise
a3ac26870a DB sync for Strategies
In this changeset, I added the ability to synchronize the strategies
into the Wather DB so that it can later be served through the Watcher
API.

Partially Implements: blueprint get-goal-from-strategy

Change-Id: Ifeaa1f6e1f4ff7d7efc1b221cf57797a49dc5bc5
2016-05-11 15:19:40 +02:00
Vincent Françoise
3b5ef15db6 Added Goal object + goal syncing
In this changeset, I added the Goal object into Watcher along with
a sync module that is responsible for syncing the goals with the
Watcher DB.

Partially Implements: blueprint get-goal-from-strategy

Change-Id: Ia3a2032dd9023d668c6f32ebbce44f8c1d77b0a3
2016-05-11 15:19:40 +02:00
Vincent Françoise
eb861f86ab Refactored DE and Applier to use oslo.service
In this PS, I have refactored the Decision Engine and the Applier
to use the oslo service utility.

Change-Id: If29158cc9b5e5e50f6c69d67c232cceeb07084f2
Closes-Bug: #1541850
2016-04-22 10:33:21 +02:00
Larry Rensing
408d6d4650 Removed unused 'alarm' field
The 'alarm' field is currently unused, so it has been removed.

Change-Id: I02fa15b06ed49dbc5dd63de54a9cde601413983c
Closes-Bug: #1550261
2016-04-18 14:12:12 +00:00
Jenkins
0f14b7635d Merge "correct the available disk, memory calculating Source data are misused in outlet temperature strategy. This patch fixes it." 2016-04-12 07:08:15 +00:00
junjie huang
bb77641aad correct the available disk, memory calculating
Source data are misused in outlet temperature strategy. This patch
fixes it.

Change-Id: I8ad5c974d7674ddfe6c4c9e3a6e3029d34a400db
Closes-bug: #1569114
2016-04-11 17:53:54 +00:00
zhangguoqing
a52d92be87 Remove unused logging import and LOG global var
In some modules the global LOG is not used any more. And the import
of logging is not used. This patch removes the unused logging import
and LOG vars.

Change-Id: I794ee719d76f04e70154cf67f726152fbb1ba15a
2016-04-06 10:34:39 +08:00
Jenkins
46d5094add Merge "Added information on plugin mechanism to glossary" 2016-04-05 07:52:55 +00:00
cima
8b77e78f3d Added missing support for resource states in unicode format in VM workload consolidation strategy
Unicode type resource state is now handled in the same fashion as resource state specified by general string.

Change-Id: I35ffa09015283b51c935515436735aecbe83a9d6
Closes-Bug: #1565764
2016-04-04 15:17:35 +02:00
Tin Lam
c67f83cce0 Added information on plugin mechanism to glossary
Added extra information regarding the plugin mechanism for:
action, strategy, and Watcher planner.

Change-Id: I9a7523282e229b83c16b06e3806ff795a0699c78
Closes-Bug: #1558470
2016-03-24 18:42:17 -05:00
Bruno Grazioli
4c924fc505 Integrated consolidation strategy with watcher
This patch adds a new load consolidation strategy based on a heuristic
algorithm which focuses on measured CPU utilization and tries to
minimize hosts which have too much or too little load.
A new goal "vm_workload_consolidation" was added which executes
the strategy "VM_WORKLOAD_CONSOLIDATION".
This work depends on the implemetation of the bug:
https://bugs.launchpad.net/watcher/+bug/1553124

Change-Id: Ide05bddb5c85a3df05b94658ee5bd98f32e554b0
Implements: blueprint basic-cloud-consolidation-integration
2016-03-24 12:00:01 +01:00