Commit Graph

106 Commits

Author SHA1 Message Date
Vincent Françoise
1c38637dff Fixed issue on compute nodes iteration
In this changeset, I fixed the issue with the basic server
consolidation strategy to now loop over all compute nodes
as expected instead of stopping after the first one.

Change-Id: If594f0df41e39dfb0ef8f0fce41822018490c4ec
Closes-bug: #1548874
2016-10-24 20:11:19 +00:00
Vincent Françoise
1eb2b517ef Refactored Tests to load scenarios from file
In this changeset, I simplified the logic that is used to create
cluster data model scenarios.

Change-Id: Ia6e138d9897190d3207a70485dc62ccc34087686
2016-10-21 17:32:42 +02:00
Jenkins
0a0f482f2d Merge "Fixed Tempest test due to notification issues" 2016-09-15 09:55:33 +00:00
Jenkins
af99b3f4eb Merge "Use parameters instead of config for workload stabilization" 2016-09-14 13:33:34 +00:00
Jenkins
5ac51efa69 Merge "Implemented GMR plugin to show CDM structures" 2016-09-14 10:09:55 +00:00
Vincent Françoise
72e6564549 Fixed Tempest test due to notification issues
Change-Id: I33a0764060600b8e3d6bec757669490b9003b345
2016-09-09 16:03:12 +02:00
Alexander Chadin
2d8650f87a Use parameters instead of config for workload stabilization
This patch set allows to use audit parameters for
workload-stabilization strategy and makes some little
refactoring.

Closes-Bug: #1620604
Change-Id: I60e34611d4dd001beed31666fd11d2ab11c1723c
2016-09-07 17:15:07 +03:00
Vincent Françoise
74989fe94e Implemented GMR plugin to show CDM structures
In this changeset, I implemented a small GMR plugin that converts
the cluster data model structure into an XML structure.

Change-Id: I75548952635a0aa3c7dbd6d068831b5765a5db1a
Closes-Bug: #1620551
2016-09-06 12:15:51 +02:00
Bin Zhou
c1cbd9ebf4 Modify use of assertTrue(A in B)
Developers should use assertIn(A, B) instead of assertTrue(A in B ).

TrivialFix

Change-Id: Ie169e731c800b5d77fd231f09ac7f92090287ef3
2016-09-02 12:43:56 +08:00
Cao Xuan Hoang
deb5cb3fc2 TrivialFix: Remove cfg import unused
This patch removes cfg import unused in
watcher/api/controllers/v1/goal.py
watcher/api/controllers/v1/strategy.py
watcher/decision_engine/cluster/history/ceilometer.py
watcher/decision_engine/model/collector/manager.py
watcher/decision_engine/strategy/selection/default.py
watcher/tests/common/test_ceilometer_helper.py
watcher/tests/decision_engine/fake_strategies.py
watcher/tests/decision_engine/strategy/selector/test_strategy_selector.py

Change-Id: I0f30a056c3efa49faed857b6d1001a2367d384ac
2016-08-29 15:01:28 +07:00
Jenkins
f42106ca34 Merge "Add unit tests for continuous.py" 2016-08-26 17:44:30 +00:00
Vincent Françoise
77b7fae41e Notification and CDM partial update
In this changeset, I implemented the notification handling (Rx only)
system for consuming incoming notifications, more especially the Nova
ones. The notifications handlers also contain the logic which
incrementally updates the Compute model.

Change-Id: Ia036a5a2be6caa64b7f180de38821b57c624300c
Partially-implements: blueprint cluster-model-objects-wrapper
2016-08-26 17:46:48 +02:00
Tomasz Kaczynski
a1cb142009 Add Scoring Module implementation
This change is adding the main logic for the scoring module,
defines entry points for the scoring engine plugins and provides
a watcher-sync tool to enable Watcher database synchronization
without needing to restart any Watcher service.

Partially-Implements: blueprint scoring-module
Change-Id: If10daae969ec27a7008af5173359992e957dcd5e
2016-08-26 07:13:39 +00:00
licanwei
da67b246a5 Add unit tests for continuous.py
Add unit tests about functions 'execute_audit'
and 'is_audit_inactive'

Change-Id: If9eb4d95166a486a59cc1535106944243bb7eb30
Closes-Bug: #1616759
2016-08-25 16:52:58 +08:00
Vincent Françoise
6be758bc5a Added strategy ID + Action Plan syncing
In this changeset, I implemented the logic which cancels
any audit or action plan whose goal has been re-synced
(upon restarting the Decision Engine).

Partially Implements: blueprint efficacy-indicator

Change-Id: I95d2739eb552d4a7a02c822b11844591008f648e
2016-08-22 10:08:05 +02:00
Viacheslav Samarin
01164b0790 Scheduler of decision_engine fix
This patch set renames 'OS-EXT-STS:instance_state' to 'OS-EXT-STS:vm_state'
for correct working of decision_engine scheduler.

Change-Id: I20805a079a991d5f3b8565f52d5f7280c2389bee
Closes-Bug: #1614511
2016-08-18 16:49:20 +03:00
Vincent Françoise
31c37342cd Refactored the compute model and its elements
In this changeset, I refactored the whole Watcher codebase to
adopt a naming convention about the various elements of the
Compute model so that it reflects the same naming convention
adopted by Nova.

Change-Id: I28adba5e1f27175f025330417b072686134d5f51
Partially-Implements: blueprint cluster-model-objects-wrapper
2016-08-03 12:10:43 +02:00
Jenkins
dbde1afea0 Merge "use parameters to set the threshold" 2016-08-03 08:28:16 +00:00
Jenkins
739b667cbc Merge "Merged metrics_engine package into decision_engine" 2016-08-02 16:34:48 +00:00
Jenkins
32be5de2f9 Merge "Added DE Background Scheduler w/ model sync jobs" 2016-08-02 16:33:51 +00:00
Jenkins
027ce5916c Merge "Loadable Cluster Data Model Collectors" 2016-08-02 16:28:51 +00:00
Jenkins
39227f6e86 Merge "Use more specific asserts" 2016-08-02 14:08:14 +00:00
Gábor Antal
cc2e805780 Use more specific asserts
Many places, there are more specific asserts which can be used.
I replaced the generic assert with more specific ones, where
it was possible.

This change enhances readibility, and on fail, more useful
message is displayed

Change-Id: I86a6baeae2cd36610a2be10ae5085555246c368a
2016-08-02 14:42:29 +02:00
Vincent Françoise
0a6841f510 Merged metrics_engine package into decision_engine
In this changeset, I merged the metrics_engine package into
the decision_engine one alongside the required changes to make
the tests pass.

Change-Id: Iac1cd266a854212bf4fa8b21c744b076c3b834a8
Partially-Implements: blueprint cluster-model-objects-wrapper
2016-08-02 12:07:35 +02:00
Vincent Françoise
06c6c4691b Added DE Background Scheduler w/ model sync jobs
In this changeset, I implemented a background scheduler service
for Watcher and more particularly for the Decision Engine where
I made it create 2 types of job per cluster data model collector
plugin:

- An initial job that is asynchronously executed upon starting the
  Decision Engine
- A periodical job that gets triggered every configurable interval
  of time

Change-Id: I3f5442f81933a19565217b894bd86c186e339762
Partially-Implements: bluprint cluster-model-objects-wrapper
2016-08-02 12:07:35 +02:00
Vincent Françoise
5a2a94fbec Loadable Cluster Data Model Collectors
In this changeset, I made BaseClusterDataModelCollector instances
pluggable. This corresponds to "part 1" of the work items detailed
in the specifications.

Change-Id: Iab1c7e264add9e2cbbbb767e3fd6e99a0c22c691
Partially-Implements: blueprint cluster-model-objects-wrapper
2016-08-02 12:07:35 +02:00
Michael Gugino
52ffc2c8e2 Implement goal_id, strategy_id and host_aggregate into Audit api
Modifying the api controller for audit objects to allow
creation of audit objects by specifying either an
audit_template uuid/id and/or a goal_id.

strategy_id is optional.

Partially Implements: blueprint persistent-audit-parameters
Change-Id: I7b3eae4d0752a11208f5f92ee13ab1018d8521ad
2016-07-28 20:12:52 -04:00
junjie huang
5dee934565 use parameters to set the threshold
use parameters to set the threshold for strategies
of Uniform Airflow,Workload balance.

Change-Id: I5a547929eb1e2413468e9a5de16d3fd42cabadf9
Close-bug: #1600707
2016-07-28 11:28:41 +00:00
Drew Thorstensen
7d704dbeec Add hacking checks to watcher
The hacking checks enforce during the pep8 run functional validations of
the code to ensure deeper filters and code consistency.  This change set
adds the hacking checks to the wathcer project.  These checks were
seeded from the neutron project, which had a good set of base defaults.

This change set also updates the watcher project to be compliant with
these new hacking checks.

Change-Id: I6f4566d384a7400bddf228aa127a53e6ecc82c2e
2016-07-25 07:52:45 -04:00
David TARDIVEL
a68eb0d619 Update unitaty tests to verify strategy and goal name
strategy and goal name must be synched with entry point names.

Change-Id: I776ed98f5c13d176fdb0b8677d609fcb1ec9a235
Closes-Bug: #1602262
2016-07-12 16:54:38 +02:00
Jenkins
e78610e7d1 Merge "Make default Planner generic to handle new action" 2016-07-12 07:35:21 +00:00
Jenkins
405cde2ff9 Merge "Uniform Airflow migration strategy implementation" 2016-07-11 07:56:26 +00:00
Alexander Chadin
1de00086f5 Add continuously optimization
This patch set adds implementation for CONTINUOUS type
of audit.

Change-Id: I5f4ec97b2082c8a6b3ccebe36b2a343fa4a67d19
Implements: blueprint continuously-optimization
2016-07-07 19:17:17 +03:00
jinquanni
ca5b849f6b Remove duplicate unittest
There are two same unit test in test_default_planner.
We need to remove one.

TrivialFix

Change-Id: Ibeec9348779dafe4c2cec2411306e09e95728ca1
2016-07-05 14:57:47 +08:00
Edwin Zhai
0b29a8394b Enable strategy parameters
Strategy provides parameters to customize algorithm behavior. End user
could query then specify parameters for their requirements.

Change-Id: Id097db5f6e79c94b57674c8e5d55b06098abf18c
Implements-bp: optimization-threshold
2016-07-01 03:01:41 +00:00
jinquanni
b4c5e2bb81 Make default Planner generic to handle new action
Currently we have to amend the code of the Planner
for each new added Watcher Action. This patch set will
modifying the DefaultPlanner class and  leverage the
plugins-parameters blueprint so that we can configure
weights via the configuration file.

Documentation and unittest also be update in this patch.

Co-Authored-By: Vincent Françoise <vincent.francoise@b-com.com>

Change-Id: Ib794488fcafd1e153a7d7b1f7253686136501872
blueprint: configurable-weights-default-planner
2016-06-28 07:15:36 +08:00
junjie huang
2381a677b6 Uniform Airflow migration strategy implementation
This is one of the algorithm of Intel thermal POC.
It's based on the Airflow of hypervisors.

Change-Id: I48bbab75b20129d85b1fb15ede823474546fe399
blueprint: uniform-airflow-migration-strategy
2016-06-27 04:46:18 +00:00
Jenkins
d7a7b3ea54 Merge "Add importing modules instead of classes" 2016-06-23 15:39:08 +00:00
Tin Lam
37595e9bb7 Centralize plugin loaders in watcher/applier
Refactor all plugin loaders in watcher/applier into
watcher/applier/loading/default.py.

Change-Id: I0f0a87d4f72ead45d34aca1b14219fd2ede42a6f
Closes-Bug: #1591095
2016-06-23 02:50:55 -05:00
Alexandr Stavitskiy
cb693e4093 Add importing modules instead of classes
This patch removes import of classes and replaces them with import of modules.

Change-Id: Id2502cf96ac7227cf1036cd54a74f3d7acd83479
Closes-Bug: #1594434
2016-06-22 13:32:34 +03:00
Tin Lam
9918f59227 Centralize plugin loaders in decision engine
Refactor planner loader in watcher/decision_engine/planner/loading/default.py
to watcher/decision_engine/loading/default.py.

Change-Id: I3d4f3668d2269b5a77a35f4470a4d1c96c3128dd
Closes-Bug: #1591092
2016-06-21 23:37:38 -05:00
jinquanni
80abcd6fd9 Use disabled/enabled to change service state
The VM workload consolidation strategy sends 'down' instead
of 'disabled' to change nova-compute service state. This patch
will correct it. The same applies for enabling nova-compute service.

Change-Id: I257411ef711b215bd9b56d0bf0479c79a9ef61d8
Closes-Bug: #1591901
2016-06-21 18:25:14 +08:00
Jean-Emile DARTOIS
7b403c0d3b Fix StrategyContext to use the strategy_id in the Audit Template
This patch fixes the StrategyContext to use the optional
attribute strategy_id.

Change-Id: Ib78581f564282de6cfc7f07495c846615ec1866a
Closed-bug: #1590357
2016-06-15 13:08:28 +02:00
Vincent Françoise
442512cd71 Added efficacy indicators to /action_plans
I this changeset, I added the efficacy indicators both at the DB
and at the API level alongside the associated logic.

Partially Implements: blueprint efficacy-indicator

Change-Id: I824553637621da67966103c1b0c01348b09bd836
2016-06-10 09:37:14 +02:00
Vincent Françoise
2b95a4cbc4 Added pre/post execution methods to strategies
In this changeset, I broke down the execute() method to sequentially
call 3 methods:

- pre_execute()
- do_execute()
- post_execute()

This changeset also removes the cluster model parameter from the
execute() method to now become a `model` property of a strategy which
is lazy loaded whenever needed.

Partially Implements: blueprint efficacy-indicator

Change-Id: I2f697938db693acfa95b2c2fbecfdc1b733c93fd
2016-06-10 09:37:11 +02:00
Vincent Françoise
f665d83657 Added efficacy specification to /goals
In this changeset, I added the new Efficacy, EfficacySpecification
and IndicatorSpecification classes which are the main components for
computing an efficacy.

Partially Implements: blueprint efficacy-indicator

Change-Id: I3a1d62569de2dd6bb6f9a52f6058313fa2b886ce
2016-06-10 09:36:59 +02:00
Vincent Françoise
2544327979 Decoupled Goal from Strategy
In this changeset, I decoupled the notion of Goal from the Strategy
by making it a distinct object. Goals are plugins that can be loaded
just like for the strategies.

Partially Implements: blueprint efficacy-indicator

Change-Id: I4378dccd508170b305aa968843228bbc8af78895
2016-06-08 14:00:43 +02:00
Alexander Chadin
ca37358cac Add fix for hardware.cpu.util meter in sd-strategy
This patch set removes normalizing for hardware.cpu.util meter
since the values that comes from ceilometer hardware.cpu.util
are already normalized.

Closes-Bug: #1588257

Change-Id: I9494f2cc9bbaa6dfd168fb515f679eb6d7f2398a
2016-06-02 13:12:48 +03:00
Jenkins
e9f237dc80 Merge "Enabled config parameters to plugins" 2016-05-30 09:40:30 +00:00
junjie huang
30bdf29002 Workload balance migration strategy implementation
This is one of the algorithm of Intel thermal POC.
It's based on the VM workloads of hypervisors.

Change-Id: I45ab0cf0f05786e6f68025bdd315f38381900a68
blueprint: workload-balance-migration-strategy
2016-05-30 07:52:05 +00:00