Commit Graph

208 Commits

Author SHA1 Message Date
Vincent Françoise
4dfe6a3fa8 Action Plan state - Changed STARTING to TRIGGERED
As the STARTING state was not clear on its meaning, I renamed it
to TRIGGERED.

Change-Id: I99cceeb57f3d7d42c1543b21fad88a6872bc4e55
Closes-Bug: #1533245
2016-01-22 15:19:04 +00:00
Vincent Françoise
b53da21b9b Tempest scenario - execute a dummy strategy
This patchset implements the first scenario for Watcher which does
the following actions:

- create an audit template with the dummy strategy
- run the audit to create an action plan
- get the action plan
- run the action plan
- get results and make sure it succeeded

Partially Implements: blueprint tempest-basic-set-up

Change-Id: Iee74ede0bd1bcd03e8938f2ec8c6884f99e7f99a
2016-01-22 15:18:44 +00:00
Vincent Françoise
a16351d352 API Tempest tests on Action plans
This patchset adds CRUD tests on Action Plans via the API.

Partially Implements: blueprint tempest-basic-set-up

Change-Id: I8ff3c3f0dbf7d301be2e3f401edf24bca44914bd
2016-01-22 15:18:15 +01:00
Vincent Françoise
6d0754bb65 API Tempest tests on Audits
This patchset adds CRUD tests on Audits via the API.
Many of them are currently skipped as they revealed some underlying
bugs which are referenced on launchpad.

Partially Implements: blueprint tempest-basic-set-up
Change-Id: I5769f601f9d1cb94bb541959f94f0fa2e17d15c9
2016-01-22 11:37:26 +01:00
Vincent Françoise
595b13a622 Refactored existing tempest API tests
We must set up Tempest for Watcher
(http://docs.openstack.org/developer/tempest/configuration.html)
to run integration tests inside devstack environment.

This patchset is a refactoring of the stale Tempest tests to now
use the latest Tempest coding standards (like using plugins and
credentials factory).

This commit will have an effect on the doc as we need to integrate
Tempest in the Watcher documentation.

DocImpact
Partially Implements: blueprint tempest-basic-set-up
Change-Id: I7600ff8a28d524b56c7dd4903ac4d203634ae412
2016-01-22 09:53:33 +00:00
Jenkins
8832ad78e2 Merge "Refactor Commands section" 2016-01-22 09:47:51 +00:00
Jenkins
be81f8aff2 Merge "Renamed Status to State" 2016-01-22 09:45:10 +00:00
Jenkins
e95548b1a9 Merge "Update the user-guide to explain the main steps" 2016-01-22 09:41:27 +00:00
Vincent Françoise
62b39fefbb Renamed Status to State
As we can see in the codebase, we have 3 "Status" enums which are
located at:

- watcher/objects/action.py
- watcher/objects/action_plan.py
- watcher/objects/audit.py

So I renamed them from "Status" to "State" to be consistent with
the DB schema.

Change-Id: If3d180c9daba6ae9083775ad6813aa4c21763dbf
Closes-Bug: #1522733
2016-01-22 09:36:21 +01:00
vmahe
f23548b92f Update the user-guide to explain the main steps
Added a new sequence diagram to explain showing the main steps
regarding Watcher usage:
- create a new audit template
- launch an audit and receive a recommended action plan
- launch the action plan

Change-Id: I1a0031b957d34908020be06870ed1d2772cd5af6
Closes-Bug: 1536731
2016-01-22 09:16:47 +01:00
Jenkins
43275537ea Merge "Removed unused parameters from api controllers" 2016-01-22 07:52:10 +00:00
David TARDIVEL
eaed650000 Refactor Commands section
Rename this section to 'Watcher Manual Pages' and add man RST
files.

Partial-Bug: #1535244
Change-Id: I1c890c8d053238aa568075d844f97b1e4289adcb
2016-01-21 19:46:04 +01:00
Jenkins
0505466ea5 Merge "Use taskflow library for building and executing action plans" 2016-01-21 17:38:14 +00:00
Jean-Emile DARTOIS
0e7bfe61bd Use taskflow library for building and executing action plans
The aim of this patchset is to integrate taskflow in
the Watcher Applier. Taskflow will help us a lot to make
Action Plan execution easy, consistent, scalable and reliable.

DocImpact

Partially implements: blueprint use-taskflow

Change-Id: I903d6509d74a61ad64e1506b8a7156e6e91abcfb
Closes-Bug: #1535326
Closes-Bug: #1531912
2016-01-21 18:13:42 +01:00
Jenkins
096354c255 Merge "Add diagrams to the architecture doc page" 2016-01-21 17:06:11 +00:00
Jenkins
afe06e2349 Merge "Reduced the complexity of the execute() method" 2016-01-21 17:05:05 +00:00
Jenkins
1d689ef410 Merge "Validate audit template UUID on audit create" 2016-01-21 16:55:20 +00:00
Gábor Antal
58fb86a3da Removed unused parameters from api controllers
In the Audit controller (watcher/api/controllers/v1/audit.py),
we have an unused "audit_uuid" parameter in both get_all()
and detail()

Similarly, the Action Plan controller
(watcher/api/controllers/v1/action_plan.py) has an unused
"action_plan_uuid" parameter in both get_all() and detail().

I also removed them from the @wsme_pecan.wsexpose decorator

Change-Id: I7f1edfd44dd95c9768249650e19b16fcbe916b89
Closes-Bug: #1534615
2016-01-21 13:55:18 +01:00
Jenkins
f675003076 Merge "Remove shadow BaseException class" 2016-01-21 08:10:55 +00:00
Taylor Peoples
e34ee792a8 Validate audit template UUID on audit create
The audit template UUID should be validated during the creation of an
audit.  An HTTP 400 error is returned to the client if an invalid audit
template UUID is passed as part of the body when creating an audit.

APIImpact
Closes-Bug: #1510188

Change-Id: I0543d22751b77f6641ddef6a7f0f4acce61180fd
2016-01-21 08:43:28 +01:00
Jenkins
037f43cd04 Merge "Missing super() in API collection controllers" 2016-01-20 14:14:08 +00:00
vmahe
8f87699910 Add diagrams to the architecture doc page
Added some data model diagrams, sequence diagrams and state machine
diagrams.
The state machine diagrams and sequence diagrams are built with
PlantUML whereas data model diagrams are built with Dia.
Also added some textual description with the sequence diagrams.

Change-Id: Iffbb47b0f2d12ce63eeaa1531a1bd1a790d69e79
Closes-Bug: #1531802
2016-01-20 07:23:03 +01:00
David TARDIVEL
c811051351 Fix Warnings generated while building of HTML docu
Same ID have been set to reference different RST blocks.
To avoid this, I added the prefix 'archi_' within ID referencing
architecture RST block.

Bad indentation warnings have been fixed.

Change-Id: I17f43f2f564ffd83fd5c345aed96fad06ee56b1d
Partial-Bug: #1522034
2016-01-20 00:58:09 +01:00
Béla Vancsics
a2750c74f9 Reduced the complexity of the execute() method
The execute() method is very large - almost 150 lines,
and McCabe's cyclomatic complexity 22.
(watcher/decision_engine/strategy/strategies/basic_consolidation.py)
I extracted some of the functionalities into helper functions
to reduce the length and complexity of execute().
http://openqa.sed.hu/dashboard/index/544838?did=1
Additionally it became more readable as well, without
changing its functionality.

Change-Id: I760929f56e258b87d7f1d4586bcc90665f1e0d8f
Closes-Bug: 1535729
2016-01-19 18:37:11 +01:00
Gábor Antal
67455c6a84 Missing super() in API collection controllers
The __init__ of these 3 classes are missing the super() call

watcher/watcher/api/controllers/v1/goal.py:113
watcher/api/controllers/v1/audit.py:217
watcher/watcher/api/controllers/v1/audit_template.py:166

Change-Id: I2fec1d5dac29a311f617ef141d5bf91f00b96219
Closes-Bug: #1535354
2016-01-19 13:15:32 +01:00
Taylor Peoples
a7455a8bf7 Remove shadow BaseException class
The BaseException class defined in exceptions.py is shadowing the
built-in BaseException class of the Python exception hierarchy, which
could potentially cause confusion.

This removes the BaseException definition and replaces it with the
existing WatcherException object.  Instantiations of the
IllegalArgumentException are also changed to use the message kwarg.

Change-Id: I20abf135805c7a354924de8a5194b59fc040460a
Closes-Bug: #1535504
2016-01-19 06:59:54 +01:00
Taylor Peoples
7fcb683404 Replace message with msg_fmt for custom exceptions
The custom exceptions were defining the message format string as the
'message' attribute, which was confusing as 'message' is also being used
as an input argument to the class.  This combined with the fact that the
the 'message' attribute of Python's BaseException has been deprecated,
msg_fmt replaces message to hold the message format of the exception.

The root cause of the bug in question was caused by the __unicode__
method returning self.message, which did not have the kwargs substituted
into the actual message format.  The fix is to use self.args[0] instead,
which will contain the message format with the kwargs substituted in
since that is passed to the super's (Exception) __init__ method.  See
PEP 0352 for more information on how Exception and BaseException work.

The _cleanse_dict method is also removed as it is not used anywhere.

Change-Id: Ie8ac96afaecc732693a184d0e06e77c56ca8eeb9
Closes-Bug: #1535473
2016-01-19 04:20:37 +01:00
Jenkins
65cf19a7e4 Merge "Removed use of deprecated LOG.warn method" 2016-01-18 17:08:58 +00:00
Jenkins
facca13dc4 Merge "Clean up flake8 ignore list" 2016-01-18 13:42:00 +00:00
Jenkins
e8576e8963 Merge "Renamed diskInfo.py" 2016-01-18 13:12:03 +00:00
Gábor Antal
dfc9a3b37d Removed use of deprecated LOG.warn method
LOG.warn is deprecated. We were still used it some places.
So I replaced the LOG.warn method to LOG.warning, which is not
deprecated.

Change-Id: I9461cec569445ad6c40db9ce2feeeba1ef0af0e3
Closes-Bug: #1508442
2016-01-18 13:12:52 +01:00
Jenkins
24dc92091c Merge "Keep py3.X compatibility for urllib" 2016-01-18 08:22:24 +00:00
Jenkins
53d6d7d131 Merge "Changed testr to os-testr" 2016-01-18 08:21:40 +00:00
Jenkins
fd6ecc1b13 Merge "Add a dynamic loading of Actions handlers in the Watcher Applier" 2016-01-15 15:44:56 +00:00
Jean-Emile DARTOIS
8bac4fd42a Add a dynamic loading of Actions handlers in the Watcher Applier
In watcher, an audit generates a set of actions which
aims at achieving a given goal (lower energy consumption, ...).
It is possible to configure different strategies in order to achieve
each goal. Each strategy is written as a Python class which produces
a set of actions. Today, the set of possible actions is fixed for a
given version of Watcher and enables optimization algorithms to
include actions such as instance migration, changing hypervisor state,
changing power state (ACPI level, ...).

The objective of this patchset is to give the ability to load
the actions dynamically in order to apply the Action Plan.

DocImpact
Partially implements: blueprint watcher-add-actions-via-conf

Change-Id: Idf295b94dca549ac65d4636e8889c8ab2ecc0df6
2016-01-15 16:08:18 +01:00
Jenkins
55d186be58 Merge "Move terminology definition to class related" 2016-01-15 14:54:48 +00:00
Jenkins
ed438d2eb2 Merge "Update API documentation for action plan" 2016-01-14 16:40:24 +00:00
David TARDIVEL
4898df89ca Update API documentation for action plan
Fix the Action Plan patch method docstring.

Change-Id: I631ba0f0754c00f49212e416f259c9158bb37d89
Closes-Bug: #1532691
2016-01-14 14:07:02 +00:00
Steve Wilkerson
ae949148ef Renamed diskInfo.py
Renamed diskInfo.py to disk_info.py and the associated test to
test_disk_info.  Also changed the usage in the test to reflect
the name change.

Closes-bug: #1533189

Change-Id: Ice63cf8ea6cd4fcc770f88952cf784e5d46cca5c
2016-01-14 07:59:55 -06:00
Jean-Emile DARTOIS
e0242b49f1 Fix extraction of _LI _LW _LE _LC for translation
In order to start translating we should gradually update the messages.
To fix this issue, we need to add _LI _LW _LE _LC in setup.cfg

Change-Id: Ia0bdea4dd3ecc12a6b804add9163926dc34bb581
Related-Bug: #1534164
2016-01-14 14:53:52 +01:00
ting.wang
5b57985686 Clean up flake8 ignore list
clean up ignore list and fix errors.

Change-Id: I3f44def50fbf44fa3cf5c49cc15a98947084c236
2016-01-14 18:42:45 +08:00
Jean-Emile DARTOIS
f6ef197473 Move terminology definition to class related
As of now, the glossary defined in our documentation reflects the
current state of the codebase. In order to avoid any discrepancy
between the codebase and each definition, the objective here is to
gather both in a single place and link it into the rst documentation
via a custom directive.
Also re-aligned the requirements with liberty for doc.

Change-Id: I3089fd9f948b948115672f10937b1500b96ce180
Partial-Bug: #1526671
2016-01-14 10:47:32 +01:00
ting.wang
61c926a10b Keep py3.X compatibility for urllib
Replace urllib with six.moves.urlparse

Change-Id: I224b7b00f7ecbe6d44ce5c9414bdaab825cbd6c7
2016-01-14 17:10:49 +08:00
Jenkins
9c33f8aaeb Merge "Remove incorrect spaces for libvirt_opts value" 2016-01-14 08:50:16 +00:00
ting.wang
08fc69cfc4 Use dict.items() dirrectly instead of six.iteritems
Replacing dict.iteritems()/.itervalues() with
six.iteritems(dict)/six.itervalues(dict) was preferred in the past,
but there was a discussion suggesting to avoid six for this.

ref:
http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html

Change-Id: I63b1e51597307862968f37803ffdbba63306d8c6
2016-01-14 16:10:37 +08:00
Jenkins
dc6b8aad7e Merge "Test: make enforce_type=True in CONF.set_override and fix error" 2016-01-14 07:59:00 +00:00
ting.wang
473cee8ad3 Test: make enforce_type=True in CONF.set_override and fix error
Each config option has limitation for type and value.
We make enforce_type=True to check whether we pass wrong type.

Also fixes a type error issue in test_db_manager.py.

Change-Id: I6e111e21588525d32b05eeba75b06583d4e605ed
Related-Bug: #1517839
2016-01-14 07:10:16 +00:00
Taylor Peoples
2ffeb48010 Remove incorrect spaces for libvirt_opts value
The documentation for the DevStack plugin suggests changing the
libvirt_opts value of /etc/default/libvirt-bin to allow for live
migration between compute nodes.  This changeset removes the incorrect
spaces surrounding the equals sign in the file, which would cause the
libvirt-bin service to crash.

Change-Id: Id073da5cdc84a900a398f260d7f7b40f027eb9d3
Closes-Bug: #1533761
2016-01-14 07:50:16 +01:00
Jean-Emile DARTOIS
86d3c2ff89 Add a generic and extensible way to describe the flow of actions
In watcher, an audit generates a set of actions which
aims at achieving a given goal (lower energy consumption, ...).
It is possible to configure different strategies in order to achieve
each goal. Each strategy is written as a Python class which produces
a set of actions. Today, the set of possible actions is fixed for a
given version of Watcher and enables optimization algorithms to
include actions such as instance migration, changing hypervisor state,
changing power state (ACPI level, ...).

This patchset propose a generic and extensible way to describe
the actions and his parameters that we want to add to Action Plan.
It also remove the static actions because they are now deprecated.

The documentation regarding strategy plugin need to be
updated (plugins.rst).

DocImpact
Partially implements: blueprint watcher-add-actions-via-conf

Change-Id: I3d641080e8ad89786abca79a942c8deb2d53355b
2016-01-13 11:18:20 +01:00
Jean-Emile DARTOIS
47759202a8 Add a dynamic loading of the Watcher Planner implementation
In watcher, an audit generates a set of actions which
aims at achieving a given goal (lower energy consumption, ...).
It is possible to configure different strategies in order to achieve
each goal. Each strategy is written as a Python class which produces
a set of actions. Today, the set of possible actions is fixed for a
given version of Watcher and enables optimization algorithms to
include actions such as instance migration, changing hypervisor state,
changing power state (ACPI level, ...).

The objective of this patchset is to give the ability to extend the
default set of planner algorithms currently available in Watcher
using Stevedore.

The doc need to explain how create a new planner.

DocImpact
Partially implements: blueprint watcher-add-actions-via-conf

Change-Id: I2fd73f8c4a457ee391d764a7a3f494deecd2634f
2016-01-13 08:26:21 +00:00