When an exception was caught and rethrown, it should call 'raise'
without any arguments because it shows the place where
an exception occured initially instead of place where
the exception re-raised
Change-Id: I662583cd3cda2424d5a510cae7e815c97a51c2fe
In this changeset, I refactored the whole Watcher codebase to
adopt a naming convention about the various elements of the
Compute model so that it reflects the same naming convention
adopted by Nova.
Change-Id: I28adba5e1f27175f025330417b072686134d5f51
Partially-Implements: blueprint cluster-model-objects-wrapper
In this changeset, I implemented a background scheduler service
for Watcher and more particularly for the Decision Engine where
I made it create 2 types of job per cluster data model collector
plugin:
- An initial job that is asynchronously executed upon starting the
Decision Engine
- A periodical job that gets triggered every configurable interval
of time
Change-Id: I3f5442f81933a19565217b894bd86c186e339762
Partially-Implements: bluprint cluster-model-objects-wrapper
A Scoring Module needs to expose a list of available
scoring engines through API and Watcher CLI. This list
is stored in database and synchronized by Decision Engine.
Partially-Implements: blueprint scoring-module
Change-Id: I32168adeaf34fd12a731204c5b58fe68434ad087
APIImpact
The hacking checks enforce during the pep8 run functional validations of
the code to ensure deeper filters and code consistency. This change set
adds the hacking checks to the wathcer project. These checks were
seeded from the neutron project, which had a good set of base defaults.
This change set also updates the watcher project to be compliant with
these new hacking checks.
Change-Id: I6f4566d384a7400bddf228aa127a53e6ecc82c2e
This patch set adds implementation for CONTINUOUS type
of audit.
Change-Id: I5f4ec97b2082c8a6b3ccebe36b2a343fa4a67d19
Implements: blueprint continuously-optimization
In this changeset, I refactored the DB filter system to support
comparison operators using a django-like syntax.
A filter can take 2 forms:
- "<FIELDNAME>" which is a syntactic sugar for "<FIELDNAME>__eq"
- "<FIELDNAME>__<OPERATOR>" where <OPERATOR> is the comparison operator
to be used.
Here is the list of the supported operators:
- 'eq' (==)
- 'neq' (!=)
- 'gt' (>)
- 'gte' (>=)
- 'lt' (<)
- 'lte' (<=)
- 'in' (in)
- 'notin' (not in)
Change-Id: I53a61d50a3253342a40f0ff87cb5612ed57a3bd1
I this changeset, I added the efficacy indicators both at the DB
and at the API level alongside the associated logic.
Partially Implements: blueprint efficacy-indicator
Change-Id: I824553637621da67966103c1b0c01348b09bd836
In this changeset, I added the new Efficacy, EfficacySpecification
and IndicatorSpecification classes which are the main components for
computing an efficacy.
Partially Implements: blueprint efficacy-indicator
Change-Id: I3a1d62569de2dd6bb6f9a52f6058313fa2b886ce
In this changeset, I created a new model named EfficacyIndicator
which is responsible for storing information regarding the efficacy
indicators that were computed by the strategy in its solution. Every
efficacy indicator should relate to a single Action Plan.
Partially Implements: blueprint efficacy-indicator
Change-Id: Ifc14ea5e16e92f032d7912c9b3fdc270af79cab6
The main purpose of this strategy is to choose the pair VM:dest_host that
minimizes the standard deviation in a cluster best.
Change-Id: I95a31b7bcab83411ef6b6e1e01818ca21ef96883
Implements: blueprint watcher-overload-sd
In this changeset, I just generate the .pot file for all the new
translations that were added during the implementation of this BP
Partially Implements: blueprint get-goal-from-strategy
Change-Id: I2192508afda037510f8f91092c5cfde0115dae1d
In this changeset, I updated the 'goal_id' field into the AuditTemplate
to now become a mandatory foreign key towards the Goal model. I also
added the 'strategy_id' field into the AuditTemplate model to be an
optional foreign key onto the Strategy model.
This changeset also includes an update of the /audit_template
Watcher API endpoint to reflect the previous changes.
As this changeset changes the API, this should be merged alongside the
related changeset from python-watcherclient.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: Ic0573d036d1bbd7820f8eb963e47912d6b3ed1a9
In this changeset, I refactored the strategy selector to now
look into the Watcher DB instead of looking into the configuration
file.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: I2bcb63542f6237f26796a3e5a781c8b62820cf6f
In this changeset, I add the Strategy model as well as the DB
functionalities we need to manipulate strategies.
This changeset implies a DB schema update.
Partially Implements: blueprint get-goal-from-strategy
Change-Id: I438a8788844fbc514edfe1e9e3136f46ba5a82f2
In this changeset, I added the Goal model into Watcher.
This implies a change into the Watcher DB schema
Partially Implements: blueprint get-goal-from-strategy
Change-Id: I5b5b0ffc7cff8affb59f17743e1af0e1277c2878
We have to check Audit Type and Audit State to make sure
these parameters are in valid status.
Also, we provide default states for the next attributes:
- 'audit_template' is required and should be either UUID or text field
- 'state' is readonly so it raises an error if submitted in POST
and is set by default to PENDING
- 'deadline' is optional and should be a datetime
- 'type' is a required text field
Change-Id: I2a7e0deec0ee2040e86400b500bb0efd8eade564
Closes-Bug: #1532843
Closes-Bug: #1533210
In watcher/applier/workflow_engine/default.py, we are checking the
return value of action.execute(). As the "TODO" above indicates it
(line 118), we should get rid of this and only flag an action as
failed if an exception was raised during its execute(). We will
need to update the related unit tests.
Change-Id: Ia8ff7abd9994c3504e733ccd1d629cafe9d4b839
Closes-Bug: #1548383
This patchset implements the purge script as specified in its
related blueprint:
- The '--age-in-days' option allows to specify the number of
days before expiry
- The '--max-number' option allows us to specify a limit on the number
of objects to delete
- The '--audit-template' option allows you to only delete objects
related to the specified audit template UUID or name
- The '--dry-run' option to go through the purge procedure without
actually deleting anything
- The '--exclude-orphans' option which allows you to exclude from the
purge any object that does not have a parent (i.e. and audit without
a related audit template)
A prompt has been added to also propose to narrow down the number of
deletions to be below the specified limit.
Change-Id: I3ce83ab95277c109df67a6b5b920a878f6e59d3f
Implements: blueprint db-purge-engine
We want a simplest way to validate the input parameters of an
Action through a schema.
APIImpact
DocImpact
Partially implements: blueprint watcher-add-actions-via-conf
Change-Id: I139775f467fe7778c7354b0cfacf796fc27ffcb2
The OpenStackClients class provides a convenient way to create and
cache client instances. The idea behind this code comes from Magnum
[0].
The OpenStackClients class will act as the manager of other project's
clients, providing an easy way to fetch instances of said clients. This
will allow the clients to be cached.
An instance of OpenStackClients is created for every call that comes
into the decision engine and the applier, using the request context to
pass needed (domain id) parameters to get a Keystone session. This
instance should be shared as much as possible to avoid additional
unneccessary connections to the other services.
This class will also allow for the version of each client to be
configurable via the watcher.conf file.
The method by which a Keystone session is also changed to use the
keystoneauth1.loading library. In order to avoid DuplicateOptErrors
with the keystone_authtoken group used for the keystonemiddleware in the
API code, a new conf group named "watcher_clients_auth" is created. A
typical configuration using a password authentication scheme will look
like:
[watcher_clients_auth]
auth_type = password
auth_url = http://<server-ip>:<port>
username = <username>
password = <password>
project_domain_id = default
user_domain_id = default
[0]: https://github.com/openstack/magnum/blob/master/magnum/common/clients.py
DocImpact
Change-Id: Iab9d0b304099686da2e9e2b19e8b1de4332ff378
Implements: blueprint external-api-versioning
Closes-Bug: #1530790
Closes-Bug: #1539670
Closes-Bug: #1522774
The InvalidParameterValue exception does not define a meaningful
msg_fmt. It is currently _("%(err)s"), which is the equivalent of
nothing and does not help with translation.
Replace InvalidParameterValue with Invalid exceptions.
Change-Id: If8b064e446cbc97e380127f360f262be9e8877a1
Closes-Bug: #1538398
The msg_fmt of ActionPlanNotFound was missing an "_" which caused
errors upon trying to format it, so I fixed it.
Change-Id: I515c2097a563f809e319d2e57480fd340b878cef
Closes-Bug: #1538065
The BaseException class defined in exceptions.py is shadowing the
built-in BaseException class of the Python exception hierarchy, which
could potentially cause confusion.
This removes the BaseException definition and replaces it with the
existing WatcherException object. Instantiations of the
IllegalArgumentException are also changed to use the message kwarg.
Change-Id: I20abf135805c7a354924de8a5194b59fc040460a
Closes-Bug: #1535504
The custom exceptions were defining the message format string as the
'message' attribute, which was confusing as 'message' is also being used
as an input argument to the class. This combined with the fact that the
the 'message' attribute of Python's BaseException has been deprecated,
msg_fmt replaces message to hold the message format of the exception.
The root cause of the bug in question was caused by the __unicode__
method returning self.message, which did not have the kwargs substituted
into the actual message format. The fix is to use self.args[0] instead,
which will contain the message format with the kwargs substituted in
since that is passed to the super's (Exception) __init__ method. See
PEP 0352 for more information on how Exception and BaseException work.
The _cleanse_dict method is also removed as it is not used anywhere.
Change-Id: Ie8ac96afaecc732693a184d0e06e77c56ca8eeb9
Closes-Bug: #1535473
In watcher, an audit generates a set of actions which
aims at achieving a given goal (lower energy consumption, ...).
It is possible to configure different strategies in order to achieve
each goal. Each strategy is written as a Python class which produces
a set of actions. Today, the set of possible actions is fixed for a
given version of Watcher and enables optimization algorithms to
include actions such as instance migration, changing hypervisor state,
changing power state (ACPI level, ...).
The objective of this patchset is to give the ability to load
the actions dynamically in order to apply the Action Plan.
DocImpact
Partially implements: blueprint watcher-add-actions-via-conf
Change-Id: Idf295b94dca549ac65d4636e8889c8ab2ecc0df6
Replacing dict.iteritems()/.itervalues() with
six.iteritems(dict)/six.itervalues(dict) was preferred in the past,
but there was a discussion suggesting to avoid six for this.
ref:
http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html
Change-Id: I63b1e51597307862968f37803ffdbba63306d8c6
In watcher, an audit generates a set of actions which
aims at achieving a given goal (lower energy consumption, ...).
It is possible to configure different strategies in order to achieve
each goal. Each strategy is written as a Python class which produces
a set of actions. Today, the set of possible actions is fixed for a
given version of Watcher and enables optimization algorithms to
include actions such as instance migration, changing hypervisor state,
changing power state (ACPI level, ...).
This patchset add a common generic dynamic loader for plugins,
such as for custom Actions, Strategies, Planners, etc.
Partially implements: blueprint watcher-add-actions-via-conf
Change-Id: I59d031b93865fff2540e3973921e1bdafa95f88e
Some Python class and packages need to be renamed
for a better compliance with the shared terminology
which provides a better understanding of Watcher objects
and components by every contributor.
Partially implements: blueprint glossary-related-refactoring
Change-Id: Ie0e33562f5e990c264a50ab3f533cfa62eac1d19
Since internationalization should be enabled in Watcher, this
patchset refactors the Watcher codebase to wrap previously
untranslatable strings with i18n translation functions so we can
import them for translation into the .pot template file.
Partially Implements: blueprint support-translation
Change-Id: I425967a60b5a7957f753894e5d2ba0d2c5009d1d
Our project should now enable its internationalization.
This patchset add the french locale to the project but also
refactors the codebase to following the oslo_i18n recommendations.
DocImpact
Implements: blueprint support-translation
Change-Id: I0e4fbf05d16afb5e25bac78438c640f147c754b1
Even though Watcher was mentioning python 3.4 as supported, it
really wasn't the case as all the unit tests were not passing in this
version of Python.
This patchset fixes all the failing tests in Python 3.4 while
keeping Watcher Python 2.7 compatible.
DocImpact
BugImpact
Change-Id: Ie74acc08ef0a2899349a4b419728c89e416a18cb
As the openstack/common is deprecated, Watcher should use oslo lib
directly to run in the openstack env. So there is much work to do
with oslo. This patch fixed the log part.
Change-Id: I3145a842fe3a22b3d08fec60d180329073edd50f
