Commit Graph

80 Commits

Author SHA1 Message Date
Alexander Chadin
95548af426 Fix watcher audit list command
This patch set adds data migration version that fills noname audits
with name like strategy.name + '-' + audit.created_at.

Closes-Bug: #1738758
Change-Id: I1d65b3110166e9f64ce5b80a34672d24d629807d
2017-12-22 08:43:28 +00:00
aditi
71730c0eaf Multiple global efficacy
This patch update the global efficacy for server consolidation
strategy, test cases and general format.

Change-Id: I62af1e4be415998669f938b3d587c1ccf4293419
Implements: blueprint multiple-global-efficacy-indicator
2017-11-09 14:51:17 +09:00
Zuul
aa72f984e4 Merge "add name for audit, changes for watcher api/db" 2017-10-30 02:11:26 +00:00
suzhengwei
d7d56cbd79 add name for audit, changes for watcher api/db
Change-Id: Ibe04f5375d741d15999fde6faf767f15311c6351
Implements:blueprint add-name-for-audit
2017-10-20 14:17:20 +08:00
Alexander Chadin
a5e7fd90c2 Fix Watcher DB schema creation
This patch set replaces create_schema with upgrade to fix
apscheduler creation issue. It also fixes pep8 warnings to
d09a5945e4a0_add_action_description_table.py

Change-Id: Ica842d585ee3a9cd67e45eb1d7bb1916573d7c9c
2017-09-15 15:30:38 +03:00
Yaguo Zhou
51dba60e01 Replace DbMigrationError with DBMigrationError
because DbMigrationError is deprecated

Change-Id: I75ef338d2e22924997804632d26ae3588c4f719b
2017-09-05 23:31:05 +08:00
licanwei
d5a7d7674c Fix show db version in README
watcher-db-manage version: Print the current version

Change-Id: Ie08eb682879b2c071f724a6847094650047bde34
2017-08-01 21:54:48 -07:00
licanwei
a24b7f0b61 dynamic action description
Add a new table to save the mapping
Add logic to update the table when action loading
Add logic to show the action description

Change-Id: Ia008a8715bcc666ab0fefe444ef612394c775e91
Implements: blueprint dynamic-action-description
2017-07-26 20:42:01 -07:00
Jenkins
aae7699bc5 Merge "Fix dbmanage upgrade and downgrade" 2017-07-19 12:37:59 +00:00
Jenkins
2266e2baa3 Merge "New cron type for audit interval" 2017-07-19 02:46:25 +00:00
Alexander Chadin
0b44492da7 New cron type for audit interval
This patch set adds cron supporting for audit.

Implements: blueprint cron-based-continuous-audits
Change-Id: I8570bebb13332dfba80185e912aeda45b6b4cd70
2017-07-14 11:50:46 +03:00
licanwei
b146e29c39 Fix dbmanage upgrade and downgrade
The argument is '--revision'

Change-Id: Icf3a37ed42dd1dfe4fc9c52525dd4f0fb851417c
2017-07-14 14:12:41 +08:00
licanwei
59649f6a81 Fix get_action_plan_list filter error
When using 'query.filter_by(deleted_at=None)'
Will generate the incorrect SQL 'audits.deleted_at', as below:
SELECT * FROM action_plans INNER JOIN audits ON audits.id =
action_plans.audit_id WHERE audits.uuid = '' AND
audits.deleted_at IS NULL ORDER BY action_plans.id ASC

The correct filter field is 'action_plans.deleted_at'

Change-Id: Ie05f35233f78e82bc7af2c26c9effd62ea5f86ab
Closes-Bug: #1698720
2017-06-19 16:23:05 +08:00
Jenkins
590bd43a1d Merge "Trivial fix typos" 2017-05-31 08:54:56 +00:00
Vu Cong Tuan
35074edaf7 Trivial fix typos
Change-Id: I4c7d3a0d815a616d1ba2c0d26135db5f2aea0c2f
2017-05-30 15:55:33 +07:00
licanwei
a33f40ec21 Deleted audit record still get by 'audit list'cmd
Audit record was deleted but the field 'state' maybe not set DELETED.
get_audit_list's filter used field 'state' will get
wrong result.
filter rule should use field 'deleted_at' instead of field 'state'.
get_action_list and get_action_plan_list have the same filter rule.

Change-Id: I08b2a005ca5fb7c2741ac5ed97c6e6b4279758ed
Closes-Bug: #1693666
2017-05-26 14:53:32 +08:00
Alexander Chadin
f40fcdc573 Add Watcher JobStore for background jobs
This patch set adds WatcherJobStore class that allows to link
jobs and services.

Partially-Implements: blueprint background-jobs-ha
Change-Id: I575887ca6dae60b3b7709e6d2e2b256e09a3d824
2017-05-03 12:07:01 +03:00
yanxubin
f605888e32 Remove log translations
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.

See:
http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html
http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html

Change-Id: I3552767976807a9851af69b1fa4f86ac25943025
2017-03-24 09:46:19 +08:00
David TARDIVEL
7ac1d0d048 Add first alembic version for db migration
Alembic provides command script to update a relational
database, using SQLAlchemy. I provide in this patchset the
initial version python script for create a Ocata db version.

Implements: blueprint db-migration

Change-Id: I7bda4286bc8141bb4fcfba3837070966ef506b5d
2017-02-01 14:40:11 +01:00
Alexander Chadin
0e440d37ee New default planner
Co-Authored-By: Vincent Francoise <Vincent.FRANCOISE@b-com.com>
Change-Id: Ide2c8fc521488e486eac8f9f89d3f808ccf4b4d7
Implements: blueprint planner-storage-action-plan
2017-01-19 13:16:57 +03:00
Alexander Chadin
d0bca1f2ab Add auto_trigger support to watcher
This patch set adds support of auto-triggering of action plans.

Change-Id: I36b7dff8eab5f6ebb18f6f4e752cf4b263456293
Partially-Implements: blueprint automatic-triggering-audit
2016-12-19 12:37:26 +03:00
Prudhvi Rao Shedimbi
74112dd7cf Implemented db config module
Implemented db config module

Partially Implements: blueprint centralise-config-opts. Also moved
basedir_def, bindir_def and state_path_def to watcher.conf.paths

Change-Id: I73d201f6a23bbdb1c6189434b11314a66620e85c
2016-12-14 15:13:31 +01:00
licanwei
1f2a854d6a Repairing unit test failures
If fieldname is 'deleted', field.type.python_type raise
NotImplementedError.

Change-Id: I47246ce9a3b0c8d2a3ea44e825d9604f5b14ed38
Closes-Bug: #1647574
2016-12-06 17:15:19 +08:00
Vincent Françoise
06682fe7c3 Fixed update of WatcherObject fields on update
I this changeset, I fixed the issue whereby object auto fields are not
being updated within the WatcherObject after an update.

Change-Id: I7e65341b386a5c0c58c2109348e39e463cf2f668
Closes-Bug: #1641955
2016-11-17 17:41:13 +01:00
Jenkins
240f849758 Merge "Implemented base + moved plugins & service conf" 2016-11-16 17:24:45 +00:00
Vincent Françoise
46f511a8c8 Implemented base + moved plugins & service conf
In this changeset, I implemented the main logic although this is
mainly a shameful copy/paste of Nova's blueprint
https://blueprints.launchpad.net/nova/+spec/centralize-config-options

Partially Implements: blueprint centralise-config-opts

Change-Id: Ib645ad5da5c706336bb6ac37e85b027d05665c32
2016-11-16 17:45:07 +01:00
Vincent Françoise
9405eb0806 Implemented audit.update notification
In this changeset, I implemented the sending of update notifications
whenever an audit is modified.

Change-Id: I5ccc2516ce896ae7d4ef542b133e8f052eaed602
Partially-Implements: blueprint audit-versioned-notifications-api
2016-11-16 13:59:00 +01:00
Vincent Françoise
fc31dae7f2 Refactored Watcher objects to use OVO
In this changeset, I modified all existing Watcher objects to now
rely on oslo.versionedobjects as a base.

Change-Id: I3c9b1ca6da529d128743b99020350f28926ea1a2
Partially-Implements: blueprint watcher-versioned-objects
2016-11-07 10:39:19 +01:00
Vincent Françoise
ed95d621f4 Removed deadline, version, extra & host_aggregate
As we are about to version the Watcher objects, we need to make sure
that upcoming model/object modifications are additive in order to
avoid having to bump the major version of the API. Therefore,
this changeset removes 4 unused DB fields that were exposed in their
associated Watcher objects (i.e. AuditTemplate and Audit).

Change-Id: Ifb0783f21cd66db16b31e3c8e376fc9d6c07dea3
Partially-Implements: blueprint watcher-versioned-objects
2016-11-07 10:39:19 +01:00
Vincent Françoise
750e6bf213 Eager loading on One-to-X foreign keys
In this changeset, I added ORM relationships to the DB models
concerning the already-declared foreign keys.
I also modified the DB query building to now handle a new 'eager'
parameter that, if True, is responsible to also fetch the data
relative to these 'parent' DB entities (no cascading).

Change-Id: Ieea181af9a4b173c54621dcc6c549161f5a35aeb
Partially-Implements: blueprint watcher-versioned-objects
2016-11-07 10:37:14 +01:00
Alexander Chadin
48cc6b2718 Add Audit Scope Handler
This patch set adds audit scope mechanism.
It also removes host_aggregate field.

Change-Id: Ia98ed180a93fc8c19599735e2b41471d322bae9a
Partially-Implements: blueprint define-the-audit-scope
2016-10-13 10:50:58 +03:00
Alexander Chadin
e7a1e148ca Add service supervisor
This patch set adds supervisor mechanism for Watcher services
to get ability to track states.

Partially-Implements: blueprint watcher-service-list
Change-Id: Iab1cefb971c79ed27b22b6a5d1bed8698e35f9a4
2016-10-12 15:52:06 +03:00
Vincent Françoise
10cbcd2432 Added composite unique name constraints
In this changeset, I added composite contraints on models that
were currently subject to an implicit uniqueness.
This composite constraint takes as an input the name and the deleted
columns.
The 'deleted' column is 0 by default and is updated with the value of
the associated 'id' column upon soft deletion.
What this means is that this composite constraint guarantees the fact
that we can hold many soft deleted records holding the exact same name.
Indeed, the value of the deleted column will be different whilst
forbidding 2 non-soft deleted records with the same name as their
associated 'deleted' value will be 0 and subsequently violates the
constraint.

Change-Id: I3ee39e23aa4ca9bc6b4ea9af8c6b7e6d67af0136
2016-10-06 16:59:41 +02:00
Vincent Françoise
6be758bc5a Added strategy ID + Action Plan syncing
In this changeset, I implemented the logic which cancels
any audit or action plan whose goal has been re-synced
(upon restarting the Decision Engine).

Partially Implements: blueprint efficacy-indicator

Change-Id: I95d2739eb552d4a7a02c822b11844591008f648e
2016-08-22 10:08:05 +02:00
Jenkins
553c5a6c4b Merge "Add scoring engines to database and API layers" 2016-08-02 08:09:39 +00:00
Tomasz Kaczynski
26d84e353e Add scoring engines to database and API layers
A Scoring Module needs to expose a list of available
scoring engines through API and Watcher CLI. This list
is stored in database and synchronized by Decision Engine.

Partially-Implements: blueprint scoring-module
Change-Id: I32168adeaf34fd12a731204c5b58fe68434ad087
APIImpact
2016-08-01 12:40:33 +00:00
Michael Gugino
52ffc2c8e2 Implement goal_id, strategy_id and host_aggregate into Audit api
Modifying the api controller for audit objects to allow
creation of audit objects by specifying either an
audit_template uuid/id and/or a goal_id.

strategy_id is optional.

Partially Implements: blueprint persistent-audit-parameters
Change-Id: I7b3eae4d0752a11208f5f92ee13ab1018d8521ad
2016-07-28 20:12:52 -04:00
Drew Thorstensen
7d704dbeec Add hacking checks to watcher
The hacking checks enforce during the pep8 run functional validations of
the code to ensure deeper filters and code consistency.  This change set
adds the hacking checks to the wathcer project.  These checks were
seeded from the neutron project, which had a good set of base defaults.

This change set also updates the watcher project to be compliant with
these new hacking checks.

Change-Id: I6f4566d384a7400bddf228aa127a53e6ecc82c2e
2016-07-25 07:52:45 -04:00
Muzammil Mueen
fd7c41fba2 Remove unused columns parameters in watcher/db/api
In watcher/db/api.py, some abstract methods are specifying a 'columns'
parameter that is actually ignored in db/sqlalchemy/api.py. Since we
do not need this parameter, realignment was done for the signatures
of these methods, by removing the 'column' parameter (and its
docstring) from every single one of the following methods.

get_audit_template_list
get_audit_list
get_action_list
get_action_plan_list

Closes-Bug: #1597641
Change-Id: If706e24d5714f0139fd135bdc41d17d0e431e302
2016-07-19 09:51:41 -07:00
Alexander Chadin
1de00086f5 Add continuously optimization
This patch set adds implementation for CONTINUOUS type
of audit.

Change-Id: I5f4ec97b2082c8a6b3ccebe36b2a343fa4a67d19
Implements: blueprint continuously-optimization
2016-07-07 19:17:17 +03:00
digambar
47ff9bb105 Fix field type to audit_type
There's a number of places where a field of an object is named "type",
which isn't good thing. Actually type is keyword in python, so wherever
I found type used in models/api/object/test, I have fixed it to
audit_type in the audit files

Change-Id: Iea6bd3acb0b2af2a833b3916701aad88f6064bba
Closes-Bug: #1533392
2016-07-06 06:26:15 +05:30
Jenkins
64903ce56c Merge "Enable strategy parameters" 2016-07-04 08:57:54 +00:00
Edwin Zhai
0b29a8394b Enable strategy parameters
Strategy provides parameters to customize algorithm behavior. End user
could query then specify parameters for their requirements.

Change-Id: Id097db5f6e79c94b57674c8e5d55b06098abf18c
Implements-bp: optimization-threshold
2016-07-01 03:01:41 +00:00
Vincent Françoise
77c07a466f Added filter operators
In this changeset, I refactored the DB filter system to support
comparison operators using a django-like syntax.
A filter can take 2 forms:

- "<FIELDNAME>" which is a syntactic sugar for "<FIELDNAME>__eq"
- "<FIELDNAME>__<OPERATOR>" where <OPERATOR> is the comparison operator
  to be used.

Here is the list of the supported operators:

- 'eq' (==)
- 'neq' (!=)
- 'gt' (>)
- 'gte' (>=)
- 'lt' (<)
- 'lte' (<=)
- 'in' (in)
- 'notin' (not in)

Change-Id: I53a61d50a3253342a40f0ff87cb5612ed57a3bd1
2016-06-17 10:37:44 +02:00
Vincent Françoise
f665d83657 Added efficacy specification to /goals
In this changeset, I added the new Efficacy, EfficacySpecification
and IndicatorSpecification classes which are the main components for
computing an efficacy.

Partially Implements: blueprint efficacy-indicator

Change-Id: I3a1d62569de2dd6bb6f9a52f6058313fa2b886ce
2016-06-10 09:36:59 +02:00
Vincent Françoise
eab47bf182 Added EfficacyIndicator model in DB
In this changeset, I created a new model named EfficacyIndicator
which is responsible for storing information regarding the efficacy
indicators that were computed by the strategy in its solution. Every
efficacy indicator should relate to a single Action Plan.

Partially Implements: blueprint efficacy-indicator

Change-Id: Ifc14ea5e16e92f032d7912c9b3fdc270af79cab6
2016-06-08 14:05:01 +02:00
Jenkins
d91d72d2c2 Merge "Add goal name as filter for strategy list cmd" 2016-05-27 15:51:22 +00:00
Jenkins
083bc2bed4 Merge "Add goal_name & strategy_name in /audit_templates" 2016-05-27 15:51:14 +00:00
David TARDIVEL
6eee64502f Add goal name as filter for strategy list cmd
This changeset add the possibility to use the goal name as a
stragegy list filter.

Change-Id: Ibaf45e694f115308f19e9bcd3023fe2e6d1750cd
2016-05-27 11:20:55 +02:00
Vincent Françoise
8c6bf734af Add goal_name & strategy_name in /audit_templates
In this changeset, I added both the 'goal_name' and the 'strategy_name'
field.

Change-Id: Ic164df84d4e23ec75b2b2f4b358cf827d0ad7fa5
Related-Bug: #1573582
2016-05-24 11:09:48 +02:00