diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst index d438522b1..479599c93 100644 --- a/doc/source/architecture.rst +++ b/doc/source/architecture.rst @@ -21,7 +21,7 @@ Overview Below you will find a diagram, showing the main components of Watcher: .. image:: ./images/architecture.svg - :width: 100% + :width: 110% .. _components_definition: @@ -37,13 +37,12 @@ AMQP Bus The AMQP message bus handles internal asynchronous communications between the different Watcher components. -.. _cluster_history_db_definition: +.. _cluster_datasource_definition: -Cluster History Database ------------------------- +Datasource +---------- -This component stores the data related to the -:ref:`Cluster History `. +This component stores the metrics related to the cluster. It can potentially rely on any appropriate storage system (InfluxDB, OpenTSDB, MongoDB,...) but will probably be more performant when using @@ -51,14 +50,6 @@ MongoDB,...) but will probably be more performant when using which are optimized for handling time series data, which are arrays of numbers indexed by time (a datetime or a datetime range). -.. _cluster_model_db_definition: - -Cluster Model Database ------------------------- - -This component stores the data related to the -:ref:`Cluster Data Model `. - .. _archi_watcher_api_definition: Watcher API @@ -193,8 +184,8 @@ data: :ref:`Managed resources ` (e.g., the data stored in the Nova database). These models gives a strategy the ability to reason on the current state of a given :ref:`cluster `. -- The data stored in the :ref:`Cluster History Database - ` which provides information about the past of +- The data stored in the :ref:`Cluster Datasource + ` which provides information about the past of the :ref:`Cluster `. Here below is a sequence diagram showing how the Decision Engine builds and @@ -452,6 +443,10 @@ state may be one of the following: - **CANCELLED** : the :ref:`Action Plan ` was in **RECOMMENDED**, **PENDING** or **ONGOING** state and was cancelled by the :ref:`Administrator ` +- **SUPERSEDED** : the :ref:`Action Plan ` was in + RECOMMENDED state and was automatically superseded by Watcher, due to an + expiration delay or an update of the + :ref:`Cluster data model ` The following diagram shows the different possible states of an diff --git a/doc/source/dev/plugin/strategy-plugin.rst b/doc/source/dev/plugin/strategy-plugin.rst index 3052954a8..b285dfff0 100644 --- a/doc/source/dev/plugin/strategy-plugin.rst +++ b/doc/source/dev/plugin/strategy-plugin.rst @@ -271,57 +271,44 @@ requires new metrics not covered by Ceilometer, you can add them through a .. _`Ceilometer plugin`: http://docs.openstack.org/developer/ceilometer/plugins.html .. _`Ceilosca`: https://github.com/openstack/monasca-ceilometer/blob/master/ceilosca/ceilometer/storage/impl_monasca.py +Read usage metrics using the Watcher Datasource Helper +------------------------------------------------------ -Read usage metrics using the Python binding -------------------------------------------- - -You can find the information about the Ceilometer Python binding on the -OpenStack `ceilometer client python API documentation -`_ - -To facilitate the process, Watcher provides the ``osc`` attribute to every -strategy which includes clients to major OpenStack services, including -Ceilometer. So to access it within your strategy, you can do the following: +The following code snippet shows how to invoke a Datasource Helper class: .. code-block:: py - # Within your strategy "execute()" - cclient = self.osc.ceilometer - # TODO: Do something here + from watcher.datasource import ceilometer as ceil + from watcher.datasource import monasca as mon + + @property + def ceilometer(self): + if self._ceilometer is None: + self._ceilometer = ceil.CeilometerHelper(osc=self.osc) + return self._ceilometer + + @property + def monasca(self): + if self._monasca is None: + self._monasca = mon.MonascaHelper(osc=self.osc) + return self._monasca Using that you can now query the values for that specific metric: .. code-block:: py - query = None # e.g. [{'field': 'foo', 'op': 'le', 'value': 34},] - value_cpu = cclient.samples.list( - meter_name='cpu_util', - limit=10, q=query) - - -Read usage metrics using the Watcher Cluster History Helper ------------------------------------------------------------ - -Here below is the abstract ``BaseClusterHistory`` class of the Helper. - -.. autoclass:: watcher.decision_engine.cluster.history.base.BaseClusterHistory - :members: - :noindex: - -The following code snippet shows how to create a Cluster History class: - -.. code-block:: py - - from watcher.decision_engine.cluster.history import ceilometer as ceil - - query_history = ceil.CeilometerClusterHistory() - -Using that you can now query the values for that specific metric: - -.. code-block:: py - - query_history.statistic_aggregation(resource_id=compute_node.uuid, - meter_name='compute.node.cpu.percent', - period="7200", - aggregate='avg' - ) + if self.config.datasource == "ceilometer": + resource_id = "%s_%s" % (node.uuid, node.hostname) + return self.ceilometer.statistic_aggregation( + resource_id=resource_id, + meter_name='compute.node.cpu.percent', + period="7200", + aggregate='avg', + ) + elif self.config.datasource == "monasca": + statistics = self.monasca.statistic_aggregation( + meter_name='compute.node.cpu.percent', + dimensions=dict(hostname=node.uuid), + period=7200, + aggregate='avg' + ) diff --git a/doc/source/glossary.rst b/doc/source/glossary.rst index 645073b3f..7ab060202 100644 --- a/doc/source/glossary.rst +++ b/doc/source/glossary.rst @@ -101,12 +101,6 @@ Cluster Data Model (CDM) .. watcher-term:: watcher.decision_engine.model.collector.base -.. _cluster_history_definition: - -Cluster History -=============== - -.. watcher-term:: watcher.decision_engine.cluster.history.base .. _controller_node_definition: diff --git a/doc/source/image_src/plantuml/action_plan_state_machine.png b/doc/source/image_src/plantuml/action_plan_state_machine.png new file mode 100644 index 000000000..69c2ad78b Binary files /dev/null and b/doc/source/image_src/plantuml/action_plan_state_machine.png differ diff --git a/doc/source/image_src/plantuml/action_plan_state_machine.txt b/doc/source/image_src/plantuml/action_plan_state_machine.txt index 93a54af35..0eab6d0f1 100644 --- a/doc/source/image_src/plantuml/action_plan_state_machine.txt +++ b/doc/source/image_src/plantuml/action_plan_state_machine.txt @@ -9,8 +9,10 @@ FAILED --> DELETED : Administrator removes\nAction Plan SUCCEEDED --> DELETED : Administrator removes\nAction Plan ONGOING --> CANCELLED : Administrator cancels\nAction Plan RECOMMENDED --> CANCELLED : Administrator cancels\nAction Plan +RECOMMENDED --> SUPERSEDED : The Watcher Decision Engine supersedes\nAction Plan PENDING --> CANCELLED : Administrator cancels\nAction Plan CANCELLED --> DELETED +SUPERSEDED --> DELETED DELETED --> [*] @enduml diff --git a/doc/source/image_src/plantuml/watcher_db_schema_diagram.png b/doc/source/image_src/plantuml/watcher_db_schema_diagram.png new file mode 100644 index 000000000..513098f98 Binary files /dev/null and b/doc/source/image_src/plantuml/watcher_db_schema_diagram.png differ diff --git a/doc/source/image_src/plantuml/watcher_db_schema_diagram.txt b/doc/source/image_src/plantuml/watcher_db_schema_diagram.txt index b827a30d4..8b56c4a9c 100644 --- a/doc/source/image_src/plantuml/watcher_db_schema_diagram.txt +++ b/doc/source/image_src/plantuml/watcher_db_schema_diagram.txt @@ -60,6 +60,7 @@ table(audits) { interval : Integer, nullable parameters : JSONEncodedDict, nullable scope : JSONEncodedList, nullable + auto_trigger: Boolean created_at : DateTime updated_at : DateTime @@ -73,7 +74,6 @@ table(action_plans) { foreign_key("audit_id : Integer, nullable") foreign_key("strategy_id : Integer") uuid : String[36] - first_action_id : Integer state : String[20], nullable global_efficacy : JSONEncodedDict, nullable @@ -91,7 +91,7 @@ table(actions) { action_type : String[255] input_parameters : JSONEncodedDict, nullable state : String[20], nullable - next : String[36], nullable + parents : JSONEncodedList, nullable created_at : DateTime updated_at : DateTime diff --git a/doc/source/images/architecture.svg b/doc/source/images/architecture.svg index abd3872bf..a70d0a809 100644 --- a/doc/source/images/architecture.svg +++ b/doc/source/images/architecture.svg @@ -1,302 +1,1365 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - AMQP - bus - - - - - - - - - - - - - - - - - - - Watcher - Decision Engine - - - - - - - - - - - - - - - - - - - - - - - - - - Watcher - Actions Applier - - - - - - - - - - - - - - - - - - - Watcher - API - - - - - - - - - - - - - - - - - - - Watcher - CLI - - - - - - - Horizon - - - - - - - - - - - - - - - - - - - Watcher - plugin - - - - - - - Nova - - - - - - - MariaDB Database - - - - - - - - - - - - - - - - - - - Ceilometer API - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Strategy - - - - - - - - - - - - - - - - - - - - - - - - Planner - - - - - - - - - - - - - - - - - - - - - - - Watcher DB - - - - - - - - Cluster Model DB - - - - - - - - Cluster History DB - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + + watcher decision engine + + + + + + watcherdb + + + + + + message bus + + watcher applier + + + nova + + + + glance + + + ceilometer + + + monasca + + + + + + datasourcedrivers + + + + + + + modeldrivers + + + + + + actiondrivers + + + + + + plannerdrivers + + + + + + strategydrivers + + + + goaldrivers + + + + + + + + + + + + + watcher api + + watcherdashboard + + watcher cli + + + + + + + + + + + scoring enginedrivers + + + + + + + + + API call + + + RPC cast + + + + notification + + + + + + + extensions + + + + + + + + + + + + + + + workflowdrivers + + + + + + + + + diff --git a/watcher/decision_engine/planner/weight.py b/watcher/decision_engine/planner/weight.py index f4b579bd8..9759a6f72 100644 --- a/watcher/decision_engine/planner/weight.py +++ b/watcher/decision_engine/planner/weight.py @@ -41,7 +41,7 @@ class WeightPlanner(base.BasePlanner): *Limitations* - This planner requires to have action_weights and parallelization configs - tuned well. + tuned well. """ def __init__(self, config):