Merge "Add a checker for the documentation"

doc/source/architecture.rst

@@ -68,8 +68,9 @@ You can also read the detailed description of `Watcher API`_.
 Watcher Applier
 ---------------
 
-This component is in charge of executing the :ref:`Action Plan <action_plan_definition>`
+This component is in charge of executing the
-built by the :ref:`Watcher Decision Engine <watcher_decision_engine_definition>`.
+:ref:`Action Plan <action_plan_definition>` built by the
+:ref:`Watcher Decision Engine <watcher_decision_engine_definition>`.
 
 It connects to the :ref:`message bus <amqp_bus_definition>` and launches the
 :ref:`Action Plan <action_plan_definition>` whenever a triggering message is
@@ -93,9 +94,11 @@ handling a given :ref:`Action <action_definition>` from the list, a
 notification message is sent on the :ref:`message bus <amqp_bus_definition>`
 indicating that the state of the action has changed to **ONGOING**.
 
-If the :ref:`Action <action_definition>` is successful, the :ref:`Watcher Applier <watcher_applier_definition>`
+If the :ref:`Action <action_definition>` is successful,
-sends a notification message on :ref:`the bus <amqp_bus_definition>` informing
+the :ref:`Watcher Applier <watcher_applier_definition>` sends a notification
-the other components of this.
+message on :ref:`the bus <amqp_bus_definition>` informing the other components
+of this.
+
 
 If the :ref:`Action <action_definition>` fails, the
 :ref:`Watcher Applier <watcher_applier_definition>` tries to rollback to the
@@ -136,8 +139,8 @@ Watcher Decision Engine
 -----------------------
 
 This component is responsible for computing a set of potential optimization
-:ref:`Actions <action_definition>` in order to fulfill the :ref:`Goal <goal_definition>`
+:ref:`Actions <action_definition>` in order to fulfill
-of an :ref:`Audit <audit_definition>`.
+the :ref:`Goal <goal_definition>` of an :ref:`Audit <audit_definition>`.
 
 It first reads the parameters of the :ref:`Audit <audit_definition>` from the
 associated :ref:`Audit Template <audit_template_definition>` and knows the
@@ -159,10 +162,13 @@ These :ref:`Actions <action_definition>` are scheduled in time by the
 In order to compute the potential :ref:`Solution <solution_definition>` for the
 Audit, the :ref:`Strategy <strategy_definition>` relies on two sets of data:
 
--   the current state of the :ref:`Managed resources <managed_resource_definition>`
+-   the current state of the
-    (e.g., the data stored in the Nova database)
+:ref:`Managed resources <managed_resource_definition>`
--   the data stored in the :ref:`Cluster History Database <cluster_history_db_definition>`
+(e.g., the data stored in the Nova database)
-    which provides information about the past of the :ref:`Cluster <cluster_definition>`
+-   the data stored in the
+:ref:`Cluster History Database <cluster_history_db_definition>`
+    which provides information about the past of the
+    :ref:`Cluster <cluster_definition>`
 
 So far, only one :ref:`Strategy <strategy_definition>` can be associated to a
 given :ref:`Goal <goal_definition>` via the main Watcher configuration file.

doc/source/cmds/watcher-db-manage.rst

@@ -54,7 +54,8 @@ Usage
 =====
 
 Options for the various :ref:`commands <db-manage_cmds>` for
-:command:`watcher-db-manage` are listed when the :option:`-h` or :option:`--help`
+:command:`watcher-db-manage` are listed when the :option:`-h` or
+:option:`--help`
 option is used after the command.
 
 For example::
@@ -81,8 +82,9 @@ If no configuration file is specified with the :option:`--config-file` option,
 Command Options
 ===============
 
-:command:`watcher-db-manage` is given a command that tells the utility what actions
+:command:`watcher-db-manage` is given a command that tells the utility
-to perform. These commands can take arguments. Several commands are available:
+what actions to perform.
+These commands can take arguments. Several commands are available:
 
 .. _create_schema:
 

doc/source/dev/glossary.rst

@@ -71,23 +71,27 @@ An :ref:`Action Plan <action_plan_definition>` is a flow of
 a given :ref:`Goal <goal_definition>`.
 
 An :ref:`Action Plan <action_plan_definition>` is generated by Watcher when an
-:ref:`Audit <audit_definition>` is successful which implies that the :ref:`Strategy <strategy_definition>`
+:ref:`Audit <audit_definition>` is successful which implies that the
+:ref:`Strategy <strategy_definition>`
 which was used has found a :ref:`Solution <solution_definition>` to achieve the
 :ref:`Goal <goal_definition>` of this :ref:`Audit <audit_definition>`.
 
-In the default implementation of Watcher, an :ref:`Action Plan <action_plan_definition>`
+In the default implementation of Watcher, an
+:ref:`Action Plan <action_plan_definition>`
 is only composed of successive :ref:`Actions <action_definition>`
 (i.e., a Workflow of :ref:`Actions <action_definition>` belonging to a unique
 branch).
 
 However, Watcher provides abstract interfaces for many of its components,
-allowing other implementations to generate and handle more complex :ref:`Action Plan(s) <action_plan_definition>`
+allowing other implementations to generate and handle more complex
+:ref:`Action Plan(s) <action_plan_definition>`
 composed of two types of Action Item(s):
 
 -  simple :ref:`Actions <action_definition>`: atomic tasks, which means it
    can not be split into smaller tasks or commands from an OpenStack point of
    view.
--  composite Actions: which are composed of several simple :ref:`Actions <action_definition>`
+-  composite Actions: which are composed of several simple
+   :ref:`Actions <action_definition>`
    ordered in sequential and/or parallel flows.
 
 An :ref:`Action Plan <action_plan_definition>` may be described using
@@ -134,7 +138,8 @@ is a role for users which allows them to run any Watcher commands, such as:
 -  Launch an :ref:`Audit <audit_definition>`
 -  Get the :ref:`Action Plan <action_plan_definition>`
 -  Launch a recommended :ref:`Action Plan <action_plan_definition>` manually
--  Archive previous :ref:`Audits <audit_definition>` and :ref:`Action Plans <action_plan_definition>`
+-  Archive previous :ref:`Audits <audit_definition>` and
+   :ref:`Action Plans <action_plan_definition>`
 
 
 The :ref:`Administrator <administrator_definition>` is also allowed to modify
@@ -163,7 +168,8 @@ be one of the following:
    event handling mechanism) and is in the queue for being processed by the
    :ref:`Watcher Decision Engine <watcher_decision_engine_definition>`
 -  **ONGOING** : the :ref:`Audit <audit_definition>` is currently being
-   processed by the :ref:`Watcher Decision Engine <watcher_decision_engine_definition>`
+   processed by the
+   :ref:`Watcher Decision Engine <watcher_decision_engine_definition>`
 -  **SUCCEEDED** : the :ref:`Audit <audit_definition>` has been executed
    successfully (note that it may not necessarily produce a
    :ref:`Solution <solution_definition>`).
@@ -191,7 +197,8 @@ An :ref:`Audit Template <audit_template_definition>` contains at least the
 
 It may also contain some error handling settings indicating whether:
 
--  :ref:`Watcher Applier <watcher_applier_definition>` stops the entire operation
+-  :ref:`Watcher Applier <watcher_applier_definition>` stops the
+   entire operation
 -  :ref:`Watcher Applier <watcher_applier_definition>` performs a rollback
 
 and how many retries should be attempted before failure occurs (also the latter
@@ -241,10 +248,12 @@ Cluster Data Model
 ==================
 
 A :ref:`Cluster Data Model <cluster_data_model_definition>` is a logical
-representation of the current state and topology of the :ref:`Cluster <cluster_definition>`
+representation of the current state and topology of the
+:ref:`Cluster <cluster_definition>`
 :ref:`Managed resources <managed_resource_definition>`.
 
-It is represented as a set of :ref:`Managed resources <managed_resource_definition>`
+It is represented as a set of
+:ref:`Managed resources <managed_resource_definition>`
 (which may be a simple tree or a flat list of key-value pairs)
 which enables Watcher :ref:`Strategies <strategy_definition>` to know the
 current relationships between the different
@@ -253,7 +262,8 @@ current relationships between the different
 and enables the :ref:`Strategy <strategy_definition>` to request information
 such as:
 
--  What compute nodes are in a given :ref:`Availability Zone <availability_zone_definition>`
+-  What compute nodes are in a given
+:ref:`Availability Zone <availability_zone_definition>`
    or a given :ref:`Host Aggregate <host_aggregates_definition>` ?
 -  What :ref:`Instances <instance_definition>` are hosted on a given compute
    node ?
@@ -263,16 +273,19 @@ such as:
 -  What is the available bandwidth on a given network link ?
 -  What is the current space available on a given virtual disk of a given
    :ref:`Instance <instance_definition>` ?
--  What is the current state of a given :ref:`Instance <instance_definition>` ?
+-  What is the current state of a given :ref:`Instance <instance_definition>`?
 -  ...
 
 In a word, this data model enables the :ref:`Strategy <strategy_definition>`
 to know:
 
 -  the current topology of the :ref:`Cluster <cluster_definition>`
--  the current capacity for each :ref:`Managed resource <managed_resource_definition>`
+-  the current capacity for each
--  the current amount of used/free space for each :ref:`Managed resource <managed_resource_definition>`
+   :ref:`Managed resource <managed_resource_definition>`
--  the current state of each :ref:`Managed resources <managed_resource_definition>`
+-  the current amount of used/free space for each
+   :ref:`Managed resource <managed_resource_definition>`
+-  the current state of each
+   :ref:`Managed resources <managed_resource_definition>`
 
 In the Watcher project, we aim at providing a generic and very basic
 :ref:`Cluster Data Model <cluster_data_model_definition>` for each
@@ -295,7 +308,8 @@ to:
    :ref:`Cluster Data Model <cluster_data_model_definition>`
    (the proposed data model acts as a pivot data model)
 
-There may be various :ref:`generic and basic Cluster Data Models <cluster_data_model_definition>`
+There may be various
+:ref:`generic and basic Cluster Data Models <cluster_data_model_definition>`
 proposed in Watcher helpers, each of them being adapted to achieving a given
 :ref:`Goal <goal_definition>`:
 
@@ -311,8 +325,10 @@ proposed in Watcher helpers, each of them being adapted to achieving a given
 
 Note however that a developer can use his/her own
 :ref:`Cluster Data Model <cluster_data_model_definition>` if the proposed data
-model does not fit his/her needs as long as the :ref:`Strategy <strategy_definition>`
+model does not fit his/her needs as long as the
-is able to produce a :ref:`Solution <solution_definition>` for the requested :ref:`Goal <goal_definition>`.
+:ref:`Strategy <strategy_definition>` is able to produce a
+:ref:`Solution <solution_definition>` for the requested
+:ref:`Goal <goal_definition>`.
 For example, a developer could rely on the Nova Data Model to optimize some
 compute resources.
 
@@ -335,16 +351,21 @@ history may be used by any :ref:`Strategy <strategy_definition>` in order to
 find the most optimal :ref:`Solution <solution_definition>` during an
 :ref:`Audit <audit_definition>`.
 
-In the Watcher project, a generic :ref:`Cluster History <cluster_history_definition>`
+In the Watcher project, a generic
+:ref:`Cluster History <cluster_history_definition>`
 API is proposed with some helper classes in order to :
 
 -  share a common measurement (events or metrics) naming based on what is
-   defined in Ceilometer. See `the full list of available measurements <http://docs.openstack.org/admin-guide-cloud/telemetry-measurements.html>`_
+   defined in Ceilometer.
+   See `the full list of available measurements <http://docs.openstack.org/admin-guide-cloud/telemetry-measurements.html>`_
 -  share common meter types (Cumulative, Delta, Gauge) based on what is
-   defined in Ceilometer. See `the full list of meter types <http://docs.openstack.org/admin-guide-cloud/telemetry-measurements.html>`_
+   defined in Ceilometer.
+   See `the full list of meter types <http://docs.openstack.org/admin-guide-cloud/telemetry-measurements.html>`_
 -  simplify the development of a new :ref:`Strategy <strategy_definition>`
--  avoid duplicating the same code in several :ref:`Strategies <strategy_definition>`
+-  avoid duplicating the same code in several
--  have a better consistency between the different :ref:`Strategies <strategy_definition>`
+:ref:`Strategies <strategy_definition>`
+-  have a better consistency between the different
+:ref:`Strategies <strategy_definition>`
 -  avoid any strong coupling with any external metrics/events storage system
    (the proposed API and measurement naming system acts as a pivot format)
 
@@ -369,8 +390,8 @@ services:
 -  Cinder scheduler: for volumes management
 -  Glance controller: for image management
 -  Neutron controller: for network management
--  Nova controller: for global compute resources management with services such as
+-  Nova controller: for global compute resources management with services
-   nova-scheduler, nova-conductor and nova-network
+   such as nova-scheduler, nova-conductor and nova-network.
 
 In many configurations, Watcher will reside on a controller node even if it
 can potentially be hosted on a dedicated machine.
@@ -388,7 +409,8 @@ Customer
 ========
 
 A :ref:`Customer <customer_definition>` is the person or company which
-subscribes to the cloud provider offering. A customer may have several :ref:`Project(s) <project_definition>`
+subscribes to the cloud provider offering. A customer may have several
+:ref:`Project(s) <project_definition>`
 hosted on the same :ref:`Cluster <cluster_definition>` or dispatched on
 different clusters.
 
@@ -484,11 +506,12 @@ measure of how much of the :ref:`Goal <goal_definition>` has been achieved in
 respect with constraints and :ref:`SLAs <sla_definition>` defined by the
 :ref:`Customer <customer_definition>`.
 
-The way efficiency is evaluated will depend on the :ref:`Goal <goal_definition>`
+The way efficiency is evaluated will depend on the
-to achieve.
+:ref:`Goal <goal_definition>` to achieve.
 
-Of course, the efficiency will be relevant only as long as the :ref:`Action Plan <action_plan_definition>`
+Of course, the efficiency will be relevant only as long as the
-is relevant (i.e., the current state of the :ref:`Cluster <cluster_definition>`
+:ref:`Action Plan <action_plan_definition>` is relevant
+(i.e., the current state of the :ref:`Cluster <cluster_definition>`
 has not changed in a way that a new :ref:`Audit <audit_definition>` would need
 to be launched).
 
@@ -551,21 +574,21 @@ which provides a good definition.
 SLA violation
 =============
 
-A :ref:`SLA violation <sla_violation_definition>` happens when a :ref:`SLA <sla_definition>`
+A :ref:`SLA violation <sla_violation_definition>` happens when a
-defined with a given :ref:`Customer <customer_definition>` could not be
+:ref:`SLA <sla_definition>` defined with a given
-respected by the cloud provider within the timeframe defined by the official
+:ref:`Customer <customer_definition>` could not be respected by the
-contract document.
+cloud provider within the timeframe defined by the official contract document.
 
 .. _slo_definition:
 
 SLO
 ===
 
-A Service Level Objective (SLO) is a key element of a :ref:`SLA <sla_definition>`
+A Service Level Objective (SLO) is a key element of a
-between a service provider and a :ref:`Customer <customer_definition>`. SLOs
+:ref:`SLA <sla_definition>` between a service provider and a
-are agreed as a means of measuring the performance of the Service Provider and
+:ref:`Customer <customer_definition>`. SLOs are agreed as a means of measuring
-are outlined as a way of avoiding disputes between the two parties based on
+the performance of the Service Provider and are outlined as a way of avoiding
-misunderstanding.
+disputes between the two parties based on misunderstanding.
 
 You can also read `the Wikipedia page for SLO <https://en.wikipedia.org/wiki/Service_level_objective>`_
 which provides a good definition.
@@ -575,9 +598,10 @@ which provides a good definition.
 Solution
 ========
 
-A :ref:`Solution <solution_definition>` is a set of :ref:`Actions <action_definition>`
+A :ref:`Solution <solution_definition>` is a set of
-generated by a :ref:`Strategy <strategy_definition>` (i.e., an algorithm) in
+:ref:`Actions <action_definition>` generated by a
-order to achieve the :ref:`Goal <goal_definition>` of an :ref:`Audit <audit_definition>`.
+:ref:`Strategy <strategy_definition>` (i.e., an algorithm) in order to achieve
+the :ref:`Goal <goal_definition>` of an :ref:`Audit <audit_definition>`.
 
 A :ref:`Solution <solution_definition>` is different from an
 :ref:`Action Plan <action_plan_definition>` because it contains the
@@ -593,8 +617,8 @@ applied.
 
 Two approaches to dealing with this can be envisaged:
 
--  **fully automated mode**: only the :ref:`Solution <solution_definition>` with
+-  **fully automated mode**: only the :ref:`Solution <solution_definition>`
-   the highest ranking (i.e., the highest
+ with the highest ranking (i.e., the highest
    :ref:`Optimization Efficiency <efficiency_definition>`)
    will be sent to the :ref:`Watcher Planner <watcher_planner_definition>` and
    translated into concrete :ref:`Actions <action_definition>`.
@@ -610,11 +634,13 @@ Strategy
 ========
 
 A :ref:`Strategy <strategy_definition>` is an algorithm implementation which is
-able to find a :ref:`Solution <solution_definition>` for a given :ref:`Goal <goal_definition>`.
+able to find a :ref:`Solution <solution_definition>` for a given
+:ref:`Goal <goal_definition>`.
 
 There may be several potential strategies which are able to achieve the same
 :ref:`Goal <goal_definition>`. This is why it is possible to configure which
-specific :ref:`Strategy <strategy_definition>` should be used for each :ref:`Goal <goal_definition>`.
+specific :ref:`Strategy <strategy_definition>` should be used for each
+:ref:`Goal <goal_definition>`.
 
 Some strategies may provide better optimization results but may take more time
 to find an optimal :ref:`Solution <solution_definition>`.
@@ -628,8 +654,9 @@ provided as well.
 Watcher Applier
 ===============
 
-This component is in charge of executing the :ref:`Action Plan <action_plan_definition>`
+This component is in charge of executing the
-built by the :ref:`Watcher Decision Engine <watcher_decision_engine_definition>`.
+:ref:`Action Plan <action_plan_definition>` built by the
+:ref:`Watcher Decision Engine <watcher_decision_engine_definition>`.
 
 See :doc:`architecture` for more details on this component.
 
@@ -658,8 +685,8 @@ Watcher Decision Engine
 =======================
 
 This component is responsible for computing a set of potential optimization
-:ref:`Actions <action_definition>` in order to fulfill the :ref:`Goal <goal_definition>`
+:ref:`Actions <action_definition>` in order to fulfill the
-of an :ref:`Audit <audit_definition>`.
+:ref:`Goal <goal_definition>` of an :ref:`Audit <audit_definition>`.
 
 It first reads the parameters of the :ref:`Audit <audit_definition>` from the
 associated :ref:`Audit Template <audit_template_definition>` and knows the

watcher/tests/test_doc_formatting.py

@@ -0,0 +1,79 @@
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+import os
+import re
+import testtools
+
+
+class TestDocFormatting(testtools.TestCase):
+    def _check_lines_wrapping(self, tpl, raw):
+        code_block = False
+        for i, line in enumerate(raw.split("\n"), start=1):
+            # NOTE(ndipanov): Allow code block lines to be longer than 79 ch
+            if code_block:
+                if not line or line.startswith(" "):
+                    continue
+                else:
+                    code_block = False
+            if "::" in line:
+                code_block = True
+            if "http://" in line or "https://" in line:
+                continue
+            # Allow lines which do not contain any whitespace
+            if re.match("\s*[^\s]+$", line):
+                continue
+            if code_block is False:
+                self.assertTrue(
+                    len(line) < 80,
+                    msg="%s:%d: Line limited to a maximum of 79 characters." %
+                        (tpl, i))
+
+    def _check_no_cr(self, tpl, raw):
+        cr = '\r'
+        matches = re.findall(cr, raw)
+        self.assertEqual(
+            len(matches), 0,
+            "Found %s literal carriage returns in file %s" %
+            (len(matches), tpl))
+
+    def _check_trailing_spaces(self, tpl, raw):
+        for i, line in enumerate(raw.split("\n"), start=1):
+            trailing_spaces = re.findall(" +$", line)
+            self.assertEqual(len(trailing_spaces), 0,
+                             "Found trailing spaces on line %s of %s" % (
+                                 i, tpl))
+
+    def _check_tab(self, tpl, raw):
+        tab = '\t'
+        matches = re.findall(tab, raw)
+        self.assertEqual(
+            len(matches), 0,
+            "Found %s tab in file %s" %
+            (len(matches), tpl))
+
+    def test_template(self):
+        doc_path = os.path.join("doc", 'source')
+        for root, dirs, files in os.walk(top=doc_path):
+            for file in files:
+                absolute_path_file = os.path.join(root, file)
+
+                if not os.path.isdir(absolute_path_file):
+                    if not absolute_path_file.endswith(".rst"):
+                        continue
+
+                    with open(absolute_path_file) as f:
+                        data = f.read()
+
+                    self._check_tab(absolute_path_file, data)
+                    self._check_lines_wrapping(absolute_path_file, data)
+                    self._check_no_cr(absolute_path_file, data)
+                    self._check_trailing_spaces(absolute_path_file, data)