initial version

Change-Id: I699e0ab082657880998d8618fe29eb7f56c6c661

watcher/decision_engine/README.md

@@ -0,0 +1,71 @@
+# Watcher Decision Engine
+
+This component is responsible for computing a list of potential optimization actions in order to fulfill the goals of an audit. 
+
+It uses the following input data :
+* current, previous and predicted state of the cluster (hosts, instances, network, ...)
+* evolution of metrics within a time frame
+
+It first selects the most appropriate optimization strategy depending on several factors :
+* the optimization goals that must be fulfilled (servers consolidation, energy consumption, license optimization, ...)
+* the deadline that was provided by the Openstack cluster admin for delivering an action plan
+* the "aggressivity" level regarding potential optimization actions :
+  * is it allowed to do a lot of instance migrations ?
+  * is it allowed to consume a lot of bandwidth on the admin network ?
+  * is it allowed to violate initial placement constraint such as affinity/anti-affinity, region, ... ? 
+
+The strategy is then executed and generates a list of Meta-Actions in order to fulfill the goals of the Audit. 
+
+A Meta-Action is a generic optimization task which is independent from the target cluster implementation (Openstack, ...). For example, an instance migration is a Meta-Action which corresponds, in the Openstack context, to a set of technical actions on the Nova, Cinder and Neutron components.
+
+Using Meta-Actions instead of technical actions brings two advantages in Watcher :
+* a loose coupling between the Watcher Decision Engine and the Watcher Applier
+* a simplification of the optimization algorithms which don't need to know the underlying technical cluster implementation
+
+Beyond that, the Meta-Actions which are computed by the optimization strategy are not necessarily ordered in time (it depends on the selected Strategy). Therefore, the Actions Planner module of Decision Engine reorganizes the list of Meta-Actions into an ordered sequence of technical actions (migrations, ...) such that all security, dependency, and performance requirements are met. An ordered sequence of technical actions is called an "Action Plan". 
+
+The Decision Engine saves the generated Action Plan in the Watcher Database. This Action Plan is loaded later by the Watcher Actions Applier.
+
+Like every Watcher component, the Decision Engine notifies its current status (learning phase, current status of each Audit, ...) on the message/notification bus. 
+
+## Watcher Compute Node Profiler
+
+This module of the Decision Engine is responsible for profiling a new compute node. When a new compute node is added to the cluster, it automatically triggers test scripts in order to extract profiling information such as :
+* the maximum I/O available on each disk
+* the evolution of energy consumption for a given workload 
+
+It stores those information in the Watcher database. They may be used by any optimization strategy that needs to rely on real metrics about a given physical machine and not only theoretical metrics.
+
+## Watcher Metrics Predictor
+
+This module of the Decision Engine is able to compute some predicted metric values according to previously acquired metrics.
+
+For instance, it may be able to predict the future CPU in the next 5 minutes for a given instance given the previous CPU load during the last 2 hours (relying on some neural network algorithm or any other machine learning system).
+
+This component pushes the new predicted metrics to the CEP in order to trigger new actions if needed.
+
+## Watcher Cluster State Collector
+
+This module of the Decision Engine provides a high level API for requesting status information from the InfluxDb database.
+
+A DSL will be provided in order to ease the development of new optimization strategies.
+
+Example of high level requests that may be provided :
+* get the difference between current cluster state and cluster state yesterday at the same time
+* get the state evolution in time of a group of instances from 9 AM to 10 AM for every day of the week
+* ... 
+
+## Watcher Resource Metrics Collector
+
+This module of the Decision Engine provides a high level API for requesting metrics information from the InfluxDb database.
+
+A DSL will be provided in order to ease the development of new optimization strategies.
+
+This component is distinct from the Cluster State Collector because it will probably have to deal with a much more important set of data and it may need a specific DSL for applying mathematical computes on metrics (min, max, average, ...).
+
+
+## Watcher Actions Planner
+
+This module of the Decision Engine translates Meta-Actions into technical actions on the Openstack modules (Nova, Cinder, ...) and builds an appropriate workflow which defines how-to schedule in time those different technical actions and for each action what are the pre-requisite conditions.
+
+Today, the Action Plan is just a simple chain of sequential actions but in later versions, we intend to rely on more complex workflow models description formats, such as [BPMN 2.0](http://www.bpmn.org/), which enable a complete definition of activity diagrams containing sequential and parallel tasks.

watcher/decision_engine/api/collector/cluster_state_collector.py

@@ -0,0 +1,22 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class ClusterStateCollector(object):
+    def get_latest_state_cluster(self):
+        raise NotImplementedError("Should have implemented this")
+    # todo(jed) think abouts needed interfaces
+    # todo(jed) stream incremental diff

watcher/decision_engine/api/collector/metrics_resource_collector.py

@@ -0,0 +1,35 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class MetricsResourceCollector(object):
+    def __init__(self):
+        pass
+
+    def get_average_usage_vm_cpu(self, uuid):
+        raise NotImplementedError("Should have implemented this")
+
+    def get_average_usage_vm_memory(self, uuid):
+        raise NotImplementedError("Should have implemented this")
+
+    def get_virtual_machine_capacity(self, uuid):
+        raise NotImplementedError("Should have implemented this")
+
+    def get_average_network_incomming(self, uuid):
+        raise NotImplementedError("Should have implemented this")
+
+    def get_average_network_outcomming(self, uuid):
+        raise NotImplementedError("Should have implemented this")

watcher/decision_engine/api/messaging/decision_engine_command.py

@@ -0,0 +1,21 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+
+class DecisionEngineCommand(object):
+    def execute(self):
+        raise NotImplementedError("Should have implemented this")

watcher/decision_engine/api/messaging/event_consumer.py

@@ -0,0 +1,27 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class EventConsumer(object):
+
+    def __init__(self):
+        self.messaging = None
+
+    def set_messaging(self, messaging):
+        self.messaging = messaging
+
+    def execute(self, request_id, context, data):
+        raise NotImplementedError('Not implemented ...')

watcher/decision_engine/api/planner/planner.py

@@ -0,0 +1,29 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class Planner(object):
+    def schedule(self, context, audit_uuid, solution):
+        """The  planner receives a solution to schedule
+
+        :param solution: the solution given by the strategy to
+        :param audit_uuid: the audit uuid
+        :return: ActionPlan ordered sequence of change requests
+        such that all security, dependency,
+        and performance requirements are met.
+        """
+        # example: directed acyclic graph
+        raise NotImplementedError("Should have implemented this")

watcher/decision_engine/api/selector/selector.py

@@ -0,0 +1,19 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class Selector(object):
+    pass

watcher/decision_engine/api/solution/solution.py

@@ -0,0 +1,35 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+
+class Solution(object):
+    def __init__(self):
+        self.modelOrigin = None
+        self.currentModel = None
+        self.efficiency = 0
+
+    def get_efficiency(self):
+        return self.efficiency
+
+    def set_efficiency(self, efficiency):
+        self.efficiency = efficiency
+
+    def set_model(self, current_model):
+        self.currentModel = current_model
+
+    def get_model(self):
+        return self.currentModel

watcher/decision_engine/api/solution/solution_comparator.py

@@ -0,0 +1,21 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+
+class Solution(object):
+    def compare(self, sol1, sol2):
+        raise NotImplementedError("Should have implemented this")

watcher/decision_engine/api/solution/solution_evaluator.py

@@ -0,0 +1,21 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+
+class SolutionEvaluator(object):
+    def evaluate(self, solution):
+        raise NotImplementedError("Should have implemented this")

watcher/decision_engine/api/strategy/meta_action.py

@@ -0,0 +1,38 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from watcher.decision_engine.api.strategy.strategy import StrategyLevel
+
+
+class MetaAction(object):
+    def __init__(self):
+        self.level = StrategyLevel.conservative
+        self.priority = 0
+
+    def get_level(self):
+        return self.level
+
+    def set_level(self, level):
+        self.level = level
+
+    def set_priority(self, priority):
+        self.priority = priority
+
+    def get_priority(self):
+        return self.priority
+
+    def __str__(self):
+        return "  "

watcher/decision_engine/api/strategy/strategy.py

@@ -0,0 +1,80 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import abc
+from oslo_log import log
+
+import six
+from watcher.decision_engine.api.strategy.strategy_level import StrategyLevel
+from watcher.decision_engine.framework.default_solution import DefaultSolution
+
+
+LOG = log.getLogger(__name__)
+
+# todo(jed) add interface
+
+
+@six.add_metaclass(abc.ABCMeta)
+class Strategy(object):
+    def __init__(self, name=None, description=None):
+        self.name = name
+        self.description = description
+        # default strategy level
+        self.strategy_level = StrategyLevel.conservative
+        self.metrics_collector = None
+        self.cluster_state_collector = None
+        # the solution given by the strategy
+        self.solution = DefaultSolution()
+
+    def get_solution(self):
+        return self.solution
+
+    def set_name(self, name):
+        self.name = name
+
+    def get_name(self):
+        return self.name
+
+    def get_strategy_strategy_level(self):
+        return self.strategy_level
+
+    def set_strategy_strategy_level(self, strategy_level):
+        """Convervative to Aggressive
+
+        the aims is to minimize le number of migrations
+        :param threshold:
+        """
+        self.strategy_level = strategy_level
+
+    @abc.abstractmethod
+    def execute(self, model):
+        """Execute a strategy
+
+        :param model:
+        :return:
+        """
+
+    def get_metrics_resource_collector(self):
+        return self.metrics_collector
+
+    def get_cluster_state_collector(self):
+        return self.cluster_state_collector
+
+    def set_metrics_resource_collector(self, metrics_collector):
+        self.metrics_collector = metrics_collector
+
+    def set_cluster_state_collector(self, cluster_state_collector):
+        self.cluster_state_collector = cluster_state_collector

watcher/decision_engine/api/strategy/strategy_context.py

@@ -0,0 +1,23 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class StrategyContext(object):
+    def __init__(self):
+        pass
+
+    def execute_strategy(self, model):
+        raise NotImplementedError("Should have implemented this")

watcher/decision_engine/api/strategy/strategy_level.py

@@ -0,0 +1,24 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from enum import Enum
+
+
+class StrategyLevel(Enum):
+    conservative = "conservative"
+    balanced = "balanced"
+    growth = "growth"
+    aggressive = "aggressive"

watcher/decision_engine/api/strategy/strategy_state.py

@@ -0,0 +1,25 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from enum import Enum
+
+
+class StrategyState(Enum):
+    INIT = 1,
+    READY = 2,
+    RUNNING = 3,
+    TERMINATED = 4,
+    ERROR = 5

watcher/decision_engine/framework/client_selector_strategy.py

@@ -0,0 +1,31 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from watcher.common.messaging.messaging_core import MessagingCore
+from watcher.decision_engine.api.selector.selector import Selector
+
+
+class ClientSelectorStrategy(Selector, MessagingCore):
+
+    """Trigger an audit (a request for optimizing a cluster)
+    :param goal: the strategy selected by the strategy selector
+    :param hosts:  the list of hypervisors where a nova-compute service
+    is running (host aggregate)
+    :return: None
+    """
+    def launch_audit(self, goal):
+        # TODO(jed):
+        # client = ClientScheduler()
+        pass

watcher/decision_engine/framework/command/trigger_audit_command.py

@@ -0,0 +1,85 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from watcher.common.messaging.events.event import Event
+from watcher.decision_engine.api.messaging.decision_engine_command import \
+    DecisionEngineCommand
+from watcher.decision_engine.framework.default_planner import DefaultPlanner
+from watcher.decision_engine.framework.messaging.events import Events
+from watcher.decision_engine.framework.strategy.StrategyManagerImpl import \
+    StrategyContextImpl
+from watcher.objects.audit import Audit
+from watcher.objects.audit import AuditStatus
+from watcher.objects.audit_template import AuditTemplate
+from watcher.openstack.common import log
+
+LOG = log.getLogger(__name__)
+
+
+class TriggerAuditCommand(DecisionEngineCommand):
+    def __init__(self, messaging, statedb, ressourcedb):
+        self.messaging = messaging
+        self.statedb = statedb
+        self.ressourcedb = ressourcedb
+        self.strategy_context = StrategyContextImpl()
+
+    def notify(self, audit_uuid, event_type, status):
+        event = Event()
+        event.set_type(event_type)
+        event.set_data({})
+        payload = {'audit_uuid': audit_uuid,
+                   'audit_status': status}
+        self.messaging.topic_status.publish_event(event.get_type().name,
+                                                  payload)
+
+    # todo(jed) remove params
+
+    def execute(self, audit_uuid, request_context):
+        LOG.debug("Execute TriggerAuditCommand ")
+
+        # 1 - change status to ONGOING
+        audit = Audit.get_by_uuid(request_context, audit_uuid)
+        audit.state = AuditStatus.ONGOING
+        audit.save()
+
+        # 2 - notify the others components of the system
+        self.notify(audit_uuid, Events.TRIGGER_AUDIT, AuditStatus.ONGOING)
+
+        # 3 - Retrieve metrics
+        cluster = self.statedb.get_latest_state_cluster()
+
+        # 4 - Select appropriate strategy
+        audit_template = AuditTemplate.get_by_id(request_context,
+                                                 audit.audit_template_id)
+
+        self.strategy_context.set_goal(audit_template.goal)
+        self.strategy_context.set_metrics_resource_collector(self.ressourcedb)
+
+        # 5 - compute change requests
+        solution = self.strategy_context.execute_strategy(cluster)
+
+        # 6 - create an action plan
+        planner = DefaultPlanner()
+        planner.schedule(request_context, audit.id, solution)
+
+        # 7 - change status to SUCCESS
+        audit = Audit.get_by_uuid(request_context, audit_uuid)
+        audit.state = AuditStatus.SUCCESS
+        audit.save()
+
+        # 8 - notify the others components of the system
+        self.notify(audit_uuid, Events.TRIGGER_AUDIT,
+                    AuditStatus.SUCCESS)

watcher/decision_engine/framework/default_planner.py

@@ -0,0 +1,170 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from enum import Enum
+from watcher.common.exception import MetaActionNotFound
+from watcher.common import utils
+from watcher.decision_engine.api.planner.planner import Planner
+
+from watcher import objects
+
+from watcher.decision_engine.framework.meta_actions.hypervisor_state import \
+    ChangeHypervisorState
+from watcher.decision_engine.framework.meta_actions.migrate import Migrate
+from watcher.decision_engine.framework.meta_actions.power_state import \
+    ChangePowerState
+from watcher.objects.action import Status as AStatus
+from watcher.objects.action_plan import Status as APStatus
+from watcher.openstack.common import log
+
+LOG = log.getLogger(__name__)
+
+# TODO(jed) The default planner is a very simple planner
+# https://wiki.openstack.org/wiki/NovaOrchestration/WorkflowEngines​
+
+
+class Primitives(Enum):
+    LIVE_MIGRATE = 'MIGRATE'
+    COLD_MIGRATE = 'MIGRATE'
+    POWER_STATE = 'POWERSTATE'
+    HYPERVISOR_STATE = 'HYPERVISOR_STATE'
+    NOP = 'NOP'
+
+
+priority_primitives = {
+    Primitives.HYPERVISOR_STATE.value: 0,
+    Primitives.LIVE_MIGRATE.value: 1,
+    Primitives.COLD_MIGRATE.value: 2,
+    Primitives.POWER_STATE.value: 3
+}
+
+
+class DefaultPlanner(Planner):
+    def create_action(self, action_plan_id, action_type, applies_to=None,
+                      src=None,
+                      dst=None,
+                      parameter=None,
+                      description=None):
+        uuid = utils.generate_uuid()
+
+        action = {
+            'uuid': uuid,
+            'action_plan_id': int(action_plan_id),
+            'action_type': action_type,
+            'applies_to': applies_to,
+            'src': src,
+            'dst': dst,
+            'parameter': parameter,
+            'description': description,
+            'state': AStatus.PENDING,
+            'alarm': None,
+            'next': None,
+        }
+        return action
+
+    def schedule(self, context, audit_id, solution):
+        LOG.debug('Create an action plan for the audit uuid')
+        action_plan = self._create_action_plan(context, audit_id)
+
+        actions = list(solution.meta_actions)
+        to_schedule = []
+
+        for action in actions:
+            if isinstance(action, Migrate):
+                # TODO(jed) type
+                primitive = self.create_action(action_plan.id,
+                                               Primitives.LIVE_MIGRATE.value,
+                                               action.get_vm().get_uuid(),
+                                               action.get_source_hypervisor().
+                                               get_uuid(),
+                                               action.get_dest_hypervisor().
+                                               get_uuid(),
+                                               description=str(action)
+                                               )
+
+            elif isinstance(action, ChangePowerState):
+                primitive = self.create_action(action_plan_id=action_plan.id,
+                                               action_type=Primitives.
+                                               POWER_STATE.value,
+                                               applies_to=action.target.
+                                               get_uuid(),
+                                               parameter=action.
+                                               get_power_state().
+                                               value, description=str(action))
+            elif isinstance(action, ChangeHypervisorState):
+                primitive = self.create_action(action_plan_id=action_plan.id,
+                                               action_type=Primitives.
+                                               HYPERVISOR_STATE.value,
+                                               applies_to=action.target.
+                                               get_uuid(),
+                                               parameter=action.get_state().
+                                               value,
+                                               description=str(action))
+
+            else:
+                raise MetaActionNotFound()
+            priority = priority_primitives[primitive['action_type']]
+            to_schedule.append((priority, primitive))
+
+        # scheduling
+        scheduled = sorted(to_schedule, reverse=False, key=lambda x: (x[0]))
+        if len(scheduled) == 0:
+            LOG.warning("The ActionPlan is empty")
+            action_plan.first_action_id = None
+            action_plan.save()
+        else:
+            parent_action = self._create_action(context,
+                                                scheduled[0][1],
+                                                None)
+            scheduled.pop(0)
+
+            action_plan.first_action_id = parent_action.id
+            action_plan.save()
+
+            for s_action in scheduled:
+                action = self._create_action(context, s_action[1],
+                                             parent_action)
+                parent_action = action
+        return action_plan
+
+    def _create_action_plan(self, context, audit_id):
+        action_plan_dict = {
+            'uuid': utils.generate_uuid(),
+            'audit_id': audit_id,
+            'first_action_id': None,
+            'state': APStatus.RECOMMENDED
+        }
+
+        new_action_plan = objects.ActionPlan(context, **action_plan_dict)
+        new_action_plan.create(context)
+        new_action_plan.save()
+        return new_action_plan
+
+    def _create_action(self, context, _action, parent_action):
+        action_description = str(_action)
+        LOG.debug("Create a action for the following resquest : %s"
+                  % action_description)
+
+        new_action = objects.Action(context, **_action)
+        new_action.create(context)
+        new_action.save()
+
+        if parent_action:
+            parent_action.next = new_action.id
+            parent_action.save()
+
+        return new_action

watcher/decision_engine/framework/default_solution.py

@@ -0,0 +1,41 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# jed <jean-emile.dartois@b-com.com>
+from watcher.decision_engine.api.solution.solution import Solution
+from watcher.openstack.common import log
+
+LOG = log.getLogger(__name__)
+
+
+class DefaultSolution(Solution):
+    def __init__(self):
+        self._meta_actions = []
+
+    def add_change_request(self, r):
+        self._meta_actions.append(r)
+
+    def __str__(self):
+        val = ""
+        for action in self._meta_actions:
+            val += str(action) + "\n"
+        return val
+
+    @property
+    def meta_actions(self):
+        """Get the current meta-actions of the solution
+
+        """
+        return self._meta_actions

watcher/decision_engine/framework/events/event_consumer_factory.py

@@ -0,0 +1,27 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class EventConsumerFactory(object):
+
+    def factory(self, type):
+        """Factory so as to create
+
+        :param type:
+        :return:
+        """
+        # return eval(type + "()")
+        raise AssertionError()

watcher/decision_engine/framework/manager_decision_engine.py

@@ -0,0 +1,97 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from concurrent.futures import ThreadPoolExecutor
+
+from oslo_config import cfg
+
+from watcher.decision_engine.framework.events.event_consumer_factory import \
+    EventConsumerFactory
+
+from watcher.common.messaging.messaging_core import \
+    MessagingCore
+from watcher.decision_engine.framework.messaging.audit_endpoint import \
+    AuditEndpoint
+from watcher.decision_engine.framework.messaging.events import Events
+
+from watcher.common.messaging.notification_handler import \
+    NotificationHandler
+from watcher.decision_engine.framework.strategy.StrategyManagerImpl import \
+    StrategyContextImpl
+from watcher.openstack.common import log
+
+LOG = log.getLogger(__name__)
+CONF = cfg.CONF
+
+WATCHER_DECISION_ENGINE_OPTS = [
+    cfg.StrOpt('topic_control',
+               default='watcher.decision.control',
+               help='The topic name used for'
+                    'control events, this topic '
+                    'used for rpc call '),
+    cfg.StrOpt('topic_status',
+               default='watcher.decision.status',
+               help='The topic name used for '
+                    'status events, this topic '
+                    'is used so as to notify'
+                    'the others components '
+                    'of the system'),
+    cfg.StrOpt('publisher_id',
+               default='watcher.decision.api',
+               help='The identifier used by watcher '
+                    'module on the message broker')
+]
+decision_engine_opt_group = cfg.OptGroup(
+    name='watcher_decision_engine',
+    title='Defines the parameters of the module decision engine')
+CONF.register_group(decision_engine_opt_group)
+CONF.register_opts(WATCHER_DECISION_ENGINE_OPTS, decision_engine_opt_group)
+
+
+class DecisionEngineManager(MessagingCore):
+    API_VERSION = '1.0'
+
+    def __init__(self):
+        MessagingCore.__init__(self, CONF.watcher_decision_engine.publisher_id,
+                               CONF.watcher_decision_engine.topic_control,
+                               CONF.watcher_decision_engine.topic_status)
+        self.handler = NotificationHandler(self.publisher_id)
+        self.handler.register_observer(self)
+        self.add_event_listener(Events.ALL, self.event_receive)
+        # todo(jed) oslo_conf
+        self.executor = ThreadPoolExecutor(max_workers=2)
+        self.topic_control.add_endpoint(AuditEndpoint(self))
+        self.context = StrategyContextImpl(self)
+
+    def join(self):
+        self.topic_control.join()
+        self.topic_status.join()
+
+    # TODO(ebe): Producer / consumer
+    def event_receive(self, event):
+        try:
+            request_id = event.get_request_id()
+            event_type = event.get_type()
+            data = event.get_data()
+            LOG.debug("request id => %s" % event.get_request_id())
+            LOG.debug("type_event => %s" % str(event.get_type()))
+            LOG.debug("data       => %s" % str(data))
+
+            event_consumer = EventConsumerFactory().factory(event_type)
+            event_consumer.set_messaging(self)
+            event_consumer.execute(request_id, data)
+        except Exception as e:
+            LOG.error("evt %s" % e.message)
+            raise e

watcher/decision_engine/framework/messaging/audit_endpoint.py

@@ -0,0 +1,43 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from watcher.decision_engine.framework.command.trigger_audit_command import \
+    TriggerAuditCommand
+from watcher.decision_engine.framework.ressourcedb_collector import RessourceDB
+from watcher.decision_engine.framework.statedb_collector import NovaCollector
+from watcher.openstack.common import log
+
+
+LOG = log.getLogger(__name__)
+
+
+class AuditEndpoint(object):
+    def __init__(self, de):
+        self.de = de
+
+    def do_trigger_audit(self, context, audit_uuid):
+        statedb = NovaCollector()
+        ressourcedb = RessourceDB()
+        audit = TriggerAuditCommand(self.de, statedb,
+                                    ressourcedb)
+        audit.execute(audit_uuid, context)
+
+    def trigger_audit(self, context, audit_uuid):
+        LOG.debug("Trigger audit %s" % audit_uuid)
+        self.de.executor.submit(self.do_trigger_audit,
+                                context,
+                                audit_uuid)
+        return audit_uuid

watcher/decision_engine/framework/messaging/events.py

@@ -0,0 +1,23 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from enum import Enum
+
+
+class Events(Enum):
+    ALL = '*',
+    ACTION_PLAN = "action_plan"
+    TRIGGER_AUDIT = "trigger_audit"

watcher/decision_engine/framework/meta_actions/hypervisor_state.py

@@ -0,0 +1,40 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from watcher.decision_engine.api.strategy.meta_action import MetaAction
+from watcher.decision_engine.framework.model.hypervisor_state import \
+    HypervisorState
+
+
+class ChangeHypervisorState(MetaAction):
+    def __init__(self, target):
+        MetaAction.__init__(self)
+        '''The target host to change the power
+
+        :param target:
+        :return:
+        '''
+        self.target = target
+        self.state = HypervisorState.ONLINE
+
+    def set_state(self, state):
+        self.state = state
+
+    def get_state(self):
+        return self.state
+
+    def __str__(self):
+        return MetaAction.__str__(self) + " ChangeHypervisorState" + str(
+            self.target) + " =>" + str(self.state)

watcher/decision_engine/framework/meta_actions/migrate.py

@@ -0,0 +1,72 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from enum import Enum
+
+from watcher.decision_engine.api.strategy.meta_action import MetaAction
+
+
+class MigrationType(Enum):
+    # Total migration time and downtime depend on memory dirtying speed
+    pre_copy = 0
+    # Postcopy transfer a page only once reliability
+    post_copy = 1
+
+
+class Migrate(MetaAction):
+    def __init__(self, vm, source_hypervisor, dest_hypervisor):
+        MetaAction.__init__(self)
+        """Request Migrate
+        :param bandwidth the bandwidth reserved for the migration
+        :param vm:       the virtual machine to migrate
+        :param source_hypervisor:
+        :param dest_hypervisor:
+        :return:
+        """
+        self.bandwidth = 0
+        self.reservedDiskIOPS = 0
+        self.remainingDirtyPages = 0
+        self.vm = vm
+        self.migration_type = MigrationType.pre_copy
+        self.source_hypervisor = source_hypervisor
+        self.dest_hypervisor = dest_hypervisor
+
+    def set_migration_type(self, type):
+        self.migration_type = type
+
+    def set_bandwidth(self, bw):
+        """Set the bandwidth reserved for the migration
+
+        :param bw: bandwidth
+        """
+        self.bandwidth = bw
+
+    def get_bandwidth(self):
+        return self.bandwidth
+
+    def get_vm(self):
+        return self.vm
+
+    def get_source_hypervisor(self):
+        return self.source_hypervisor
+
+    def get_dest_hypervisor(self):
+        return self.dest_hypervisor
+
+    def __str__(self):
+        return MetaAction.__str__(self) + " Migrate " + str(
+            self.vm) + " from " + str(
+            self.source_hypervisor) + " to " + str(self.dest_hypervisor)

watcher/decision_engine/framework/meta_actions/power_state.py

@@ -0,0 +1,39 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from watcher.decision_engine.api.strategy.meta_action import MetaAction
+from watcher.decision_engine.framework.model.power_state import PowerState
+
+
+class ChangePowerState(MetaAction):
+    def __init__(self, target):
+        MetaAction.__init__(self)
+        """The target host to change the power
+
+        :param target:
+        :return:
+        """
+        self.target = target
+        self.power_state = PowerState.g0
+
+    def set_power_state(self, state):
+        self.power_state = state
+
+    def get_power_state(self):
+        return self.power_state
+
+    def __str__(self):
+        return MetaAction.__str__(self) + "ChangePowerState " + str(
+            self.target) + " => " + str(self.power_state)

watcher/decision_engine/framework/model/diskInfo.py

@@ -0,0 +1,53 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class DiskInfo(object):
+    def __init__(self):
+        self.name = ""
+        self.major = 0
+        self.minor = 0
+        self.size = 0
+        self.scheduler = ""
+
+    def set_size(self, size):
+        """DiskInfo
+
+        :param size:   Size in bytes
+        """
+        self.size = size
+
+    def get_size(self):
+        return self.size
+
+    def set_scheduler(self, scheduler):
+        """DiskInfo
+
+        I/O Scheduler noop cfq deadline
+        :param scheduler:
+        :return:
+        """
+        self.scheduler = scheduler
+
+    def set_device_name(self, name):
+        """Device name
+
+        :param name:
+        """
+        self.name = name
+
+    def get_device_name(self):
+        return self.name

watcher/decision_engine/framework/model/hypervisor.py

@@ -0,0 +1,31 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from watcher.decision_engine.framework.model.hypervisor_state import \
+    HypervisorState
+from watcher.decision_engine.framework.model.named_element import NamedElement
+from watcher.decision_engine.framework.model.power_state import PowerState
+
+
+class Hypervisor(NamedElement):
+    def __init__(self):
+        self.state = HypervisorState.ONLINE
+        self.power_state = PowerState.g0
+
+    def set_state(self, state):
+        self.state = state
+
+    def get_state(self):
+        return self.state

watcher/decision_engine/framework/model/hypervisor_state.py

@@ -0,0 +1,22 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from enum import Enum
+
+
+class HypervisorState(Enum):
+    ONLINE = 'ONLINE'
+    OFFLINE = 'OFFLINE'

watcher/decision_engine/framework/model/mapping.py

@@ -0,0 +1,119 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from threading import Lock
+
+from watcher.openstack.common import log
+
+LOG = log.getLogger(__name__)
+
+
+class Mapping(object):
+    def __init__(self, model):
+        self.model = model
+        self._mapping_hypervisors = {}
+        self.mapping_vm = {}
+        self.lock = Lock()
+
+    def map(self, hypervisor, vm):
+        """Select the hypervisor where the instance are launched
+
+        :param hypervisor: the hypervisor
+        :param vm: the virtual machine or instance
+        """
+        try:
+            self.lock.acquire()
+
+            # init first
+            if hypervisor.get_uuid() not in self._mapping_hypervisors.keys():
+                self._mapping_hypervisors[hypervisor.get_uuid()] = []
+
+            # map node => vms
+            self._mapping_hypervisors[hypervisor.get_uuid()].append(
+                vm.get_uuid())
+
+            # map vm => node
+            self.mapping_vm[vm.get_uuid()] = hypervisor.get_uuid()
+
+        finally:
+            self.lock.release()
+
+    def unmap(self, hypervisor, vm):
+        """Remove the instance from the hypervisor
+
+        :param hypervisor: the hypervisor
+        :param vm: the virtual machine or instance
+        """
+        self.unmap_from_id(hypervisor.get_uuid(), vm.get_uuid())
+
+    def unmap_from_id(self, node_uuid, vm_uuid):
+        try:
+            self.lock.acquire()
+            if str(node_uuid) in self._mapping_hypervisors:
+                self._mapping_hypervisors[str(node_uuid)].remove(str(vm_uuid))
+            else:
+                LOG.warn("trying to delete the virtual machine " + str(
+                    vm_uuid) + " but it was not found")
+        finally:
+            self.lock.release()
+
+    def get_mapping(self):
+        return self._mapping_hypervisors
+
+    def get_mapping_vm(self):
+        return self.mapping_vm
+
+    def get_node_from_vm(self, vm):
+        return self.get_node_from_vm_id(vm.get_uuid())
+
+    def get_node_from_vm_id(self, vm_uuid):
+        """Getting host information from the guest VM
+
+        :param vm: the uuid of the instance
+        :return: hypervisor
+        """
+        return self.model.get_hypervisor_from_id(
+            self.get_mapping_vm()[str(vm_uuid)])
+
+    def get_node_vms(self, hypervisor):
+        """Get the list of instances running on the hypervisor
+
+        :param hypervisor:
+        :return:
+        """
+        return self.get_node_vms_from_id(hypervisor.get_uuid())
+
+    def get_node_vms_from_id(self, node_uuid):
+        if str(node_uuid) in self._mapping_hypervisors.keys():
+            return self._mapping_hypervisors[str(node_uuid)]
+        else:
+            # empty
+            return []
+
+    def migrate_vm(self, vm, src_hypervisor, dest_hypervisor):
+        """Migrate single instance from src_hypervisor to dest_hypervisor
+
+        :param vm:
+        :param src_hypervisor:
+        :param dest_hypervisor:
+        :return:
+        """
+        if src_hypervisor == dest_hypervisor:
+            return False
+        # unmap
+        self.unmap(src_hypervisor, vm)
+        # map
+        self.map(dest_hypervisor, vm)
+        return True

watcher/decision_engine/framework/model/model_root.py

@@ -0,0 +1,80 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from watcher.common.exception import HypervisorNotFound
+from watcher.common.exception import VMNotFound
+from watcher.decision_engine.framework.model.hypervisor import Hypervisor
+from watcher.decision_engine.framework.model.mapping import Mapping
+from watcher.decision_engine.framework.model.vm import VM
+from watcher.openstack.common import log
+
+LOG = log.getLogger(__name__)
+
+
+class ModelRoot(object):
+    def __init__(self):
+        self._hypervisors = {}
+        self._vms = {}
+        self.mapping = Mapping(self)
+        self.resource = {}
+
+    def assert_hypervisor(self, hypervisor):
+        if not isinstance(hypervisor, Hypervisor):
+            raise Exception("assert_vm")
+
+    def assert_vm(self, vm):
+        if not isinstance(vm, VM):
+            raise Exception("assert_vm")
+
+    def add_hypervisor(self, hypervisor):
+        self.assert_hypervisor(hypervisor)
+        self._hypervisors[hypervisor.get_uuid()] = hypervisor
+
+    def remove_hypervisor(self, hypervisor):
+        self.assert_hypervisor(hypervisor)
+        if str(hypervisor.get_uuid()) not in self._hypervisors.keys():
+            raise HypervisorNotFound(hypervisor.get_uuid())
+        else:
+            del self._hypervisors[hypervisor.get_uuid()]
+
+    def add_vm(self, vm):
+        self.assert_vm(vm)
+        self._vms[vm.get_uuid()] = vm
+
+    def get_all_hypervisors(self):
+        return self._hypervisors
+
+    def get_hypervisor_from_id(self, hypervisor_uuid):
+        if str(hypervisor_uuid) not in self._hypervisors.keys():
+            raise HypervisorNotFound(hypervisor_uuid)
+        return self._hypervisors[str(hypervisor_uuid)]
+
+    def get_vm_from_id(self, uuid):
+        if str(uuid) not in self._vms.keys():
+            raise VMNotFound(uuid)
+        return self._vms[str(uuid)]
+
+    def get_all_vms(self):
+        return self._vms
+
+    def get_mapping(self):
+        return self.mapping
+
+    def create_resource(self, r):
+        self.resource[str(r.get_name())] = r
+
+    def get_resource_from_id(self, id):
+        return self.resource[str(id)]

watcher/decision_engine/framework/model/named_element.py

@@ -0,0 +1,30 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class NamedElement(object):
+
+    def __init__(self):
+        self.uuid = ""
+
+    def set_uuid(self, uuid):
+        self.uuid = uuid
+
+    def get_uuid(self):
+        return self.uuid
+
+    def __str__(self):
+        return "[" + str(self.uuid) + "]"

watcher/decision_engine/framework/model/power_state.py

@@ -0,0 +1,31 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from enum import Enum
+
+
+class PowerState(Enum):
+    # away mode
+    g0 = "g0"
+    # power on suspend (processor caches are flushed)
+    # The power to the CPU(s) and RAM is maintained
+    g1_S1 = "g1_S1"
+    # CPU powered off. Dirty cache is flushed to RAM
+    g1_S2 = "g1_S2"
+    # Suspend to RAM
+    g1_S3 = "g1_S3"
+    # Suspend to Disk
+    g1_S4 = "g1_S4"
+    # switch outlet X OFF on the PDU (Power Distribution Unit)
+    switch_off = "switch_off"
+    # switch outlet X ON on the PDU (Power Distribution Unit)
+    switch_on = "switch_on"

watcher/decision_engine/framework/model/resource.py

@@ -0,0 +1,52 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from enum import Enum
+
+
+class ResourceType(Enum):
+    cpu_cores = 'num_cores'
+    memory = 'memory'
+    disk = 'disk'
+
+
+class Resource(object):
+    def __init__(self, name, capacity=None):
+        """Resource
+
+        :param name: ResourceType
+        :param capacity: max
+        :return:
+        """
+        self.name = name
+        self.capacity = capacity
+        self.mapping = {}
+
+    def get_name(self):
+        return self.name
+
+    def set_capacity(self, element, value):
+        self.mapping[element.get_uuid()] = value
+
+    def get_capacity_from_id(self, uuid):
+        if str(uuid) in self.mapping.keys():
+            return self.mapping[str(uuid)]
+        else:
+            # TODO(jed) throw exception
+            return None
+
+    def get_capacity(self, element):
+        return self.get_capacity_from_id(element.get_uuid())

watcher/decision_engine/framework/model/vm.py

@@ -0,0 +1,28 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from watcher.decision_engine.framework.model.named_element import NamedElement
+from watcher.decision_engine.framework.model.vm_state import VMState
+
+
+class VM(NamedElement):
+    def __init__(self):
+        self.state = VMState.INIT
+
+    def set_state(self, state):
+        self.state = state
+
+    def get_state(self):
+        return self.state

watcher/decision_engine/framework/model/vm_state.py

@@ -0,0 +1,26 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from enum import Enum
+
+
+class VMState(Enum):
+    INIT = 1,
+    READY = 2,
+    RUNNING = 3,
+    SLEEPING = 4,
+    KILLED = 5,
+    LIVE_MIGRATION = 6

watcher/decision_engine/framework/ressourcedb_collector.py

@@ -0,0 +1,117 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+import ceilometerclient.v2 as c_client
+import keystoneclient.v3.client as ksclient
+from oslo_config import cfg
+CONF = cfg.CONF
+
+from watcher.decision_engine.api.collector.metrics_resource_collector import \
+    MetricsResourceCollector
+
+CONF.import_opt('admin_user', 'keystonemiddleware.auth_token',
+                group='keystone_authtoken')
+CONF.import_opt('admin_tenant_name', 'keystonemiddleware.auth_token',
+                group='keystone_authtoken')
+CONF.import_opt('admin_password', 'keystonemiddleware.auth_token',
+                group='keystone_authtoken')
+CONF.import_opt('auth_uri', 'keystonemiddleware.auth_token',
+                group='keystone_authtoken')
+
+
+class RessourceDB(MetricsResourceCollector):
+    def __init__(self):
+        creds = \
+            {'auth_url': CONF.keystone_authtoken.auth_uri,
+             'username': CONF.keystone_authtoken.admin_user,
+             'password': CONF.keystone_authtoken.admin_password,
+             'project_name': CONF.keystone_authtoken.admin_tenant_name,
+             'user_domain_name': "default",
+             'project_domain_name': "default"}
+        self.keystone = ksclient.Client(**creds)
+
+        self.ceilometer = c_client.Client(
+            endpoint=self.get_ceilometer_uri(),
+            token=self.keystone.auth_token)
+
+    def make_query(user_id=None, tenant_id=None, resource_id=None,
+                   user_ids=None, tenant_ids=None, resource_ids=None):
+
+        """Returns query built form given parameters.
+    This query can be then used for querying resources, meters and
+    statistics.
+    :Parameters:
+    - `user_id`: user_id, has a priority over list of ids
+    - `tenant_id`: tenant_id, has a priority over list of ids
+    - `resource_id`: resource_id, has a priority over list of ids
+    - `user_ids`: list of user_ids
+    - `tenant_ids`: list of tenant_ids
+    - `resource_ids`: list of resource_ids
+    """
+        user_ids = user_ids or []
+        tenant_ids = tenant_ids or []
+        resource_ids = resource_ids or []
+        query = []
+        if user_id:
+            user_ids = [user_id]
+        for u_id in user_ids:
+            query.append({"field": "user_id", "op": "eq", "value": u_id})
+        if tenant_id:
+            tenant_ids = [tenant_id]
+        for t_id in tenant_ids:
+            query.append({"field": "project_id", "op": "eq", "value": t_id})
+        if resource_id:
+            resource_ids = [resource_id]
+        for r_id in resource_ids:
+            query.append({"field": "resource_id", "op": "eq", "value": r_id})
+        return query
+
+    def get_ceilometer_uri(self):
+        a = self.keystone.services.list(**{'type': 'metering'})
+        e = self.keystone.endpoints.list()
+        for s in e:
+            if s.service_id == a[0].id and s.interface == 'internal':
+                return s.url
+        raise Exception("Ceilometer Metering Service internal not defined")
+
+    def get_average_usage_vm_cpu(self, instance_uuid):
+        """The last VM CPU usage values to average
+
+        :param uuid:00
+        :return:
+        """
+        # query influxdb stream
+        query = self.make_query(resource_id=instance_uuid)
+        cpu_util_sample = self.ceilometer.samples.list('cpu_util',
+                                                       q=query)
+        cpu_usage = 0
+        count = len(cpu_util_sample)
+        for each in cpu_util_sample:
+            # print each.timestamp, each.counter_name, each.counter_volume
+            cpu_usage = cpu_usage + each.counter_volume
+        if count == 0:
+            return 0
+        else:
+            return cpu_usage / len(cpu_util_sample)
+
+    def get_average_usage_vm_memory(self, uuid):
+        # Obtaining Memory Usage is not implemented for LibvirtInspector
+        # waiting for kilo memory.resident
+        return 1
+
+    def get_average_usage_vm_disk(self, uuid):
+        # waiting for kilo disk.usage
+        return 1

watcher/decision_engine/framework/rpcapi.py

@@ -0,0 +1,90 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+from oslo_config import cfg
+import oslo_messaging as om
+
+from watcher.common import exception
+from watcher.common import utils
+
+
+from watcher.common.messaging.messaging_core import MessagingCore
+from watcher.common.messaging.notification_handler import NotificationHandler
+from watcher.common.messaging.utils.transport_url_builder import \
+    TransportUrlBuilder
+from watcher.decision_engine.framework.events.event_consumer_factory import \
+    EventConsumerFactory
+from watcher.decision_engine.framework.manager_decision_engine import \
+    decision_engine_opt_group
+from watcher.decision_engine.framework.manager_decision_engine import \
+    WATCHER_DECISION_ENGINE_OPTS
+
+from watcher.decision_engine.framework.messaging.events import Events
+
+from watcher.openstack.common import log
+
+LOG = log.getLogger(__name__)
+CONF = cfg.CONF
+
+CONF.register_group(decision_engine_opt_group)
+CONF.register_opts(WATCHER_DECISION_ENGINE_OPTS, decision_engine_opt_group)
+
+
+class DecisionEngineAPI(MessagingCore):
+    # This must be in sync with manager.DecisionEngineManager's.
+    MessagingCore.API_VERSION = '1.0'
+
+    def __init__(self):
+        MessagingCore.__init__(self, CONF.watcher_decision_engine.publisher_id,
+                               CONF.watcher_decision_engine.topic_control,
+                               CONF.watcher_decision_engine.topic_status)
+        self.handler = NotificationHandler(self.publisher_id)
+        self.handler.register_observer(self)
+        self.add_event_listener(Events.ALL, self.event_receive)
+        self.topic_status.add_endpoint(self.handler)
+
+        transport = om.get_transport(CONF, TransportUrlBuilder().url)
+        target = om.Target(
+            topic=CONF.watcher_decision_engine.topic_control,
+            version=MessagingCore.API_VERSION)
+
+        self.client = om.RPCClient(transport, target,
+                                   serializer=self.serializer)
+
+    def trigger_audit(self, context, audit_uuid=None):
+        if not utils.is_uuid_like(audit_uuid):
+            raise exception.InvalidUuidOrName(name=audit_uuid)
+
+        return self.client.call(
+            context.to_dict(), 'trigger_audit', audit_uuid=audit_uuid)
+
+    # TODO(ebe): Producteur / consommateur implementer
+    def event_receive(self, event):
+
+        try:
+            request_id = event.get_request_id()
+            event_type = event.get_type()
+            data = event.get_data()
+            LOG.debug("request id => %s" % event.get_request_id())
+            LOG.debug("type_event => %s" % str(event.get_type()))
+            LOG.debug("data       => %s" % str(data))
+
+            event_consumer = EventConsumerFactory.factory(event_type)
+            event_consumer.execute(request_id, self.context, data)
+        except Exception as e:
+            LOG.error("evt %s" % e.message)
+            raise e

watcher/decision_engine/framework/statedb_collector.py

@@ -0,0 +1,104 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+
+from keystoneclient.auth.identity import v3
+from keystoneclient import session
+
+from watcher.applier.framework.command.wrapper.nova_wrapper import NovaWrapper
+from watcher.decision_engine.api.collector.cluster_state_collector import \
+    ClusterStateCollector
+from watcher.decision_engine.framework.model.hypervisor import Hypervisor
+from watcher.decision_engine.framework.model.model_root import ModelRoot
+from watcher.decision_engine.framework.model.resource import Resource
+from watcher.decision_engine.framework.model.resource import ResourceType
+from watcher.decision_engine.framework.model.vm import VM
+from watcher.openstack.common import log
+
+from oslo_config import cfg
+CONF = cfg.CONF
+LOG = log.getLogger(__name__)
+
+CONF.import_opt('admin_user', 'keystonemiddleware.auth_token',
+                group='keystone_authtoken')
+CONF.import_opt('admin_tenant_name', 'keystonemiddleware.auth_token',
+                group='keystone_authtoken')
+CONF.import_opt('admin_password', 'keystonemiddleware.auth_token',
+                group='keystone_authtoken')
+CONF.import_opt('auth_uri', 'keystonemiddleware.auth_token',
+                group='keystone_authtoken')
+
+
+class NovaCollector(ClusterStateCollector):
+    def get_latest_state_cluster(self):
+        try:
+            creds = \
+                {'auth_url': CONF.keystone_authtoken.auth_uri,
+                 'username': CONF.keystone_authtoken.admin_user,
+                 'password': CONF.keystone_authtoken.admin_password,
+                 'project_name': CONF.keystone_authtoken.admin_tenant_name,
+                 'user_domain_name': "default",
+                 'project_domain_name': "default"}
+            auth = v3.Password(auth_url=creds['auth_url'],
+                               username=creds['username'],
+                               password=creds['password'],
+                               project_name=creds['project_name'],
+                               user_domain_name=creds[
+                                   'user_domain_name'],
+                               project_domain_name=creds[
+                                   'project_domain_name'])
+            sess = session.Session(auth=auth)
+            wrapper = NovaWrapper(creds, session=sess)
+
+            cluster = ModelRoot()
+            mem = Resource(ResourceType.memory)
+            num_cores = Resource(ResourceType.cpu_cores)
+            disk = Resource(ResourceType.disk)
+            cluster.create_resource(mem)
+            cluster.create_resource(num_cores)
+            cluster.create_resource(disk)
+
+            flavor_cache = {}
+            hypervisors = wrapper.get_hypervisors_list()
+            for h in hypervisors:
+                i = h.hypervisor_hostname.index('.')
+                name = h.hypervisor_hostname[0:i]
+                # create hypervisor in stateDB
+                hypervisor = Hypervisor()
+                hypervisor.set_uuid(name)
+                # set capacity
+                mem.set_capacity(hypervisor, h.memory_mb)
+                disk.set_capacity(hypervisor, h.disk_available_least)
+                num_cores.set_capacity(hypervisor, h.vcpus)
+                cluster.add_hypervisor(hypervisor)
+                vms = wrapper.get_vms_by_hypervisor(str(name))
+                for v in vms:
+                    # create VM in stateDB
+                    vm = VM()
+                    vm.set_uuid(v.id)
+                    # set capacity
+                    wrapper.get_flavor_instance(v, flavor_cache)
+                    mem.set_capacity(vm, v.flavor['ram'])
+                    disk.set_capacity(vm, v.flavor['disk'])
+                    num_cores.set_capacity(vm, v.flavor['vcpus'])
+                    # print(dir(v))
+                    cluster.get_mapping().map(hypervisor, vm)
+                    cluster.add_vm(vm)
+            return cluster
+        except Exception as e:
+            LOG.error("nova collector " + unicode(e))
+            return None

watcher/decision_engine/framework/strategy/StrategyManagerImpl.py

@@ -0,0 +1,56 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from watcher.decision_engine.api.strategy.strategy_context import \
+    StrategyContext
+from watcher.decision_engine.framework.default_planner import DefaultPlanner
+from watcher.decision_engine.framework.strategy.strategy_selector import \
+    StrategySelector
+from watcher.openstack.common import log
+
+LOG = log.getLogger(__name__)
+
+
+class StrategyContextImpl(StrategyContext):
+    def __init__(self, broker=None):
+        LOG.debug("Initializing decision_engine Engine API ")
+        self.strategies = {}
+        self.selected_strategies = []
+        self.broker = broker
+        self.planner = DefaultPlanner()
+        self.strategy_selector = StrategySelector()
+        self.goal = None
+        self.metrics_resource_collector = None
+
+    def add_strategy(self, strategy):
+        self.strategies[strategy.name] = strategy
+        self.selected_strategy = strategy.name
+
+    def remove_strategy(self, strategy):
+        pass
+
+    def set_goal(self, goal):
+        self.goal = goal
+
+    def set_metrics_resource_collector(self, metrics_resource_collector):
+        self.metrics_resource_collector = metrics_resource_collector
+
+    def execute_strategy(self, model):
+        # todo(jed) create thread + refactoring
+        selected_strategy = self.strategy_selector.define_from_goal(self.goal)
+        selected_strategy.set_metrics_resource_collector(
+            self.metrics_resource_collector)
+        return selected_strategy.execute(model)

watcher/decision_engine/framework/strategy/strategy_loader.py

@@ -0,0 +1,56 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from oslo_config import cfg
+from watcher.decision_engine.strategies.basic_consolidation import \
+    BasicConsolidation
+from watcher.openstack.common import log
+
+LOG = log.getLogger(__name__)
+CONF = cfg.CONF
+
+strategies = {
+    'basic': 'watcher.decision_engine.strategies.'
+             'basic_consolidation::BasicConsolidation'
+}
+WATCHER_STRATEGY_OPTS = [
+    cfg.DictOpt('strategies',
+                default=strategies,
+                help='Strategies used for the optimization ')
+]
+strategies_opt_group = cfg.OptGroup(
+    name='watcher_strategies',
+    title='Defines strategies available for the optimization')
+CONF.register_group(strategies_opt_group)
+CONF.register_opts(WATCHER_STRATEGY_OPTS, strategies_opt_group)
+
+
+class StrategyLoader(object):
+
+    def __init__(self):
+        '''Stevedor loader
+
+        :return:
+        '''
+
+        self.strategies = {
+            None: BasicConsolidation("basic", "Basic offline consolidation"),
+            "basic": BasicConsolidation(
+                "basic",
+                "Basic offline consolidation")
+        }
+
+    def load(self, model):
+        return self.strategies[model]

watcher/decision_engine/framework/strategy/strategy_selector.py

@@ -0,0 +1,53 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from oslo_config import cfg
+from watcher.decision_engine.framework.strategy.strategy_loader import \
+    StrategyLoader
+from watcher.objects.audit_template import Goal
+from watcher.openstack.common import log
+
+LOG = log.getLogger(__name__)
+CONF = cfg.CONF
+
+goals = {
+    'SERVERS_CONSOLIDATION': 'basic',
+    'MINIMIZE_ENERGY_CONSUMPTION': 'basic',
+    'BALANCE_LOAD': 'basic',
+    'MINIMIZE_LICENSING_COST': 'basic',
+    'PREPARE_PLANNED_OPERATION': 'basic'
+}
+WATCHER_GOALS_OPTS = [
+    cfg.DictOpt('goals',
+                default=goals, help='Goals used for the optimization ')
+]
+goals_opt_group = cfg.OptGroup(name='watcher_goals',
+                               title='Goals available for the optimization')
+CONF.register_group(goals_opt_group)
+CONF.register_opts(WATCHER_GOALS_OPTS, goals_opt_group)
+
+
+class StrategySelector(object):
+
+    def __init__(self):
+        self.strategy_loader = StrategyLoader()
+
+    def define_from_goal(self, goal_name):
+        if goal_name is None:
+            goal_name = Goal.SERVERS_CONSOLIDATION
+
+        strategy_to_load = CONF.watcher_goals.goals[goal_name]
+        return self.strategy_loader.load(strategy_to_load)

watcher/decision_engine/strategies/README.md

@@ -0,0 +1,23 @@
+
+Basic Consolidation is based on Sercon : 
+
+Sercon: Server Consolidation Algorithm using Live Migration of Virtual Machines for Green Computing 
+Virtualization technologies changed the way data centers of enterprises utilize their server resources.
+ 
+Instead of using dedicated servers for each type of application, virtualization allows viewing resources as a pool of
+unified resources, thereby reducing complexity and easing manageability.
+ 
+Server consolidation technique,which deals with reducing the number of servers used by consolidating applications, is one of the main
+applications of virtualization in data centers.
+The latter technique helps to use computing resources more effectively and has many benefits,such as reducing costs of power, cooling and, hence, contributes to the
+Green IT initiative. 
+In a dynamic data center environment, where applications encapsulated as virtual machines
+are mapped to and released from the nodes frequently, reducing the number of server nodes used can be
+achieved by migrating applications without stopping their services, the technology known as live migration.
+
+However, live migration is a costly operation; hence, how to perform periodic server consolidation operation
+in a migration-aware way is a challenging task. 
+Sercon not only minimizes the overall number of used servers, but also minimizes the number of migrations. 
+
+Aziz Murtazaev and Sangyoon Oh
+Department of Computer and Information Engineering, Ajou University, Suwon, Korea

watcher/decision_engine/strategies/basic_consolidation.py

@@ -0,0 +1,426 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from watcher.common.exception import ClusterEmpty
+from watcher.common.exception import ClusteStateNotDefined
+from watcher.common.exception import MetricCollectorNotDefined
+from watcher.decision_engine.api.strategy.strategy import Strategy
+from watcher.decision_engine.api.strategy.strategy import StrategyLevel
+from watcher.decision_engine.framework.meta_actions.hypervisor_state import \
+    ChangeHypervisorState
+from watcher.decision_engine.framework.meta_actions.migrate import Migrate
+from watcher.decision_engine.framework.meta_actions.migrate import \
+    MigrationType
+from watcher.decision_engine.framework.meta_actions.power_state import \
+    ChangePowerState
+from watcher.decision_engine.framework.meta_actions.power_state import \
+    PowerState
+from watcher.decision_engine.framework.model.hypervisor_state import \
+    HypervisorState
+from watcher.decision_engine.framework.model.resource import ResourceType
+from watcher.openstack.common import log
+
+LOG = log.getLogger(__name__)
+
+
+class BasicConsolidation(Strategy):
+    def __init__(self, name=None, description=None):
+        """Basic offline Consolidation using live migration
+
+    The basic consolidation algorithm has several limitations.
+    It has been developed only for tests.
+    eg: The BasicConsolidation assumes that the virtual mahine and
+    the compute node are on the same private network.
+
+    Good Strategy :
+    The workloads of the VMs are changing over the time
+    and often tend to migrate from one physical machine to another.
+    Hence, the traditional and offline heuristics such as bin packing
+    are not applicable for the placement VM in cloud computing.
+    So, the decision Engine optimizer provide placement strategy considering
+    not only the performance effects but also the workload characteristics of
+    VMs and others metrics like the power consumption and
+    the tenants constraints (SLAs).
+
+    The watcher optimizer use an online VM placement technique
+    based on machine learning and meta-heuristics that must handle :
+    - multi-objectives
+    - Contradictory objectives
+    - Adapt to changes dynamically
+    - Fast convergence
+
+        :param name: the name of the strategy
+        :param description: a description of the strategy
+        """
+        Strategy.__init__(self, name, description)
+
+        # set default value for the number of released nodes
+        self.number_of_released_nodes = 0
+        # set default value for the number of migrations
+        self.number_of_migrations = 0
+        # set default value for number of allowed migration attempts
+        self.migration_attempts = 0
+
+        # set default value for the efficiency
+        self.efficiency = 100
+
+        # TODO(jed) improve threshold overbooking ?,...
+        self.threshold_mem = 0.90
+        self.threshold_disk = 0.80
+        self.threshold_cores = 1
+
+        # TODO(jed) target efficiency
+        self.target_efficiency = 60
+
+        # TODO(jed) weight
+        self.weight_cpu = 1
+        self.weight_mem = 1
+        self.weight_disk = 1
+
+        # TODO(jed) bound migration attempts (80 %)
+        self.bound_migration = 0.80
+
+    def compute_attempts(self, size_cluster):
+        """Upper bound of the number of migration
+
+        :param size_cluster:
+        """
+        self.migration_attempts = size_cluster * self.bound_migration
+
+    def check_migration(self, model,
+                        src_hypervisor,
+                        dest_hypervisor,
+                        vm_to_mig):
+        '''check if the migration is possible
+
+        :param model: current state of the cluster
+        :param src_hypervisor: the current of the virtual machine
+        :param dest_hypervisor:the destination of the virtual machine
+        :param vm_to_mig: the virtual machine
+        :return: True if the there is enough place otherwise false
+        '''
+        if src_hypervisor == dest_hypervisor:
+            return False
+
+        total_cores = 0
+        total_disk = 0
+        total_mem = 0
+        cap_cores = model.get_resource_from_id(ResourceType.cpu_cores)
+        cap_disk = model.get_resource_from_id(ResourceType.disk)
+        cap_mem = model.get_resource_from_id(ResourceType.memory)
+
+        for vm_id in model.get_mapping().get_node_vms(dest_hypervisor):
+            vm = model.get_vm_from_id(vm_id)
+            total_cores += cap_cores.get_capacity(vm)
+            total_disk += cap_disk.get_capacity(vm)
+            total_mem += cap_mem.get_capacity(vm)
+
+        # capacity requested by hypervisor
+        total_cores += cap_cores.get_capacity(vm_to_mig)
+        total_disk += cap_disk.get_capacity(vm_to_mig)
+        total_mem += cap_mem.get_capacity(vm_to_mig)
+
+        return self.check_threshold(model,
+                                    dest_hypervisor,
+                                    total_cores,
+                                    total_disk,
+                                    total_mem)
+
+    def check_threshold(self, model,
+                        dest_hypervisor,
+                        total_cores,
+                        total_disk,
+                        total_mem):
+        """Check threshold
+
+        check the threshold value defined by the ratio of
+        aggregated CPU capacity of VMS on one node to CPU capacity
+        of this node must not exceed the threshold value.
+        :param dest_hypervisor:
+        :param total_cores
+        :param total_disk
+        :param total_mem
+        :return: True if the threshold is not exceed
+        """
+        cap_cores = model.get_resource_from_id(ResourceType.cpu_cores)
+        cap_disk = model.get_resource_from_id(ResourceType.disk)
+        cap_mem = model.get_resource_from_id(ResourceType.memory)
+        # available
+        cores_available = cap_cores.get_capacity(dest_hypervisor)
+        disk_available = cap_disk.get_capacity(dest_hypervisor)
+        mem_available = cap_mem.get_capacity(dest_hypervisor)
+        if cores_available >= total_cores * self.threshold_cores \
+                and disk_available >= total_disk * self.threshold_disk \
+                and mem_available >= total_mem * self.threshold_mem:
+            return True
+        else:
+            return False
+
+    def get_allowed_migration_attempts(self):
+        """Allowed migration
+
+        Maximum allowed number of migrations this allows us to fix
+        the upper bound of the number of migrations
+        :return:
+        """
+        return self.migration_attempts
+
+    def get_threshold_cores(self):
+        return self.threshold_cores
+
+    def set_threshold_cores(self, threshold):
+        self.threshold_cores = threshold
+
+    def get_number_of_released_nodes(self):
+        return self.number_of_released_nodes
+
+    def get_number_of_migrations(self):
+        return self.number_of_migrations
+
+    def calculate_weight(self, model, element, total_cores_used,
+                         total_disk_used, total_memory_used):
+        """Calculate weight of every
+
+        :param model:
+        :param element:
+        :param total_cores_used:
+        :param total_disk_used:
+        :param total_memory_used:
+        :return:
+        """
+        cpu_capacity = model.get_resource_from_id(
+            ResourceType.cpu_cores).get_capacity(element)
+
+        disk_capacity = model.get_resource_from_id(
+            ResourceType.disk).get_capacity(element)
+
+        memory_capacity = model.get_resource_from_id(
+            ResourceType.memory).get_capacity(element)
+
+        score_cores = (1 - (float(cpu_capacity) - float(total_cores_used)) /
+                       float(cpu_capacity))
+
+        score_disk = (1 - (float(disk_capacity) - float(total_disk_used)) /
+                      float(disk_capacity))
+
+        score_memory = (
+            1 - (float(memory_capacity) - float(total_memory_used)) /
+            float(memory_capacity))
+        # todo(jed) take in account weight
+        return (score_cores + score_disk + score_memory) / 3
+
+    def calculate_score_node(self, hypervisor, model):
+        """calculate the score that reprensent the utilization level
+
+            :param hypervisor:
+            :param model:
+            :return:
+            """
+        metrics_collector = self.get_metrics_resource_collector()
+        if metrics_collector is None:
+            raise MetricCollectorNotDefined()
+        total_cores_used = 0
+        total_memory_used = 0
+        total_disk_used = 0
+
+        for vm_id in model.get_mapping().get_node_vms(hypervisor):
+            total_cores_used += metrics_collector.get_average_usage_vm_cpu(
+                vm_id)
+            total_memory_used += metrics_collector.get_average_usage_vm_memory(
+                vm_id)
+            total_disk_used += metrics_collector.get_average_usage_vm_disk(
+                vm_id)
+
+        return self.calculate_weight(model, hypervisor, total_cores_used,
+                                     total_disk_used,
+                                     total_memory_used)
+
+    def calculate_migration_efficiency(self):
+        """Calculate migration efficiency
+
+        :return: The efficiency tells us that every VM migration resulted
+         in releasing on node
+        """
+        if self.number_of_migrations > 0:
+            return (float(self.number_of_released_nodes) / float(
+                self.number_of_migrations)) * 100
+        else:
+            return 0
+
+    def calculate_score_vm(self, vm, model):
+        """Calculate Score of virtual machine
+
+        :param vm_id: the id of virtual machine
+        :param model: the model
+        :return: score
+        """
+        # todo(jed) inject ressource metric
+        metric_collector = self.get_metrics_resource_collector()
+        if metric_collector is None:
+            raise MetricCollectorNotDefined()
+
+        if model is None:
+            raise ClusteStateNotDefined()
+
+        vm = model.get_vm_from_id(vm.get_uuid())
+        cores_used = metric_collector.get_average_usage_vm_cpu(vm.get_uuid())
+        memory_used = metric_collector.get_average_usage_vm_memory(
+            vm.get_uuid())
+        disk_used = metric_collector.get_average_usage_vm_disk(vm.get_uuid())
+
+        return self.calculate_weight(model, vm, cores_used,
+                                     disk_used,
+                                     memory_used)
+
+    def print_utilization(self, model):
+        if model is None:
+            raise ClusteStateNotDefined()
+        for node_id in model.get_all_hypervisors():
+            builder = node_id + " utilization  " + str(
+                (self.calculate_score_node(
+                    model.get_hypervisor_from_id(node_id),
+                    model)) * 100) + " %"
+            LOG.debug(builder)
+
+    def execute(self, orign_model):
+        LOG.debug("initialize Sercon Consolidation")
+
+        if orign_model is None:
+            raise ClusteStateNotDefined()
+
+        # todo(jed) clone model
+        current_model = orign_model
+
+        self.efficiency = 100
+        unsuccessful_migration = 0
+
+        first = True
+        self.print_utilization(current_model)
+        size_cluster = len(current_model.get_all_hypervisors())
+        if size_cluster == 0:
+            raise ClusterEmpty()
+
+        self.compute_attempts(size_cluster)
+
+        while self.get_allowed_migration_attempts() >= unsuccessful_migration:
+            if first is not True:
+                self.efficiency = self.calculate_migration_efficiency()
+                if self.efficiency < float(self.target_efficiency):
+                    break
+            first = False
+            score = []
+
+            ''' calculate score of nodes based on load by VMs '''
+            for hypevisor_id in current_model.get_all_hypervisors():
+                hypevisor = current_model.get_hypervisor_from_id(hypevisor_id)
+                result = self.calculate_score_node(hypevisor, current_model)
+                if result != 0:
+                    score.append((hypevisor_id, result))
+
+            ''' sort compute nodes by Score decreasing '''''
+            s = sorted(score, reverse=True, key=lambda x: (x[1]))
+            LOG.debug("Hypervisor(s) BFD {0}".format(str(s)))
+
+            ''' get Node to be released '''
+            if len(score) == 0:
+                LOG.warning(
+                    "The workloads of the compute nodes"
+                    " of the cluster is zero.")
+                break
+
+            node_to_release = s[len(score) - 1][0]
+
+            ''' get List of VMs from Node '''
+            vms_to_mig = current_model.get_mapping().get_node_vms_from_id(
+                node_to_release)
+
+            vm_score = []
+            for vm_id in vms_to_mig:
+                vm = current_model.get_vm_from_id(vm_id)
+                vm_score.append(
+                    (vm_id, self.calculate_score_vm(vm, current_model)))
+
+            ''' sort VM's by Score '''
+            v = sorted(vm_score, reverse=True, key=lambda x: (x[1]))
+            LOG.debug("VM(s) BFD {0}".format(str(v)))
+
+            m = 0
+            tmp_vm_migration_schedule = []
+            for vm in v:
+                for j in range(0, len(s)):
+                    mig_vm = current_model.get_vm_from_id(vm[0])
+                    mig_src_hypervisor = current_model.get_hypervisor_from_id(
+                        node_to_release)
+                    mig_dst_hypervisor = current_model.get_hypervisor_from_id(
+                        s[j][0])
+
+                    result = self.check_migration(current_model,
+                                                  mig_src_hypervisor,
+                                                  mig_dst_hypervisor, mig_vm)
+                    if result is True:
+
+                        ''' create migration VM '''
+                        if current_model.get_mapping(). \
+                                migrate_vm(mig_vm, mig_src_hypervisor,
+                                           mig_dst_hypervisor):
+                            live_migrate = Migrate(mig_vm,
+                                                   mig_src_hypervisor,
+                                                   mig_dst_hypervisor)
+                            # live migration
+                            live_migrate.set_migration_type(
+                                MigrationType.pre_copy)
+                            live_migrate.set_level(
+                                StrategyLevel.conservative)
+
+                            tmp_vm_migration_schedule.append(live_migrate)
+
+                        if len(current_model.get_mapping().get_node_vms(
+                                mig_src_hypervisor)) == 0:
+                            # TODO(jed) how to manage strategy level
+                            # from conservative to aggressive
+                            change_power = ChangePowerState(mig_src_hypervisor)
+                            change_power.set_power_state(PowerState.g1_S1)
+                            change_power.set_level(
+                                StrategyLevel.conservative)
+                            tmp_vm_migration_schedule.append(change_power)
+
+                            h = ChangeHypervisorState(mig_src_hypervisor)
+                            h.set_level(StrategyLevel.aggressive)
+
+                            h.set_state(HypervisorState.OFFLINE)
+                            tmp_vm_migration_schedule.append(h)
+
+                            self.number_of_released_nodes += 1
+
+                        m += 1
+                        break
+            if m > 0:
+                self.number_of_migrations = self.number_of_migrations + m
+                unsuccessful_migration = 0
+                for a in tmp_vm_migration_schedule:
+                    self.solution.add_change_request(a)
+            else:
+                unsuccessful_migration += 1
+        self.print_utilization(current_model)
+        infos = {
+            "number_of_migrations": self.number_of_migrations,
+            "number_of_nodes_released": self.number_of_released_nodes,
+            "efficiency": self.efficiency
+        }
+        LOG.debug(infos)
+        self.solution.set_model(current_model)
+        self.solution.set_efficiency(self.efficiency)
+        return self.solution

watcher/decision_engine/strategies/dummy_strategy.py

@@ -0,0 +1,25 @@
+# -*- encoding: utf-8 -*-
+# Copyright (c) 2015 b<>com
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+# implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from watcher.decision_engine.api.strategy.strategy import Strategy
+from watcher.openstack.common import log
+
+LOG = log.getLogger(__name__)
+
+
+class DummyStrategy(Strategy):
+    def execute(self, model):
+        return self.get_solution()