Commit Graph

2099 Commits

Author SHA1 Message Date
Dantali0n
cebee2c4d7 improve OptGroup consistency across configuration
This is a follow-up to: https://review.opendev.org/#/c/666897/
and makes sure titles and help information get rendered in
the configuration documentation and configuration samples.

The options for the placement_client group are already changed
and left untouched as a result. The changes to grafana_client
are already done in another patch and also untouched.

Change-Id: Ia33cd4576e4b55e651f3f3779a01f2867126138d
2019-07-01 09:29:14 +02:00
Zuul
4d35aabfdf Merge "Fix invalid assert states" 2019-07-01 06:24:53 +00:00
zhufl
8fc4a9cbee Fix invalid assert states
"self.assertTrue(action.state, objects.action.State.SUCCEEDED)"
and "self.assertTrue(action.state, objects.action.State.FAILED)"
should use assertEqual.

Co-Authored-By: Canwei Li <li.canwei2@zte.com.cn>
Change-Id: I8e28d651938ca6ed8d12e8a6f5ecf775cf01a39c
2019-07-01 03:20:19 +00:00
Zuul
1b41d92b9e Merge "Add uWSGI support" 2019-06-28 02:29:16 +00:00
Zuul
c90c4989f6 Merge "Configure nova notification format in non-grenade CI jobs" 2019-06-28 02:29:14 +00:00
Zuul
f335b6dff2 Merge "improve the process of instance_created.end" 2019-06-28 02:29:11 +00:00
Zuul
0915d991b4 Merge "Add name for instance in Watcher datamodel" 2019-06-28 02:20:27 +00:00
Zuul
f75cf1ddff Merge "Improve the configuration parameters for grafana" 2019-06-27 07:03:21 +00:00
licanwei
c1a5e443fe Add uWSGI support
This patch implements uWSGI support for Watcher API service.
Because mod_wsgi is deprecated, using uwsgi to replace of mod_wsgi.
Most of Openstack projects have finished it.

Closes-Bug: #1834392
Change-Id: I3fad8d30a15aba493fb91da9337c2515ddea5167
2019-06-27 14:56:52 +08:00
Zuul
550b10b586 Merge "Documentation configuring grafana datasource" 2019-06-27 06:30:32 +00:00
chenke
b62965c2bf Add name for instance in Watcher datamodel
Now Watcher's datamodel uses human_id to store the display_name
of the intance. But the value of human_id is not reliable. About
the reason, please see[1].

The solution is to add a 'name' field to save the display_name of
the instance, and ensure that the value of this field is the same
when the datamodel is created and when the datamodel is updated.

About the 'human_id', We will remove it in the future.

References:
[1]. https://bugs.launchpad.net/watcher/+bug/1833665

20190619 Watcher meeting IRC Log:
[1]. http://eavesdrop.openstack.org/irclogs/%23openstack-watcher/%23openstack-watcher.2019-06-19.log.html
[2]. http://eavesdrop.openstack.org/meetings/watcher/2019/watcher.2019-06-19-08.00.log.html#l-47

Change-Id: I6976759629a4feedee09261cc1dac935e050202a
Closes-Bug: #1833665
2019-06-26 22:29:12 +08:00
Dantali0n
fa1642e323 Documentation configuring grafana datasource
Documentation for administrators on how to configure the Grafana
datasource.

Change-Id: I5d1d3129b5d225f0f2fc86d149c046f9aab94d47
2019-06-26 10:58:14 +02:00
Zuul
aa70cd8b1a Merge "Fix placement_client group help docs generation" 2019-06-26 07:29:05 +00:00
Matt Riedemann
5f521471e1 Fix placement_client group help docs generation
To get the placement_client OptGroup help text to generate
in the docs, we have to use the OptGroup object in the
list_opts() method rather than a string for the group name.

https://docs.openstack.org/oslo.config/latest/cli/generator.html#defining-option-discovery-entry-points

Change-Id: Ie728081caa205ded2435c6b95f5e8d4bbd23372c
2019-06-24 14:52:02 -04:00
Dantali0n
d08d7c396e Improve the configuration parameters for grafana
This improves the documentation on configuration parameters for the
Grafana datasource.

Follow-up: If1f27dc01e853c5b24bdb21f1e810f64eaee2e5c
Depends-on: I5d1d3129b5d225f0f2fc86d149c046f9aab94d47
Change-Id: Ifd8be7491669c429482d880fdf0219be5ef03163
2019-06-24 14:38:42 -04:00
Matt Riedemann
966a4dfa5f Configure nova notification format in non-grenade CI jobs
Nova used to emit versioned and unversioned notiifcations
by default but that changed in https://review.opendev.org/603079/
so now nova emits only unversioned notifications by default.
Watcher listens for versioned notifications so we need to configure
nova to emit both versioned (for Watcher) and unversioned
(for Ceilometer) notifications explicitly.

This adds an override-defaults file so devstack will load up
the nova devstack variable to set the notification_format before
importing and stacking the nova lib script.

Note that this only fixes the non-grenade CI jobs since grenade
requires separate handling for overriding defaults which is proving
hard to do and will be addressed in a separate change.

Partial-Bug: #1831917

Change-Id: I7e441608b38338eecd80e663ed3abe66a89e504f
2019-06-24 13:09:34 -04:00
Zuul
899e534761 Merge "check instance state for instance.update" 2019-06-24 03:54:52 +00:00
Zuul
d046606a7e Merge "remove tail_log" 2019-06-22 02:31:12 +00:00
licanwei
dd321e9f21 improve the process of instance_created.end
In the process of handling instance_created.end,
there is a KeyError exception output log. This is because
invoking get_instance_by_uuid before creating the instance
in the data model.
During the review of https://review.opendev.org/#/c/663489/,
reviewers think that it's better to remove the KeyError exception.
This patche seperates the process of instance_created.end from
other Nova notifications and removes the call of get_instance_by_uuid.

Change-Id: Ie9e2d4f5b32ee7a5b52bbcd50abfa81dcabab7bb
2019-06-21 16:53:25 +08:00
Zuul
d98f51c452 Merge "Implement the configuration for Grafana datasource" 2019-06-21 02:52:24 +00:00
Zuul
5effcde632 Merge "Update strategy doc" 2019-06-21 01:37:39 +00:00
licanwei
9b8d1445f1 remove tail_log
tail_log is deprecated and should be removed
https://github.com/openstack/devstack/blob/master/functions-common#L1611

Change-Id: I6012fd63aa6b0cdb8ed1b278880f8f6c3e4d38cf
2019-06-20 15:28:33 +08:00
licanwei
90291923a1 Update strategy doc
Ceilometer removed cpu_util metric in [1].
Another metric compute.node.cpu.percent need to set
compute_monitors option to cpu.virt_driver in the
nova.conf, we should remind user about these.
[1]: https://review.opendev.org/#/c/580709/

Change-Id: I89306ef7c26fa2927945bd4f3ee88b670511d147
2019-06-20 10:57:05 +08:00
Dantali0n
06f8aa712a Implement the configuration for Grafana datasource
This implements the configuration parameters to implement
Grafana as a datasource including the influxdb translator

Change-Id: If1f27dc01e853c5b24bdb21f1e810f64eaee2e5c
Partially-implements: blueprint grafana-proxy-datasource
2019-06-19 15:58:18 +02:00
zhufl
37b11fa404 Fix missing print format
This is to add missing print format in placement_helper.py.

Change-Id: I23492fd33f3df8c8709f6d6317c936221f1108d4
2019-06-19 11:43:18 +08:00
Zuul
fd04c67ed8 Merge "Map instance to its node" 2019-06-19 02:44:58 +00:00
Zuul
08622c6c80 Merge "typo ceilometer url" 2019-06-19 02:44:57 +00:00
licanwei
e4fc5a08ed typo ceilometer url
Change-Id: I2cb24249ed1e82dd85c0f586532e5a7cbc57f2c0
2019-06-18 19:29:00 +08:00
Zuul
413028e527 Merge "Fix base enable_plugin branch for grenade run" 2019-06-18 07:53:31 +00:00
Zuul
f8fef7d774 Merge "Replace removed exceptions and prevent regression" 2019-06-18 05:04:24 +00:00
Zuul
81fc8ac6a8 Merge "Define a new InstanceNotMapped exception" 2019-06-18 02:32:33 +00:00
Zuul
5d385db8a1 Merge "Cleanup ConfFixture" 2019-06-16 08:32:41 +00:00
Zuul
a1dd90bb74 Merge "Fix property access in test_global_preference* tests" 2019-06-14 20:17:17 +00:00
Dantali0n
15754a14dd Replace removed exceptions and prevent regression
Replaces the NoSuchMetric exception that was replaced. The exception
is replaced with MetricNotAvailable and test cases are added to prevent
regression.

The changes in the exceptions were introduced in:
https://review.opendev.org/#/c/658127/

Change-Id: Id0f872e916aaa5dec59ed1ae6c0f653553b3fe46
2019-06-14 22:00:41 +02:00
Zuul
667d2d661a Merge "Move datasource query_retry into baseclass." 2019-06-14 09:04:15 +00:00
Zuul
b2111baf91 Merge "Backwards compatibility for node parameter" 2019-06-14 07:42:09 +00:00
licanwei
a4d978b893 Define a new InstanceNotMapped exception
In get_node_by_instance_uuid, an exception ComputeNodeNotFound
will be thrown if can't find a node through instance uuid.
But the exception information replaces the node name with
instance uuid, which is misleading, so we define a new exception.

Closes-Bug: #1832156

Change-Id: Ic6c44ae44da7c3b9a1c20e9b24a036063af266ba
2019-06-14 10:51:20 +08:00
Zuul
6495e42a60 Merge "Optimize NovaHelper.get_compute_node_by_hostname" 2019-06-14 02:31:13 +00:00
Zuul
e4f80b5461 Merge "Optimize hypervisor API calls" 2019-06-14 02:28:51 +00:00
Zuul
2a2f7902bd Merge "Improve DevStack documentation to support metrics" 2019-06-14 02:28:48 +00:00
Zuul
6502435dc5 Merge "Remove dead code" 2019-06-14 02:28:47 +00:00
Zuul
5f126cffe0 Merge "Add Placement helper" 2019-06-14 02:21:44 +00:00
Dantali0n
584eeefdc8 Move datasource query_retry into baseclass.
Moves the query_retry method into the baseclass and makes the query
retry and timeout options part of the watcher_datasources config group.
This makes the query_retry behavior uniform across all datasources.

A new baseclass method named query_retry_reset is added so datasources
can define operations to perform when recovering from a query error.
Test cases are added to verify the behavior of query_retry.

The query_max_retries and query_timeout config parameters are
deprecated in the gnocchi_client group and will be removed in a future
release.

Change-Id: I33e9dc2d1f5ba8f83fcf1488ff583ca5be5529cc
2019-06-13 15:52:53 +02:00
Matt Riedemann
28df60e275 Fix base enable_plugin branch for grenade run
We should be starting from stable/stein on the "old" side
of grenade runs now. Rather than hard-code the branch, just
use the BASE_DEVSTACK_BRANCH variable.

Change-Id: I1b0406f870ed0ae5622cfa7421a6cca00d0f891c
2019-06-13 09:51:19 -04:00
licanwei
7281f6184f Remove dead code
get_node_by_instance_uuid will never return None,
so the OR condition is dead code.

Change-Id: I26c553e1067a3cbeac6c0afe1c4bfdee4d939055
2019-06-13 17:31:49 +08:00
licanwei
79a57f67e6 Map instance to its node
When receiving Nova notification instance.create.end,
map instance to its node after adding instance to datamodel.
Related-Bug: #1832156

Change-Id: I6f39e8d935195c611f668f71590e1d9ff52ced0d
2019-06-13 15:58:28 +08:00
chenming
731d4bfdf2 update contraints url
http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006478.html

Change-Id: I1929fc98cd728eb0dae66762481880e23cd793c7
2019-06-12 19:11:53 +00:00
Dantali0n
dd119ca1f8 Backwards compatibility for node parameter
Adds backwards compatibility for node parameter used by strategies. If
the node value is set by the user configuration it will override the
value for compute_node which is the value used by the strategies now.

This change was introduced in: https://review.opendev.org/#/c/656622/
Resolution discussed in the meeting on the 5th of June 2019
https://eavesdrop.openstack.org/meetings/watcher/2019/watcher.2019-06-05-08.00.log.html

Change-Id: Idaea062789a6b169e64f556fecc34cfbaaee5076
2019-06-12 16:23:58 +00:00
chenke
00f20ab1d4 Fix property access in test_global_preference* tests
In Python, when we use @property, the method will be
decorated by property.

When we call method self.strategy.datasource_backend()[1],
Actually it did two things:
1. call self.strategy.datasource_backend()
2. according to  the method's return value[2], call self._datasource_backend()

[1]. https://github.com/openstack/watcher/blob/bd8636f3f/watcher/tests/decision_engine/strategy/strategies/test_base.py#L87
[2]. https://github.com/openstack/watcher/blob/bd8636f3f/watcher/decision_engine/strategy/strategies/base.py#L368

But in this part, we just want it to perform the first step.
So we have to use self.strategy.datasource_backend instead of
self.strategy.datasource_backend()

The reason why the unittest does not report an error is
because the returned value is a mock object, and the second step
is executed without error, for example:

python -m unittest  watcher.tests.decision_engine.strategy.strategies.test_base
(Pdb) x=self.strategy.datasource_backend
(Pdb) type(x)
<class 'mock.mock.MagicMock'>
(Pdb) x
<MagicMock name='DataSourceManager().get_backend()' id='139740418102608'>
(Pdb) x()
<MagicMock name='DataSourceManager().get_backend()()' id='139740410824976'>
(Pdb) self.strategy.datasource_backend()
<MagicMock name='DataSourceManager().get_backend()()' id='139740410824976'>

To make the tests more robust, the underlying backend function
is mocked to be not callable.

Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>

Change-Id: I3305d9afe8ed79e1dc3affe02ba067ac06cece42
2019-06-12 12:00:45 -04:00
licanwei
b57feba5e8 Add Placement helper
This patch added Placement to Watcher
We plan to improve the data model and strategies in
the future specs.

Change-Id: I7141459eef66557cd5d525b5887bd2a381cdac3f
Implements: blueprint support-placement-api
2019-06-12 11:11:13 +08:00