The [nova_client]/api_version defaults to 2.56 since
change Idd6ebc94f81ad5d65256c80885f2addc1aaeaae1. There
is compatibility code for that change but if 2.56 is
not available watcher_non_live_migrate_instance will
still fail if a destination host is used.
Since 2.56 has been available since the Queens version of
nova it should be reasonable to require at least that
version of nova is running for using Watcher.
This adds code which enforces the minimum version along
with a release note and "watcher-status upgrade check"
check method.
Note that it's kind of weird for watcher to have a config
option like nova_client.api_version since compute API
microversions are per API request even though novaclient
is constructed with the single configured version. It should
really be something the client (watcher in this case) determines
using version discovery and gracefully enables features if
the required nova API version is available, but that's a bigger
change.
Change-Id: Id34938c7bb8a5ca934d997e52cac3b365414c006
As a follow up to the recent test improvements for Ceilometer this
patch ensures that the same test pattern is used for Gnocchi and
Monasca as well. This ensures that the mocked functions will be called
with matching signatures.
Change-Id: Ic14a4c087f3961a4b4f373e2e3d792aba71868f6
Override the metric map of each datasource as soon as it is created by
the manager. This override comes from a file whose path is provided by
a setting in config file.
Loading at creation time allows the correct datasource be used when
get_backend is called, this allows loading a datasource whose metric
names get updated outside the watcher's codebase.
The function 'load_metric_map' returns empty-dict in any error case.
Also in case the file is empty where safe_load is unable finds any
yaml documents, it will return None. [1]
Some minor refactoring in the test_manager file for readability and
added tests for file load and metric override.
1 - https://pyyaml.org/wiki/PyYAMLDocumentation
Change-Id: I1df16245f4c7dfd34066f3ab0553cd67154faa58
Implements: blueprint file-based-metric-map
Change I25b4cb0e1b85379ff0c4da9d0c1474380d75ce3a in
Queens refactored the statistic_aggregation method
and renamed the "aggregate" kwarg to "aggregation",
presumably to match the signature of the GnocchiHelper
statistic_aggregation method (the commit message does
not give details) so a base method could be added to
the parent class for all datasource helpers.
As a result, the CeilometerHelper calls to its
statistic_aggregation started passing the new
"granularity" kwarg but failed to match the rename
to the "aggregation" kwarg, which breaks the
CeilometerHelper. This was missed by the unit tests
because the tests were just asserting the erroneous
call that the runtime code made.
This change fixes the kwarg typo and makes the
tests more robust by using the mock spec kwarg
to define a spec for the statistic_aggregation
mock so that it must be called with the correct
parameters defined in the method signature. The
test is refactored to reduce duplicate mocking.
The same test hardening can and should be done
in the gnocchi and monasca helper tests but that
should be done in a separate change.
Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>
Closes-Bug: #1829542
Change-Id: Idfd099f718873d9056fdc35a97954771c9ae5762
As of change Ic4659d1f18af181203439a8bf1b38805ff34c309 the
nova CDM will not be built until an audit is performed.
Instances and services (compute hosts) can be created and
deleted before an audit is performed which will attempt
to use the notification callback function which relies
on the CDM being built already, and if not results in
an AttributeError.
This change side-steps that issue by checking to see that the
nova CDM exists before trying to call the notification
callback function.
An alternative to this is forcefully create the nova CDM when
notifications are received before an audit which is what happend
before change Ic4659d1f18af181203439a8bf1b38805ff34c309.
Change-Id: I16990afb82019821c443c9df26d3e515e52efa69
Closes-Bug: #1828582
_post_live_migration[1] runs on the source host and calls
post_live_migration_at_destination on the dest host which
emits the instance.live_migration_post_dest.end notification:[2]
But it's not the last notification for the live migration operation.
so we should use instance.live_migration_post.end instead of
instance.live_migration_post_dest.end notification.
[1]daa2ac2287/nova/compute/manager.py (L6907)
[2]daa2ac2287/nova/compute/manager.py (L7035)
Change-Id: Id1e2d98f56d5a95d49e32f98d2910660b9f48ce6
The version of bandit in lower-constraints (1.4.0) does
not match the version in test-requirements (1.6.0) however
bandit is a test-only dependency and there is no test coverage
for bandit in the lower-constraints tox job target, so there
is really no good reason to have bandit in lower-constraints.
As such, this change simply removes it from lower-constraints.
Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>
Change-Id: I35f66994e9a3a334b342232587d84491542da755
Sphinx 2.0 no longer works on python 2.7, so we need to start capping
it there as well.
The errors are as follow:
Requirement(package='sphinx', location='', specifiers='!=1.6.6,!=1.6.7,>=1.6.5'
does not match "python_version>='3.4'"
Requirement(package='sphinx', location='', specifiers='!=1.6.6,!=1.6.7,>=1.6.5'
does not match "python_version=='2.7'"
Could not find a global requirements entry to match package sphinx. If the package
is already included in the global list, the name or platform markers there may not
match the local settings.
Change-Id: I6dad56ffbb9e85e36cacea1a89565c2fc8248fbf
The final Stein version of Watcher was 2.0.0
so this fixes the version mentioned in the
watcher-status man page docs.
Change-Id: I7fce35471cf31222f9cdafc35e5a7b287bc4598e
The _add_virtual_layer and _add_virtual_servers methods
have not been used since Ic4659d1f18af181203439a8bf1b38805ff34c309
in Stein so this change removes them.
Change-Id: I8c05f29c3c03aa5897cb182bb492948771c42881
This enhances the [collector]/collector_plugins
config option help text to mention the storage
and baremetal in-tree collectors and the ability
to load out-of-tree collectors via extension point.
While doing this, the help text is formatted for
prettier rst rendering in the docs.
Change-Id: Ifd32c95c664c4e9586c250e6bceaeaba2e2df417
CeilometerClient has been deprecated and is no longer available for
master. Without ceilometer client installed docs fail to build with
an exception [1].
This patch marks the import optional.
1 -
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/sphinx/config.py", line 368, in
eval_config_file
execfile_(filename, namespace)
File "/usr/lib/python2.7/site-packages/sphinx/util/pycompat.py", line
150, in execfile_
exec_(code, _globals)
File "/usr/lib/python2.7/site-packages/six.py", line 709, in exec_
exec(""exec _code_ in _globs_, _locs_"")
File "<string>", line 1, in <module>
File
"/home/abuild/rpmbuild/BUILD/python-watcher-2.1.0.dev45/doc/source/conf.py",
line 20, in <module>
objects.register_all()
File
"/home/abuild/rpmbuild/BUILD/python-watcher-2.1.0.dev45/watcher/objects/__init__.py",
line 31, in register_all
__import__('watcher.objects.action_plan')
File
"/home/abuild/rpmbuild/BUILD/python-watcher-2.1.0.dev45/watcher/objects/action_plan.py",
line 78, in <module>
from watcher import conf
File
"/home/abuild/rpmbuild/BUILD/python-watcher-2.1.0.dev45/watcher/conf/__init__.py",
line 28, in <module>
from watcher.conf import datasources
File
"/home/abuild/rpmbuild/BUILD/python-watcher-2.1.0.dev45/watcher/conf/datasources.py",
line 21, in <module>
from watcher.datasources import manager
File
"/home/abuild/rpmbuild/BUILD/python-watcher-2.1.0.dev45/watcher/datasources/manager.py",
line 19, in <module>
from watcher.datasources import ceilometer as ceil
File
"/home/abuild/rpmbuild/BUILD/python-watcher-2.1.0.dev45/watcher/datasources/ceilometer.py",
line 21, in <module>
from ceilometerclient import exc
ImportError: No module named ceilometerclient
)
Change-Id: Idcf582c2495aab39aacf691b687759405bb94dca
Currently default config files are being for initialization of CONF from
oslo_config. However default config dirs are not being passed as a
result watcher components (eg: decision-engine) are unable to load
files from default directories (eg: /etc/watcher/watcher.conf.d)
supported by oslo_config. This is a short-coming on watcher's side.
Also this forces user to have multiple config for each component.
Without this default set, oslo_config will search for conf with string
'python-watcher' in it, eg: /etc/python-watcher/.... Since there is a
because project=python-watcher a couple of lines below
This patch adds the option after evaluating using project as 'watcher'
which is similar to evaluation of default_config_files and also allows
it to be passed in as a function parameter.
Change-Id: I013f9d03978f8716847f8d1ee6888629faf5779b
This fixes the wrong installation guide link from the
user guide which was pointing to the watcherclient docs
for some reason, maybe it was just a copy/paste error.
Change-Id: I38f536e187245523ac37d70054a2df8cdfcbe4b2
Closes-Bug: #1828584
Hard-coding watcher.openstack.common to warning level logging
only makes it hard to debug watcher's interactions with other
services, like when it's triggering and monitoring a server live
migration.
Since debug logging is controlled via the "debug" configuration
option, we can just rely on that to filter out debug logs within
watcher itself.
Note this has been this way since change
I699e0ab082657880998d8618fe29eb7f56c6c661 back in 2015 and there
was no explanation why the watcher.openstack.common logging
was set to WARN level.
Change-Id: I939403a4ae36d1aa9ea83badb9404bc37d18a1a6
Related-Bug: #1828598
The -x option for bandit changed in 1.6.0 and now
supports glob patterns so use that to correctly
exclude test code from bandit scans.
Since this change requires bandit>=1.6.0, we have
to also fix the networkx requirement to pass the
requirements-check job so that the networkx requirement
matches what is in global-requirements from change
I0a9700926c9a0db93e782c853c33f1aaee3d4876.
Change-Id: I4fc1166daee5d8739296419216d11d684be27c0a
Closes-Bug: #1828419
Allows to define a global preference for metric datasources with the
ability for strategy specific overrides. In addition, strategies which
do not require datasources have the config options removed this is
done to prevent confusion.
Some documentation that details the inner workings of selecting
datasources is updated.
Imports for some files in watcher/common have been changed to resolve
circular dependencies and now match the overall method to import
configuration.
Addtional datasources will be retrieved by the manager if the
datasource throws an error.
Implements: blueprint global-datasource-preference
Change-Id: I6fc455b288e338c20d2c4cfec5a0c95350bebc36
To get log formatting like the other openstack projects
running in devstack the setup_logging function should be
used. This will also give us the "Display level" formatting
in the logs via the os-loganalyze packaged used by infra.
Needed by: https://review.opendev.org/657652
Change-Id: I5e9bd5a142e45804e8d915b370746a2142243088
