Commit Graph

2318 Commits

Author SHA1 Message Date
chenke
cbc2b9eb37 Fix var src_extra_specs error
This error was discovered by tool coverity. If we don't
initialize this var src_extra_specs, line 225 may sometimes
raise an error.

Change-Id: I992b56b64d56f35c8355b22707c3db5112964b31
2019-08-06 11:34:59 +08:00
chenke
251ad35c8b Remove stale comment in method execute()
The code associated with virtual has been removed before,
and the relevant comments should be removed here.

Change-Id: I7104c1a6752ad0b8c9837a643e51b0a13194a81b
2019-08-05 09:10:13 +08:00
Zuul
f2020a9283 Merge "Getting data from placement when updating datamodel" 2019-07-29 06:03:32 +00:00
Zuul
503e67f82e Merge "Check resource class before using" 2019-07-29 03:45:25 +00:00
Zuul
c0bba3ecf3 Merge "replace disk_capacity by disk" 2019-07-29 03:43:52 +00:00
licanwei
0b25c884e6 Add resource capacity property
Resource(VCPU, memory and disk) capacity need to be calculated
through formula: capacity = (total-reserved)*ratio.

Partially Implements: blueprint improve-compute-data-model

Change-Id: I15ca66dd2c3a21c5acfebf6f04fa6601aff7918f
2019-07-27 15:41:04 +08:00
licanwei
86d9cf17a2 Getting data from placement when updating datamodel
We have some new fields(vcpus_ratio, vcpus_used, ...)
in the Watcher ComputeNode. During the process of updating
data model by notifications, we need to get data from
placement.

Partially Implements: blueprint improve-compute-data-model

Change-Id: I10587e93bb3e7be6af78bb3a50509d82d8228f78
2019-07-27 15:03:13 +08:00
Zuul
9b4693a105 Merge "set disk field to disk capacity" 2019-07-27 03:05:43 +00:00
licanwei
86ea9c8e7b replace disk_capacity by disk
Partially Implements: blueprint improve-compute-data-model

Change-Id: I9af8d3e5ad3288d56f9ef5ef998b56f9a3e6622d
2019-07-27 10:23:21 +08:00
Zuul
4ae51c58bd Merge "Optimize method list_opts() in watcher/conf/opts.py" 2019-07-26 10:12:38 +00:00
Zuul
56ab717d25 Merge "Remove useless gconfig process in watcher/api/scheduling.py" 2019-07-26 10:12:15 +00:00
licanwei
6cc9ea7cfb set disk field to disk capacity
The node.free_disk_gb does not take allocation ratios used
for overcommit into account so this value may be negative.
We do not need this field and plan to set disk to total disk
capacity and then remove disk_capacity.

Partially Implements: blueprint improve-compute-data-model

Change-Id: I72c4490f5a8d0fbd1039f70ff20f07b743b6bb2d
2019-07-26 17:23:18 +08:00
licanwei
0986168fe6 Check resource class before using
check if the resource class(VCPU, memory, disk) in the return
dictionary. If they are, don't need to use dict.get() with a
default value because the parameters are required.

Partially Implements: blueprint improve-compute-data-model

Change-Id: Icb8c672d0e87e6e5f030a2222f928d1bbd069e3c
2019-07-26 14:26:08 +08:00
Zuul
e0f70cb87c Merge "remove id field from CDM" 2019-07-24 07:30:00 +00:00
Zuul
672066d982 Merge "Update api-ref location" 2019-07-24 07:29:58 +00:00
licanwei
4b83bf33e2 remove id field from CDM
There are 3 related fields(id, uuid and hostname) in ComputeNode[1].
according to [2], after nova api 2.53, the id of the hypervisor as a UUID.
and service.host is equal to hypervisor name for compute node.
so we can remove id and only keep uuid then set uuid to node.id

[1]:https://github.com/openstack/watcher/blob/master/watcher/decision_engine/model/collector/nova.py#L306
[2]:https://developer.openstack.org/api-ref/compute/?expanded=list-hypervisors-details-detail#list-hypervisors-details

Change-Id: Ie1d1ad56808270d936ec25186061f7f12cc49fdc
Closes-Bug: #1835192
Depends-on: I752fbfa560313e28e87d83e46431c283b4db4f23
Depends-on: I0975500f359de92b6d6fdea2e01614cf0ba73f05
2019-07-23 10:28:47 +08:00
Andreas Jaeger
54344208fc Update api-ref location
The api documentation is now published on docs.openstack.org instead
of developer.openstack.org. Update all links that are changed to the
new location.

Note that redirects will be set up as well but let's point now to the
new location.

For details, see:
http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007828.html

Change-Id: I4101eced9c4bd26741f760e5651204f5d2dfea0f
2019-07-22 19:00:11 +02:00
licanwei
3d741d05aa Improve Compute Data Model
The fields(vcpus, memory and disk_capacity) in the Watcher ComputeNode
do not take allocation ratios used for overcommit into account so there
may be disparity between this and the used count.
This patch added some new fields to solve this problem.

Partially Implements: blueprint improve-compute-data-model

Change-Id: Id33496f368fb23cb8e744c7e8451e1cd1397866b
2019-07-22 10:22:35 +08:00
Zuul
cd86e85ae8 Merge "Add reource_name for save_energy in action input parameter field" 2019-07-20 02:59:24 +00:00
Zuul
885ec20c94 Merge "Add call_retry for ModelBuilder for error recovery" 2019-07-20 02:58:10 +00:00
Zuul
1f9abe6785 Merge "Replace human_id with name in grafana doc" 2019-07-20 02:31:19 +00:00
Dantali0n
cadc000f32 Add call_retry for ModelBuilder for error recovery
Add call_retry method for ModelBuilder classes along with configuration
options. This allows ModelBuilder classes to reattempt any failed calls
to external services such as Nova or Ironic.

Change-Id: Ided697adebed957e5ff13b4c6b5b06c816f81c4a
2019-07-19 16:09:18 +02:00
Zuul
1af7ac107c Merge "Baseclass for ModelBuilder with audit scope" 2019-07-19 13:34:42 +00:00
Zuul
3a1e83dd9d Merge "Move datasources folder into decision_engine" 2019-07-19 13:27:10 +00:00
chenke
361f22638f Remove useless gconfig process in watcher/api/scheduling.py
Change-Id: I3849453286a735fbe95984e8496fd8a2fcb156b4
2019-07-18 16:05:24 +08:00
chenke
458ad08693 Optimize method list_opts() in watcher/conf/opts.py
Actually list_opts() return a list like[1], So we don't need to
convert list to dict and then convert to list[2].

The reason why we need to convert it before is to put together
the same group of configuration objects, but we don't need it
actually.

Now, the list_opts()'s result like this[3].

Reference:
[1]. [(Group1,[cfgObj1,cfgObj2....]),(Group2,[cfgObj3,cfgObj3....])..]
[2]. 375ae32fad/watcher/conf/opts.py (L51-L52)
[3]. [(Group1,[cfgObj1]),(Group1,[cfgObj2]),(Group2,[cfgObj3,cfgObj3....])..]

Change-Id: I50fcc5f812be42038852662639fb10c6dd2f6f72
2019-07-18 13:23:04 +08:00
Zuul
375ae32fad Merge "Fix watcher/conf/applier.py default worker value" 2019-07-18 03:08:56 +00:00
chenke
d5126174c0 Fix watcher/conf/applier.py default worker value
The default value of worker should be int type.

Change-Id: Ie2aa0c34ffbb7fa86154cc5c8ce2c365579d3fb1
2019-07-17 13:34:56 +08:00
chenke
904720b020 Remove useless _opts.py
This file has not been used.

Change-Id: Ie53dca6673f87bfeded74e336f3d60fbf500caf8
2019-07-16 19:36:16 +08:00
Zuul
4b8fe2745d Merge "Remove redundant human_id fields when creating and updating datamodel" 2019-07-16 11:11:26 +00:00
Dantali0n
933bc59b39 Baseclass for ModelBuilder with audit scope
This lets all the ModelBuilder classes use one baseclass and forces
ClusterDataModelCollector's to pass the scope.

The scopes are still unused in the case of Ironic and Cinder.

The idea is to do several follow ups to this and in the end have a
similar method to query_retry in the datasources baseclass.

Change-Id: Ibbdedd3087fef5298d7f4c9d3abdba05d1fbb2f0
2019-07-15 22:32:14 +02:00
Zuul
7cc5aef03f Merge "Add get_compute_node_by_uuid" 2019-07-13 08:42:08 +00:00
Dantali0n
433eabb8d1 Move datasources folder into decision_engine
The datasources are only used by the decision_engine, however, they
are placed in a directory one level higher. This patch moves the
datasources code into the decision_engine folder.

Change-Id: Ia54531fb899b79a59bb77adea079ff27c0d518fa
2019-07-12 08:54:09 +02:00
chenke
0727c0e302 Add reource_name for save_energy in action input parameter field
(Partial implement)
Implements: blueprint add-resource-name-in-action-input-parameter-field

Change-Id: Icd9ee099bb769679b0313025e841c898a0fa2c6f
2019-07-12 13:42:45 +08:00
licanwei
3009716ded Add get_compute_node_by_uuid
We want to set the value of uuid field of Watcher ComputeNode
to hypversion id(as uuid). So we need to get hypervisor
information by uuid.

Change-Id: I752fbfa560313e28e87d83e46431c283b4db4f23
Related-Bug: #1835192
2019-07-12 10:27:47 +08:00
Zuul
233a2b5585 Merge "Releasenote for grafana datasource" 2019-07-12 02:19:17 +00:00
Zuul
2365418e1b Merge "Grafana proxy datasource to retrieve metrics" 2019-07-12 02:19:16 +00:00
Zuul
cd70ad0d01 Merge "Add reource_name for zone_migration in action input parameter field" 2019-07-12 02:19:15 +00:00
Zuul
3bc426a590 Merge "remove baremetal nodes when building CDM" 2019-07-12 02:18:25 +00:00
Zuul
a4cbe69d57 Merge "Add get_node_by_name" 2019-07-12 02:18:24 +00:00
Dantali0n
80e8d0002a Resolve aggregate error in workload_stabilization
This error is caused because the condition "is not '':" is not always
true. Sometimes self.aggregation_method['node'] is u'' instead of ''.
This patch ensures that in both cases the behavior is the same.

Change-Id: I7453678cc76892ebeacca23c3501a10a08725d1d
Closes-bug: #1836195
2019-07-11 14:54:04 +02:00
Zuul
5259e5b332 Merge "Add marker option for get_instance_list()" 2019-07-11 09:21:17 +00:00
chenke
6dd35a0058 Remove redundant human_id fields when creating and updating datamodel
For the reason, please see:
[1]. http://eavesdrop.openstack.org/irclogs/%23openstack-watcher/%23openstack-watcher.2019-06-19.log.html
[2]. http://eavesdrop.openstack.org/meetings/watcher/2019/watcher.2019-06-19-08.00.log.html#l-47

Change-Id: I4284397aa987565f4cfc2697907a879d7d6492e9
Related-Bug: #1833665
2019-07-10 15:21:40 +08:00
chenke
8cb7158790 Replace human_id with name in grafana doc
This patch does two things:

1. replace instance's human_id with name.
2. remove ComputeNode human_id.

Now name field in Watcher Compute Data Model is availible.
Use name is better than human_id. For the reason, please see[1].

[1]. https://bugs.launchpad.net/watcher/+bug/1833665

Change-Id: I04f40e7d2a2bda48e9a362f9d0b23f449c40324e
2019-07-10 15:10:59 +08:00
Zuul
a01b4ddc81 Merge "Add reource_name in action input parameter field" 2019-07-10 06:57:36 +00:00
chenke
502ed741d6 Add marker option for get_instance_list()
Change-Id: Iee31369876052a22e5f3263cd5e7fad5d068f68d
2019-07-10 14:42:45 +08:00
licanwei
256104a38a remove baremetal nodes when building CDM
aggregate list and availability_zone list may return ironic type
compute nodes. When building compute data model we should check
the hypervisor_type and remove ironic compute nodes.

Change-Id: Idf404c104c30368baf95ef7d05ad8fc3e7adca38
Related-Bug: #1835183
2019-07-10 14:03:31 +08:00
chenke
612fc12af1 Add reource_name for zone_migration in action input parameter field
(Partial implement)
Implements: blueprint add-resource-name-in-action-input-parameter-field

Depends-on: I708cf63ff1d9a989604e1d5b834c8b7e5b087892
Change-Id: I5428fb7a1195a39f15f36509997ff5ad6fda4bb7
2019-07-10 09:57:42 +08:00
Dantali0n
0541d8c25b Grafana proxy datasource to retrieve metrics
New datasource to retrieve metrics that can be configured in a
flexible way depending on the deployment. Current implemenation only
works with InfluxDB. Slight changes to datasource manager were
necessary because grafana metric_map can only be built at runtime.

The yaml configuration file can still be used to define metrics
but will require that five different attributes are specified per
metric.

Specific databases accesible through grafana can be defined by
creating 'translators' for these specific databases. This patch
introduces a base class for these translators and their methods.

In addition the first translator specific for InfluxDB is
created.

Depends-on: I68475883529610e514aa82f1881105ab0cf24ec3
Depends-on: If1f27dc01e853c5b24bdb21f1e810f64eaee2e5c
Implements: blueprint grafana-proxy-datasource
Change-Id: Ib12b6a7882703e84a27c301e821c1a034b192508
2019-07-09 16:22:00 +02:00
chenke
dc2c361d04 Add reource_name in action input parameter field
(Partial implement)
Implements: blueprint add-resource-name-in-action-input-parameter-field

Depends-on: I51d879e31dee03652ee9d0d94a7f3168012cc060
Change-Id: I708cf63ff1d9a989604e1d5b834c8b7e5b087892
2019-07-09 18:40:49 +08:00