develop #1

Merged
Arnike merged 21 commits from develop into main 2026-02-07 18:06:51 +03:00
Showing only changes of commit 6836e0535b - Show all commits

146
README.md
View File

@@ -1,5 +1,151 @@
# watcher-visio
Web dashboard (**SWatcher**) for monitoring an OpenStack cluster and visualising OpenStack Watcher audits: region and host counts, physical/virtual CPU and RAM usage, VM stats, top flavors, and audit recommendations (migrations) with CPU load charts per host.
Data sources:
- **OpenStack** (SDK, `clouds.yaml`): compute region, servers, flavors; Watcher API for audits, action plans, and actions.
- **Prometheus**: node_exporter, libvirt, and placement metrics for pCPU/vCPU, pRAM/vRAM, and VM counts.
---
## Configuration
### Environment variables
Copy [.env.example](.env.example) to `.env` and set as needed. For Docker Compose you can use `env_file: [.env]` in `docker-compose.yml`.
| Variable | Description |
|----------|-------------|
| `PROMETHEUS_URL` | Prometheus base URL (e.g. `http://10.0.0.1:9090/`). |
| `OPENSTACK_CLOUD` | Cloud name from `clouds.yaml` (e.g. `distlab`). |
| `OPENSTACK_REGION_NAME` | OpenStack region (e.g. `cl2k1distlab`). |
| `USE_MOCK_DATA` | Set to `true`/`1`/`yes` to serve mock data (no OpenStack/Prometheus). Useful for local/dev. |
| `SECRET_KEY` | Django secret key; override in production. |
Defaults for Prometheus and OpenStack are in [watcher_visio/settings.py](watcher_visio/settings.py).
### OpenStack (`clouds.yaml`)
Authentication uses OpenStacks standard `clouds.yaml`. The cloud name must match `OPENSTACK_CLOUD`. Place `clouds.yaml` in the project root (or standard OpenStack config location). **Do not commit real credentials;** use a local or CI-specific file and keep production secrets out of the repo.
---
## Running locally
1. Create a virtualenv and install dependencies:
```bash
python -m venv .venv
source .venv/bin/activate # or .venv\Scripts\activate on Windows
pip install -r requirements.txt
```
2. Optionally build frontend CSS (see [Frontend build](#frontend-build)).
3. Configure `clouds.yaml` and environment (e.g. `.env` or export `PROMETHEUS_URL`, `OPENSTACK_CLOUD`, `OPENSTACK_REGION_NAME`). For development without OpenStack/Prometheus, set `USE_MOCK_DATA=true`.
4. Run migrations and start the server:
```bash
python manage.py migrate
python manage.py runserver
```
Open http://127.0.0.1:8000/ (or the port shown). With `USE_MOCK_DATA=true`, the dashboard is filled with mock data; otherwise the page loads a skeleton and fetches data from the API.
---
## Running with Docker
**Production-like** (built image, no volume mount):
```bash
docker compose up --build
```
App is available at http://localhost:8080. Healthcheck hits `GET /`.
**Development** (mounted code, mock data, no OpenStack/Prometheus):
```bash
docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
```
Uses `USE_MOCK_DATA=true` and mounts the project directory for live code changes. Build CSS before building the image so `static/css/output.css` is present, or run `npm run build` locally before `docker compose ... up --build`.
---
## Frontend build
CSS is built with Tailwind and DaisyUI ([package.json](package.json)).
- Install: `npm install`
- One-off build: `npm run build`
- Watch: `npm run dev`
Source: [static/css/main.css](static/css/main.css). Output: `static/css/output.css`. For Docker, run `npm run build` before building the image so the image includes `output.css`.
---
## API
| Endpoint | Description |
|----------|-------------|
| `GET /` | Dashboard page. With `USE_MOCK_DATA=true`, rendered with mock context; otherwise skeleton, with data loaded via the API. |
| `GET /api/stats/` | JSON: region, pCPU/vCPU, pRAM/vRAM, VM stats, top flavors. Cached for `DASHBOARD_CACHE_TTL` seconds (see settings). |
| `GET /api/audits/` | JSON: `{ "audits": [ ... ] }` — list of Watcher audits with migrations and chart data (host labels, cpu_current, cpu_projected). Same cache TTL. |
---
## Repository structure
| Path | Description |
|------|-------------|
| `watcher_visio/` | Django project: settings, root URL config, WSGI/ASGI. |
| `dashboard/` | Main app: views (index, api_stats, api_audits), `openstack_utils` (connect, flavor, audits), `prometheus_utils` (query), `mock_data`, `templatetags` (mathfilters), tests. |
| `templates/`, `static/` | HTML templates and static assets (Tailwind output, Chart.js, etc.). |
| `clouds.yaml` | OpenStack config (do not commit production secrets). |
| [Dockerfile](Dockerfile), [docker-entrypoint.sh](docker-entrypoint.sh) | Image build and entrypoint (migrate then run server). |
| [docker-compose.yml](docker-compose.yml), [docker-compose.dev.yml](docker-compose.dev.yml) | Compose: base (prod-like) and dev override (mount + mock). |
---
## Architecture
```mermaid
flowchart LR
subgraph sources [Data sources]
OS[OpenStack SDK]
Prom[Prometheus]
Watcher[Watcher API]
end
subgraph app [Django]
Views[views]
Cache[(Cache)]
end
subgraph out [Output]
HTML[HTML]
API[JSON API]
end
subgraph frontend [Frontend]
Chart[Chart.js]
end
OS --> Views
Prom --> Views
Watcher --> Views
Views --> Cache
Cache --> Views
Views --> HTML
Views --> API
HTML --> Chart
API --> Chart
```
OpenStack (region, servers, flavors), Prometheus (metrics), and the Watcher API (audits, action plans, actions) are queried in Django views; results are cached. The dashboard page is either rendered with mock/skeleton data or loads stats and audits via `/api/stats/` and `/api/audits/`; Chart.js draws the CPU and other charts.
---
## Running tests
From the project root (with Django and dependencies installed, e.g. in a virtualenv):