Add Promtail service to Docker Compose and update Loki configuration
- Introduced a new Promtail service in `docker-compose.observability.yml` for log collection. - Configured Promtail with necessary volumes and command for log processing. - Updated `loki-config.yml` to include `allow_structured_metadata` setting for enhanced metadata handling.
This commit is contained in:
89
docs/en/guides/observability.md
Normal file
89
docs/en/guides/observability.md
Normal file
@@ -0,0 +1,89 @@
|
||||
# Observability Stack Guide
|
||||
|
||||
This guide explains how to use the observability stack (Grafana, Prometheus, Loki, Promtail) included in the infrastructure.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The stack consists of the following components:
|
||||
|
||||
- **Prometheus**: Collects metrics from services.
|
||||
- **Loki**: Collects logs.
|
||||
- **Promtail**: Scrapes logs from Docker containers and pushes them to Loki.
|
||||
- **Grafana**: Visualization dashboard for metrics (from Prometheus) and logs (from Loki).
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Docker and Docker Compose installed.
|
||||
- Existing `microservices-network` (created by the main application stack or manually).
|
||||
|
||||
### Starting the Stack
|
||||
|
||||
You can easily start the stack using the provided script:
|
||||
|
||||
```bash
|
||||
./scripts/observability/start.sh
|
||||
```
|
||||
|
||||
Or manually:
|
||||
|
||||
```bash
|
||||
# Ensure network exists
|
||||
docker network create microservices-network || true
|
||||
|
||||
cd infra/observability
|
||||
docker-compose -f docker-compose.observability.yml up -d
|
||||
```
|
||||
|
||||
Check if all containers are running:
|
||||
|
||||
```bash
|
||||
docker ps
|
||||
```
|
||||
|
||||
You should see `grafana`, `prometheus`, `loki`, and `promtail`.
|
||||
|
||||
## Accessing Services
|
||||
|
||||
| Service | URL | Credentials (if applicable) | Description |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| **Grafana** | [http://localhost:3001](http://localhost:3001) | `admin` / `admin` | Main dashboard for visualization. |
|
||||
| **Prometheus** | [http://localhost:9090](http://localhost:9090) | N/A | Raw metrics and target status. |
|
||||
| **Loki** | [http://localhost:3100](http://localhost:3100) | N/A | Log aggregation API (no UI). |
|
||||
|
||||
## Using Grafana
|
||||
|
||||
1. **Login**: Access [http://localhost:3001](http://localhost:3001) and login with `admin`/`admin`.
|
||||
2. **Explore Data**:
|
||||
- Go to **Explore** (compass icon) in the sidebar.
|
||||
- Select **Loki** from the datasource dropdown to search logs.
|
||||
- Select **Prometheus** from the datasource dropdown to query metrics.
|
||||
|
||||
### Viewing Logs (Loki)
|
||||
|
||||
In the **Explore** view with **Loki** selected:
|
||||
|
||||
1. Click **Label browser**.
|
||||
2. Select a label, e.g., `container`.
|
||||
3. Choose a specific container (e.g., `auth-service` or `traefik`).
|
||||
4. Click **Show logs**.
|
||||
|
||||
You can also write LogQL queries manually, for example:
|
||||
|
||||
```logql
|
||||
{container="auth-service"}
|
||||
```
|
||||
|
||||
### Viewing Metrics (Prometheus)
|
||||
|
||||
In the **Explore** view with **Prometheus** selected:
|
||||
|
||||
1. Type a metric name in the query field (e.g., `up`, `container_memory_usage_bytes`).
|
||||
2. Click **Run query**.
|
||||
|
||||
## Configuration
|
||||
|
||||
- **Prometheus**: Rules and targets are configured in `infra/observability/prometheus/prometheus.yml`.
|
||||
- **Promtail**: Log scraping rules are configured in `infra/observability/promtail/promtail-config.yml`.
|
||||
- **Grafana**: Datasources and dashboards provisioning are in `infra/observability/grafana/`.
|
||||
Reference in New Issue
Block a user