Add Promtail service to Docker Compose and update Loki configuration

- Introduced a new Promtail service in `docker-compose.observability.yml` for log collection.
- Configured Promtail with necessary volumes and command for log processing.
- Updated `loki-config.yml` to include `allow_structured_metadata` setting for enhanced metadata handling.
This commit is contained in:
Ho Ngoc Hai
2025-12-27 10:47:35 +07:00
parent 5ff8035013
commit d92cfb3bf3
8 changed files with 268 additions and 3 deletions

View File

@@ -0,0 +1,89 @@
# Observability Stack Guide
This guide explains how to use the observability stack (Grafana, Prometheus, Loki, Promtail) included in the infrastructure.
## Architecture Overview
The stack consists of the following components:
- **Prometheus**: Collects metrics from services.
- **Loki**: Collects logs.
- **Promtail**: Scrapes logs from Docker containers and pushes them to Loki.
- **Grafana**: Visualization dashboard for metrics (from Prometheus) and logs (from Loki).
## Getting Started
### Prerequisites
- Docker and Docker Compose installed.
- Existing `microservices-network` (created by the main application stack or manually).
### Starting the Stack
You can easily start the stack using the provided script:
```bash
./scripts/observability/start.sh
```
Or manually:
```bash
# Ensure network exists
docker network create microservices-network || true
cd infra/observability
docker-compose -f docker-compose.observability.yml up -d
```
Check if all containers are running:
```bash
docker ps
```
You should see `grafana`, `prometheus`, `loki`, and `promtail`.
## Accessing Services
| Service | URL | Credentials (if applicable) | Description |
| :--- | :--- | :--- | :--- |
| **Grafana** | [http://localhost:3001](http://localhost:3001) | `admin` / `admin` | Main dashboard for visualization. |
| **Prometheus** | [http://localhost:9090](http://localhost:9090) | N/A | Raw metrics and target status. |
| **Loki** | [http://localhost:3100](http://localhost:3100) | N/A | Log aggregation API (no UI). |
## Using Grafana
1. **Login**: Access [http://localhost:3001](http://localhost:3001) and login with `admin`/`admin`.
2. **Explore Data**:
- Go to **Explore** (compass icon) in the sidebar.
- Select **Loki** from the datasource dropdown to search logs.
- Select **Prometheus** from the datasource dropdown to query metrics.
### Viewing Logs (Loki)
In the **Explore** view with **Loki** selected:
1. Click **Label browser**.
2. Select a label, e.g., `container`.
3. Choose a specific container (e.g., `auth-service` or `traefik`).
4. Click **Show logs**.
You can also write LogQL queries manually, for example:
```logql
{container="auth-service"}
```
### Viewing Metrics (Prometheus)
In the **Explore** view with **Prometheus** selected:
1. Type a metric name in the query field (e.g., `up`, `container_memory_usage_bytes`).
2. Click **Run query**.
## Configuration
- **Prometheus**: Rules and targets are configured in `infra/observability/prometheus/prometheus.yml`.
- **Promtail**: Log scraping rules are configured in `infra/observability/promtail/promtail-config.yml`.
- **Grafana**: Datasources and dashboards provisioning are in `infra/observability/grafana/`.