This article presents how to integrate Aviatrix CoPilot with Prometheus and Grafana.
The design provided here guides you on how to create a centralized monitoring solution for your environment. Having that allows you to gain full visibility of your Aviatrix and non-Aviatrix environments/applications integrated into one monitoring system to quickly identify the health (errors, bottlenecks, etc.) of all of your resources. What is more, it gives you the visibility of your entire environment.
What is Prometheus
Prometheus is an open-source monitoring and alerting system. Prometheus pulls the metrics from different Targets (API calls). Those Targets are specified in the Prometheus configuration file. Prometheus will initiate the connection with all the Targets and scrape the metrics at specific intervals. In our case, Aviatrix CoPilot will be the Target.
According to https://prometheus.io/docs/introduction/overview/ “Prometheus collects and stores its metrics as time series data, i.e. metrics information is stored with the timestamp at which it was recorded, alongside optional key-value pairs called labels”.
One important note is that the Prometheus job is querying the data and storing it. It is not for visualizing anything. This is where Grafana plays its role.
Prometheus has its Alertmanager that can be set up to send the Alerts (Email Notifications) based on specific events. However, I will not dig into that in this article.
What is Grafana
Grafana ( https://grafana.com/ ) is an open-source tool (WebUI) that provides interactive data/metrics analysis and visualization. The user can create fully customizable dashboards and share them with other people. Grafana is using PromQL to query the metrics from the Prometheus server.
Aviatrix integration with Prometheus and Grafana
The diagram below presents the integration between Aviatrix CoPilot, Prometheus, and Grafana (image taken from https://prometheus.io/docs/introduction/overview/ and modified by me).
Aviatrix CoPilot Metrics API
The Aviatrix CoPilot Metrics API (https://docs.aviatrix.com/copilot/latest/monitoring-troubleshooting/metrics-api-enable.html) allows you to retrieve metrics for your Aviatrix Controller and Aviatrix Gateways. There are two types of Collections you can use:
- Status API
- Metrics API
Please follow the steps below to activate Metrics API and Download the Collections:
Go to CoPilot > Settings > Configuration > License and confirm the CoPilot API Access under Add-on Features is enabled
Go to CoPilot > Settings > Configuration > General and Reset the API Key if needed
Download the Metrics and Status API Collections
Deploying Prometheus and Grafana containers
In my Lab, I am running the Prometheus and Grafana as Docker containers. To deploy them you must have the proper configuration files. The following screenshot presents what the structure of files (for both Prometheus and Grafana) looks like.
Content of the compose.yaml file:
services: prometheus: image: prom/prometheus container_name: prometheus command: - '--config.file=/etc/prometheus/prometheus/prometheus.yml' ports: - 9090:9090 restart: unless-stopped volumes: - ./prometheus:/etc/prometheus/prometheus - prom_data:/prometheus/prometheus grafana: image: grafana/grafana container_name: grafana ports: - 3000:3000 restart: unless-stopped environment: - GF_SECURITY_ADMIN_USER=admin - GF_SECURITY_ADMIN_PASSWORD=grafana volumes: - ./grafana:/etc/grafana/provisioning/datasources volumes: prom_data:
Content of the prometheus.yml file:
global: scrape_interval: 1m scrape_timeout: 10s evaluation_interval: 1m scrape_configs: - job_name: jakub-copilot honor_labels: true honor_timestamps: false scrape_interval: 30s scrape_timeout: 10s metrics_path: /metrics-api/v1/gateways scheme: https authorization: type: Bearer credentials: tls_config: insecure_skip_verify: true follow_redirects: true enable_http2: true static_configs: - targets: - - job_name: jakub-copilot-status honor_labels: true honor_timestamps: false scrape_interval: 30s scrape_timeout: 10s metrics_path: /status-api/v1 scheme: https authorization: type: Bearer credentials: tls_config: insecure_skip_verify: true follow_redirects: true enable_http2: true static_configs: - targets: -
Content of the datasource.yml file:
apiVersion: 1 datasources: - name: Prometheus type: prometheus url: http://prometheus:9090 isDefault: true access: proxy editable: true
My containers are presented below. As you can see the Prometheus can be accessed via port 9090, while Grafana via 3000.
Go to http://<your-server-ip>:9090/targets?search= to verify that Prometheus can connect to the Target(s).
Now you must set up Grafana to connect to Prometheus to request the metrics. Prometheus will be a Data Source for Grafana.
- Go to Home > Connections > Data Sources
- Choose “+ Add a new data source“.
- Click on “Prometheus“
- Create a new connection. Provide the name and the URL. Your URL will be: http://<your-server-ip>:9090
- Click on “Save & test” once done
- You are ready to create your dashboard. Go to Home > Dashboards and select “New” and “New Dashboard“
Supported Aviatrix Metrics:
The sample dashboard I created for my Aviatrix environment: