Prometheus integration with Grafana

Steps to configure a Grafana server with a Prometheus data source in a simple Docker Compose configuration

6 min readAug 14, 2023

Introduction

Grafana is an open-source data visualization and monitoring tool that allows users to create, customize, and display interactive dashboards for analyzing and tracking various metrics and data sources in real-time. It supports integration with a wide range of data storage systems, such as databases, time-series databases, cloud services, and APIs, enabling users to collect and display data from diverse sources. Grafana’s user-friendly interface and extensive library of visualization options make it popular for monitoring and analyzing performance, infrastructure, and application metrics, making complex data more understandable and actionable through visually appealing graphs, charts, and alerts.

Grafana finds widespread usage among DevOps and Site Reliability Engineering (SRE) experts within Cloud Native ecosystems, often working in tandem with Prometheus. Prometheus gathers metrics from diverse origins and presents them to Grafana for visualization and analysis.

In this post, we will build upon our previous guide for configuring Prometheus through Docker Compose. This time, we will expand our setup by incorporating a Grafana server. This Grafana instance will be linked to our Prometheus server and pre-populated with ready-to-use dashboards. All code samples can be found here and it is based on the book Prometheus: Up & Running, 2nd Edition.

Pre-requisites

Consult the official Docker documentation to set up Docker and Docker Compose on your system.

Step 1: Setup Prometheus with Node Exporter

In this step, we will proceed with a straightforward configuration of the Prometheus container with Node Exporter. For additional insights, refer to this guide.

Prometheus setup

The prometheus.yml configuration file including the Node Exporter job should look like this:

global:
  scrape_interval: 10s
scrape_configs:
 - job_name: prometheus
   static_configs:
    - targets:
       - prometheus:9090
 - job_name: node
   static_configs:
    - targets:
       - node-exporter:9100

Docker Compose setup

The docker-compose.yaml file configuration with the container’s and network setup should look like this:

version: '3'
services:
  prometheus:
    image: prom/prometheus
    volumes:
      - "./prometheus.yml:/etc/prometheus/prometheus.yml"
    networks:
      - localprom
    ports:
      - 9090:9090
  node-exporter:
    image: prom/node-exporter
    networks:
      - localprom
    ports:
      - 9100:9100
  grafana:
    image: grafana/grafana
    volumes:
      - ./provisioning:/etc/grafana/provisioning
    networks:
      - localprom
    ports:
      - 3000:3000
networks:
  localprom:
    driver: bridge

Note that we’ve incorporated the Grafana server into the docker-compose.yaml file to initiate a container using the official Grafana image from Docker Hub, which will operate on port 3000. It’s important to mention that we’re also mounting the provisioning directory into /etc/grafana/provisioning. This facilitates the loading of configurations into the container, a process that will be elaborated upon in step 2.

Step 2: Setup Grafana

In this step, we will proceed to configure your Grafana container. This configuration entails the automated addition of the Prometheus server’s data source and a basic dashboard that loads upon container startup, a process known as Provisioning. While provisioning is optional, it eliminates the need for manual intervention each time you redeploy your orchestration. If you want to manually setup your Grafana server, you can refer to the official Grafana documentation for more details.

Provisioning setup

The provisioning configuration files are automatically loaded within your Grafana’s container’s /etc/grafana/provisioning directory. Within this directory, Prometheus will initiate the specified setup. Go ahead and create a directory structure resembling the following:

provisioning/
├─ dashboards/
│  ├─ dashboard.yaml
│  ├─ my-new-dashboard-1.json
│  ├─ my-new-dashboard-2.json
│  ├─ ...
│  ├─ my-new-dashboard-n.json
├─ datasources/
│  ├─ datasources.yaml

Now, we will configure data sources and dashboards used for provisioning.

Data sources

The datasources.yaml file is used to define a list datasourcesof data sources, which are connections to various backend data providers that Grafana can use to fetch and visualize metrics, logs, and other data. Each data source contains

name: Assigns a name to the data source, in this case, Prometheus.
type: Specifies the type of data source, which is set as prometheus to indicate integration with Prometheus.
url: Identifies the URL endpoint where the Prometheus server is accessible. In this example, it's set to http://prometheus:9090, which assumes that the Prometheus server configured in our orchestration.
access: Defines the method of access to the data source, which is proxy here, indicating that Grafana will proxy requests to the data source.
isDefault: Indicates whether this data source should be set as the default. In this snippet, it's set to true, implying that the Prometheus data source is the default choice for visualization unless explicitly specified otherwise.

apiVersion: 1
datasources:
  - name: Prometheus
    type: prometheus
    url: http://prometheus:9090
    access: proxy
    isDefault: true

Dashboards

The dashboards.yaml file is used to define and configure dashboards in Grafana. It contains a list of dashboard providers that are responsible to load dashboards into Grafana. Each provider contains:

name: Specifies a descriptive name for the dashboard provider, in this case, Dashboard provider.
orgId: Specifies the organization ID that the dashboards will be associated with. The default is 1 .
type: Indicates the type of provider, which is filein this case. This means that the dashboards will be sourced from local files.
path: Included in the options section, it denotes the directory path from which dashboards will be loaded. In this example, the dashboards will be loaded from the /etc/grafana/provisioning/dashboards directory.

apiVersion: 1

providers:
  - name: "Dashboard provider"
    orgId: 1
    type: file
    options:
      path: /etc/grafana/provisioning/dashboards

Dashboard JSON files

They are structured configuration files that captures the settings and layout of a dashboard in a machine-readable format. Each file includes details such as the dashboard’s title, unique identifier, panels with visualization types and data sources, as well as organizational information. These files enable users to share, collaborate, and backup dashboard configurations, allowing for consistency across different Grafana instances and simplifying the process of recreating complex dashboards. For this example, we provide some sample dashboard JSON files that you can download from this git repository.

Step 3: Accessing your Grafana server

Once you have completed all the configurations, proceed to start your orchestration:

docker compose up -d

To access Grafana, simply open http://localhost:3000/ in your web browser, where you will be prompted to log in. The default admin account’s username and password are both set as admin. Upon logging in, you will be prompted to update this information, although you skip this step if preferred.

Upon reaching the homepage, you can start exploring the features Grafana provides. In the Grafana’s homepage, you will encounter documentation and learning resources, along with a compilation of available Dashboards.

You can review the list of data sources at http://localhost:3000/connections/datasources. On this page, you will find a pre-configured data source for your Prometheus server. Additionally, you have the option to add new data sources from this location.

Lastly, you can access the comprehensive compilation of dashboards at http://localhost:3000/dashboards. To view a particular dashboard, simply click on its name. Alternatively, you create a new dashboard directly from this page.

If you click on “My First Dashboard,” you should be presented with a dashboard resembling the one depicted here, showcasing data extracted from both the Node Exporter container and your Prometheus server.

For more information on how to use Grafana once it is installed, you can refer to the official documentation.

Conclusion

This guide offers step-by-step instructions for setting up a compact configuration involving Prometheus, Node Exporter, and Grafana using Docker Compose. It's important to note that this setup is designed for developmental and testing purposes, serving as an initial glimpse into the potential for automating your monitoring arrangement. I hope you find this guide to be of value.