preloader

Send Metrics From Proxmox to InfluxDB2 Using Telegraf

  • Thursday, Jul 23, 2020
blog-image

Enabling InfluxDB Metrics in Proxmox

First, we need to configure Proxmox to send InfluxDB metrics somewhere. To do that, we need to SSH into our Proxmox node and add the following lines to /etc/pve/status.cfg by running nano /etc/pve/status.cfg

influxdb: shift-rmm
   server 127.0.0.1
   port 8089

This will start sending metrics via UDP to localhost on port 8089. If you are using InfluxDB1 without authentication and don’t care about security, you can change the server from 127.0.0.1 to the IP of your InfluxDB server and ignore the rest of this article.

Installing Telegraf

Now that we are sending metrics to localhost, we need something to collect those metrics and ship them off to InfluxDB2. First, we need to allow packages signed by InfluxData to be installed on Proxmox. To do this, run wget -qO- https://repos.influxdata.com/influxdb.key | apt-key add - to add their signing key. To add the InfluxData repository, run echo "deb https://repos.influxdata.com/debian buster stable" | tee /etc/apt/sources.list.d/influxdb.list. Now that their repository is added, we can refresh our repositories and install Telegraf by running apt update && apt install telegraf.

Configuring and Testing Telegraf

Since Telegraf is now on our system, we need to tell it what to collect and where to get it. The default config for Telegraf is really handy since it provides examples of all the things Telegraf can do and is worth looking through. In this case, we just want to forward the metrics from Proxmox. Delete it by typing rm /etc/telegraf/telegraf.conf. To add in the new config, run nano /etc/telegraf/telegraf.conf. Paste in the text below and replace the bucket, domain, organization, and URLs with the values for your instance. If you are not sure how or were to get a token, refer to the InfluxDB docs

[agent]
  interval = "10s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = ""
  hostname = ""
  omit_hostname = false

# Configuration for sending metrics to InfluxDB
[[outputs.influxdb_v2]]
  urls = ["https://influxdb.example.com:9999"]
  token = "GET_YOUR_OWN_TOKEN_YOU_ROTTEN_KIDS"
  organization = "muh org"
  bucket = "proxmox"

# Gather metrics from proxmox based on what is in /etc/pve/setup.cfg
[[inputs.socket_listener]]
  service_address = "udp://:8089"

Once that is done, you can test out your config by running telegraf --debug. If everything is going well, you should see something like the output below after 10 or seconds. If you see that zero metrics are being shipped, check that the port in the setup.cfg file and the telegraf.conf file are the same. You can kill the process by pressing ctrl + c

2020-07-24T04:45:49Z D! [agent] Initializing plugins
2020-07-24T04:45:49Z D! [agent] Connecting outputs
2020-07-24T04:45:49Z D! [agent] Attempting connection to [outputs.influxdb_v2]
2020-07-24T04:45:49Z D! [agent] Successfully connected to outputs.influxdb_v2
2020-07-24T04:45:49Z D! [agent] Starting service inputs
2020-07-24T04:45:49Z I! [inputs.socket_listener] Listening on udp://[::]:8089
2020-07-24T04:46:00Z D! [outputs.influxdb_v2] Wrote batch of 25 metrics in 67.17539ms
2020-07-24T04:46:00Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics

Enabling Telegraf

Starting and enabling Telegraf to start at boot simply requires starting and enabling the Telegraf service systemctl enable --now telegraf

But Why Do I Have to Use Telegraf?

You can send Proxmox metrics directly to an InfluxDB1 instance, but you cannot send metrics to an InfluxDB2 instance over UDP. Even if you could, it would not be a great idea to send info such as which VMs are running where over plain text UDP. Piping the metrics through Telegraf allows for sending them over a secured TLS connection. You could also use Telegraf as a way to send the same metrics to one of the many Telegraf output plugins.

How can I setup InfluxDB2?

I am glad you asked, Reader! You can setup InfluxDB2 as well as Meshcentral by setting up Shift RMM. If you are just looking for a place to send all your metrics and manage all your boxes via a web interface, please Contact Us. We are more than happy to setup Shift RMM on your infrastructure or manage it on the cloud provider of your choice.

Sources