I am trying to setup Prometheus as a service on Linux to monitor external Linux nodes. However I design the systemd service (on my monitor node) seems to fail. My OS is Ubuntu 20.04.3 LTS and I’m runing Prometheus version prometheus-2.31.1.linux-amd64.
My systemd service is setup like this:
[Unit]
Description=Prometheus Server
Wants=network-online.target
After=network-online.target
Requires=prometheus.service
[Service]
User=usr
Group=usr
Type=simple
ExecStart= /home/usr/Downloads/prometheus-2.31.1.linux-amd64/prometheus --config.file=/home/usr/Downloads/prometheus-2.31.1.linux-amd64/prometheus.yml
[Install]
WantedBy=multi-user.target
When I try to run the service manually I get the following error:
prometheus.service - Prometheus Server
Loaded: loaded (/etc/systemd/system/prometheus.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sun 2022-01-02 18:50:38 CST; 17min ago
Main PID: 1411 (code=exited, status=2)
Jan 02 18:50:38 monitor prometheus[1411]: ts=2022-01-03T00:50:38.975Z caller=query_logger.go:109 level=error component=activeQueryTracker msg="Failed to create directory for logging active queries"
Jan 02 18:50:38 monitor prometheus[1411]: ts=2022-01-03T00:50:38.975Z caller=query_logger.go:87 level=error component=activeQueryTracker msg="Error opening query log file" file=data/queries.active err="open data/queries.active: no such file or directory"
Jan 02 18:50:38 monitor prometheus[1411]: panic: Unable to create mmap-ed active query log
Jan 02 18:50:38 monitor prometheus[1411]: goroutine 1 [running]:
Jan 02 18:50:38 monitor prometheus[1411]: github.com/prometheus/prometheus/promql.NewActiveQueryTracker({0x2bda6f0, 0x5}, 0x14, {0x34442c0, 0xc0003f2320})
Jan 02 18:50:38 monitor prometheus[1411]: /app/promql/query_logger.go:117 +0x3d7
Jan 02 18:50:38 monitor prometheus[1411]: main.main()
Jan 02 18:50:38 monitor prometheus[1411]: /app/cmd/prometheus/main.go:491 +0x6bbf
Jan 02 18:50:38 monitor systemd[1]: prometheus.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jan 02 18:50:38 monitor systemd[1]: prometheus.service: Failed with result 'exit-code'.
Can anyone help me to understand where I am going wrong? The user has permissions to execute all the files and I have tried adding new user profiles just for the service with little success. Any help on better diagnosing the issue or possible fixes would be greatly appreciated.