Servers and HypervisorsUTMStack Self-Monitoring

UTMStack Self-Monitoring

The UTMStack self-monitoring integration sends UTMStack's own logs back into UTMStack v12. A small collector installed on the UTMStack server reads the logs of the platform's Docker containers and ships them in, so you can search and watch your SIEM's own activity like any other source.

Connection details

SettingValue
Card on the Integrations pageUtmstack (category SIEM)
Data typeutmstack
Service on the UTMStack serverUTMStackCollector
Install folder/opt/utmstack-collector
Collector log file/opt/utmstack-collector/logs/utmstack_collector.log
Parsing pipelineutmstack/utmstack.yaml (version 1.0.2). The Filters button in the drawer opens it.
Detection rulesNone. No shipped rule uses the utmstack data type.

Before you begin

This collector runs on the UTMStack server only. It can only be installed on a UTMStack instance running Ubuntu or a Red Hat family distribution. Install it directly on the UTMStack server, not on another host.

  • You need root rights (sudo) on the UTMStack server.

  • You need the install command from your own UTMStack instance. It contains your connection key, which only administrators can see.

Install the collector

  1. 1

    Open the Utmstack card

    In UTMStack, go to Configure > Integrations and open the Utmstack card.

    UTMStack self-monitoring setup guide, showing How it works, the Runs on the UTMStack host only warning and the Install the collector command with the Ubuntu and CentOS / Red Hat choices
  2. 2

    Choose the operating system

    Under Install the collector, pick the operating system of your UTMStack server: Ubuntu (the command uses apt) or CentOS / Red Hat (the command uses dnf).

  3. 3

    Run the command on the UTMStack server

    Copy the command and run it on the UTMStack server. Treat it like a password: it contains a sensitive connection token.

For Ubuntu, the command looks like this:

sudo bash -c "
  apt update -y && apt install wget -y && 
  mkdir -p /opt/utmstack-collector && 
  wget --no-check-certificate -P /opt/utmstack-collector 
    https://<server>/private/dependencies/collector/utmstack_collector && 
  chmod 755 /opt/utmstack-collector/utmstack_collector && 
  /opt/utmstack-collector/utmstack_collector install <server> <connection-key> yes
"

On CentOS / Red Hat, the command starts with dnf update -y, which installs every pending package update on the UTMStack server, not just a package list refresh as apt update does on Ubuntu. If you do not want that, remove dnf update -y && from the command before you run it; dnf install wget -y alone is enough.

The install prints "Checking server connection", "Downloading Version", "Configuring collector" and "Creating service", and ends with "UTMStackCollector service installed correctly".

What it collects

  • The collector reads the logs that the UTMStack Docker containers on the server write from the moment it starts. It does not read older container logs.

  • It skips containers whose names contain any of these words: postgres, logstash, mutate, node1, filebrowser, user-auditor, web-pdf, frontend.

  • It looks for new containers every 5 minutes and follows containers that Docker replaces.

  • Each log line is cut at 32,768 characters. JSON log lines get a containerName field with the name of the container they came from.

  • Every event is stored under the utmstack data type, with the UTMStack server's host name as its data source.

The pipeline reads the platform's structured JSON lines (those that contain both msg and args) into fields under log., such as log.msg. For other lines it keeps the text in log.message.

Check that logs arrive

  1. On the UTMStack server, check the service and its log:

    sudo systemctl status UTMStackCollector
    sudo tail -n 50 /opt/utmstack-collector/logs/utmstack_collector.log

    A working collector logs "Connected to Docker daemon successfully", "Discovered N containers" and "Starting log stream for container: …" for each container it follows.

  2. In the Utmstack drawer, click View logs. Log Explorer opens on the logs data set with the utmstack data type for the last 24 hours. You can also open Log Explorer and choose the data type yourself.

  3. Within about 20 minutes, the UTMStack server appears on Configure > Data Sources under Direct, with the data type utmstack.

You can also check from Log Explorer in SQL mode:

SELECT `@timestamp`, dataSource, raw
FROM logs
WHERE dataType = 'utmstack'
  AND `@timestamp` > now() - INTERVAL 1 HOUR
ORDER BY `@timestamp` DESC
LIMIT 20

Uninstall the collector

Expand Uninstall the collector in the Utmstack card, or run this on the UTMStack server:

sudo bash -c "
  /opt/utmstack-collector/utmstack_collector uninstall && 
  (systemctl stop UTMStackCollector 2>/dev/null || true) && 
  (systemctl disable UTMStackCollector 2>/dev/null || true) && 
  rm -rf /opt/utmstack-collector && 
  rm -f /etc/systemd/system/UTMStackCollector.service && 
  (systemctl daemon-reload 2>/dev/null || true)
"

The collector's own uninstall removes its registration in UTMStack and stops and removes the UTMStackCollector service. It prints "UTMStackCollector service uninstalled correctly".

The first command can end with an error status even when it worked, and then the rest of the line does not run. Check that /opt/utmstack-collector is gone. If it is still there, run the clean-up yourself:

sudo rm -rf /opt/utmstack-collector
sudo rm -f /etc/systemd/system/UTMStackCollector.service
sudo systemctl daemon-reload

Troubleshooting

Symptom or log messageLikely causeWhat to do
The install stops at "Error trying to connect to server"The server address in the command does not answer on 443 from the server itselfCheck that the UTMStack server can reach its own address on 443 (for example a firewall or DNS name that only works from outside).
The install says "UTMStackCollector service is already installed"An earlier install is still thereUninstall it first (see above), then run the install command again.
"failed to connect to Docker" or "failed to create Docker client"The collector cannot reach the Docker daemon on the serverCheck that Docker is running (sudo systemctl status docker) and restart the collector with sudo systemctl restart UTMStackCollector.
A container you expect has no logsIts name contains one of the skipped words, or it started less than 5 minutes agoCheck the list in "What it collects". New containers are picked up within 5 minutes.
The server is not listed on the Data Sources pageSources sent through a collector appear only after events arrive, within about 20 minutesWait 20 minutes after the first events, then click Refresh.