UTMStack Forwarder
The UTMStack Forwarder is a Linux service that receives logs from your network devices and other systems and forwards them to UTMStack v12 in real time. The web interface calls it the Collector. One Collector can serve many integrations at the same time, so you usually install it once and then enable each integration on it.
In UTMStack v12 the agent no longer receives syslog. Every integration that sends syslog, NetFlow or webhooks (firewalls, switches, VMware ESXi, antivirus consoles, Suricata, Oracle, GitHub and others) goes through a Forwarder.
| Name you will see | Where |
|---|---|
| Collector | The web interface ("Install the UTMStack Collector", "Add Collector", the Collector list) |
utmstack_forwarder | The program on the host, in /opt/utmstack-forwarder |
UTMStackForwarder | The service on the host |
How it works
A device sends its logs to the Forwarder host, on the port of its integration. Syslog (a standard way for devices to send log lines over the network) is the most common format.
The Forwarder tags every log with the integration's data type (for example
firewall-fortigate-traffic) and with the address of the device that sent it.It streams the logs to the UTMStack server over port 443, on an encrypted connection.
UTMStack parses each log with the pipeline for that data type, runs the detection rules and stores the result. See How Data Reaches UTMStack.
| Listener | Protocols | Typical use |
|---|---|---|
| Syslog | UDP, TCP, TLS | Firewalls, switches, routers, hypervisors, security consoles |
| NetFlow | UDP (NetFlow v5, v9 and IPFIX) | Network traffic records |
| HTTP / HTTPS | HTTP, HTTPS | Webhooks and applications that post logs |
The Forwarder does not parse logs and does not store them on disk. It keeps up to 10,000 logs in memory while it sends them.
The Forwarder does not buffer logs on disk. If the UTMStack server cannot be reached, logs that arrive during the outage are lost. Keep the connection between the Forwarder and UTMStack reliable.
Requirements
A Linux host with an x86-64 (AMD64) processor. UTMStack v12 publishes the Forwarder for AMD64 only.
The install command in the product uses
apt, so it is written for Debian or Ubuntu.A direct connection from the host to your UTMStack server on 443/TCP. The install command checks this first. The Forwarder has no proxy setting.
Inbound access from your devices on the ports of the integrations you enable. See Ports and Network Requirements.
Root rights (
sudo) on the host.Only one Forwarder can run on a host: the service name and the folder are fixed.
UTMStack does not publish sizing figures for the Forwarder. For where to install it, how many to install and what to watch, see Forwarder Placement and Sizing.
Where to install it
You can install the Forwarder on the UTMStack server itself or on another Linux host in your network that your devices can reach.
Several device guides in the product (for example FortiGate, Palo Alto, pfSense, SonicWall, VMware ESXi and GitHub) fill in your UTMStack server's address as the place to send logs. That address is right only when the Forwarder runs on the UTMStack server. If your Forwarder runs on another host, use that host's address instead.
Multi-tenant installations
With the multi-tenant licence, a Forwarder belongs to the tenant whose install command you used. UTMStack stores every log it receives in that tenant, whatever device sent it. To collect for several tenants, install one Forwarder per tenant, each on its own host, with the command copied inside that tenant. See Forwarder Placement and Sizing and Multi-Tenancy Overview.
Install the Forwarder
- 1
Find the install command
In UTMStack, go to Configure > Integrations and open any card that uses the Collector, for example Cisco. Expand Install the UTMStack Collector near the bottom of the guide. If no Collector exists yet, you can also choose Add Collector in the Collector list, which opens the same section.
- 2
Copy the command
The command contains your server address and connection key. Only administrators see the key; other users see
*******, and a command copied with*******fails to register. - 3
Run it on the Forwarder host
Paste the command into a terminal on the host and press Enter.
- 4
Check that it is online
Open any device guide again. The new Collector appears in the Collector list with its host name and IP address, and the count of online Collectors goes up by one.
The command looks like this:
sudo bash -c "
apt update -y && apt install wget -y &&
mkdir -p /opt/utmstack-forwarder &&
wget --no-check-certificate -P /opt/utmstack-forwarder
https://<server>/private/dependencies/collector/forwarder/utmstack_forwarder &&
chmod 755 /opt/utmstack-forwarder/utmstack_forwarder &&
/opt/utmstack-forwarder/utmstack_forwarder install <server> <connection-key> yes
"The last argument is "skip TLS certificate validation": yes accepts the server's self-signed certificate, no requires a certificate the host trusts.
A successful run prints Checking server connection ... [OK], Downloading version info ... [OK], Configuring collector ... [OK], Creating service ... [OK] and UTMStackForwarder service installed correctly. If a step fails, see Forwarder Troubleshooting.
Lock the Forwarder against remote changes (optional)
If you add --no-remote-control at the end of the install line, the Forwarder ignores every configuration change sent from the web interface. You then manage it only with the command line on the host. The guide then says: "This forwarder was installed with remote control disabled…". This can only be undone by reinstalling.
Enable an integration
Each device page explains its own steps. The common part is to enable the integration on a Collector, either from the web interface or from the command line on the Forwarder host. Changes apply at once; the service does not need a restart.
From the web interface
Step 1 of every device guide is Enable this integration remotely: "Pick a Collector, choose the protocol and port, then enable the integration — no SSH needed."
| Field | What to enter |
|---|---|
| Collector | The Forwarder that should listen. The list shows how many Collectors are online, offline and in total, and ends with Add Collector. It lists only the Collectors of your current tenant. |
| Protocol | UDP, TCP or TLS for syslog sources; UDP only for NetFlow; HTTP or HTTPS for webhooks. |
| Port | Filled in with the default port. For built-in integrations, leave it at the port the Forwarder uses (see "Use a different port" below). |
| Authentication, Path, Signature header | Only for HTTP and HTTPS listeners. |
| TLS certificate | Only for TLS and HTTPS. Shows whether a certificate is already loaded, and lets you upload Certificate, Private key and an optional CA certificate with Upload certificate. |
Click Enable integration. The line under the fields shows the result, for example Currently: enabled · udp · port 514. To stop the listener, choose the same protocol you enabled and click Disable integration.
While an integration is enabled on the selected Collector, its protocol, port and other fields are locked. Disable it first to change them.
You need the
integrations.writepermission. Without it the panel says: "You don't have permission to manage remote integrations…".An offline Collector cannot receive changes: "This Collector is offline and can't receive configuration changes right now."
If the Forwarder refuses the change, a message explains why, for example
port 514 is already in use by integration 'firewall-cisco-asa'. See Forwarder Troubleshooting.
The Currently line shows the last setting sent from the web interface, including the port typed in the panel. It is not read back from the Forwarder, so it does not show the port the Forwarder really listens on. Changes you make with the command line on the host do not appear there.
From the command line
Run the commands on the Forwarder host with root rights. For a built-in integration you only need the data type and the protocol:
sudo /opt/utmstack-forwarder/utmstack_forwarder enable-integration firewall-fortigate-traffic udpThe command prints the port it actually opened, for example Integration "firewall-fortigate-traffic" enabled on udp port 7005.
| Command | What it does |
|---|---|
enable-integration <name> [port] <tcp|udp|tls|http|https> [--auth bearer|hmac] [--signature-header <header>] | Starts a listener. For a built-in data type the port comes from the Forwarder's settings. A name that is not a built-in data type creates a new custom data type ("New data type … created."); then the port is required. |
disable-integration <name> <protocol> | Stops a listener. Use the protocol you enabled. |
change-port <name> <protocol> <port> | Changes the port a data type uses. |
load-tls-certs <cert> <key> [ca] | Loads the certificate used by TLS and HTTPS listeners. |
list-datatypes | Lists built-in and custom data types (columns NAME, KIND and ORIGIN). |
remove-datatype <name> | Deletes a custom data type. Disable it first. Built-in types cannot be removed. |
uninstall | Stops and removes the service. |
Use a different port
For built-in data types, the Forwarder always listens on the port saved in its own settings file. It is the default port unless you change it on the host. The port typed in the panel, or given to enable-integration, does not change it. To move a built-in integration to another port:
- 1
Change the port on the Forwarder host
sudo /opt/utmstack-forwarder/utmstack_forwarder change-port cisco-switch udp 5140The command prints
Port changed from 514 to 5140 for integration "cisco-switch" (udp). - 2
Enable the integration
Enable it in the panel and type the same port (5140), so the page shows the port that is really in use. Or run
enable-integration cisco-switch udpon the host. - 3
Point the device at the new port
Change the device's syslog destination to the new port.
You need this when two integrations share a default port. The four Cisco integrations all use 514/UDP and 1470/TCP, and only one data type can listen on a given port and protocol of one Forwarder.
Use TLS
TLS encrypts syslog between the device and the Forwarder. It uses the TCP port of the integration and accepts TLS 1.2 and 1.3 only.
Get a certificate and private key for the Forwarder host, in PEM format. Use one your devices trust. For a test, you can make a self-signed one on the host:
sudo openssl req -x509 -newkey rsa:2048 -nodes -days 825 -keyout /root/forwarder.key -out /root/forwarder.crt -subj "/CN=forwarder.example.com" -addext "subjectAltName=DNS:forwarder.example.com,IP:10.0.0.20"Replace the name and address with the Forwarder host's own. Devices that check certificates must trust it, so import
forwarder.crton them.Load the certificate on the Forwarder, once per Collector. Either upload it in the TLS certificate box of the web interface, or run:
sudo /opt/utmstack-forwarder/utmstack_forwarder load-tls-certs /root/forwarder.crt /root/forwarder.keyYou can add a CA certificate as a third argument. Without it, the certificate itself is used as the CA. The files are copied to
certs/integration.crt,certs/integration.keyandcerts/integration_ca.crtin the Forwarder folder. The Forwarder sends exactly what is in the certificate file, so if your certificate comes from a certificate authority, put the full chain in that file.Enable the integration with the protocol TLS (
tlson the command line).Configure the device to send syslog over TLS to the Forwarder's TCP port.
A TCP listener without TLS refuses TLS connections, and a TLS listener refuses plain connections. Both are logged in the Forwarder's log file.
HTTP and HTTPS listeners
HTTP and HTTPS listeners accept logs posted by applications and webhooks. The body can be in any format (JSON, plain text or anything else) up to 8 MB per request. It is passed to UTMStack as it is.
| Setting | Built-in github type | New custom types |
|---|---|---|
| Listens on | All addresses | 127.0.0.1 only (the Forwarder host itself) |
| Path | /github | /logs |
| Authentication | HMAC signature in X-Hub-Signature-256 | None, unless you choose one |
Bearer token: the sender adds
Authorization: Bearer <token>.HMAC signature: the sender signs each request with a shared secret and puts the signature in the signature header. GitHub, for example, works this way.
When you enable a listener with a bearer token or HMAC, the Forwarder creates a secret token and shows it once ("Store this token securely — it won't be shown again."). It is kept in certs/integration-http-<name>.token in the Forwarder folder.
For sources outside the Forwarder host, use syslog (UDP, TCP or TLS) or the GitHub listener. For your own sources, see Custom Integrations.
Message size and format
| Path | What the Forwarder accepts |
|---|---|
| Syslog over UDP | One message per packet. The Forwarder reads at most 2,048 bytes of each message; the rest is cut off. Use TCP or TLS for sources that send long messages. |
| Syslog over TCP or TLS, newline framing | One message per line. Lines normally start with the syslog priority, for example <134>. A line that starts with a digit is read as a length prefix (see the next row), so do not send plain lines that start with a date or an address. |
| Syslog over TCP or TLS, octet counting | <length> <message>, as defined in RFC 5425. Messages up to 8,192 bytes. |
| HTTP or HTTPS | One request body per log, up to 8 MB. |
| NetFlow | NetFlow v5, v9 and IPFIX over UDP. v9 and IPFIX records are decoded once the exporter has sent its template. |
Built-in data types and default ports
| Data type | Source | UDP | TCP / TLS |
|---|---|---|---|
firewall-cisco-asa | Cisco ASA | 514 | 1470 |
firewall-cisco-firepower | Cisco Firepower | 514 | 1470 |
firewall-meraki | Cisco Meraki | 514 | 1470 |
cisco-switch | Cisco Switch | 514 | 1470 |
vmware-esxi | VMware ESXi | 7002 | 7002 |
antivirus-esmc-eset | ESET PROTECT | 7003 | 7003 |
antivirus-kaspersky | Kaspersky Security Center | 7004 | 7004 |
firewall-fortigate-traffic | Fortinet FortiGate | 7005 | 7005 |
firewall-paloalto | Palo Alto Networks | 7006 | 7006 |
firewall-mikrotik | MikroTik | 7007 | 7007 |
firewall-sophos-xg | Sophos Firewall (XG) | 7008 | 7008 |
firewall-sonicwall | SonicWall | 7009 | 7009 |
deceptive-bytes | Deceptive Bytes | 7010 | 7010 |
antivirus-sentinel-one | SentinelOne | 7012 | 7012 |
syslog | Any device that sends standard syslog, CEF or LEEF (no card of its own; see Generic Syslog, CEF and LEEF) | 7014 | 7014 |
ibm-aix | IBM AIX | 7016 | 7016 |
firewall-pfsense | pfSense | 7017 | 7017 |
firewall-fortiweb | Fortinet FortiWeb | 7018 | 7018 |
suricata | Suricata | 7019 | 7019 |
oracle | Oracle Database | 7021 | 7021 |
netflow | NetFlow exporters | 2055 | — |
github | GitHub webhooks | — | 7020 (HTTPS) |
Files on the Forwarder host
All files are in /opt/utmstack-forwarder.
| File | What it holds |
|---|---|
collector-config.yml | The connection to UTMStack (encrypted). |
collector-uuid.yml | The Forwarder's unique installation ID. |
log-collector-config.json | The integrations, their ports and protocols, and whether each is enabled. The Forwarder watches this file and applies changes at once. |
log-collector-catalog.json | Your custom data types. |
certs/ | The TLS certificate and key, and the tokens of HTTP and HTTPS listeners. |
logs/utmstack_forwarder.log | The Forwarder's own log file. Check it first when something does not work. |
Manage the service
sudo systemctl status UTMStackForwarder
sudo systemctl restart UTMStackForwarder
sudo tail -f /opt/utmstack-forwarder/logs/utmstack_forwarder.logWhen a listener starts, the log file shows a line such as Server firewall-fortigate-traffic listening in port: 7005 protocol: UDP. When the connection to UTMStack is up, it shows Log stream connected.
Update or reinstall the Forwarder
The Forwarder does not update itself. To install a newer build, reinstall it:
Write down the integrations that are enabled on it, with their protocols and ports (
list-datatypeslists the data types; the device guides show the Currently line for each).Keep a copy of your certificate and key if you use TLS or HTTPS.
Run the uninstall command (below), then the install command from the product.
Load the certificate again, and run
change-portagain for any port you had changed. When the reinstalled Forwarder keeps the same entry (see below), it applies the settings last sent from the web interface again as soon as it connects; settings you made on the command line must be made again. If a listener did not come back (for example a TLS or HTTPS one, which needs the certificate first), click Disable integration and then Enable integration in its guide.
Reinstalling on the same host, with the same host name and IP address, keeps the same entry in the Collector list. See Forwarder Troubleshooting if the install stops with "hostname has already been registered".
Uninstall the Forwarder
Expand Optional — Uninstall the Collector in any device guide, or run:
sudo bash -c "
/opt/utmstack-forwarder/utmstack_forwarder uninstall &&
rm -rf /opt/utmstack-forwarder
"This stops the service and deletes all its files, including certificates, tokens and integration settings. The Forwarder's entry stays in the Collector list and shows as offline.
Related pages
Forwarder Placement and Sizing: where to install Forwarders, how many, and high availability.
Forwarder Troubleshooting: no data, wrong integration, parsing problems and error messages.
Generic Syslog, CEF and LEEF: devices without a card of their own.
Ports and Network Requirements: every port in one place.