AWS CloudWatch
The AWS Cloudwatch integration brings your AWS account activity into UTMStack. AWS CloudTrail records API calls and management events, sends them to a CloudWatch log group, and UTMStack reads that log group with read-only credentials. Nothing is installed in AWS.
Since UTMStack v12.0.2, this integration refers to CloudWatch instead of IAM User. This page describes the v12.0.2 setup.
How it works
CloudTrail captures API calls and management events across your AWS account and forwards them to a CloudWatch log group.
UTMStack connects with the access key of an IAM user (AWS Identity and Access Management, the AWS service that controls who can use which AWS resources). The user only needs read-only access to CloudWatch Logs. UTMStack reads every log stream in the log group you name, and looks for new log streams every 5 minutes.
UTMStack reads events from the moment collection starts. Events that were already in the log group before you saved the config group are not read.
Events are stored with the data type
awsand parsed by the shipped pipelineaws/aws.yaml(version 1.1.2). UTMStack v12 ships 81 detection rules for this data type; see AWS in the Rules tab.The same pipeline also understands Amazon GuardDuty findings that reach the log group through Amazon EventBridge. One of the AWS rules, AWS GuardDuty High or Critical Finding, uses them.
Before you start
In AWS, you need permission to create IAM users and access keys, CloudWatch log groups and CloudTrail trails.
In UTMStack, you need the Administrator role (the
integrations.writepermission). Configuration is saved in the tenant you are working in.The UTMStack server must be able to reach AWS over HTTPS (outbound port 443).
Step 1: Create an IAM user for UTMStack
- 1
Open the IAM dashboard
Open the AWS console, go to the IAM dashboard and select Users.
- 2
Add a user
Click Create user (Add user in older versions of the console), enter a descriptive user name, for example
utmstack-reader, and continue to the next step. The user does not need access to the AWS console. - 3
Attach the read-only policy
Under Permissions, choose Attach policies directly and attach the AWS managed policy CloudWatchLogsReadOnlyAccess. It gives read-only access to CloudWatch Logs.
Optional: a narrower policy instead
UTMStack uses three CloudWatch Logs actions: it lists log groups when you save the config group, then lists the log streams of your log group and reads their events. If your security policy asks for the least access possible, attach this inline policy instead of CloudWatchLogsReadOnlyAccess. Replace the region, the account ID and the log group name with yours.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "logs:DescribeLogGroups", "Resource": "*" }, { "Effect": "Allow", "Action": ["logs:DescribeLogStreams", "logs:GetLogEvents"], "Resource": [ "arn:aws:logs:us-east-1:111122223333:log-group:utmstack", "arn:aws:logs:us-east-1:111122223333:log-group:utmstack:*" ] } ] } - 4
Create the user
Review the settings and click Create user.
- 5
Create an access key
Open the new user, select the Security credentials tab and click Create access key. Choose the option for an application running outside AWS, then confirm. AWS generates an Access Key ID and a Secret Access Key.
- 6
Save the credentials
Download the credentials file or copy both values to a safe place.
AWS shows the Secret Access Key only once.
Step 2: Send CloudTrail events to a CloudWatch log group
- 1
Create a CloudWatch log group
Open the CloudWatch console, choose Log management and click Create log group.
- 2
Name the log group and set retention
Enter the log group name
utmstack, set the retention to 1 day and click Create log group. - 3
Create a CloudTrail trail
Open the CloudTrail console and click Create trail.
- 4
Configure the trail
On the trail form, set these options:
Trail name and scope: enter a descriptive name. Turn on Apply trail to all accounts in my organization if you want to collect from your whole AWS organization.
S3 destination: choose Create new S3 bucket, or select an existing bucket, for raw log delivery.
Encryption: turn on Log file SSE-KMS and select the KMS key to use.
Log file validation: turn it on (recommended) so the delivered logs can be checked for changes.
CloudWatch Logs: turn on delivery to CloudWatch, select Use existing log group and enter the exact name
utmstack.IAM role: provide or create the role that CloudTrail uses to write to S3 and to CloudWatch.
- 5
Select the event types
Management events: turn them on and select All to record all API activity that creates, changes or deletes AWS resources.
Data events: add a data event, choose S3 and select Log all events to record object-level operations such as GetObject and PutObject.
Network activity events: add at least one service, for example
ec2.amazonaws.com, and select Log all events.
- 6
Review and create the trail
On the Configure event aggregation screen, keep the defaults or adjust them if needed. Review all settings and click Create trail.
Step 3: Connect AWS in UTMStack
Each AWS connection is a config group. Create the log group (Step 2) before you save the config group, because UTMStack checks that the log group exists.
- 1
Open the integration
Go to Configure > Integrations, search for
AWSand open the AWS Cloudwatch card. The credentials form is in the setup guide, after the IAM user steps. - 2
Fill in the form
Field What to enter Config group name A name for this connection, for example acme-prod. It cannot be changed later, and it becomes the data source name on the Data Sources page.AWS access key ID The Access Key ID from Step 1 (it starts with AKIA).AWS secret access key The Secret Access Key from Step 1. AWS default region The AWS region where the log group is, for example us-east-1.Log group name The CloudWatch log group that receives the CloudTrail events, for example utmstack.All fields are required.
- 3
Save
Click Add config group. UTMStack checks the credentials against AWS before it saves them. When the check passes, you see "Config group created successfully."
If the save fails
When the check fails, the config group is not saved and a message starting with "credential verification failed:" explains why.
| Message | What to check |
|---|---|
| Invalid Access Key ID | The Access Key ID is wrong or was deleted. |
| Invalid Secret Access Key | The secret does not belong to that Access Key ID. |
| AWS credentials have expired | Create a new access key for the IAM user. |
| AWS credentials are invalid | The key pair was rejected for another reason. Check both values for extra spaces, or create a new access key. |
| Invalid Default Region | The region code is wrong, for example us-east1 instead of us-east-1. |
| AWS credentials do not have permission to access CloudWatch in region ... | The IAM user is missing the CloudWatchLogsReadOnlyAccess policy. |
| Cannot access CloudWatch Log Groups in region ... | UTMStack could not list log groups in that region. Check the region code, the IAM policy, and that the UTMStack server reaches AWS on port 443. |
| The CloudWatch Log Group ... was not found in region ... | The log group name is wrong (names are case-sensitive), or the log group is in another region. |
Check that events arrive
- 1
Create some activity in AWS
Do something that CloudTrail records, for example sign in to the AWS console or open the IAM user list. CloudTrail usually delivers events to CloudWatch Logs a few minutes after the action.
- 2
Check the log group in AWS
In the CloudWatch console, open the
utmstacklog group. You should see at least one log stream with recent events. If the log group is empty, the trail is not delivering to CloudWatch Logs yet; check the trail settings first. - 3
Open the logs in UTMStack
Open the AWS Cloudwatch card again and click View logs. Log Explorer opens with the
awsdata type for the last 24 hours. UTMStack looks for new log streams every 5 minutes, so allow up to 10 minutes after you save. - 4
Check the data source
Go to Configure > Data Sources and open the Integrations tab. Your config group is listed by its name, for example
acme-prod. The row is created when you save, and turns Live after the first events are counted, which can take up to 20 more minutes.
Troubleshooting
Errors that happen after the config group is saved, for example a deleted access key, do not appear in the web interface. They are written to the logs of the UTMStack event processor on the server, where the integration runs.
| Symptom | Likely cause | Fix |
|---|---|---|
| The save fails with one of the messages above | The credentials, region or log group do not match. | Fix the value the message names and click Add config group again. |
The config group saved, but no aws events arrive | The trail is not sending to the log group, or it sends to a log group in another region. | In CloudTrail, open the trail and check that CloudWatch Logs is enabled with the log group utmstack. Check that the AWS default region in UTMStack is the region of that log group. |
| Events stopped after some time | The access key was deactivated, deleted or rotated, or the policy was removed. | Create a new access key, then edit the config group and paste both values. |
| Older events are missing | UTMStack reads from the moment collection starts and does not import history. | Search older activity in the CloudTrail console or in the trail's S3 bucket. |
| Some accounts of your AWS organization are missing | The trail covers only the account where it was created. | Turn on Apply trail to all accounts in my organization on the trail, or add one config group per account. |
| The Integrations row shows Offline although events arrive | The data source's "last seen" time is updated from the ingestion counts, which can take up to about 20 minutes. | Wait 20 minutes and click Refresh. If it stays Offline, check that events in Log Explorer show your config group name as the data source. |
Manage config groups
Add more connections: save one config group per AWS account, region or log group you want to read.
Edit: click the pencil icon next to a config group. The name cannot be changed. Secret fields show a masked value; leave it as it is to keep the stored secret. Click Update config group.
Delete: click the bin icon. The config group is deleted at once, without a confirmation, and collection stops. The data source it created stays on the Data Sources page; delete it there if you no longer need it. Events already stored stay searchable.