CloudWatch Logs is a powerful AWS service for collecting, monitoring, and analyzing log data in real-time. In a streaming ETL (Extract, Transform, Load) workflow, it acts as both a data source and a monitoring tool, enabling seamless integration of log data into pipelines. By leveraging CloudWatch Logs, organizations can gain actionable insights and maintain continuous data flow to support real-time analytics and system optimization.

Connecting Cloudwatch (IAM Role Setup - General Setup)

To integrate Cloudwatch into your ProxyHook you first need to create an IAM role for ProxyHook within your AWS account. If you've already done this, you can skip to the next section

Log into your AWS account and head to 'Identity and Access Management (IAM)'

Click 'Create Role'

Select 'AWS account' and select 'Another AWS Account' with the value 121269227306

Skip the permissions policy page by clicking 'Next'

Provide a name for the role (e.g. ProxyHook) and click Create Role

Note the ARN Role for your Cloudwatch Settings in the ProxyHook Settings page

Connecting Cloudwatch (IAM Role Setup - Permissions)

In order to have records posted to your Cloudwatch account, you will need to ensure your ProxyHook IAM account has permissions. To do so, follow these steps:

Log into your AWS account and head to 'Identity and Access Management (IAM)' > 'Roles'

Select your role made from above and under Permissions, Select Add permissions > Create Inline Police

Click JSON and paste the following, editing the code where variables exist

{

"Version": "2012-10-17",

"Statement": [

{

"Effect": "Allow",

"Action": [ "logs:*"],

"Resource": "arn:aws:logs:YOUR_REGION:YOUR_ACCOUNT_ID:log-group:YOUR_GROUP_NAME:*"

}

]

}

Click next, create a policy name and hit save. If done correctly, the IAM role will now be able to write to cloudwatch