AWS Kinesis Streams is a real-time data streaming service that enables organizations to build scalable and high-throughput data pipelines. In a streaming ETL (Extract, Transform, Load) workflow, Kinesis Streams serves as the backbone for ingesting and processing large volumes of real-time data. Its ability to integrate with various AWS services and custom applications makes it a versatile tool for dynamic and responsive data systems.

Connecting Kinesis Data Firehose (IAM Role Setup - General Setup)

To integrate Kinesis Data Firehose into your ProxyHook you first need to create an IAM role for ProxyHook within your AWS account. If you've already done this, you can skip to the next section

1. Log into your AWS account and head to 'Identity and Access Management (IAM)'
2. Click 'Create Role'
3. Select 'AWS account' and select 'Another AWS Account' with the value 121269227306
4. Skip the permissions policy page by clicking 'Next'
5. Provide a name for the role (e.g. ProxyHook) and click Create Role
6. Note the ARN Role for your destination settings in the ProxyHook Settings page

Connecting Kinesis Data Firehose (IAM Role Setup - Permissions)

In order to have records posted to your Kinesis Data Firehose, you will need to ensure your ProxyHook IAM account has permissions. To do so, follow these steps:

1. Log into your AWS account and head to 'Identity and Access Management (IAM)' > 'Roles'
2. Select your role made from above and under Permissions, Select Add permissions > Create Inline Police
3. Click JSON and paste the following, editing the code where variables exist

{

"Version": "2012-10-17",

"Statement": [

{

"Effect": "Allow",

"Action": [

"firehose:PutRecord"

],

"Resource": [

"arn:aws:firehose:YOUR_REGION:YOUR_ACCOUNT_ID:stream/YOUR_STREAM_NAME"

]

}

]

}

Click next, create a policy name and hit save.