To create a stream
- Open the Amazon Kinesis console at https://console.aws.amazon.com/kinesis.
- Choose Go to Streams.
- In the navigation bar, expand the region selector and choose a region.
- Choose Create Stream.
- Type a name for your stream (for example,
StockTradeStream). - Type
1for the number of shards, but leave Estimate the number of shards you’ll need collapsed. - Choose Create.
Create IAM Policy and User
Producer
| Actions | Resource | Purpose |
|---|---|---|
DescribeStream |
Amazon Kinesis stream | Before attempting to write records, the producer should check if the stream exists and is active. |
PutRecord, PutRecords |
Amazon Kinesis stream | Write records to Streams. |
Consumer
| Actions | Resource | Purpose |
|---|---|---|
DescribeStream |
Amazon Kinesis stream | Before attempting to read records, the consumer checks if the stream exists and is active, and if the shards are contained in the stream. |
GetRecords, GetShardIterator |
Amazon Kinesis stream | Read records from a Streams shard. |
CreateTable, DescribeTable,GetItem, PutItem,PutMetricData, Scan, UpdateItem |
Amazon DynamoDB table | If the consumer is developed using the Kinesis Client Library (KCL), it needs permissions to a DynamoDB table to track the processing state of the application. The first consumer started creates the table. |
DeleteItem |
Amazon DynamoDB table | For when the consumer performs split/merge operations on Streams shards. |
PutMetricData |
Amazon CloudWatch log | The KCL also uploads metrics to CloudWatch, which are useful for monitoring the application. |
To create an IAM policy
- Determine the Amazon Resource Name (ARN) for the new stream. The ARN format is as follows:
arn:aws:kinesis:region:account:stream/name- region
- The region code; for example, us-west-2. For more information, see Region and Availability Zone Concepts.
- account
- The AWS account ID, as shown in Account Settings.
- name
- The name of the stream from Step 1: Create a Stream, which is
StockTradeStream.
- Choose Select next to Policy Generator.
- Choose Amazon Kinesis as the AWS service.
- Select
DescribeStream,GetShardIterator,GetRecords,PutRecord, andPutRecordsas the allowed actions. - Type the ARN that you created in Step 1.
- Use Add Statement for each of the following:
AWS Service Actions ARN Amazon DynamoDB CreateTable,DeleteItem,DescribeTable,GetItem,PutItem,PutMetricData,Scan,UpdateItemThe ARN you created in Step 2 Amazon CloudWatch PutMetricData*The asterisk (
*) is used when specifying an ARN is not required. In this case, it’s because there is no specific resource in CloudWatch on which thePutMetricDataaction is invoked.