Correct option:

Create an IAM Role for EC2

IAM roles have been incorporated so that your applications can securely make API requests from your instances, without requiring you to manage the security credentials that the applications use. Instead of creating and distributing your AWS credentials, you can delegate permission to make API requests using IAM roles.

Amazon EC2 uses an instance profile as a container for an IAM role. When you create an IAM role using the IAM console, the console creates an instance profile automatically and gives it the same name as the role to which it corresponds.

Incorrect options:

EC2 User Data - You can specify user data when you launch an instance and you would not want to hard code the AWS credentials in the user data.

Run aws configure on the EC2 instance - When you first configure the CLI you have to run this command, afterward you should not need to if you want to obtain credentials to authenticate to other AWS services. An IAM role will receive temporary credentials for you so you can focus on using the CLI to get access to other AWS services if you have the permissions.

CloudFormation - AWS CloudFormation gives developers and businesses an easy way to create a collection of related AWS and third-party resources and provision them in an orderly and predictable fashion.

How CloudFormation Works:

via - https://aws.amazon.com/cloudformation/

Reference:

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html

Correct option:

Create an S3 event to invoke a Lambda function that inserts records into DynamoDB

The Amazon S3 notification feature enables you to receive notifications when certain events happen in your bucket. To enable notifications, you must first add a notification configuration that identifies the events you want Amazon S3 to publish and the destinations where you want Amazon S3 to send the notifications. You store this configuration in the notification subresource that is associated with a bucket.

Amazon S3 APIs such as PUT, POST, and COPY can create an object. Using these event types, you can enable notification when an object is created using a specific API, or you can use the s3:ObjectCreated:* event type to request notification regardless of the API that was used to create an object.

For the given use-case, you would create an S3 event notification that triggers a Lambda function whenever we have a PUT object operation in the S3 bucket. The Lambda function in turn would execute custom code to inserts records into DynamoDB.

via - https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html

Incorrect options:

Write a cron job that will execute a Lambda function at a scheduled time and insert the records into DynamoDB - This is not efficient because there may not be any unprocessed file in the S3 bucket when the cron triggers the Lambda on schedule. So this is not the correct option.

Set up an event with Amazon CloudWatch Events that will monitor the S3 bucket and then insert the records into DynamoDB - The CloudWatch event cannot directly insert records into DynamoDB as it's not a supported target type. The CloudWatch event needs to use something like a Lambda function to insert the records into DynamoDB.

Develop a Lambda function that will poll the S3 bucket and then insert the records into DynamoDB - This is not efficient because there may not be any unprocessed file in the S3 bucket when the Lambda function polls the S3 bucket at a given time interval. So this is not the correct option.

Reference:

https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html

Correct option:

The partition key that you have selected isn't distributed enough

Amazon Kinesis Data Streams enables you to build custom applications that process or analyze streaming data for specialized needs.

A Kinesis data stream is a set of shards. A shard is a uniquely identified sequence of data records in a stream. A stream is composed of one or more shards, each of which provides a fixed unit of capacity.

The partition key is used by Kinesis Data Streams to distribute data across shards. Kinesis Data Streams segregates the data records that belong to a stream into multiple shards, using the partition key associated with each data record to determine the shard to which a given data record belongs.

Kinesis Data Streams Overview:

via - https://docs.aws.amazon.com/streams/latest/dev/key-concepts.html

For the given use-case, as the partition key is not distributed enough, all the data is getting skewed at a few specific shards and not leveraging the entire cluster of shards.

You can also use metrics to determine which are your "hot" or "cold" shards, that is, shards that are receiving much more data, or much less data, than expected. You could then selectively split the hot shards to increase capacity for the hash keys that target those shards. Similarly, you could merge cold shards to make better use of their unused capacity.

Incorrect options:

Metrics are slow to update - Metrics are a CloudWatch concept. This option has been added as a distractor.

You have too many shards - Too many shards is not the issue as you would see a LimitExceededException in that case.

The data retention period is too long - Your streaming data is retained for up to 365 days. The data retention period is not an issue causing this error.

References:

https://docs.aws.amazon.com/streams/latest/dev/key-concepts.html

https://docs.aws.amazon.com/streams/latest/dev/kinesis-using-sdk-java-resharding-strategies.html

Correct option:

Create an IAM role with an AmazonDynamoDBReadOnlyAccess policy and apply it to the EC2 instance profile

As an AWS security best practice, you should not create an IAM user and pass the user's credentials to the application or embed the credentials in the application. Instead, create an IAM role that you attach to the EC2 instance to give temporary security credentials to applications running on the instance. When an application uses these credentials in AWS, it can perform all of the operations that are allowed by the policies attached to the role.

So for the given use-case, you should create an IAM role with an AmazonDynamoDBReadOnlyAccess policy and apply it to the EC2 instance profile.

via - https://docs.aws.amazon.com/IAM/latest/UserGuide/id.html

Incorrect options:

Create a new IAM user with access keys. Attach an inline policy to the IAM user with read-only access to DynamoDB. Place the keys in the code. For security, redeploy the code whenever the keys rotate

Create an IAM user with Administrator access and configure AWS credentials for this user on the given EC2 instance

Run application code with AWS account root user credentials to ensure full access to all AWS services

As mentioned in the explanation above, it is dangerous to pass an IAM user's credentials to the application or embed the credentials in the application. The security implications are even higher when you use an IAM user with admin privileges or use the AWS account root user. So all three options are incorrect.

Reference:

https://docs.aws.amazon.com/IAM/latest/UserGuide/id.html

Correct option:

Enable SQS KMS encryption

Server-side encryption (SSE) lets you transmit sensitive data in encrypted queues. SSE protects the contents of messages in queues using keys managed in AWS Key Management Service (AWS KMS).

AWS KMS combines secure, highly available hardware and software to provide a key management system scaled for the cloud. When you use Amazon SQS with AWS KMS, the data keys that encrypt your message data are also encrypted and stored with the data they protect.

You can choose to have SQS encrypt messages stored in both Standard and FIFO queues using an encryption key provided by AWS Key Management Service (KMS).

Incorrect options:

Use the SSL endpoint - The given use-case needs encryption at rest. When using SSL, the data is encrypted during transit, but the data needs to be encrypted at rest as well, so this option is incorrect.

Use Client-side encryption - For additional security, you can build your application to encrypt messages before they are placed in a message queue but will require a code change, so this option is incorrect.

*Use Secrets Manager * - AWS Secrets Manager enables you to easily rotate, manage, and retrieve database credentials, API keys, and other secrets throughout their lifecycle. Users and applications retrieve secrets with a call to Secrets Manager APIs, eliminating the need to hardcode sensitive information in plain text. Secrets Manager offers secret rotation with built-in integration for Amazon RDS, Amazon Redshift, and Amazon DocumentDB. Secrets Manager cannot be used for encrypting data at rest.

Reference:

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-server-side-encryption.html

Correct option:

Yes, the max message size is 256KB

The minimum message size is 1 byte (1 character). The maximum is 262,144 bytes (256 KB).

via - https://aws.amazon.com/sqs/faqs/

Incorrect options:

Yes, the max message size is 512KB - The max size is 256KB

No, the max message size is 128KB - The max size is 256KB

No, the max message size is 64KB - The max size is 256KB

Reference:

https://aws.amazon.com/sqs/faqs/

Correct option:

You create a CloudWatch custom metric and build an alarm to scale your ASG

Here we need to scale on the metric "number of requests per minute", which is a custom metric we need to create, as it's not readily available in CloudWatch.

Metrics produced by AWS services are standard resolution by default. When you publish a custom metric, you can define it as either standard resolution or high resolution. When you publish a high-resolution metric, CloudWatch stores it with a resolution of 1 second, and you can read and retrieve it with a period of 1 second, 5 seconds, 10 seconds, 30 seconds, or any multiple of 60 seconds.

Incorrect options:

Attach an Elastic Load Balancer - This is not what you need for auto-scaling. An Elastic Load Balancer distributes workloads across multiple compute resources and checks your instances' health status to name a few, but it does not automatically increase and decrease the number of instances based on the application requirement.

Attach additional Elastic File Storage - This is a file storage service designed for performance. Amazon Elastic File System (Amazon EFS) provides a simple, scalable, fully managed elastic NFS file system for use with AWS Cloud services and on-premises resources. It is built to scale on-demand to petabytes without disrupting applications, growing and shrinking automatically as you add and remove files, eliminating the need to provision and manage capacity to accommodate growth. This cannot be used to facilitate auto-scaling.

How EFS Works:

via - https://aws.amazon.com/efs/

You enable detailed monitoring and use that to scale your ASG - The detailed monitoring metrics won't provide information about database /application-level requests per minute, so this option is not correct.

Reference:

https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html

Correct option:

Use parallel scans

By default, the Scan operation processes data sequentially. Amazon DynamoDB returns data to the application in 1 MB increments, and an application performs additional Scan operations to retrieve the next 1 MB of data. The larger the table or index being scanned, the more time the Scan takes to complete. To address these issues, the Scan operation can logically divide a table or secondary index into multiple segments, with multiple application workers scanning the segments in parallel.

To make use of a parallel Scan feature, you will need to run multiple worker threads or processes in parallel. Each worker will be able to scan a separate partition of a table concurrently with the other workers.

How DynamoDB parallel Scan works:

via - https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Scan.html#Scan.ParallelScan

Incorrect options:

Use a ProjectionExpression - A projection expression is a string that identifies the attributes you want. To retrieve a single attribute, specify its name. For multiple attributes, the names must be comma-separated

Use a FilterExpression - If you need to further refine the Scan results, you can optionally provide a filter expression. A filter expression determines which items within the Scan results should be returned to you. All of the other results are discarded.

A filter expression is applied after a Scan finishes, but before the results are returned. Therefore, a Scan consumes the same amount of read capacity, regardless of whether a filter expression is present.

Use a Query - This could work if we were able to create an index, but the question says: "There is no possibility of creating indexes to retrieve these items predictably". As such, we cannot use a Query.

Reference:

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Scan.html#Scan.ParallelScan

Correct option:

Use an IAM Instance Role

An instance profile is a container for an IAM role that you can use to pass role information to an EC2 instance when the instance starts. The AWS SDK will use the EC2 metadata service to obtain temporary credentials thanks to the IAM instance role. This is the most secure and common setup when deploying any kind of applications onto an EC2 instance.

Incorrect options:

Use environment variables - This is another option if you configure AWS CLI on the EC2 instance. When configuring the AWS CLI you will set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables. This practice may not be bad for one instance but once you start running more EC2 instances this is not a good practice because you may have to change credentials on each instance whereas an IAM Role gets temporary permissions.

Hardcode the credentials in the application code - It will work for sure, but it's not a good practice from a security point of view.

Use the SSM parameter store - With parameter store you can store data such as passwords. The problem is that you need the SDK to access parameter store and without credentials, you cannot use the SDK. Use parameter store for other uses such as database connection strings or other secret codes when you have already authenticated to AWS.

Reference:

https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html

Correct option:

Install and configure Kinesis Agent on each of the instances

Kinesis Agent is a stand-alone Java software application that offers an easy way to collect and send data to Kinesis Data Streams. The agent continuously monitors a set of files and sends new data to your stream. The agent handles file rotation, checkpointing, and retry upon failures. It delivers all of your data in a reliable, timely, and simple manner. It also emits Amazon CloudWatch metrics to help you better monitor and troubleshoot the streaming process.

You can install the agent on Linux-based server environments such as web servers, log servers, and database servers. After installing the agent, configure it by specifying the files to monitor and the stream for the data. After the agent is configured, it durably collects data from the files and reliably sends it to the stream.

The agent can also pre-process the records parsed from monitored files before sending them to your stream. You can enable this feature by adding the dataProcessingOptions configuration setting to your file flow. One or more processing options can be added and they will be performed in the specified order.

Incorrect options:

Run cron job on each of the instances to collect log data and send it to Kinesis Data Streams - This solution is possible, though not an optimal one. This solution requires writing custom code and tracking file/log changes, retry failures and so on. Kinesis Agent is built to handle all these requirements and integrates with Data Streams.

Install AWS SDK on each of the instances and configure it to send the necessary files to Kinesis Data Streams - Kinesis Data Streams APIs that are available in the AWS SDKs, helps you manage many aspects of Kinesis Data Streams, including creating streams, resharding, and putting and getting records. You will need to write custom code to handle new data in the log files and send it over to your stream. Kinesis Agent does it easily, as it is designed to continuously monitor a set of files and send new data to your stream.

Use Kinesis Producer Library (KPL) to collect and ingest data from each EC2 instance - The KPL is an easy-to-use, highly configurable library that helps you write to a Kinesis data stream. It acts as an intermediary between your producer application code and the Kinesis Data Streams API actions. This is not optimal compared to Kinesis Agent which is designed to continuously monitor a set of files and send new data to your stream.

Reference:

https://docs.aws.amazon.com/streams/latest/dev/writing-with-agents.html