Using Bedrock models with cross-region inference requires inference profile IDs

[inariku]

Description

The current default configuration uses amazon.nova-pro-v1:0 as the model identifier for Amazon Bedrock's Nova-Pro model. This works in primary regions (us-east-1, us-gov-west-1), but causes API invocation errors when deployed in regions that require cross-region inference.

Error Details

When attempting to invoke the Nova-Pro model in non-primary regions with the direct model ID, the following error occurs:

ValidationException: Invocation of model ID amazon.nova-pro-v1:0 with on-demand throughput isn't supported. Retry your request with the ID or ARN of an inference profile that contains this model.

This is a known issue also reported in AWS repost.

Technical Background

According to the AWS documentation:

1. Amazon Bedrock base model IDs (on-demand throughput) lists the models that support direct invocation.

2. Supported Regions and models for cross-region inference shows that Nova-Pro requires cross-region inference in most regions (marked with *).

3. The boto3 documentation for converse_stream explains that:

   If you use a base model, specify the model ID or its ARN.
   If you use an inference profile, specify the inference profile ID or its ARN.

Reproduction Steps

1. Deploy the stack in a non-primary region (e.g., us-west-2) using the default configuration with bedrockModel: "amazon.nova-pro-v1:0"
2. Attempt to use the Nova-Pro model functionality
3. Observe the API invocation error

Proposed Solution

To make the stack work in all regions, update the configuration to use the inference profile ID:

const config: StackConfig = {
  bedrockRegion: "us-east-1", // Works in any supported region
  bedrockModel: "us.amazon.nova-pro-v1:0", // Works in both primary and non-primary regions
  // ...
};

Additionally, we should add a note in the README to explain this requirement for cross-region inference, helping users avoid this common error when deploying to different regions.

Alternatively, add logic to select the appropriate identifier based on the deployment region by checking if cross-region inference is required for the selected region.