Bedrock Titan Validation Malformed input request: string does not match pattern ^(\|+|User:)$

[mtrefilek]

Describe the bug

The AWS Bedrock Inference docs only specify the stopSequence parameter for titan models as an array of strings.
https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-titan-text.html

The following code:

# Run inference on a model
modelId = 'arn:aws:bedrock:us-west-2::foundation-model/amazon.titan-text-express-v1'

body = json.dumps({"inputText": "explain black holes to 8th graders",
                   "textGenerationConfig": {
                        "maxTokenCount": 4096,
                        "stopSequences": ["Human:"],
                        "temperature":0.5,
                        "topP":1
                    }})

Returns an error:
ValidationException: An error occurred (ValidationException) when calling the InvokeModel operation: Malformed input request: string [Human:] does not match pattern ^(\|+|User:)$, please reformat your input and try again.

This validation is not required in any of the documentation.

Expected Behavior

Should produce a LLM Response from the Bedrock Runtime

Current Behavior

---------------------------------------------------------------------------
ValidationException                       Traceback (most recent call last)
Cell In[15], [line 21](vscode-notebook-cell:?execution_count=15&line=21)
     [19](vscode-notebook-cell:?execution_count=15&line=19) accept = 'application/json'
     [20](vscode-notebook-cell:?execution_count=15&line=20) contentType = 'application/json'
---> [21](vscode-notebook-cell:?execution_count=15&line=21) response = bedrock_runtime.invoke_model(body=body, modelId=modelId, accept=accept, contentType=contentType)
     [22](vscode-notebook-cell:?execution_count=15&line=22) response_body = json.loads(response.get('body').read())
     [23](vscode-notebook-cell:?execution_count=15&line=23) print(response)

File [/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:553](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:553), in ClientCreator._create_api_method.<locals>._api_call(self, *args, **kwargs)
    [549](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:549)     raise TypeError(
    [550](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:550)         f"{py_operation_name}() only accepts keyword arguments."
    [551](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:551)     )
    [552](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:552) # The "self" in this scope is referring to the BaseClient.
--> [553](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:553) return self._make_api_call(operation_name, kwargs)

File [/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:1009](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:1009), in BaseClient._make_api_call(self, operation_name, api_params)
   [1005](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:1005)     error_code = error_info.get("QueryErrorCode") or error_info.get(
   [1006](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:1006)         "Code"
   [1007](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:1007)     )
   [1008](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:1008)     error_class = self.exceptions.from_code(error_code)
-> [1009](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:1009)     raise error_class(parsed_response, operation_name)
   [1010](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:1010) else:
   [1011](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/botocore/client.py:1011)     return parsed_response

ValidationException: An error occurred (ValidationException) when calling the InvokeModel operation: Malformed input request: string [Human:] does not match pattern ^(\|+|User:)$, please reformat your input and try again.

Reproduction Steps

Run the following in a python environment:

# Run inference on a model
modelId = 'arn:aws:bedrock:us-west-2::foundation-model/amazon.titan-text-express-v1'

body = json.dumps({"inputText": "explain black holes to 8th graders",
                   "textGenerationConfig": {
                        "maxTokenCount": 4096,
                        "stopSequences": ["Human:"],
                        "temperature":0.5,
                        "topP":1
                    }})

Possible Solution

No response

Additional Information/Context

No response

SDK version used

1.34.21

Environment details (OS name and version, etc.)

Mac Sonoma M1 Chip

[yifanmai]

I am running into this issue as well. The official Titan documentation says that stopSequences is an array of string:

{
    "inputText": string,
    "textGenerationConfig": {
        "temperature": float,  
        "topP": float,
        "maxTokenCount": int,
        "stopSequences": [string]
    }
}

but then goes on to say that the stop sequences should be concatenated using the | character:

Stop sequences (stopSequences) – Specify character sequences to indicate where the model should stop. Use the | (pipe) character to separate different sequences (maximum 20 characters).

which seems strange - if stopSequences is an array of sequences, then there should be no need to concatenate the sequences with the | character.

[basaldella]

+1, same issue here. The only stopSequences parameters that work are using either User: or |, which makes sense, given that the validation regex is ^(\|+|User:)$. Adding more than 1 element to the array (e.g. 5) returns Malformed input request: expected maximum item count: 1, found: 5, please reformat your input and try again.. So it seems like atm you can't use Human: or similar as stop sequences.

[manwithaplandy]

Same issue here, this makes using CoT prompting methods or Langchain agents impossible to use with Bedrock because of what is likely a mistyped regular expression and confusing documentation. Please fix the validation to allow for stop sequences other than User: or |, list the actual regular expression used to validate within the documentation, and clarify whether the stop sequences should be an array of strings, or an array with a single string of | separated words.

[RyanFitzSimmonsAK]

Thank you for providing your use cases and more details. I've reached out to the service team about the following issues.

1. Validation regex is not documented in the User Guide or the API reference
2. Validation regex is too restrictive
3. Only one stop sequence is allowed
4. Unclear syntax for using multiple sequences if they were allowed

As this is a service team issue (not unique to the Python SDK), I've created a tracking issue in our cross-SDK repository (aws/aws-sdk#692). Please continue to discuss your use cases and other details in the new issue.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.