Considering adding AWS Bedrock APIs support

[kalimu]

Are you considering adding AWS Bedrock backend as well?

We use it in a corporate environment, so we have some working code for doing that.
However, there is some additional complexity with interacting with the cloud (different regions, different endpoints, signing each request with V4 signature, etc.).
Not sure how that fits to your plans for this package.

[hadley]

Documentation at https://docs.aws.amazon.com/bedrock/latest/APIReference/welcome.html

Specifically https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html and https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html

Looks like roughly equivalent work to adding claude and gemini support (maybe a bit extra to load up the various AWS complexities) so it seems worthwhile to me.

[hadley]

This is more work than I expected because AWS uses a special signing format: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv.html

And a special streaming format: paws-r/paws#842 (comment)

I will still do this eventually, but it's probably a week of work.

[kalimu]

I haven't explored streaming format, so I cannot help, but I did some work that might be helpful regarding signing requests.

I needed a function that could sign my httr2 requests to AWS API but didn't want to have additional dependencies (like to paws).

There is an R package: https://github.com/cloudyr/aws.signature, but it is not maintained. It has some flaws, and it does not correctly sign some requests to Bedrock.

So I've created a function (below) that can sign any httr2 requests like:

base_url <-
    paste0("https://", "bedrock-runtime.",
           aws_region, ".amazonaws.com")
req <-
  httr2::request(base_url =  base_url) %>%
  httr2::req_method("POST") %>%
  httr2::req_headers("Content-type" = "application/json") %>%
  httr2::req_url_path_append("model") %>%
  httr2::req_url_path_append("anthropic.claude-3-5-sonnet-20240620-v1:0") %>%
  httr2::req_url_path_append("converse") %>%
  httr2::req_body_json(
    list(
      'messages' = list(list(
          role    = 'user',
          content = list(
            list(text = 'Write a sentence about impact of AI.')
       ))) %>%
  req_sign_aws_v4_auth(aws_service  = "bedrock")

There is just one thing that I haven't implemented fully. It is this line:

 path <- gsub(x = path, pattern = ":", replacement = "%3A")

The request path should be URL encoded, but it is a bit tricky, because it's not clear from documentation how UriEncode() should work for URL path. I've found that it should encode ":" in "anthropic.claude-3-5-sonnet-20240620-v1:0" but not forward slashes "/" as (Encode the forward slash character, '/', everywhere except in the object key name. For example, if the object key name is photos/Jan/sample.jpg, the forward slash in the key name is not encoded.) https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv-create-signed-request.html
So the implementation of encoding should yet be completed, but in this form it works for me for my Bedrock and others AWS API calls already.

req_sign_aws_v4_auth <- function(
    req,
    aws_service,
    aws_access_key_id     = Sys.getenv("AWS_ACCESS_KEY_ID"),
    aws_secret_access_key = Sys.getenv("AWS_SECRET_ACCESS_KEY"),
    aws_session_token     = Sys.getenv("AWS_SESSION_TOKEN"),
    aws_region            = Sys.getenv("AWS_DEFAULT_REGION"),
    current_time          = Sys.time(),
    debug                 = FALSE
    ) {

  host     <- httr2::url_parse(req$url)$hostname
  req_time <- format(current_time, "%Y%m%dT%H%M%SZ", tz = "UTC")

  if (is.null(req$body)) {

    body <- ""

  } else {

    body <-
      req$body$data %>%
      jsonlite::toJSON(auto_unbox = TRUE)
  }

  content_sha256 <- digest::digest(body, algo = "sha256", serialize = FALSE)

  if (is.null(req$method)) {
    req <-
      req %>%
      httr2::req_method("GET")
  }

  req <-
    req %>%
    httr2::req_headers("Accept" = "application/json") %>%
    httr2::req_headers("host"                 = host,
                       "x-amz-date"           = req_time,
                       "x-amz-content-sha256" = content_sha256,
                       "x-amz-security-token" = aws_session_token)

  query_args <- httr2::url_parse(req$url)$query

  if (length(query_args)) {

    query_args   <- unlist(query_args[order(names(query_args))])

    query_string <- paste0(sapply(names(query_args),
                                  URLencode,
                                  reserved = TRUE),
                           "=",
                           sapply(as.character(query_args),
                                  URLencode,
                                  reserved = TRUE))

    query_string <- paste(query_string, sep = "", collapse = "&")

  } else {

    query_string <- ""
  }

  canonical_headers        <- req$headers
  names(canonical_headers) <- tolower(names(canonical_headers))

  canonical_headers <- canonical_headers[order(names(canonical_headers))]
  trimmed_headers   <- gsub("[[:space:]]{2,}", " ", trimws(canonical_headers))

  header_string  <- paste0(names(canonical_headers), ":", trimmed_headers, "\n",
                          collapse = "")
  signed_headers <- paste(names(canonical_headers),
                          sep      = "",
                          collapse = ";")

  path <- httr2::url_parse(req$url)$path
  path <- ifelse(is.null(path), "/", path)
  path <- gsub(x = path, pattern = ":", replacement = "%3A")

  canonical_request <- paste(req$method,
                             path,
                             query_string,
                             header_string,
                             signed_headers,
                             content_sha256,
                             sep = "\n")

  if (debug) {
    print("Canonical request:")
    cat(canonical_request, "\n")
  }

  canonical_request_hash <- digest::digest(canonical_request,
                                           algo      = "sha256",
                                           serialize = FALSE)
  algorithm <- "AWS4-HMAC-SHA256"
  region    <- aws_region
  service   <- aws_service

  string_to_sign <- paste(
    algorithm,
    req_time,
    paste(substring(req_time, 1, 8),
          region,
          service,
          "aws4_request",
          sep = "/"),
    canonical_request_hash,
    sep = "\n")

  if (debug) {
    print("String to sign:")
    cat(string_to_sign, "\n")
  }

  date <- format(current_time, "%Y%m%d")

  hash_date <-
    digest::hmac(key    = paste0("AWS4", aws_secret_access_key),
                 object = date,
                 algo   = "sha256",
                 raw    = TRUE)

  hash_region <-
    digest::hmac(key    = hash_date,
                 object = region,
                 algo   = "sha256",
                 raw    = TRUE)

  hash_service <-
    digest::hmac(key    = hash_region,
                 object = service,
                 algo   = "sha256",
                 raw    = TRUE)

  hash_aws4_request <-
    digest::hmac(key    = hash_service,
                 object = "aws4_request",
                 algo   = "sha256",
                 raw    = TRUE)

  signature <-
    digest::hmac(key    = hash_aws4_request,
                 object = string_to_sign,
                 algo   = "sha256")


  if (debug) {
    print("Signature:")
    cat(signature, "\n")
  }

  credential <-  paste(aws_access_key_id,
                       date,
                       region,
                       service,
                       "aws4_request",
                       sep = "/")

  authorization_header <-
    paste(algorithm,
          paste(
            paste0("Credential=",    credential),
            paste0("SignedHeaders=", signed_headers),
            paste0("Signature=",     signature),
            sep = ","))

  if (debug) {
    print("Authorization header:")
    cat(authorization_header, "\n")
  }


  req <-
    req %>%
    httr2::req_headers(
      Authorization = authorization_header)

  req
}

[kalimu]

However you might consider to add function like this (or based on curl functionality) to httr2 directly.

[hadley]

@kalimu awesome thank you! Do I have your permission to include that function (or something based on it) into httr2?

[kalimu]

@kalimu awesome thank you! Do I have your permission to include that function (or something based on it) into httr2?

Of course! I would be happy to contribute any way I can. It wasn't trivial to figure out how it should work, but once I did, it proved to be very useful in my projects. I'm pretty sure it should be useful to others as well. And if elmer gains Bedrock support thanks to that I would be even happier! 🙂

[hadley]

@kalimu in my experiments I'm consistently getting this response back:

$Output
$Output$`__type`
[1] "com.amazon.coral.service#UnknownOperationException"


$Version
[1] "1.0"

Have you seen this before? Any idea what I'm doing wrong? My code looks like this:

creds <- paws.common::locate_credentials()
model_id <- "anthropic.claude-3-5-sonnet-20240620-v1:0"
req <- request("https://bedrock.us-east-1.amazonaws.com")
req <- req_url_path_append(req, "model", model_id, "converse")
req <- req_body_json(req, list(
  messages = list(
    role = "user",
    content = list(list(text = "What's your name?"))
  )
))
req <- req_method(req, "POST")
req <- req_sign_aws_v4_auth(
  req,
  "bedrock",
  aws_access_key_id = creds$access_key_id,
  aws_secret_access_key = creds$secret_access_key,
  aws_session_token = creds$session_token,
  aws_region = creds$region,
  debug = TRUE
)
resp <- req_perform(req)
resp_body_json(resp)

[hadley]

Ooooh, I need to use the bedrock-runtime endpoint

[kalimu]

Ooooh, I need to use the bedrock-runtime endpoint

Did it work? I hope I didn't share code that's not working... but it has been working for me ;)

[hadley]

@kalimu and done! Thanks for you help with this and please let me know if you encounter any problems.

[kalimu]

That's great! We will be testing it soon.