Skip to content

Add api args to Bedrock provider - #295

Merged
hadley merged 11 commits into
tidyverse:mainfrom
billsanto:add-api-args
Feb 6, 2025
Merged

hadley merged 11 commits into
tidyverse:mainfrom
billsanto:add-api-args

Conversation

@billsanto

Copy link
Copy Markdown
Contributor

Added api_args to the Bedrock provider. api_args is a list for parameters such as temperature, top_p, top_k, max_tokens, and stop_sequences. Added validation for each of these parameters. Updated the Roxygen documentation. (Removed the verbose parameter and relevant code/doc comments).

William Santo and others added 4 commits February 2, 2025 07:39
…ved verbose printing. Compacted request body. Updated Roxygen documentation.

@hadley hadley left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this! A few comments below.

Comment thread R/provider-bedrock.R Outdated
cache <- aws_creds_cache(profile)
credentials <- paws_credentials(profile, cache = cache)

# Validate api_args if present

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please omit this for now? As I mentioned, we're going to do something for all providers in the next release.

Comment thread R/provider-bedrock.R Outdated
# https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html
req <- req_body_json(req, list(
# Build request body
body <- compact(list(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please copy the generic api_args interface used by the other providers?

Comment thread R/provider-bedrock.R Outdated
Comment on lines +79 to +96
#' # The echo argument, "none", "text", and "all" determines whether
#' # input and/or output is echoed to the console. Also of note, "none" uses a
#' # non-streaming endpoint, whereas "text", "all", or TRUE uses a streaming endpoint.
#' chat <- chat_bedrock()
#' chat$chat("What is 1 + 1?") # Streaming response
#' resp <- chat$chat("What is 1 + 1?", echo = "none") # Non-streaming response
#' resp # View response
#'
#' # Use echo = "none" in the client constructor to suppress streaming response
#' chat <- chat_bedrock(echo = "none")
#' resp <- chat$chat("What is 1 + 1?") # Non-streaming response
#' resp # View response
#' chat$chat("What is 1 + 1?", echo=TRUE) # Overrides client echo arg, uses streaming
#'
#' # $stream returns a generator, requiring concatentation of the streamed responses.
#' resp <- chat$stream("What is the capital of France?") # resp is a generator object
#' chunks <- coro::collect(resp) # returns list of partial text responses
#' complete_response <- paste(chunks, collapse="") # Full text response, no echo

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to document this all generically, so can you please omit for now?

@hadley
hadley merged commit 064e527 into tidyverse:main Feb 6, 2025
@hadley

hadley commented Feb 6, 2025

Copy link
Copy Markdown
Member

Thanks @billsanto, I simplified this a bit more in the light of the fact that we'll invest more in cross-provider parameters in the future.

@billsanto

Copy link
Copy Markdown
Contributor Author

I see--that is a bit simpler. Thanks for adding it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants