Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot perform unittest without valid key #10

Open
Domejko opened this issue Apr 16, 2024 · 5 comments
Open

Cannot perform unittest without valid key #10

Domejko opened this issue Apr 16, 2024 · 5 comments
Labels
enhancement New feature or request

Comments

@Domejko
Copy link
Contributor

Domejko commented Apr 16, 2024

While I wanted to perform tests I did encounter this error on 12 from 16 test cases:

TypeError: "Could not resolve authentication method. Expected either api_key or auth_token to be set. Or for one of the `X-Api-Key` or `Authorization` headers to be explicitly omitted"

Test could not be performed without Claude account with resources on it. I think that for general testing purposes we will need to mock Anthropic client.messages.create.

@alat-rights
Copy link
Owner

Hi!

Yes, tests are currently run live via the Anthropic API.

While I'm open to mocking, this is not currently considered a bug.

Can you get yourself an Anthropic API key here, and then add os.environ["ANTHROPIC_API_KEY"] = your_api_key to your local copy of the test code?

Definitely want to automate this with GitHub Actions in the future.

@alat-rights alat-rights added the enhancement New feature or request label Apr 16, 2024
@Domejko Domejko changed the title Bug: Cannot perform unittest without valid key Cannot perform unittest without valid key Apr 16, 2024
@Domejko
Copy link
Contributor Author

Domejko commented Apr 16, 2024

I have already set up an account and used valid generated api_key but then it was demanding to top up my credits, there is a option go get 5$ for free but I'm not convinced to give my phone number left and right.

I'll will try to mock that return value so that testing will be independent of valid api_key.

@alat-rights
Copy link
Owner

Ah, I see!

I think a graceful way to handle this would be to keep the existing functions as-is and add new ones with mocked outputs.

Proposal:

  • We can have two classes (maybe TestFunctionsLocal and TestFunctionsAPI?), and then take an argument from the terminal to decide whether to run both or to only run TestFunctionsLocal.

@Domejko
Copy link
Contributor Author

Domejko commented Apr 16, 2024

Yes if you want to keep test on live API responses it can be done this way. We can also have 2 test files, one for live testing and the other for local and run them independently (but this only assuming that in future all test will be still kept in one-file format).

Choice is up to you.

@alat-rights
Copy link
Owner

Yup! Let's keep it all in simple_tests.py for now. We can always split into multiple files in the future.

@Domejko Domejko mentioned this issue Apr 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants