-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow to retry a model when it errors #125
Conversation
b5129de
to
3decf21
Compare
3decf21
to
280e54e
Compare
280e54e
to
4a67cb3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome!
) | ||
}, | ||
After: func(t *testing.T, logger *log.Logger, resultPath string) { | ||
delete(provider.Providers, "testing-provider") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a source of problems, when we register a provider temporarily, we can run into concurrency problems. Better to refactor the code, make the provider a parameter and mock that parameter instead. Easier to type. Easier to understand. Less code. Cleaner.
Name: "Single try fails", | ||
|
||
Before: func(t *testing.T, logger *log.Logger, resultPath string) { | ||
queryMock.On("Query", mock.Anything, "testing-provider/testing-model", mock.Anything).Return("", errors.New("empty response from model")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not copy errors, create a variable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you copy, don't. Create an API for it.
@@ -124,3 +147,8 @@ func (m *Model) GenerateTestsForFile(logger *log.Logger, language language.Langu | |||
|
|||
return assessment, nil | |||
} | |||
|
|||
// SetAttempts sets the number of attempts to perform when a model errors in the process of solving a task. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When we implement an interface, we always explicitly check that the interface is implemented it in the code. Even if it is just one method.
Part of #123