-
Notifications
You must be signed in to change notification settings - Fork 0
2022 11 28 defaults #4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: base-sha/125b313bd333a294888cfa7a463a8819f109b9a7
Are you sure you want to change the base?
Conversation
|
This is a benchmark review for experiment This pull request was cloned from Experiment configurationreview_config:
# User configuration for the review
# - benchmark - use the user config from the benchmark reviews
# - <value> - use the value directly
user_config:
enable_ai_review: true
enable_rule_comments: false
enable_complexity_comments: benchmark
enable_docstring_comments: benchmark
enable_security_comments: benchmark
enable_tests_comments: benchmark
enable_comment_suggestions: benchmark
enable_functionality_review: benchmark
enable_approvals: true
ai_review_config:
# The model responses to use for the experiment
# - benchmark - use the model responses from the benchmark reviews
# - llm - call the language model to generate responses
model_responses:
comments_model: benchmark
comment_validation_model: benchmark
comment_suggestion_model: benchmark
complexity_model: benchmark
docstrings_model: benchmark
functionality_model: benchmark
security_model: benchmark
tests_model: benchmark
# The pull request dataset to run the experiment on
pull_request_dataset:
- https://github.com/gdsfactory/gdsfactory/pull/2714
- https://github.com/gdsfactory/cspdk/pull/31
- https://github.com/gdsfactory/cspdk/pull/32
- https://github.com/allthingslinux/tux/pull/204
- https://github.com/rybalka1/devmetrics/pull/9
- https://github.com/albumentations-team/albumentations/pull/1705
- https://github.com/albumentations-team/albumentations/pull/1706
- https://github.com/BuczynskiRafal/stormwater-analysis/pull/16
- https://github.com/BuczynskiRafal/stormwater-analysis/pull/17
- https://github.com/nbhirud/system_update/pull/26
- https://github.com/nbhirud/system_update/pull/27
- https://github.com/nbhirud/system_update/pull/30
- https://github.com/nbhirud/system_update/pull/29
- https://github.com/nbhirud/system_update/pull/28
- https://github.com/writememe/motherstarter/pull/241
- https://github.com/gdsfactory/gdsfactory/pull/2715
- https://github.com/Anselmoo/spectrafit/pull/1282
- https://github.com/Remi-Gau/bids2cite/pull/84
- https://github.com/2lambda123/google-python-fire/pull/5
- https://github.com/2lambda123/google-python-fire/pull/6
- https://github.com/2lambda123/google-python-fire/pull/11
- https://github.com/2lambda123/google-python-fire/pull/14
- https://github.com/2lambda123/analogdevicesinc-hdl/pull/1
- https://github.com/2lambda123/analogdevicesinc-hdl/pull/3
- https://github.com/2lambda123/analogdevicesinc-hdl/pull/5
- https://github.com/2lambda123/analogdevicesinc-hdl/pull/7
- https://github.com/2lambda123/analogdevicesinc-hdl/pull/8
- https://github.com/2lambda123/analogdevicesinc-hdl/pull/9
- https://github.com/2lambda123/deepscan-vscode-deepscan/pull/2
- https://github.com/2lambda123/ultralytics-ultralytics/pull/3
- https://github.com/2lambda123/ultralytics-ultralytics/pull/4
- https://github.com/2lambda123/ultralytics-ultralytics/pull/7
- https://github.com/2lambda123/ultralytics-ultralytics/pull/8
- https://github.com/2lambda123/ultralytics-ultralytics/pull/10
- https://github.com/2lambda123/ultralytics-ultralytics/pull/11
- https://github.com/2lambda123/mcafee2cash/pull/6
- https://github.com/2lambda123/OpenBioLink-ThoughtSource/pull/1
- https://github.com/2lambda123/OpenBioLink-ThoughtSource/pull/5
- https://github.com/kod-kristoff/parallel-corpus-rs/pull/4
- https://github.com/ANIALLATOR114/API-Artisan/pull/2
- https://github.com/ignition-api/8.1/pull/273
- https://github.com/New-dev0/SpotifyIG/pull/1
- https://github.com/supabase-community/postgrest-py/pull/425
- https://github.com/supabase-community/auth-py/pull/488
- https://github.com/LLotme/vscode-surround/pull/4
- https://github.com/Nuitka/Nuitka/pull/2837
review_comment_labels:
- label: correct
question: Is this comment correct?
- label: helpful
question: Is this comment helpful?
- label: comment-type
question: Is the comment type correct?
- label: comment-area
question: Is the comment area correct?
- label: llm-test
question: |
What type of LLM test could this comment become?
- 👍 - this comment is really good/important and we should always make it
- 👎 - this comment is really bad and we should never make it
- no reaction - don't turn this comment into an LLM test
# Benchmark reviews generated by running
# python -m scripts.experiment benchmark <experiment_name>
benchmark_reviews: []
|
SourceryAI
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @sourcery-ai-experiments-bot - I've reviewed your changes and they look great!
Here's what I looked at during the review
- 🟡 General issues: 1 issue found
- 🟢 Security: all looks good
- 🟡 Testing: 1 issue found
- 🟢 Complexity: all looks good
Help me be more useful! Please click 👍 or 👎 on each comment to tell me if it was helpful.
| if flag in spec.kwonlydefaults: | ||
| return repr(spec.kwonlydefaults[flag]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
suggestion (bug_risk): Consider handling the case where kwonlydefaults is None.
If spec.kwonlydefaults is None, this code will raise an AttributeError. It might be safer to check if kwonlydefaults is not None before checking if flag is in kwonlydefaults.
| if flag in spec.kwonlydefaults: | |
| return repr(spec.kwonlydefaults[flag]) | |
| if spec.kwonlydefaults and flag in spec.kwonlydefaults: | |
| return repr(spec.kwonlydefaults[flag]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this comment correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this comment helpful?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the comment type correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the comment area correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What type of LLM test could this comment become?
- 👍 - this comment is really good/important and we should always make it
- 👎 - this comment is really bad and we should never make it
- no reaction - don't turn this comment into an LLM test
| output = helptext.HelpText(component, trace=t) | ||
| self.assertIn('NAME\n FunctionMixedDefaults', output) | ||
| self.assertIn('FunctionMixedDefaults <flags>', output) | ||
| self.assertIn('--alpha=ALPHA (required)', output) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
suggestion (testing): Consider adding assertions for edge cases in testHelpTextFunctionMixedDefaults.
The test currently checks for the presence of flags with and without defaults. It would be beneficial to include assertions for edge cases such as missing flags or incorrect default values to ensure robustness.
| self.assertIn('--alpha=ALPHA (required)', output) | |
| self.assertIn('--alpha=ALPHA (required)', output) | |
| self.assertIn('--beta=BETA\n Default: \'0\'', output) | |
| self.assertNotIn('--gamma=GAMMA', output) | |
| self.assertIn('--beta=BETA\n Default: \'1\'', output, msg="Check incorrect default for beta") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this comment correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this comment helpful?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the comment type correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the comment area correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What type of LLM test could this comment become?
- 👍 - this comment is really good/important and we should always make it
- 👎 - this comment is really bad and we should never make it
- no reaction - don't turn this comment into an LLM test
Description
Related Issue
Types of changes
Checklist: