Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VCR Fix for debug_code_challenge_a #3926

Closed

Conversation

gravelBridge
Copy link
Contributor

Background

When using @pytest.vcr.mark, and then using docker, the test hangs as vcr intercepts the docker call, and is stuck inside an infinite loop. This is due to the vcr library checking if the instance is part of the vcr class, and if it isn't it's stuck inside a while loop. The test is from @merwanehamadi

Changes

I modified the vcr library to skip the while loop if the instance is docker. This prevents the hang, and tests using @pytest.vcr.mark work correctly now. As a result, I modified requirements.txt to use my slightly modified version of vcr with the fix, find pr here: kevin1024/vcrpy#692

Documentation

The test comes from @merwanehamadi, and I modified the vcrpy library to allow @pytest.mark.vcr to work without hanging the test.

Test Plan

I have tested the test with my version of vcrpy, and everything works perfectly.

PR Quality Checklist

  • My pull request is atomic and focuses on a single change.
  • I have thoroughly tested my changes with multiple different prompts.
  • I have considered potential risks and mitigations for my changes.
  • I have documented my changes clearly and comprehensively.
  • I have not snuck in any "extra" small tweaks changes

I have not added tests as all I have done is fix the vcr hanging issue, which will not affect the functionality of vcr if the instance is not docker.

@vercel
Copy link

vercel bot commented May 6, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 6, 2023 9:11pm

@github-actions
Copy link
Contributor

github-actions bot commented May 6, 2023

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

@vercel vercel bot temporarily deployed to Preview May 6, 2023 21:11 Inactive
@dschonholtz
Copy link
Contributor

Fantastic!
I'll try to test this later today. Most likely for tomorrow.

@dschonholtz dschonholtz self-requested a review May 12, 2023 16:47
@github-actions
Copy link
Contributor

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

@github-actions github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label May 13, 2023
@dschonholtz
Copy link
Contributor

A bajilion years later. This appears to work.
The agent removes all errors by re-writing the code.py with a single print statement, but the test still passes for some reason.
I'm gonna try to figure out why.

@dschonholtz
Copy link
Contributor

You must manually remove the existing vcrpy lib though, so we will need to figure out why that is and how to mitigate that going forward

nums = [2, 7, 11, 15]
target = 9
result = two_sum(nums, target)
print(result) # Output: [0, 1]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't tell it what the expected output is, as otherwise the agent might simplify this to just be print('[0, 1]')


with contextlib.suppress(SystemExit):
create_code_agent.start_interaction_loop()
assert "[0, 1]" in execute_python_file(file_path), "Expected the output to be [0, 1]"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason this doesn't appear to be executing.
I have a file that was re-written to print('Hello World')
And this still appears to be passing.
I'm about to get on a plane. But if you could debug that that'd be sweet

@dschonholtz dschonholtz mentioned this pull request May 18, 2023
6 tasks
@github-actions github-actions bot removed the conflicts Automatically applied to PRs with merge conflicts label May 25, 2023
@github-actions
Copy link
Contributor

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

@github-actions
Copy link
Contributor

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

@github-actions github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label May 30, 2023
@Pwuts
Copy link
Member

Pwuts commented Jun 14, 2023

Already picked up in another PR according to Wayne. Closing :)

@Pwuts Pwuts closed this Jun 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
conflicts Automatically applied to PRs with merge conflicts size/xl testing
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

5 participants