Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip flaky challenges #4573

Merged

Conversation

waynehamadi
Copy link
Contributor

Background

Information retrieval challenge B and debug challenge A are flaky, they don't pass reliably.
We're going to skip them for now to not waste people's time, but we will put them back as soon as possible.

Changes

Documentation

Test Plan

PR Quality Checklist

  • My pull request is atomic and focuses on a single change.
  • I have thoroughly tested my changes with multiple different prompts.
  • I have considered potential risks and mitigations for my changes.
  • I have documented my changes clearly and comprehensively.
  • I have not snuck in any "extra" small tweaks changes.
  • I have run the following commands against my code to ensure it passes our linters:
    black .
    isort .
    mypy
    autoflake --remove-all-unused-imports --recursive --ignore-init-module-imports autogpt tests --in-place

@vercel
Copy link

vercel bot commented Jun 4, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
docs ⬜️ Ignored (Inspect) Jun 4, 2023 2:39pm

@github-actions github-actions bot added the size/s label Jun 4, 2023
@codecov
Copy link

codecov bot commented Jun 4, 2023

Codecov Report

Patch coverage has no change and project coverage change: -0.09 ⚠️

Comparison is base (79ba85a) 69.65% compared to head (2e68786) 69.57%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4573      +/-   ##
==========================================
- Coverage   69.65%   69.57%   -0.09%     
==========================================
  Files          72       72              
  Lines        3523     3523              
  Branches      562      562              
==========================================
- Hits         2454     2451       -3     
- Misses        881      884       +3     
  Partials      188      188              

see 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@Pwuts Pwuts merged commit 59d31b0 into Significant-Gravitas:master Jun 4, 2023
11 checks passed
kuwork added a commit to kuwork/Auto-GPT that referenced this pull request Jun 12, 2023
* cn-dev: (64 commits)
  针对国内的情况,翻译成中文
  Update challenge scores
  Update version numbers for v0.4.0 release
  Add `replace_in_file` command (Significant-Gravitas#4565)
  Update bulletin with highlights for v0.4.0 release (Significant-Gravitas#4576)
  Skip flaky challenges (Significant-Gravitas#4573)
  Fix `test_web_selenium` (Significant-Gravitas#4554)
  Clean up CI git logic
  remove information retrieval challenge b from beaten challenges
  Fix CI git authentication and cassettes
  debug
  Fix CI git diff
  Fix CI git authorization
  Update submodule reference
  Update current score
  Cache Python Packages in the CI pipeline (Significant-Gravitas#4488)
  Fix pushing cassettes in CI
  Remove news about config (Significant-Gravitas#4553)
  Fix CI for internal PRs with CI changes (Significant-Gravitas#4552)
  Update BULLETIN.md
  ...

# Conflicts:
#	BULLETIN.md
#	CONTRIBUTING.md
#	autogpt/agent/agent.py
#	autogpt/app.py
#	autogpt/llm/llm_utils.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

3 participants