Skip to content

Improve QA bots with GH Actions #238

@evgenydmitriev

Description

@evgenydmitriev

Our crowdsourced Crypto Attack Wiki has a number of bots in charge of automated quality checks. Things have changed since we first implemented the quality checks, and now most of the functionality is covered by GPT-like services. It should be possible to refactor existing quality checks to fit the whole pull request (instead of splitting it into chunks) into one API request. Anything that goes beyond LLM's context window, can safely be cut out and produce a warning to submit smaller PRs. Although OpenAI API still lacks web browsing functionality, there are alternative solutions offered by Goggle or Azure. Reach out if you need test API keys.

To participate in the challenge, submit a pull request that refactors existing bots into one GPT-like API request and request a review from this issue assignees. Below are some improvement ideas, but feel free to suggest other functionality.

### Tasks
- [ ] Fact-checking. [Azure OpenAI Service](https://learn.microsoft.com/en-us/answers/questions/1345055/can-the-gpt-models-in-azure-openai-service-access#:~:text=Yes%2C%20the-,GPT%20models%20in%20Azure%20OpenAI%20Service%20can%20access%20the%20internet,-.%20This%20is%20a) and [reverse-engineered Bard libraries](https://pypi.org/project/bardapi/) have that functionality, but there are plenty of alternative solutions.
- [ ] Spellcheck can safely be moved to GPT with a carefully crafted prompt.
- [ ] Hugo SSG formatting checker. The fact that an article is a Markdown document with headers for Hugo SSG would probably need to be specified in the prompt anyway for adding the spellcheck functionality. Asking the LLM to check formatting should be an easy addition.
- [ ] Existing articles' check. Getting the directory listing is the easiest way to do it, and you already implemented it.
- [ ] Article [submission guidelines](https://github.com/1712n/dn-institute/issues/277#:~:text=Submission%20Guidelines,-Ensure%20your%20submission) checks. A good idea might be adding a markdown file with guidelines to load guidelines from there instead of hard-coding them in the prompt.
- [ ] Plagiarism. Probably requires additional testing of existing LLM services, but there's nothing fundamental that would prevent models with internet access to spot copy-paste work.

All submissions will be reviewed by the wiki maintainers, and additional changes to your pull request may be asked of you to bring your submission to improve code quality, security, and/or efficiency.

How to Claim Bounty

Email your bitcoin or stablecoin payment address to challenge.bounties@dn.institute. Expect all payouts to be completed by the end of the month.


🎉 @Kseymur manged to figure out most of the needed improvements using Claude and Brave API.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions