Regression Test: Auto-GPT should always be able to build this simple contact form

[waynehamadi]

We need your help to build an ambitious regression test.

As suggested by @Boostrix, one of the tasks Auto-GPT is able to perform right now is building a simple contact form:
#2775 (comment)

STEP 1: Create the same set of instructions and run this agent in a test.

STEP 2: Test whether the website is working (not easy)

@Adam-Peterson has created a php website using these instructions.

ai_name: PHP-GPT
ai_role: An AI agent that can take the defined goals and generate a PHP project that can accomplish them.
ai_goals:
  - All generated code should be valid PHP code with the .php file extension.
  - Read the contents of project.txt to understand the project requirements and deliverables.
  - Do not use any external libraries or frameworks.
  - Do not checkout any code from any source control system.
  - Do not attempt to configure any web-server or PHP interpreter.
  - Only write the project files and assume they work.

projects.txt

Project Brief: Simple PHP Contact Form

Objective:
Create a simple PHP contact form that allows users to submit their name, email address, and message. The form should validate user input and send an email to the specified recipient address upon successful submission.

Requirements:

Frontend:
- Create an HTML form with the following input fields:
--Name (text input)
--Email (email input)
--Message (textarea)
- Include a submit button to submit the form.
- Add basic CSS styling to make the form visually appealing.

Backend:
- Create a PHP script to handle the form submission.
- Validate user input:
-- Ensure that all fields are filled out.
-- Validate the email address format.
-- Limit the message length to a reasonable value (e.g., 500 characters).
- If the validation fails, display an error message and return the user to the form with the submitted values pre-filled.
- If the validation is successful, send an email to a specified recipient address with the user's submitted information.
- Display a success message to the user after successful form submission.

Security:
- Implement basic security measures to protect against common vulnerabilities, such as:
-- Cross-Site Scripting (XSS): sanitize user input before displaying it on the page.
-- Email header injection: validate and sanitize email input before using it in email headers.

Deliverables:
- An HTML file (contact.html) containing the contact form and basic CSS styling.
- A PHP script (submit.php) to handle form submission, input validation, and email sending.
- A README file with instructions on how to set up and use the contact form.

Note: You can assume that the server has PHP and an email-sending function (e.g., mail()) correctly configured.

[Boostrix]

STEP 2: Test whether the website is working (not easy)

should be fairly doable, just ask GPT to generate a set of unit tests in Python for the project (adapt the specs as needed to provide sufficient info for the tests, i.e. remove the last phrase in the yaml file, and for testing purposes, just make the backend in python):

• check the paths/file names and file extensions involved
• files should exist, be valid PHP/HTML respectively
• when running an XPath expression on the HTML, the widgets and CSS styling must be found

Likewise for the backend.

If in doubt, create a set of unit tests that fire up a Python HTTPServer instance to actually "test" the website that way (again, I believe Selenium would be overkill)

@merwanehamadi apparently you misattributed this to the wrong person, it was posted by @adam-paterson in the other issue - and I am sure we can rework the project specs to use a Python back-end for unit testing purposes (even though Python seems to have CGIHTTPServer module that does serve Php CGIs)

[waynehamadi]

yeah probably easier to do in python.
I quoted adam in the post.

[Boostrix]

Seems you can assign that to @adam-paterson and myself, as per: #2775 (comment)

Draft at: https://gist.github.com/Boostrix/7111a99b6d890b920a6cf9f5c160ece7

I quoted adam in the post.
Once again, it's @adam-paterson :-)

[waynehamadi]

@Boostrix I think adam needs to participate to the conversation for me to be able to assign him.

Do you know how to write challenges ? A challenge is just a pytest test.
You can see many examples under tests/integration/challenges

In this case the challenge will be a regression test, meaning it's already working.

[Boostrix]

understood, I think at least for starters, we're going to tinker around with adapting project.txt (its specs) to also produce the required pytest stubs based on the deliverables and specs and then take it from there.

Obviously, this would require some stubs to actually run auto-gpt as part of pytest, to get an agent to process a custom ai_settings.yaml file and then open project.txt to generate the project.

If you can provide us with pointers, it would obviously help enormously, to get a handle to the number of steps taken (at least thinking steps)

[waynehamadi]

@Boostrix yeah the challenges are all written in
tests/integration/challenges
you can copy the tests written there and adapt them for this challenge.

[github-actions]

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

[github-actions]

This issue was closed automatically because it has been stale for 10 days with no activity.