Adding support for other programming languages

[mwos-xebia]

Duplicates

• I have searched the existing issues

Summary 💡

I have tried asking Agent-GPT to write a simple test in PHP (and in several other languages, different from Python) and every time the application creates its codes in *.py files and only empty comment files.

For example - if I ask Auto-GPT to create a PHP script to do some simple thing, I would like to receive a *.php file in workspace with the content with the expected code. At the moment it looks more or less like this:

<?php

// Code for the DES-HyperNEAT algorithm implementation goes here

?>

<?php

// This is a sample implementation of the DES-HyperNEAT algorithm in PHP

?>

<?php

// Start writing the code for the DES-HyperNEAT algorithm implementation here

?>

<?php

// PoC code for DES-HyperNEAT algorithm in PHP

?>

The best one:

import unittest

class TestMyCode(unittest.TestCase):
    def test_addition(self):
        self.assertEqual(1 + 1, 2)

    def test_subtraction(self):
        self.assertEqual(5 - 3, 2)

    def test_multiplication(self):
        self.assertEqual(2 * 3, 6)

    def test_division(self):
        self.assertEqual(10 / 2, 5)

if __name__ == '__main__':
    unittest.main()
<?php

class DesHyperNEATTests extends PHPUnit_Framework_TestCase
{
    public function testExample()
    {
        $this->assertTrue(true);
    }
}
<?php\n\n// Tests for the DES-HyperNEAT algorithm implementation go here\n\n?>

For this script, despite being asked to create a working script in PHP, he created himself a working script in Python, despite my forbidding him to compile code in that language:

import unittest
from des_hyperneat import DESHyperNEAT

class DESHyperNEATTest(unittest.TestCase):
    def test_initialization(self):
        des = DESHyperNEAT()
        self.assertIsInstance(des, DESHyperNEAT)

    def test_generate_cppn(self):
        des = DESHyperNEAT()
        cppn = des.generate_cppn()
        self.assertIsInstance(cppn, CPPN)

    def test_generate_network(self):
        des = DESHyperNEAT()
        cppn = des.generate_cppn()
        network = des.generate_network(cppn)
        self.assertIsInstance(network, Network)

    def test_evaluate_network(self):
        des = DESHyperNEAT()
        cppn = des.generate_cppn()
        network = des.generate_network(cppn)
        input = [1, 2, 3, 4, 5]
        output = des.evaluate_network(network, input)
        self.assertIsNotNone(output)

if __name__ == '__main__':
    unittest.main()

After which he assessed that the job was done and the work could be finished :)

Examples 🌈

I do not have links to implementation

Motivation 🔦

I would like this tool to be 'multipurpose'. Don't let it close only to the needs of people associated with one programming language. I suspect that people writing in Go, Java or other languages have similar problems.

[mwos-xebia]

@k-boikov I don't think I would agree that this is unrelated to Auto-GPT and that it is somehow a limitation of AI.

One of the Auto-GPT commands is "execute_python_script" and some of the prompts directly require the code returned by the model to be in Python code. In the same way, Python scripts are created completely unnecessarily in the workspace of the application, which are not required but GPT is asked for in the sequence of prompts sent to the model by the application.

[adam-paterson]

@mwos-xebia This is more to do with the initial parameters passed to the agent rather than AI or AutoGPT itself.

$ agpt -C php-test/ai_settings.yaml --debug --continuous

php-test/ai_settings.yaml

ai_name: PHP-GPT
ai_role: An AI agent that can take the defined goals and generate a PHP project that can accomplish them.
ai_goals:
  - All generated code should be valid PHP code with the .php file extension.
  - Read the contents of project.txt to understand the project requirements and deliverables.
  - Do not use any external libraries or frameworks.
  - Do not checkout any code from any source control system.
  - Do not attempt to configure any web-server or PHP interpreter.
  - Only write the project files and assume they work.

project.txt

Project Brief: Simple PHP Contact Form

Objective:
Create a simple PHP contact form that allows users to submit their name, email address, and message. The form should validate user input and send an email to the specified recipient address upon successful submission.

Requirements:

Frontend:
- Create an HTML form with the following input fields:
--Name (text input)
--Email (email input)
--Message (textarea)
- Include a submit button to submit the form.
- Add basic CSS styling to make the form visually appealing.

Backend:
- Create a PHP script to handle the form submission.
- Validate user input:
-- Ensure that all fields are filled out.
-- Validate the email address format.
-- Limit the message length to a reasonable value (e.g., 500 characters).
- If the validation fails, display an error message and return the user to the form with the submitted values pre-filled.
- If the validation is successful, send an email to a specified recipient address with the user's submitted information.
- Display a success message to the user after successful form submission.

Security:
- Implement basic security measures to protect against common vulnerabilities, such as:
-- Cross-Site Scripting (XSS): sanitize user input before displaying it on the page.
-- Email header injection: validate and sanitize email input before using it in email headers.

Deliverables:
- An HTML file (contact.html) containing the contact form and basic CSS styling.
- A PHP script (submit.php) to handle form submission, input validation, and email sending.
- A README file with instructions on how to set up and use the contact form.

Note: You can assume that the server has PHP and an email-sending function (e.g., mail()) correctly configured.

Output Archtecture

auto_gpt_workspace/
├── contact.html
├── file_logger.txt
├── project.txt
├── send_mail.php
├── style.css
└── submit.php

contact.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Contact Form</title>
    <link rel="stylesheet" href="style.css">
    <style>
        /* Add basic CSS styling to the contact form */
    </style>
</head>
<body>
    <form method="post" action="submit.php">
        <label for="name">Name:</label>
        <input type="text" id="name" name="name" required>
        <label for="email">Email:</label>
        <input type="email" id="email" name="email" required>
        <label for="message">Message:</label>
        <textarea id="message" name="message" required></textarea>
        <input type="submit" value="Submit">
    </form>
</body>
</html>

submit.php

<?php
if($_SERVER["REQUEST_METHOD"] == "POST"){
  $name = $_POST["name"];
  $email = $_POST["email"];
  $message = $_POST["message"];

  // Validate user input
  if(empty($name) || empty($email) || empty($message)){
    $error = "All fields are required.";
    echo $error;
    exit();
  }

  if(!filter_var($email, FILTER_VALIDATE_EMAIL)){
    $error = "Invalid email format.";
    echo $error;
    exit();
  }

  if(strlen($message) > 500){
    $error = "Message is too long.";
    echo $error;
    exit();
  }

  // Send email
  $to = "your-email@example.com";
  $subject = "New Contact Form Submission";
  $body = "Name: $name\nEmail: $email\nMessage: $message";

  if(mail($to, $subject, $body)){
    $success = "Your message has been sent!";
    echo $success;
  } else {
    $error = "Something went wrong. Please try again later.";
    echo $error;
  }
}
?>

send_mail.php

<?php

$name = $_POST['name'];
$email = $_POST['email'];
$message = $_POST['message'];

// validate input
$name = filter_var($name, FILTER_SANITIZE_STRING);
$email = filter_var($email, FILTER_SANITIZE_EMAIL);
$message = filter_var($message, FILTER_SANITIZE_STRING);

// set email parameters
$to = 'you@example.com';
$subject = 'New Contact Form Submission';
$headers = 'From: ' . $email . '\r\n' .
           'Reply-To: ' . $email . '\r\n' .
           'X-Mailer: PHP/' . phpversion();

// send email
if (mail($to, $subject, $message, $headers)) {
    echo '<p>Thank you for your message!</p>';
} else {
    echo '<p>Oops! Something went wrong. Please try again later.</p>';
}

?>

[Boostrix]

I also cannot confirm Auto-GPT being restricted to just Python, been using it a lot to code up stuff in other languages. So it's an issue of priming it correctly - obviously, that's easier if you've got a coding background and are familiar with the corresponding lingo to use. But if in doubt, just ask Auto-GPT/GPT to rephrase everything for you to ensure that indeed a PhP x.xx project is generated.

Other than that, I would suggest to close this RFE for now ?

(What you see above is a masterpiece of using proper lingo to make the GPT LLM look up the correct stuff and parameterize it internally as needed - in fact, @adam-paterson should consider getting involved in helping with prompt engineering/benchmarking Auto-GPT, or at the very least help us implement the wizard feature, so that working workflows [like his!] can be well formalized and offered to less-experienced users: #3820 )

[adam-paterson]

Absolutely I would love to be involved. I've halted my plug-in development while the re-architecture project is going ahead so perhaps I can focus on assisting here.

[Boostrix]

My suggestion for now would then be to take your existing work and adapt it to be used as the foundation for the benchmarks, as outlined by @merwanehamadi

I would change the specs/project.txt to create a python backend instead.
Once we have that working, we would add a new paragraph about "unit testing" the whole shebang.
This would check for file names, extensions and types - check if the HTML file is valid. And probably run XPath queries for the HTML elements that the agent needs to add to the HTML page.

• Help us build challenges! #3835 (comment)
• Regression Test: Auto-GPT should always be able to build this simple contact form #3901 (comment)

The final phase would then be actual testing by firing up a Python HTTPServer and using a bunch of HTTP requests to test whether the backend is working as expected. @merwanehamadi also mentioned Selenium being an option, but I believe for starters using just the unit test approach should work fine, and would work in a headless environment.

Once all that is working, we should open a PR to get this committed to the repo.
Afterwards we could look at adding an outer agent to mutate some of those specs in project.txt to use different file names/elements and generate the matching unit tests. That way, we'd end up with good testing coverage at the mere cost of running a few outer loops to mutate those specs.

For instance, each HTML element could use 5 different colors - and that's where the deliverables should differ obviously, and a corresponding XPath query should also return the color.

As long as we spell out everything in excruciating detail, Auto-GPT should be quite capable of coming up with the unit testing portion of the code, and even come up with a way to mutate those specs (for instance by writing to 10 different project folders inside the work space).

I suppose, helping this way will have a more immediate effect/benefit to the project, compared to starting yet another plugin or another redundant PR ...

PS: If everything works as expected, all we'll be doing is editing project.txt :-)

[github-actions]

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

[github-actions]

This issue was closed automatically because it has been stale for 10 days with no activity.