Hello Stacklok!

This is a simple Next.js project that will take an open source package in a particular ecosystem, and use an LLM (by default, ChatGPT gpt-4o-mini) to determine whether the package is good or bad.

Getting Started

First, install the dependencies:

npm install

Second, get an OpenAI API key and configure it in your environment:

export OPENAI_API_KEY="aa-proj-foo-bar-baz-42"

Finally, run the development server:

npm run dev

Now, if you visit http://localhost:3000 with your browser, you should see the page.

Engineering the prompt

By default, this project does query OpenAI's API, but it does not query it in a useful way to evaluate a package. You'll need to update the prompt to get that result from the LLM.

Open app/api/evaluate/route.js, and navigate to line 20. You should see a large comment that indicates that you should:

/****************** MODIFY THIS PROMPT ******************/

By default, the application simply asks the LLM to return back the given string. Update that prompt to interrogate the LLM.

Now, navigate to line 30. You should see another large comment that indicates that you should:

/****************** ANALYZE THE RESPONSE ******************/

Here, you should take the response and use it to make a determination about whether the package is good or bad. Then you should return that to the end user in the response JSON object.

The response object should look like:

{
  result: "good", /* "good" or "bad" */
  message: "an optional detailed message"
}

Notes

• If you're using console.log-style debugging, note that the app/page.js page runs on the client-side, so console debugging statements will show up in your Browser's console. The app/api/evaluate/route.js API runs on the server-side, so console debugging statements will show up in the console where you started the application.

• By default, Next.js will try to hot-reload any changes that you make to either the frontend page (app/page.js) or the backend API component (app/api/evaluate/route.js). When you save a change to either file, you should see in your console:

  ✓ Compiled / in 42ms
  

  or:

  ✓ Compiled /api/evaluate in 42ms
  

  But stop the application (with Ctrl-C) and restart it (npm run dev) if you're making changes that aren't reloading.