Added support for runtime analysis and add input to test results#13
Conversation
|
Actually, I'll just include input for sample test cases only, wait for next commit. This means test results entries will have a property saying whether they are samples or not Also another note, perhaps a pop up showing their runtime after all test cases a solved would be a way to display the runtime. |
weebao
left a comment
weebao
left a comment
There was a problem hiding this comment.
Use response_format for OpenAI chat.completions.create function. Check the inline code review comment for more details
| {"role": "system", "content": "You are a code analysis assistant. Analyze the user given code and ONLY provide a single time complexity rating e.g. O(n^2), O(log n), etc. Nothing else. Do not provide any other data or analysis."}, | ||
| {"role": "user", "content": f"Analyze this code:\n\n```python\n{code}\n```"} | ||
| ], | ||
| temperature=0 |
There was a problem hiding this comment.
@hdngo Restricting the format by prompt would fail sometimes, so it's best to use response_format param for this. You can create a pydantic model and put it in there
https://platform.openai.com/docs/guides/structured-outputs
from pydantic import BaseModel
from openai import OpenAI
client = OpenAI()
class CalendarEvent(BaseModel):
name: str
date: str
participants: list[str]
completion = client.beta.chat.completions.parse(
model="gpt-4o-2024-08-06",
messages=[
{"role": "system", "content": "Extract the event information."},
{"role": "user", "content": "Alice and Bob are going to a science fair on Friday."},
],
response_format=CalendarEvent,
)
event = completion.choices[0].message.parsedThere was a problem hiding this comment.
Thanks, I didn't know about that, please check the latest commit 130ba87 🙂
|
Closes #12 |
2 participants
For now, runtime analysis only happens when all test cases are cleared since I don't wanna make unneeded API calls with errored code for example. Let me know if you want this adjusted.
Based on testing with the TA seems like showing the input for tests is more intuitive for the users so we'll go along with that.