First pass at generating Question Summaries for Assistants. Run by
calling `poetry run python -m pingpong export summarize {class_id}
{days}`. This will output the JSON print of AIAssistantSummaries.
## Prompt
The input is formatted as JSON for easier parsing. The Chat Completion
call uses Structured Output to enable better management of the model's
output.
```json
{
"threads": [
{
"id": 122,
"user_messages": ["User message 1", "User message 2"]
},
]
}
```
### Output predictability
Despite the extensive prompt instructions, the original prompt was prone
to hallucination by coming up with non-existent questions and user
messages.
I added additional language like "Do not output any JSON if there are no
valuable questions or threads.", which seems to have resolved the issue.
We need to closely monitor results.
Edit: Added additional language so that the same thread isn't picked as
a representative quote more than once.
<details><summary>Example</summary>
```
{
"assistant_id": 43,
"questions": [
{
"question": "How do I debug my program efficiently?",
"relevant_threads": [
{
"thread_id": 148,
"quote": "User message 5"
},
{
"thread_id": 293,
"quote": "User message 9"
},
{
"thread_id": 510,
"quote": "User message 12"
}
]
}
]
}
```
</details>
## Next Steps
- [ ] Prepare email template for each assistant summary
- [ ] Prepare email template for summarization email
- [ ] Write parser function that creates the final summarization email
- [ ] Add checks for private classes
- [ ] Create summarizer function that sends out summarization emails for
all eligible classes
- [ ] Add UserClassRole table field for unsubscribing from summarization
emails.
- [ ] Add UI for unsubscribing from summarization emails.