[bug] LLM can generated extremely long titles

[brendan-kellam]

Example:

We should just cut off titles at some character limit, e.g., ~50 characters. Additionally, the title probably shouldn't wrap and instead should be truncated with a "..."

[TheDropZone]

After grabbing the latest openai compatible, I connected my sourcebot to a vLLM served instance of Qwen 3 32B (thinking) and ran into the exact same thing as above

<think> Okay, let's see. The user wants to convert the question "How is the database backed up?" into a short topic title following specific rules. First, I need to check the rules again. No question words like how, where, etc. So "how" is out. The title should focus on the subject, which here is the database backup. Also, no question mark at the end. Capitalize the first letter and make it sound like a filename or category. The original question is about the method of backing up the database. The key terms are "database" and "backup." Since the question is "how is the database backed up," the answer would likely discuss the process or method. So the topic title should be something like "Database Backup Process" or "Database Backup Method." Wait, the example given for "How to setup the database?" becomes "Database Setup." So maybe the structure is [Subject] [Action]. Here, the action is "backup," so "Database Backup." But does that capture the "how"? The user is asking about the method, but the example for "How to setup..." uses just "Setup" without "Method." So maybe "Database Backup" is sufficient. Let me check the example again. "How to setup the database?" → "Database Setup." So the answer is about the setup process, but the title is just the subject and action. Similarly, "How is the database backed up?" would be "Database Backup." But wait, maybe "Database Backup Procedure" is better? However, the example uses "Setup" instead of "Procedure." The user might prefer a shorter title. Let me think. The maximum is 50 characters. "Database Backup" is 14 characters. "Database Backup Process" is 20. "Database Backup Method" is 19. All under 50. But the example for "How to setup the database?" uses "Database Setup," which is 13 characters. So maybe just "Database Backup" is acceptable. But the original question is about the method, so maybe "Backup Method" is too generic. However, the example for "What are the API endpoints?" becomes "API Endpoints," which is just the subject. So following that, "Database Backup" would be the subject and the action. The user is asking how it's done, so the title should indicate the topic, which is the backup of the database. So "Database Backup" seems right. Alternatively, maybe "Database Backup Process" is better, but the example for "How to setup..." uses "Setup" instead of "Setup Process." So perhaps the rule is to omit the process or method part. So the answer is "Database Backup." </think> Database Backup

Tried to be placed as the title. It does a GREAT job at title construction, "Database backup", but the issue being it includes the thinking portion in the title content. Seems to do the same in the above example. Looks like the AI call just needs to only take the response and ignore reasoning content for title?

digging in to

    const result = await generateText({
        model,
        prompt,
    });

the result only contains a content of type text, and text includes ALL the output, reasoning and normal output. Checking result for .reasoning or .reasoningText fields shows they are empty, which would indicate that the backend service for the LLM is probably not properly parsing content into the reasoning portion of the response.

Tested with a call to v1/chat/completions on the backend and sure enough, no reasoning parsing. After adjusting some backend settings, reasoning parsing is acting as desired and the title generation shows as expected.

[owlesia]

+1 It would be nice if there was an option not to display "think" output from the model. Frequently, "think" output (...)is so big that the final response is not being displayed in the UI, and that makes Ask Sourcebot unusable

[brendan-kellam]

I believe what we want to do is to use the extractReasoningMiddleware to extract out reasoning output (denoted by the think tags)

[brendan-kellam]

@owlesia @TheDropZone I shipped a fix in #582 - do you mind giving it a try and seeing if it resolves the issue using the container tagged main? Depending on your model, you may need to adjust the reasoningTag in the config

[owlesia]

works now, thank you!!

[TheDropZone]

Thanks @brendan-kellam. Planning to update our deployment next week and will give it a try.