[BUG] o3-mini token counts not getting correctly sent to Langfuse

[dkindlund]

Describe the bug
When using o3-mini on any ChatFlow or AgentFlow in 2.2.7-patch1, I'm seeing only zero (0) token counts in Langfuse.

To Reproduce
Steps to reproduce the behavior:

1. Create a ChatFlow using o3-mini
2. Configure ChatFlow to send trace data to Langfuse
3. See trace in Langfuse with zero token counts

Expected behavior
I'd expect to see Langfuse traces with accurate token counts.

Screenshots

Setup

• Installation [docker]
• Flowise Version [2.2.7-patch1]
• OS: [Linux]
• Browser [chrome]

[GittyFriend]

I've encountered a problem with Flowise version 2.2.7-patch1 when using the o3-mini model in both ChatFlow and AgentFlow. Specifically, when I configure the flows to send trace data to Langfuse, the token counts are recorded as zero, which isn't expected.

Steps to reproduce:

Create a ChatFlow using the o3-mini model.
Configure the ChatFlow to send trace data to Langfuse.
Observe that the token counts in Langfuse are recorded as zero.​
Expected behavior: I would expect to see accurate token counts in Langfuse traces.​

Environment:

Flowise Version: 2.2.7-patch1
OS: Linux
Browser: Chrome​
GitHub
GitHub
+2
GitHub
+2
GitHub
+2
GitHub
+3
GitHub
+3
GitHub
+3

[HenryHengZJ]

I think its because of the tiktoken library, perhaps need to update that