Skip to content

fix document converter with large transcript #989

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
May 21, 2025
Merged

fix document converter with large transcript #989

merged 1 commit into from
May 21, 2025

Conversation

lebaudantoine
Copy link
Contributor

Problem:

  • Default Express JSON parser limit (100kb) is insufficient for larger transcription files
  • 2-hour audio transcriptions slightly exceed the 100kb limit, causing request failures

Solution:

  • Implemented custom middleware to apply different JSON parser configurations based on route
  • Applied 500kb limit specifically for transcription conversion endpoints
  • Maintained default limits for all other routes to preserve security

Technical notes:

  • Could not find a built-in Express solution to specify parser config per route
  • Custom middleware conditionally applies the appropriate parser configuration

@lunika lunika requested a review from AntoLC May 21, 2025 10:15
@lunika lunika assigned lunika and lebaudantoine and unassigned lunika May 21, 2025
@lunika lunika added enhancement improve an existing feature bug Something isn't working labels May 21, 2025
@lebaudantoine lebaudantoine force-pushed the fix-yjs-server branch 7 times, most recently from d621c7b to 3cdfdd4 Compare May 21, 2025 10:52
AntoLC
AntoLC approved these changes May 21, 2025
View reviewed changes
Copy link
Collaborator

@AntoLC AntoLC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch ^^

@lebaudantoine lebaudantoine marked this pull request as ready for review May 21, 2025 13:25
@lebaudantoine
🐛(y-provider) increase JSON size limits for transcription conversion
3f2d84b 
Problem:
- Default Express JSON parser limit (100kb) is insufficient for larger
 transcription files
- 2-hour audio transcriptions slightly exceed the 100kb limit, causing request
 failures

Solution:
- Implemented custom middleware to apply different JSON parser configurations
 based on route
- Applied 500kb limit specifically for transcription conversion endpoints
- Maintained default limits for all other routes to preserve security

Technical notes:
- Could not find a built-in Express solution to specify parser config per route
- Custom middleware conditionally applies the appropriate parser configuration
@lebaudantoine lebaudantoine changed the title [Wip] fix document converter fix document converter with large transcript May 21, 2025
@lebaudantoine lebaudantoine merged commit 3f2d84b into main May 21, 2025
20 of 21 checks passed
@lebaudantoine lebaudantoine deleted the fix-yjs-server branch May 21, 2025 23:13
@AntoLC AntoLC mentioned this pull request May 22, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AntoLC AntoLC AntoLC approved these changes

Assignees

@lebaudantoine lebaudantoine

Labels
bug Something isn't working enhancement improve an existing feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@lebaudantoine @AntoLC @lunika