[DRAFT]feat: boost revise_paragraph_classification

[prnake]

This is almost a draft for discussion, including several changes:

• max_paragraph_length, to prevent too large paragraphs
• fix issue in Preprocessing of header blocks #45
• a much faster revise_paragraph_classification implemention

Base on https://gist.github.com/prnake/e40fb3dd9b0af1f7a5fc73f2ee5236e9, it's much faster than original implementation with the same result, reduce time complexity from O(n^2) to O(n).

===== Test data size: 1000 =====
Performance comparison for 1000 paragraphs:
Original function time: 0.055953 seconds
Optimized function time: 0.001080 seconds
Performance improvement: 51.79x
Results consistency: Pass

===== Test data size: 5000 =====
Performance comparison for 5000 paragraphs:
Original function time: 1.181464 seconds
Optimized function time: 0.005710 seconds
Performance improvement: 206.91x
Results consistency: Pass

===== Test data size: 10000 =====
Performance comparison for 10000 paragraphs:
Original function time: 4.699047 seconds
Optimized function time: 0.011343 seconds
Performance improvement: 414.28x
Results consistency: Pass

===== Test data size: 20000 =====
Performance comparison for 20000 paragraphs:
Original function time: 18.514682 seconds
Optimized function time: 0.022034 seconds
Performance improvement: 840.28x
Results consistency: Pass

[miso-belica]

Thank you for the changes. Can you write tests for these, please? It seems like a lot of code with quite a big impact. Also, it's 3 different changes so maybe we could split the smaller ones into PRs to merge them faster.

[prnake]

Thank you for the changes. Can you write tests for these, please? It seems like a lot of code with quite a big impact. Also, it's 3 different changes so maybe we could split the smaller ones into PRs to merge them faster.

Sure, I’ll do these later.