[Question]:About updown_concat_xgb.model

[eshine996]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (Language Policy).
• Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• Please do not modify this template :) and fill in all the required fields.

Describe your problem

What model is updown_comcat-xgb.model? What problem is being solved?

[dosubot]

The updown_concat_xgb.model in RAGFlow is an XGBoost machine learning model used within the PDF parser component. Its primary function is to help determine whether two text boxes in a PDF should be concatenated vertically (i.e., whether they belong to the same logical block of text across rows or pages). This is part of the document structure analysis process, where the parser decides how to group and merge text fragments for more accurate document understanding.

The model is invoked in the _concat_downward function of pdf_parser.py, where features describing the relationship between two text boxes are computed and passed to the model for prediction. If the model predicts a high enough probability, the boxes are merged as part of the same block. This helps improve the accuracy of text extraction and logical structure reconstruction from PDFs, especially in complex layouts. The model file should be located at rag/res/deepdoc/updown_concat_xgb.model and is automatically downloaded from the InfiniFlow/text_concat_xgb_v1.0 repository during setup if missing [source] [source].

If you need to troubleshoot missing model errors, ensure the file exists in the correct directory as described above.

To continue talking to Dosu, mention @dosu-bot.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[eshine996]

@dosu-bot What is the principle of updown_comcat-xgb.model and how is it trained?

[dosubot]

The principle behind the updown_concat_xgb.model is to predict whether two text boxes in a PDF should be concatenated vertically, meaning whether they belong to the same logical block of text. This is achieved by computing a set of features that describe the relationship between two text boxes—such as their relative positions, sizes (mean height and width), layout type, and layout number—and passing these features to the model for prediction. If the model predicts a probability above a certain threshold (0.5), the boxes are merged as part of the same block. This approach helps improve the accuracy of text extraction and logical structure reconstruction from PDFs, especially in complex layouts. The model is used within the _concat_downward function in the PDF parser component of RAGFlow [source].

Regarding how the model is trained: I was unable to find specific documentation or code comments in the repository that describe the exact training process, dataset, or feature engineering steps used to train the updown_concat_xgb.model. The available information confirms that it is an XGBoost model and suggests it is trained on labeled examples of text box pairs, but details about the data and methodology are not provided in the accessible sources.

To continue talking to Dosu, mention @dosu-bot.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}