Skip to content

[Feature Request]: The chunking configurations available in the RAGFlow backend are more comprehensive than those provided via the HTTP API. We hope that more chunking configuration parameters can be added to the HTTP API. #12307

Description

@viphonestsong

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (Language Policy).
  • Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
  • Please do not modify this template :) and fill in all the required fields.

Is your feature request related to a problem?

Describe the feature you'd like

I hope that the configuration options for these features can be added to the parser_config parameter of the provided HTTP API interface. This will allow my business system to implement more precise chunking settings based on specific rules, thereby achieving the goal of accurate content retrieval.

Describe implementation you've considered

When performing file chunking in the RAGFlow backend, different chunking methods are equipped with targeted chunk configurations. For example, the General chunking method includes features such as keyword extraction and TOCEnhance. However, the HTTP API interface does not fully provide the configuration parameters for these chunk settings—currently, I can only configure the parameter {"use_raptor": false/true}.

Within the settings provided by the RAGFlow backend, I can manually configure TOC Enhance, as well as the generation quantities for Auto-keyword and Auto-question features

From the content of the HTTP API documentation below, I am unable to configure settings such as whether to enable TOC Enhance, or the quantity for automatic keyword extraction/automatic question extraction.

"
Configuration settings for the knowledge base parser. The properties in this JSON object vary depending on the selected "chunk_method".
If the "chunk_method" is set to "naive", the "parser_config" object contains the following properties:
"chunk_token_num": Defaults to 256.
"layout_recognize": Defaults to true.
"html4excel": Indicates whether to convert Excel documents to HTML format. Defaults to false.
"delimiter": Defaults to "\n".
"task_page_size": Defaults to 12. PDF only.
"raptor": RAPTOR-specific settings. Defaults to: {"use_raptor": false}.
If the "chunk_method" is set to "qa", "manuel", "paper", "book", "laws", or "presentation", the "parser_config" object contains the following property:
"raptor": RAPTOR-specific settings. Defaults to: {"use_raptor": false}.
If the "chunk_method" is set to "table", "picture", "one", or "email", the "parser_config" is an empty JSON object.
"

Documentation, adoption, use case

Additional information

ragflow verison v0.22

Metadata

Metadata

Assignees

Labels

💞 featureFeature request, pull request that fullfill a new feature.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions