Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #913: FileSystemDocumentLoader: ignore empty/blank documents, improved error/warn messages #920

Merged
merged 10 commits into from Apr 16, 2024

Conversation

langchain4j
Copy link
Owner

@langchain4j langchain4j commented Apr 11, 2024

Context

When loading with FileSystemDocumentLoader and it encounters an empty file, a WARN message in the logs looks like an exception and is confusing.
See #913

Change

  • Ignore empty/blank documents when loading multiple files with DocumentIsBlankException
  • Changed log message to look not like an exception

Checklist

Before submitting this PR, please check the following points:

  • I have added unit and integration tests for my change
  • All unit and integration tests in the module I have added/changed are green
  • All unit and integration tests in the core and main modules are green
  • I have added/updated the documentation
  • I have added an example in the examples repo (only for "big" features)
  • I have added my new module in the BOM (only when a new module is added)

Checklist for adding new embedding store integration

  • I have added a {NameOfIntegration}EmbeddingStoreIT that extends from either EmbeddingStoreIT or EmbeddingStoreWithFilteringIT

@langchain4j langchain4j changed the title Fix #913: FileSystemDocumentLoader: improved error/warn messages Fix #913: FileSystemDocumentLoader: ignore empty/blank documents, improved error/warn messages Apr 15, 2024
@langchain4j langchain4j merged commit 03528af into main Apr 16, 2024
6 checks passed
@langchain4j langchain4j deleted the fix_913 branch April 16, 2024 06:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant