community[minor]: add exclude parameter to DirectoryLoader #17316
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
dosubot
bot
added
size:S
libs/community/langchain_community/document_loaders/directory.py
Outdated
Show resolved
Hide resolved
| @@ -51,6 +52,7 @@ def __init__( | |||
| path: Path to directory. | |||
| glob: Glob pattern to use to find files. Defaults to "**/[!.]*" | |||
| (all files except hidden). | |||
| exclude: patterns to exclude from results, use glob syntax | |||
There was a problem hiding this comment.
Could you provide an example of how to exclude something?
There was a problem hiding this comment.
@eyurtsev this is a direct copy from the docstring in the file system loader:
Should I update all occurrences to keep them consistent?
There was a problem hiding this comment.
I added some docstring examples based on other loaders.
|
Thanks for the quick review @eyurtsev! Just noticed the tests also need some optional deps in CI, will rework this ASAP. |
nejch
force-pushed
the
feat/exclude-directoryloader
branch
from
167616c
to
6f524c4
Compare
dosubot
bot
added
size:M
nejch
force-pushed
the
feat/exclude-directoryloader
branch
4 times, most recently
from
7011967
to
143d007
Compare
dosubot
bot
added
the
lgtm
|
Can merge if tests pass |
nejch
force-pushed
the
feat/exclude-directoryloader
branch
from
143d007
to
bd784e5
Compare
|
Going to help resolve conflicts so we can merge |
eyurtsev
changed the title
|
Can merge when tests pass |
|
Sorry got the wrong lockfile updated there, I was going to get back to the dependency issue but I see you already got around that by mocking it out @eyurtsev, I guess next time this should be green ;) |
|
@nejch awesome! thank you. We need to investigate what's going with the extra dependency, but didn't want that to block from getting your contribution merged in! |
|
Perfect, thanks a lot @eyurtsev. Looks like it's green now 👍 |
…-ai#17316) - **Description:** adds an `exclude` parameter to the DirectoryLoader class, based on similar behavior in GenericLoader - **Issue:** discussed in langchain-ai#9059 and I think in some other issues that I cannot find at the moment 🙇 - **Dependencies:** None - **Twitter handle:** don't have one sorry! Just https://github/nejch --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
excludeparameter to the DirectoryLoader class, based on similar behavior in GenericLoader