Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Crawler support for Indexing Pipelines #1322

Closed
oryx1729 opened this issue Aug 6, 2021 · 0 comments · Fixed by #1360
Closed

Add Crawler support for Indexing Pipelines #1322

oryx1729 opened this issue Aug 6, 2021 · 0 comments · Fixed by #1360
Labels
topic:pipeline type:feature New feature or request

Comments

@oryx1729
Copy link
Contributor

oryx1729 commented Aug 6, 2021

In the current implementation, Crawler.crawl() writes documents to JSON files and returns their paths.

Adding an option to return documents will enable connecting a Crawler in an Indexing Pipeline where crawled documents can be indexed directly to a document store.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
topic:pipeline type:feature New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add Crawler support for indexing pipeline
2 participants
@tholor @oryx1729