Cookboook with Firecrawl #200#206
Conversation
AnandKrishnamoorthy1
requested review from
CoderOMaster,
HarshaNalluru,
r4ghu and
yatharthk2
as code owners
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 8 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "#pip install firecrawl-py moss python-dotenv" |
| " id=f\"firecrawl-{index}\",\n", | ||
| " text=page.markdown,\n", | ||
| " metadata={\"source_url\": page.url, \"title\": page.title or \"\"},\n", |
| "async def prepare_knowledge_base(urls: list[str], limit: int = 10) -> tuple[MossClient, str]:\n", | ||
| " validate_configuration(urls)\n", | ||
| " crawled_pages = crawl_urls(urls, limit=limit)\n", | ||
| " documents = crawled_pages_to_moss_docs(crawled_pages)\n", | ||
| "\n", | ||
| " if not documents:\n", | ||
| " raise RuntimeError(\"Firecrawl returned no markdown content to index.\")\n", | ||
| "\n", | ||
| " index_name = f\"firecrawl-cookbook-{uuid.uuid4().hex[:8]}\"\n", | ||
| " client = MossClient(MOSS_PROJECT_ID, MOSS_PROJECT_KEY)\n", | ||
| "\n", |
| ├──> Markdown Normalization | ||
| │ (clean text, remove chrome) | ||
| │ |
There was a problem hiding this comment.
@AnandKrishnamoorthy1, can you please have look into this copilot comment?
| # Optional: default index name used by the notebook | ||
| MOSS_INDEX_NAME=firecrawl-demo | ||
|
|
| "display_name": "Python [conda env:base] *", | ||
| "language": "python", | ||
| "name": "conda-base-py" |
| " print(f\" {item.text[:200].strip()}\")\n", | ||
| "\n", | ||
| "\n", | ||
| "# Build knowledgebase and query it in one step\n", |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
|
Hi @AnandKrishnamoorthy1 , thank you for working on this pr. were you able to solve the ai comments ? |
@yatharthk2 Yes, I have. Lmk if you have any other comments, or else you can merge and close this PR |
yatharthk2
left a comment
yatharthk2
left a comment
There was a problem hiding this comment.
added two small comments, will approve after this
| ├──> Markdown Normalization | ||
| │ (clean text, remove chrome) | ||
| │ |
There was a problem hiding this comment.
@AnandKrishnamoorthy1, can you please have look into this copilot comment?
| ## Installation | ||
|
|
||
| ```bash | ||
| pip install firecrawl-py moss python-dotenv |
There was a problem hiding this comment.
can you please add pyproject
Pull Request Checklist
Please ensure that your PR meets the following requirements:
Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context.
Fixes #200
Type of Change