Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add script to check imports #19611

Merged
merged 11 commits into from Mar 29, 2024
Merged

add script to check imports #19611

merged 11 commits into from Mar 29, 2024

Conversation

ccurme
Copy link
Collaborator

@ccurme ccurme commented Mar 26, 2024 •
edited

A bit messy.

Currently outputs:

Found bad imports:
File: llmlingua.ipynb
langchain_community.retrievers.LLMLinguaCompressor
File: jaguar.ipynb
langchain_community.vectorstores.Jaguar
langchain_community.vectorstores.Jaguar
File: cohere-reranker.ipynb
langchain.retrievers.CohereRerank
File: apify.ipynb
langchain_community.document_loaders.base.Document
File: google_drive.ipynb
langchain_community.tools.google_drive.tool.GoogleDriveSearchTool
langchain_community.utilities.google_drive.GoogleDriveAPIWrapper
File: xata_chat_message_history.ipynb
langchain_community.vectorstores.XataVectorStore
File: dappier.ipynb
langchain_community.chat_models.dappier.ChatDappierAI
File: xorbits.ipynb
langchain.agents.create_xorbits_agent
File: airbyte_structured_qa.ipynb
langchain.agents.create_pandas_dataframe_agent
File: spark.ipynb
langchain.agents.create_spark_dataframe_agent
File: duckdb.ipynb
langchain.vectorstores.DuckDB
File: vikingdb.ipynb
langchain.vectorstores.vikingdb.VikingDB
langchain.vectorstores.vikingdb.VikingDBConfig
File: apify_dataset.ipynb
langchain_community.document_loaders.base.Document
File: quip.ipynb
langchain_community.document_loaders.QuipLoader
File: llm_caching.ipynb
langchain.cache.AzureCosmosDBSemanticCache
File: model_laboratory.ipynb
langchain.chains.SelfAskWithSearchChain
File: graph_gremlin_cosmosdb_qa.ipynb
langchain.chains.graph_qa.GremlinQAChain
File: async_callbacks.ipynb
langchain_core.messages.LLMResult

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Mar 26, 2024
@vercel Vercel
Copy link

vercel bot commented Mar 26, 2024 •
edited

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchain ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 29, 2024 4:15pm

@dosubot dosubot bot added Ɑ: agent Related to agents module 🤖:docs Changes to documentation and examples, like .md, .rst, .ipynb files. Changes to the docs/ folder labels Mar 26, 2024
@ccurme ccurme mentioned this pull request Mar 27, 2024
Merged
ccurme added a commit that referenced this pull request Mar 27, 2024
@ccurme
docs: Fix broken imports in documentation (#19655)
4e9b358 
Found via script in #19611
@ccurme
Copy link
Collaborator Author

ccurme commented Mar 27, 2024

All of these should now be fixed following #19655

@baskaryan
Copy link
Collaborator

should we add this to CI now?

@ccurme ccurme changed the title RFC: add script to check imports add script to check imports Mar 28, 2024
baskaryan
baskaryan reviewed Mar 28, 2024
View reviewed changes
.github/workflows/_test_doc_imports.yml Outdated

- name: Install langchain editable
run: |
poetry run pip install -e libs/langchain libs/community libs/experimental
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we install core editable as well?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could also check partner packages (and install those here)

eyurtsev
eyurtsev reviewed Mar 29, 2024
View reviewed changes
.github/workflows/_test_doc_imports.yml Outdated Show resolved Hide resolved
docs/scripts/check_imports.py Show resolved Hide resolved
docs/scripts/check_imports.py Outdated Show resolved Hide resolved
docs/scripts/check_imports.py Outdated
getattr(imported_module, item)
except Exception as e:
offending_imports.append((module, item))
except Exception as e:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why a broad exception?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in specific exceptions I was expecting. These are broad in case I missed something among ImportError, AttributeError, or ModuleNotFoundError.

docs/scripts/check_imports.py Outdated
offending_imports.append((module, item))
else:
importlib.import_module(module)
except Exception as e:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this import error?

.github/workflows/_test_doc_imports.yml Outdated Show resolved Hide resolved
@ccurme
Copy link
Collaborator Author

ccurme commented Mar 29, 2024

Test takes about 1-2 min. to run if dependencies aren't cached, 40s otherwise. Script itself takes 4s.

Errors look like:
Screenshot 2024-03-29 at 12 07 44 PM

@dosubot dosubot bot added the lgtm PR looks good. Use to confirm that a PR is ready for merging. label Mar 29, 2024
@ccurme ccurme merged commit 0dbd5f5 into master Mar 29, 2024
98 checks passed
@ccurme ccurme deleted the cc/check_imports branch March 29, 2024 17:30
bechbd pushed a commit to bechbd/langchain that referenced this pull request Mar 29, 2024
@ccurme
docs: Fix broken imports in documentation (langchain-ai#19655)
2315676 
Found via script in langchain-ai#19611
bechbd pushed a commit to bechbd/langchain that referenced this pull request Mar 29, 2024
@ccurme
add script to check imports (langchain-ai#19611)
cd9412e
gkorland pushed a commit to FalkorDB/langchain that referenced this pull request Mar 30, 2024
@ccurme @gkorland
docs: Fix broken imports in documentation (langchain-ai#19655)
bd395cf 
Found via script in langchain-ai#19611
gkorland pushed a commit to FalkorDB/langchain that referenced this pull request Mar 30, 2024
@ccurme @gkorland
add script to check imports (langchain-ai#19611)
ba5b7a6
marlenezw pushed a commit to marlenezw/langchain that referenced this pull request Apr 2, 2024
@ccurme @marlenezw
docs: Fix broken imports in documentation (langchain-ai#19655)
4b3a934 
Found via script in langchain-ai#19611
marlenezw pushed a commit to marlenezw/langchain that referenced this pull request Apr 2, 2024
@ccurme @marlenezw
add script to check imports (langchain-ai#19611)
ca740d3
Je-Cp pushed a commit to Je-Cp/jcp-langchain that referenced this pull request Apr 2, 2024
@ccurme @Je-Cp
docs: Fix broken imports in documentation (#19655)
f20ab53 
Found via script in langchain-ai/langchain#19611
hinthornw pushed a commit that referenced this pull request Apr 26, 2024
@ccurme @hinthornw
docs: Fix broken imports in documentation (#19655)
a0c0221 
Found via script in #19611
hinthornw pushed a commit that referenced this pull request Apr 26, 2024
@ccurme @hinthornw
add script to check imports (#19611)
af26389
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eyurtsev eyurtsev eyurtsev left review comments

@baskaryan baskaryan baskaryan approved these changes

@rlancemartin rlancemartin Awaiting requested review from rlancemartin

@jacoblee93 jacoblee93 Awaiting requested review from jacoblee93

Assignees
No one assigned
Labels
Ɑ: agent Related to agents module 🤖:docs Changes to documentation and examples, like .md, .rst, .ipynb files. Changes to the docs/ folder lgtm PR looks good. Use to confirm that a PR is ready for merging. size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@ccurme @baskaryan @eyurtsev