fix: OpenAIEncoder, tests, and docs

[tolgadevAI]

PR Type

Enhancement, Documentation, Tests

---

Description

• Refactored document truncation logic in OpenAIEncoder to use list comprehension for improved readability and performance.
• Fixed import statement in OpenAI encoder integration tests to ensure proper module import.
• Reformatted the entire dynamic routes documentation notebook to enhance readability and presentation.
• Added a new author to the project and updated the black dependency to include Jupyter extras in the pyproject.toml file.
• Fixed a typo in the function calling example notebook to ensure correct route creation.
• Improved readability in the unstructured element splitter example notebook by adding a missing newline.

---

Changes walkthrough 📝

	Relevant files
Enhancement	openai.py Refactor document truncation logic using list comprehension. semantic_router/encoders/openai.py Refactored document truncation logic to use list comprehension. +1/-2
Tests	test_openai_integration.py Fix import statement in OpenAI encoder integration tests. tests/integration/encoders/test_openai_integration.py Fixed import statement for OpenAIEncoder. +1/-3
Documentation	02-dynamic-routes.ipynb Reformat dynamic routes documentation notebook.                    docs/02-dynamic-routes.ipynb Reformatted the entire notebook to improve readability. +1043/-1043 function_calling.ipynb Fix typo in function calling example notebook.                      docs/examples/function_calling.ipynb Fixed typo in route creation. +3/-1      unstructured-element-splitter.ipynb Improve readability in unstructured element splitter example. docs/examples/unstructured-element-splitter.ipynb Added missing newline for better readability. +1/-0
Configuration changes	pyproject.toml Update authors and `black` dependency in project configuration. pyproject.toml Added a new author. Updated black dependency to include Jupyter extras. +3/-2

---

💡 PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[github-actions]

PR Review 🔍

⏱️ Estimated effort to review [1-5]	3, because the PR includes multiple changes across different files including Python scripts and Jupyter notebooks. The changes involve refactoring, fixing imports, and updating dependencies which require a careful review to ensure compatibility and functionality.
🧪 Relevant tests	No
⚡ Possible issues	Possible Bug: The change from Route.from_dynamic_route(llm=llm, entity=function) to Route.from_dynamic_route(llm=llm, entities=function) in 'docs/examples/function_calling.ipynb' might introduce a bug if the from_dynamic_route method does not support or expect a parameter named entities. This could potentially break the functionality unless the method definition is also updated to accommodate this change.
🔒 Security concerns	No

Code feedback:

---

relevant file	semantic_router/encoders/openai.py
suggestion	Consider adding error handling for the list comprehension used in document truncation to manage cases where _truncate might fail or return unexpected results. This could prevent the entire encoding process from failing due to issues with individual documents. [important]
relevant line	docs = [self._truncate(doc) for doc in docs]

---

relevant file	tests/integration/encoders/test_openai_integration.py
suggestion	Ensure that the modified import statement is correctly reflecting the new structure or location of OpenAIEncoder. If the encoder is not correctly imported, it could lead to runtime errors during testing. Consider adding a specific test to verify that the import and instantiation of OpenAIEncoder work as expected. [important]
relevant line	from semantic_router.encoders.openai import OpenAIEncoder

---

relevant file	docs/examples/function_calling.ipynb
suggestion	Verify the change from entity=function to entities=function in the Route.from_dynamic_route method call. If the method's signature or expected parameters have not been updated to handle entities as a keyword, this will raise an error. [important]
relevant line	" route = Route.from_dynamic_route(llm=llm, entities=function)\n"

---

relevant file	pyproject.toml
suggestion	Ensure that the version constraints for the black package with Jupyter extras (extras = ["jupyter"]) do not conflict with other dependencies or the project's requirements. It's crucial to test this change to avoid dependency resolution issues. [medium]
relevant line	black = {extras = ["jupyter"], version = ">=23.12.1,<24.5.0"}

---

[github-actions]

PR Code Suggestions ✨

Category	Suggestion	Score
Possible issue	Verify the correctness of the import path to avoid import errors The import statement for 'OpenAIEncoder' has been changed to a more specific path. Ensure that this new path is correctly reflecting the module's location and is not leading to any import errors in the test environment. tests/integration/encoders/test_openai_integration.py [2] -from semantic_router.encoders.openai import OpenAIEncoder +from semantic_router.encoders import OpenAIEncoder # Use this if the previous import path was correct Suggestion importance[1-10]: 9 Why: Correct import paths are essential for the functionality of the tests. The suggestion addresses a potential issue that could cause import errors, which is critical for the test environment's reliability.	9
Possible bug	Verify and correct the parameter name in the method call to prevent bugs Ensure that the variable name change from 'entity' to 'entities' in the method call is intentional and correctly reflects the expected parameter in the 'Route.from_dynamic_route' method. If 'entity' was correct, revert this change to avoid potential bugs. docs/examples/function_calling.ipynb [147] -route = Route.from_dynamic_route(llm=llm, entities=function) +route = Route.from_dynamic_route(llm=llm, entity=function) # Assuming 'entity' was the correct parameter Suggestion importance[1-10]: 8 Why: Ensuring the correct parameter name is crucial to avoid potential bugs. The suggestion correctly identifies a possible issue that could lead to runtime errors if the parameter name change was not intentional.	8
Compatibility	Check the 'black' version compatibility with the project requirements The version constraint for 'black' has been changed to a range that includes a lower version than previously specified. Ensure this change is compatible with other dependencies and does not introduce any conflicts or deprecated features being used in the project. pyproject.toml [66] -black = {extras = ["jupyter"], version = ">=23.12.1,<24.5.0"} +black = {extras = ["jupyter"], version = "^24.0.0"} # Reverting to previous version if needed Suggestion importance[1-10]: 7 Why: The suggestion to verify the compatibility of the 'black' version is important for maintaining project stability. However, it is more of a cautionary note rather than a direct improvement to the code.	7
Performance	Optimize list comprehension by integrating conditionals directly Consider using a list comprehension with a conditional check instead of filtering after the list comprehension. This approach avoids creating an intermediate list that is not needed, which can be more efficient, especially with large lists. semantic_router/encoders/openai.py [82] -docs = [self._truncate(doc) for doc in docs] +docs = [self._truncate(doc) for doc in docs if condition] # Replace 'condition' with the actual condition needed Suggestion importance[1-10]: 3 Why: The suggestion to integrate conditionals directly into the list comprehension is valid for performance optimization. However, the original code does not indicate any specific condition, making the suggestion somewhat speculative without additional context.	3

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.13%. Comparing base (a8969d4) to head (2a4d747).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #295      +/-   ##
==========================================
- Coverage   81.14%   81.13%   -0.01%     
==========================================
  Files          45       45              
  Lines        2705     2704       -1     
==========================================
- Hits         2195     2194       -1     
  Misses        510      510              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.