Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Semantic text notebook #271

Merged
merged 20 commits into from
Jul 5, 2024
Merged

Conversation

Mikep86
Copy link
Contributor

@Mikep86 Mikep86 commented Jun 17, 2024

Adds a search notebook demonstrating basic, end-to-end usage of the semantic_text field type

Copy link

gitnotebooks bot commented Jun 17, 2024

Found 1 changed notebook. Review the changes at https://gitnotebooks.com/elastic/elasticsearch-labs/pull/271

"source": [
"# Semantic Search with Semantic Text\n",
"\n",
"<a target=\"_blank\" href=\"https://colab.research.google.com/github/Mikep86/elasticsearch-labs/blob/semantic-text-notebook/notebooks/search/09-semantic-text.ipynb\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fix this link before merging

Copy link
Member

@carlosdelest carlosdelest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍 - some minor comments

notebooks/search/09-semantic-text.ipynb Outdated Show resolved Hide resolved
notebooks/search/09-semantic-text.ipynb Show resolved Hide resolved
notebooks/search/09-semantic-text.ipynb Outdated Show resolved Hide resolved
notebooks/search/09-semantic-text.ipynb Show resolved Hide resolved
@miguelgrinberg
Copy link
Collaborator

miguelgrinberg commented Jun 18, 2024

@Mikep86 To prevent this notebook for running (and failing) in 8.14 and older CI runs, add a EXEMPT_NOTEBOOKS__8_14 section in bin/find-notebooks-to-test.sh with this notebook and then 8.14 and older runs will skip it.

Comment on lines 271 to 287
"while True:\n",
" status = client.ml.get_trained_models_stats(\n",
" model_id=model_id,\n",
" )\n",
"\n",
" deployment_stats = status[\"trained_model_stats\"][0].get(\"deployment_stats\")\n",
" if deployment_stats is None:\n",
" print(\"ELSER Model is currently being deployed.\")\n",
" continue\n",
"\n",
" nodes = deployment_stats.get(\"nodes\")\n",
" if nodes is not None and len(nodes) > 0:\n",
" print(\"ELSER Model has been successfully deployed.\")\n",
" break\n",
" else:\n",
" print(\"ELSER Model is currently being deployed.\")\n",
" time.sleep(5)"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joemcelroy I used the ELSER notebook as a guide for this code and modified it to work with inference endpoints. However, I found two issues when doing so:

  • The notebook CI currently doesn't check against a serverless environment
  • The status check used in the ELSER notebook breaks when run against a serverless environment. trained_model_stats.0.deployment_stats.state doesn't exist in the serverless response to client.ml.get_trained_models_stats. Instead, I checked if the nodes list is populated, which seems to work in both hosted & serverless environments.

@Mikep86
Copy link
Contributor Author

Mikep86 commented Jul 3, 2024

@miguelgrinberg Seems like CI is broken on the notebooks/langchain/langchain-using-own-model.ipynb notebook

@Mikep86 Mikep86 merged commit f6e00c4 into elastic:main Jul 5, 2024
2 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants