Skip to content

Commit

Permalink
chore: update examples
Browse files Browse the repository at this point in the history
  • Loading branch information
@SimsonW @jameszyao
SimsonW authored and jameszyao committed May 13, 2024
1 parent 6f144d2 commit 07c0c2f
Showing 1 changed file with 40 additions and 37 deletions.
77 changes: 40 additions & 37 deletions examples/retrieval/semantic_search.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@
"cells": [
{
"cell_type": "code",
"execution_count": null,
"id": "initial_id",
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import taskingai\n",
"# Load TaskingAI API Key from environment variable\n",
"from taskingai.retrieval import Collection\n",
"from taskingai.retrieval.text_splitter import TokenTextSplitter"
]
],
"outputs": [],
"execution_count": null
},
{
"cell_type": "markdown",
Expand All @@ -37,21 +37,19 @@
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"# choose an available text_embedding model from your project\n",
"embedding_model_id = \"YOUR_EMBEDDING_MODEL_ID\""
],
"metadata": {
"collapsed": false
},
"id": "388eb6fa46f66b52"
"id": "388eb6fa46f66b52",
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"# create a collection\n",
"def create_collection() -> Collection:\n",
Expand All @@ -67,12 +65,12 @@
"metadata": {
"collapsed": false
},
"id": "7c7d4e2cc2f2f494"
"id": "7c7d4e2cc2f2f494",
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"# Check collection status. \n",
"# Only when status is \"READY\" can you insert records and query chunks.\n",
Expand All @@ -82,16 +80,17 @@
"metadata": {
"collapsed": false
},
"id": "eb5dee18aa83c5e4"
"id": "eb5dee18aa83c5e4",
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"# create record 1 (machine learning)\n",
"taskingai.retrieval.create_record(\n",
" collection_id=collection.collection_id,\n",
" title=\"Machine Learning\",\n",
" type=\"text\",\n",
" content=\"Machine learning is a subfield of artificial intelligence (AI) that involves the development of algorithms that allow computers to learn from and make decisions or predictions based on data. The term \\\"machine learning\\\" was coined by Arthur Samuel in 1959. In other words, machine learning enables a system to automatically learn and improve from experience without being explicitly programmed. This is achieved by feeding the system massive amounts of data, which it uses to learn patterns and make inferences. There are three main types of machine learning: 1. Supervised Learning: This is where the model is given labeled training data and the goal of learning is to generalize from the training data to unseen situations in a principled way. 2. Unsupervised Learning: This involves training on a dataset without explicit labels. The goal might be to discover inherent groupings or patterns within the data. 3. Reinforcement Learning: In this type, an agent learns to perform actions based on reward/penalty feedback to achieve a goal. It's commonly used in robotics, gaming, and navigation. Deep learning, a subset of machine learning, uses neural networks with many layers (\\\"deep\\\" structures) and has been responsible for many recent breakthroughs in AI, including speech recognition, image recognition, and natural language processing. It's important to note that machine learning is a rapidly developing field, with new techniques and applications emerging regularly.\",\n",
" text_splitter={\"type\": \"token\", \"chunk_size\": 100, \"chunk_overlap\": 10},\n",
Expand All @@ -100,30 +99,30 @@
"metadata": {
"collapsed": false
},
"id": "f783de4624047df7"
"id": "f783de4624047df7",
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"# create record 2 (Michael Jordan)\n",
"taskingai.retrieval.create_record(\n",
" collection_id=collection.collection_id,\n",
" type=\"text\",\n",
" content=\"Michael Jordan, often referred to by his initials MJ, is considered one of the greatest players in the history of the National Basketball Association (NBA). He was known for his scoring ability, defensive prowess, competitiveness, and clutch performances. Born on February 17, 1963, Jordan played 15 seasons in the NBA, primarily with the Chicago Bulls, but also with the Washington Wizards. His professional career spanned two decades from 1984 to 2003, during which he won numerous awards and set multiple records. Here are some key highlights of his career: - Scoring: Jordan won the NBA scoring title a record 10 times. He also has the highest career scoring average in NBA history, both in the regular season (30.12 points per game) and in the playoffs (33.45 points per game). - Championships: He led the Chicago Bulls to six NBA championships and was named Finals MVP in all six of those Finals (1991-1993, 1996-1998). - MVP Awards: Jordan was named the NBA's Most Valuable Player (MVP) five times (1988, 1991, 1992, 1996, 1998). - Defensive Ability: He was named to the NBA All-Defensive First Team nine times and won the NBA Defensive Player of the Year award in 1988. - Olympics: Jordan also won two Olympic gold medals with the U.S. basketball team, in 1984 and 1992. - Retirements and Comebacks: Jordan retired twice during his career. His first retirement came in 1993, after which he briefly played minor league baseball. He returned to the NBA in 1995. He retired a second time in 1999, only to return again in 2001, this time with the Washington Wizards. He played two seasons for the Wizards before retiring for good in 2003. After his playing career, Jordan became a team owner and executive. As of my knowledge cutoff in September 2021, he is the majority owner of the Charlotte Hornets. Off the court, Jordan is known for his lucrative endorsement deals, particularly with Nike. The Air Jordan line of sneakers is one of the most popular and enduring in the world. His influence also extends to the realms of film and fashion, and he is recognized globally as a cultural icon. In 2000, he was inducted into the Basketball Hall of Fame.\",\n",
" text_splitter={\"type\": \"token\", \"chunk_size\": 100, \"chunk_overlap\": 10},\n",
" content=\"Michael Jordan, often referred to by his initials MJ, is considered one of the greatest players in the history of the National Basketball Association (NBA). He was known for his scoring ability, defensive prowess, competitiveness, and clutch performances. Born on February 17, 1963, Jordan played 15 seasons in the NBA, primarily with the Chicago Bulls, but also with the Washington Wizards. His professional career spanned two decades from 1984 to 2003, during which he won numerous awards and set multiple records. \\n\\n Here are some key highlights of his career: - Scoring: Jordan won the NBA scoring title a record 10 times. He also has the highest career scoring average in NBA history, both in the regular season (30.12 points per game) and in the playoffs (33.45 points per game). - Championships: He led the Chicago Bulls to six NBA championships and was named Finals MVP in all six of those Finals (1991-1993, 1996-1998). - MVP Awards: Jordan was named the NBA's Most Valuable Player (MVP) five times (1988, 1991, 1992, 1996, 1998). - Defensive Ability: He was named to the NBA All-Defensive First Team nine times and won the NBA Defensive Player of the Year award in 1988. - Olympics: Jordan also won two Olympic gold medals with the U.S. basketball team, in 1984 and 1992. \\n\\n - Retirements and Comebacks: Jordan retired twice during his career. His first retirement came in 1993, after which he briefly played minor league baseball. He returned to the NBA in 1995. He retired a second time in 1999, only to return again in 2001, this time with the Washington Wizards. He played two seasons for the Wizards before retiring for good in 2003. After his playing career, Jordan became a team owner and executive. As of my knowledge cutoff in September 2021, he is the majority owner of the Charlotte Hornets. Off the court, Jordan is known for his lucrative endorsement deals, particularly with Nike. \\n\\n The Air Jordan line of sneakers is one of the most popular and enduring in the world. His influence also extends to the realms of film and fashion, and he is recognized globally as a cultural icon. In 2000, he was inducted into the Basketball Hall of Fame.\",\n",
" text_splitter={\"type\": \"separator\", \"chunk_size\": 200, \"chunk_overlap\": 10, \"separators\": [\"\\n\\n\"]}\n",
")"
],
"metadata": {
"collapsed": false
},
"id": "e23ee88246ffc350"
"id": "e23ee88246ffc350",
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"# create record 3 (Granite)\n",
"taskingai.retrieval.create_record(\n",
Expand All @@ -136,7 +135,9 @@
"metadata": {
"collapsed": false
},
"id": "73458e8086bec5bd"
"id": "73458e8086bec5bd",
"outputs": [],
"execution_count": null
},
{
"cell_type": "markdown",
Expand All @@ -150,8 +151,6 @@
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"# Check record status. \n",
"# Only when status is \"READY\", the record chunks can appear in query results.\n",
Expand All @@ -163,48 +162,50 @@
"metadata": {
"collapsed": false
},
"id": "f6140ba9ae4e3f91"
"id": "f6140ba9ae4e3f91",
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"# query chunks 1\n",
"chunks = taskingai.retrieval.query_chunks(\n",
" collection_id=collection.collection_id,\n",
" query_text=\"Basketball\",\n",
" top_k=2\n",
" top_k=10,\n",
" score_threshold=0.5,\n",
")\n",
"print(chunks)"
],
"metadata": {
"collapsed": false
},
"id": "cd499d7869e8445c"
"id": "cd499d7869e8445c",
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"# query chunks 2\n",
"chunks = taskingai.retrieval.query_chunks(\n",
" collection_id=collection.collection_id,\n",
" query_text=\"geology\",\n",
" top_k=2\n",
" top_k=10,\n",
" max_tokens=300,\n",
")\n",
"print(chunks)"
],
"metadata": {
"collapsed": false
},
"id": "b6fd67f81af404b2"
"id": "b6fd67f81af404b2",
"outputs": [],
"execution_count": null
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"# query chunks 3\n",
"chunks = taskingai.retrieval.query_chunks(\n",
Expand All @@ -217,7 +218,9 @@
"metadata": {
"collapsed": false
},
"id": "fc9c1fa12d893dd1"
"id": "fc9c1fa12d893dd1",
"outputs": [],
"execution_count": null
}
],
"metadata": {
Expand Down

0 comments on commit 07c0c2f

Please sign in to comment.