Skip to content

Commit

Permalink
Fixed up the renaming and migrated over to chain-of-density to be con…
Browse files Browse the repository at this point in the history 
…sistent with the other notebooks
  • Loading branch information
@ivanleomk
ivanleomk committed Nov 20, 2023
1 parent a1ca42c commit 54536f6
Show file tree
Hide file tree
Showing 2 changed files with 39 additions and 81 deletions.
120 changes: 39 additions & 81 deletions .../6 - Chain Of Density Summarization.ipynb → tutorials/6.chain-of-density.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 1,
"id": "0dbdda0a-2648-4e0f-8633-ea19bef4a460",
"metadata": {},
"outputs": [
Expand All @@ -133,9 +133,6 @@
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n",
"\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
"\u001b[38;5;2m✔ Download and installation successful\u001b[0m\n",
"You can now load the package via spacy.load('en_core_web_sm')\n"
]
Expand Down Expand Up @@ -192,7 +189,7 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": 2,
"id": "bd6ebf95-60c6-4ec8-be17-d5ab436a67fd",
"metadata": {},
"outputs": [
Expand All @@ -202,7 +199,7 @@
"['My', 'favourite', 'type', 'of', 'Sashimi', 'is', 'Toro']"
]
},
"execution_count": 12,
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -224,7 +221,7 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": 3,
"id": "8a87b231-57b0-426c-98d5-cd7d8b512121",
"metadata": {},
"outputs": [
Expand All @@ -234,7 +231,7 @@
"['I', \"'m\", 'fascinated', 'by', 'machine', 'learning', '!']"
]
},
"execution_count": 15,
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -255,7 +252,7 @@
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": 4,
"id": "c905dff4-5753-4274-90fe-44aa3393ff0f",
"metadata": {},
"outputs": [
Expand Down Expand Up @@ -287,7 +284,7 @@
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": 5,
"id": "47a4a8f6-295d-4040-beb1-3c8e9ff3bf99",
"metadata": {},
"outputs": [],
Expand All @@ -301,7 +298,7 @@
},
{
"cell_type": "code",
"execution_count": 20,
"execution_count": 6,
"id": "51197222-2124-46f8-9a57-555d43836401",
"metadata": {},
"outputs": [
Expand All @@ -311,7 +308,7 @@
"(Apple, U.K., $1 billion)"
]
},
"execution_count": 20,
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -333,7 +330,7 @@
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": 7,
"id": "9c2ad5a0-2f24-442e-a46a-3a265ef873f6",
"metadata": {},
"outputs": [
Expand All @@ -343,7 +340,7 @@
"()"
]
},
"execution_count": 21,
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -358,7 +355,7 @@
},
{
"cell_type": "code",
"execution_count": 22,
"execution_count": 8,
"id": "dc7964d3-61f6-436e-bfb0-080cd46c41bf",
"metadata": {},
"outputs": [
Expand All @@ -368,7 +365,7 @@
"(J.K., one, Harry Potter')"
]
},
"execution_count": 22,
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
Expand Down Expand Up @@ -402,7 +399,7 @@
},
{
"cell_type": "code",
"execution_count": 41,
"execution_count": 9,
"id": "15accf59-a264-4e1c-9b77-8b486e423f95",
"metadata": {},
"outputs": [],
Expand All @@ -420,7 +417,7 @@
},
{
"cell_type": "code",
"execution_count": 42,
"execution_count": 10,
"id": "648206dc-a734-49eb-bd2e-8b46a914cacf",
"metadata": {},
"outputs": [
Expand All @@ -430,7 +427,7 @@
"(17, 0, 0.0)"
]
},
"execution_count": 42,
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -444,7 +441,7 @@
},
{
"cell_type": "code",
"execution_count": 43,
"execution_count": 11,
"id": "9fd5717f-202a-4b39-976c-a32d0f1a4b29",
"metadata": {},
"outputs": [
Expand All @@ -454,7 +451,7 @@
"(11, 3, 0.273)"
]
},
"execution_count": 43,
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -477,7 +474,7 @@
},
{
"cell_type": "code",
"execution_count": 46,
"execution_count": 12,
"id": "ae27bcc5-da32-4aaa-9ebb-dbc21700ee14",
"metadata": {},
"outputs": [
Expand All @@ -487,7 +484,7 @@
"((82, 11, 0.134), (71, 17, 0.239))"
]
},
"execution_count": 46,
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
Expand Down Expand Up @@ -538,18 +535,18 @@
},
{
"cell_type": "code",
"execution_count": 59,
"execution_count": 13,
"id": "2ac40d98-2843-4c9c-bc18-50ab1d4ffa94",
"metadata": {},
"outputs": [],
"source": [
"from pydantic import BaseModel,Field\n",
"from pydantic import BaseModel,Field,field_validator\n",
"from typing import List"
]
},
{
"cell_type": "code",
"execution_count": 61,
"execution_count": 14,
"id": "486e85fc-3fc8-4143-bdf4-d7cef91a37cf",
"metadata": {},
"outputs": [],
Expand Down Expand Up @@ -581,23 +578,23 @@
},
{
"cell_type": "code",
"execution_count": 57,
"execution_count": 15,
"id": "609a9edd-7c4e-4586-a5be-037c4c3c7ff7",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'description': 'This is an initial summary which should be long ( 4-5 sentences, ~80 words)\\nyet highly non-specific, containing little information beyond the entities marked as missing.\\nUse overly verbose languages and fillers (Eg. This article discusses) to reach ~80 words.',\n",
" 'properties': {'summary': {'description': 'This is a summary of the article provided which is overly verbose and uses fillers. It should be roughly 80 words in length',\n",
" 'properties': {'summary': {'description': 'This is a summary of the article provided which is overly verbose and uses fillers. It should be roughly 80 words in length',\n",
" 'title': 'Summary',\n",
" 'type': 'string'}},\n",
" 'required': ['summary'],\n",
" 'title': 'InitialSummary',\n",
" 'type': 'object'}"
]
},
"execution_count": 57,
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -624,7 +621,7 @@
},
{
"cell_type": "code",
"execution_count": 81,
"execution_count": 16,
"id": "d3d589ca-00cd-42cc-9a7a-a8f0620b4ea1",
"metadata": {},
"outputs": [],
Expand Down Expand Up @@ -685,7 +682,7 @@
},
{
"cell_type": "code",
"execution_count": 82,
"execution_count": 17,
"id": "8f81f281-0950-4973-81b6-e1acd8b35aa0",
"metadata": {},
"outputs": [],
Expand Down Expand Up @@ -778,7 +775,7 @@
},
{
"cell_type": "code",
"execution_count": 67,
"execution_count": 18,
"id": "fc66ffcc-db30-429a-8007-4d4a24bf2426",
"metadata": {},
"outputs": [],
Expand All @@ -792,7 +789,7 @@
" summary_chain = []\n",
" # We first generate an initial summary\n",
" summary: InitialSummary = client.chat.completions.create( \n",
" model=\"gpt-4-0613\",\n",
" model=\"gpt-4-1106-preview\",\n",
" response_model=InitialSummary,\n",
" messages=[\n",
" {\n",
Expand Down Expand Up @@ -821,7 +818,7 @@
" ]\n",
" )\n",
" new_summary: RewrittenSummary = client.chat.completions.create( \n",
" model=\"gpt-4-0613\",\n",
" model=\"gpt-4-1106-preview\",\n",
" messages=[\n",
" {\n",
" \"role\": \"system\",\n",
Expand Down Expand Up @@ -869,30 +866,21 @@
},
{
"cell_type": "code",
"execution_count": 74,
"execution_count": 19,
"id": "6044c72b-fdc7-4cea-893b-a408c7b60230",
"metadata": {},
"outputs": [],
"source": [
"with open(\"./article.txt\",\"r+\") as file:\n",
"with open(\"./assets/article.txt\",\"r+\") as file:\n",
" article = file.readline()"
]
},
{
"cell_type": "code",
"execution_count": 75,
"execution_count": null,
"id": "2302dedc-f22a-41e9-b9c2-1579a4e8f623",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 459 ms, sys: 17.5 ms, total: 477 ms\n",
"Wall time: 59.9 s\n"
]
}
],
"outputs": [],
"source": [
"%%time\n",
"\n",
Expand All @@ -909,21 +897,10 @@
},
{
"cell_type": "code",
"execution_count": 77,
"execution_count": null,
"id": "99f7361c-2737-44ef-8515-1919e009e718",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Article 1 -> Results (Tokens: 104, Entity Count: 0, Density: 0.0)\n",
"Article 2 -> Results (Tokens: 66, Entity Count: 6, Density: 0.091)\n",
"Article 3 -> Results (Tokens: 74, Entity Count: 7, Density: 0.095)\n",
"Article 4 -> Results (Tokens: 66, Entity Count: 11, Density: 0.167)\n"
]
}
],
"outputs": [],
"source": [
"for index,summary in enumerate(summaries):\n",
" tokens,entity,density = calculate_entity_density(summary)\n",
Expand All @@ -940,29 +917,10 @@
},
{
"cell_type": "code",
"execution_count": 78,
"execution_count": null,
"id": "e7149f4d-41ca-4cb1-8438-65cd97cb4246",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"This article discusses the instances of multiple renowned public figures and their respective candid encounters with body shaming. It delves into how these notable personalities, more specifically, famous females, have confidently demonstrated their authenticity on social media platforms, challenging societal constructs around body image. Furthermore, the article highlights the overwhelmingly positive public response and encouragement received from this display of genuine body acceptance, attempting to reshape conventional perspectives on beauty standards. Lastly, the article includes several individual narratives, illustrating each celebrity's unique approach in addressing body shaming and promoting body positivity.\n",
"\n",
"\n",
"Chrissy Teigen, Pink, Kelly Clarkson, Lena Dunham, Tyra Banks, and Gabourey Sidibe confront body shaming, displaying their bodies on social media and challenging body image norms. With diverse positivity approaches, they receive public backing. The struggle against body shaming by Banks and Sidibe, despite not being the focus of the piece, was also highlighted.\n",
"\n",
"\n",
"Chrissy Teigen, Pink, Kelly Clarkson, Lena Dunham, Tyra Banks, and Gabourey Sidibe are rallying against body shaming, proudly showcasing their bodies on social media. Each celebrity takes a unique approach to body positivity, generating public praise. Tyra Banks and Gabourey Sidibe particularly combat body shaming, with their stories noted in the report, underscoring the issue's widespread nature in Hollywood and beyond.\n",
"\n",
"\n",
"Stars Chrissy Teigen, Pink, Kelly Clarkson, Lena Dunham, Tyra Banks, and Gabourey Sidibe defy body shaming through social media platforms like Instagram and Twitter. Their unique approaches to body positivity earn public commendation. Tyra Banks and Gabourey Sidibe's stories, including Sidibe's response to Golden Globes critics, underscore the pervasive issue in Hollywood and beyond.\n",
"\n"
]
}
],
"outputs": [],
"source": [
"for summary in summaries:\n",
" print(f\"\\n{summary}\\n\")"
Expand Down
0 tutorials/article.txt → tutorials/assets/article.txt
View file
File renamed without changes.

0 comments on commit 54536f6

Please sign in to comment.