Skip to content

Add example to list the data rows in a batch #613

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 18, 2022
Merged

Add example to list the data rows in a batch #613

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2d2377a
Add example to list the data rows in a batch
paulnoirel Jun 24, 2022
cf16c7a
Amend the example with dataset.export_data_rows for efficiency
paulnoirel Jun 24, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
198 changes: 107 additions & 91 deletions examples/basics/batches.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,16 +32,20 @@
},
{
"cell_type": "markdown",
"source": [
"## Batches (*Currently in Public Beta*)"
],
"id": "Lup2QNWjaxKg",
"metadata": {
"id": "Lup2QNWjaxKg"
},
"id": "Lup2QNWjaxKg"
"source": [
"## Batches (*Currently in Public Beta*)"
]
},
{
"cell_type": "markdown",
"id": "KONWmRQkadPf",
"metadata": {
"id": "KONWmRQkadPf"
},
"source": [
"* A Batch is collection of datarows picked out of a Data Set.\n",
"* A Datarow cannot be part of more than one batch in a project.\n",
Expand All @@ -50,59 +54,55 @@
"* Batches may have Datarows from multiple Datasets.\n",
"* Datarows can only be attached to a Project as part of a single Batch.\n",
"* You can set priority for each Batch."
],
"metadata": {
"id": "KONWmRQkadPf"
},
"id": "KONWmRQkadPf"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "HoW5ypnyzpqb",
"metadata": {
"id": "HoW5ypnyzpqb"
},
"outputs": [],
"source": [
"!pip install labelbox[data]"
],
"id": "HoW5ypnyzpqb"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6-Us9Gj1zpqc",
"metadata": {
"id": "6-Us9Gj1zpqc"
},
"outputs": [],
"source": [
"from labelbox import DataRow, Client\n",
"import random"
],
"id": "6-Us9Gj1zpqc"
]
},
{
"cell_type": "markdown",
"id": "qQiozm-dzpqd",
"metadata": {
"id": "qQiozm-dzpqd"
},
"source": [
"Set the following cell with your data to run this notebook:"
],
"id": "qQiozm-dzpqd"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "84Zna5c0zpqd",
"metadata": {
"id": "84Zna5c0zpqd"
},
"outputs": [],
"source": [
"PROJECT_NAME = \"Batch Queue Demo\" #text project\n",
"DATASET_NAME = \"Batch Queue Demo Data\""
],
"id": "84Zna5c0zpqd"
]
},
{
"cell_type": "markdown",
Expand All @@ -118,6 +118,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "Ge-dfNh-zpqe",
"metadata": {
"id": "Ge-dfNh-zpqe"
},
Expand All @@ -126,12 +127,12 @@
"# Add your api key\n",
"API_KEY = None\n",
"client = Client(api_key=API_KEY)"
],
"id": "Ge-dfNh-zpqe"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "nMVtBYQmzpqe",
"metadata": {
"id": "nMVtBYQmzpqe"
},
Expand All @@ -146,161 +147,181 @@
" 'row_data': 'https://picsum.photos/200/300'\n",
" })\n",
"dataset.create_data_rows(uploads)"
],
"id": "nMVtBYQmzpqe"
]
},
{
"cell_type": "markdown",
"source": [
"# Ensure project is in batch mode:"
],
"id": "61CvCD3C7qv6",
"metadata": {
"id": "61CvCD3C7qv6"
},
"id": "61CvCD3C7qv6"
"source": [
"# Ensure project is in batch mode:"
]
},
{
"cell_type": "code",
"source": [
"project = client.create_project(name=PROJECT_NAME)\n",
"project.update(queue_mode=project.QueueMode.Batch)"
],
"execution_count": null,
"id": "tqtT4q31787T",
"metadata": {
"id": "tqtT4q31787T"
},
"id": "tqtT4q31787T",
"execution_count": null,
"outputs": []
"outputs": [],
"source": [
"project = client.create_project(name=PROJECT_NAME)\n",
"project.update(queue_mode=project.QueueMode.Batch)"
]
},
{
"cell_type": "markdown",
"source": [
"# Collect Datarow id's:"
],
"id": "Xti9AoZWELrq",
"metadata": {
"id": "Xti9AoZWELrq"
},
"id": "Xti9AoZWELrq"
"source": [
"# Collect Datarow id's:"
]
},
{
"cell_type": "markdown",
"source": [
"### Select All Data Rows from dataset."
],
"id": "9JVLsXdevywS",
"metadata": {
"id": "9JVLsXdevywS"
},
"id": "9JVLsXdevywS"
"source": [
"### Select All Data Rows from dataset."
]
},
{
"cell_type": "code",
"source": [
"data_rows = [dr.uid for dr in list(dataset.export_data_rows())]"
],
"execution_count": null,
"id": "U4C1ZyJ2EgTS",
"metadata": {
"id": "U4C1ZyJ2EgTS"
},
"id": "U4C1ZyJ2EgTS",
"execution_count": null,
"outputs": []
"outputs": [],
"source": [
"data_row_ids = [dr.uid for dr in dataset.export_data_rows()]"
]
},
{
"cell_type": "markdown",
"id": "6699941a",
"metadata": {},
"source": []
},
{
"cell_type": "markdown",
"id": "B0UqO_O1V8ei",
"metadata": {
"id": "B0UqO_O1V8ei"
},
"source": [
"### Randomly sample\n",
"\n",
"Rather than selecting all of the data we sample 5 data rows at random"
],
"metadata": {
"id": "B0UqO_O1V8ei"
},
"id": "B0UqO_O1V8ei"
]
},
{
"cell_type": "code",
"source": [
"sample = random.sample(data_rows, 5)"
],
"execution_count": null,
"id": "WJAXBf1bV-td",
"metadata": {
"id": "WJAXBf1bV-td"
},
"id": "WJAXBf1bV-td",
"execution_count": null,
"outputs": []
"outputs": [],
"source": [
"sample = random.sample(data_rows, 5)"
]
},
{
"cell_type": "markdown",
"source": [
"# Batch Manipulation"
],
"id": "UPdaTqkgYyvt",
"metadata": {
"id": "UPdaTqkgYyvt"
},
"id": "UPdaTqkgYyvt"
"source": [
"# Batch Manipulation"
]
},
{
"cell_type": "markdown",
"source": [
"### Create a Batch:"
],
"id": "Al-K1lBBEjtb",
"metadata": {
"id": "Al-K1lBBEjtb"
},
"id": "Al-K1lBBEjtb"
"source": [
"### Create a Batch:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "resH3xqeErVv",
"metadata": {
"id": "resH3xqeErVv"
},
"outputs": [],
"source": [
"batch = project.create_batch(\n",
" \"first batch\", # Each batch in a project must have a unique name\n",
" sample, # A list of data rows or data row ids\n",
" 5 # priority between 1(Highest) - 5(lowest)\n",
")"
],
"metadata": {
"id": "resH3xqeErVv"
},
"id": "resH3xqeErVv",
"execution_count": null,
"outputs": []
]
},
{
"cell_type": "code",
"source": [
"# number of data rows in the batch\n",
"batch.size"
],
"execution_count": null,
"id": "gFio7ONOWYdJ",
"metadata": {
"id": "gFio7ONOWYdJ"
},
"id": "gFio7ONOWYdJ",
"execution_count": null,
"outputs": []
"outputs": [],
"source": [
"# number of data rows in the batch\n",
"batch.size"
]
},
{
"cell_type": "markdown",
"id": "8Cj64Isxzpqe",
"metadata": {
"id": "8Cj64Isxzpqe"
},
"source": [
"### List DataRows in a Batch (Not supported yet)\n",
"Note: You can view your batch through in the Data Row table of the project"
],
"id": "8Cj64Isxzpqe"
"### List DataRows in a Batch\n",
"Note: You can view your batch through in the *Data Row tab* of the project"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0a7d1d3e",
"metadata": {},
"outputs": [],
"source": [
"data_rows = [dr for dr in batch.export_data_rows()]"
]
},
{
"cell_type": "markdown",
"id": "rU7iddSQzpqg",
"metadata": {
"id": "rU7iddSQzpqg"
},
"source": [
"### Remove queued data rows by batch (Not supported yet)\n",
"Note: You can do this through the batch management pane on the data rows tab of the project"
],
"id": "rU7iddSQzpqg"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "Batches.ipynb",
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
Expand All @@ -317,13 +338,8 @@
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
},
"colab": {
"name": "Batches.ipynb",
"provenance": [],
"collapsed_sections": []
}
},
"nbformat": 4,
"nbformat_minor": 5
}
}