Skip to content
This repository was archived by the owner on Sep 3, 2022. It is now read-only.
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 9 additions & 13 deletions samples/ML Toolbox/Classification/Iris/1 Local End to End.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,7 @@
"source": [
"# About this notebook\n",
"\n",
"This notebook uses the datalab structured data package for building and running a Tensorflow classification model locally. This notebook uses the classic <a href=\"https://en.wikipedia.org/wiki/Iris_flower_data_set\">Iris flower data set.</a>\n",
"\n",
"In the notebooks that follow, an example of running preprocessing, training, and prediction using the Google Cloud Machine Learning Engine services are given. Note that running the cloud versions of preprocessing, training, and prediction take longer than the local versions. The performance advantage of using the cloud applies to very large data sets, and you don't see it with this sample because the data is small and run time is dominated by setup overhead."
"This notebook uses the datalab structured data package for building and running a Tensorflow classification model locally. This notebook uses the classic <a href=\"https://en.wikipedia.org/wiki/Iris_flower_data_set\">Iris flower data set.</a>"
]
},
{
Expand All @@ -29,7 +27,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Lets look at the versions of datalab_structured_data and TF we have. Make sure TF and SD are 1.0.0"
"Let's look at the versions of TensorFlow and the structured data package we have. Make sure TF and SD are 1.0.0"
]
},
{
Expand Down Expand Up @@ -74,7 +72,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook will write files during preprocessing, training, and prediction. Please give a root folder you wish to use."
"This notebook will write files during preprocessing, training, and prediction into a folder called 'iris_notebook_workspace'. Edit the next cell if you want to write files to a different location."
]
},
{
Expand All @@ -90,8 +88,7 @@
"# already exists, it will be deleted.\n",
"LOCAL_ROOT = './iris_notebook_workspace'\n",
"\n",
"# No need to edit anything else in this cell. But if you do, you \n",
"# might need to chagne the global variables in the cloud notebooks.\n",
"# No need to edit anything else in this cell.\n",
"LOCAL_PREPROCESSING_DIR = os.path.join(LOCAL_ROOT, 'preprocessing')\n",
"LOCAL_TRAINING_DIR = os.path.join(LOCAL_ROOT, 'training')\n",
"LOCAL_BATCH_PREDICTION_DIR = os.path.join(LOCAL_ROOT, 'batch_prediction')\n",
Expand Down Expand Up @@ -407,7 +404,6 @@
" } \n",
"]\n",
"\n",
"# Write schema to a file so that the cloud notebooks can use it.\n",
"file_io.write_string_to_file(\n",
" LOCAL_SCHEMA_FILE,\n",
" json.dumps(schema, indent=2))"
Expand Down Expand Up @@ -466,7 +462,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The output of preprocessing is a numerical_analysis file that contains analysis from the numerical columns, and a vocab file from each categorical column. The files preoduced by preprocessing are consumed in training, and you should not have to worry about these files. Just for fun, lets look at them."
"The output of analyze is a stats file that contains analysis from the numerical columns, and a vocab file from each categorical column. The files preoduced by analyze are consumed in training, and you should not have to worry about these files. Just for fun, lets look at them."
]
},
{
Expand Down Expand Up @@ -558,7 +554,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The files in the output folder of preprocessing are consumed by the trainer. Training requires a transform config file to describe what transforms to apply on the data. The key and target transform are the only required transform, a default transform will be applied to every other column if it is not listed in the transforms."
"The files in the output folder of analyze are consumed by the trainer. Training requires a features file to describe what transforms to apply on the data. The key and target transform are the only required transform, a default transform will be applied to every other column if it is not listed in the features dict."
]
},
{
Expand All @@ -576,7 +572,7 @@
" \"flower\": {\"transform\": \"target\"}\n",
" }\n",
"\n",
"# Write the features to a file so that the cloud notebooks can use the same features.\n",
"# Write the features to a file.\n",
"file_io.write_string_to_file(\n",
" LOCAL_FEATURES_FILE,\n",
" json.dumps(features, indent=2)\n",
Expand Down Expand Up @@ -861,7 +857,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Local batch prediction runs prediction on batched input data. This is ideal if the input dataset is very large or you have limited available main memory. However, for very large datasets, it is better to run batch prediction using the Google Cloud Machine Learning Engine services. Two output formats are supported, csv and json. The output may also be shardded. Another feature of batch prediction is the option to run evaluation--prediction on data that contains the target column. Like local_predict, the input data must batch the schema used for training."
"Local batch prediction runs prediction on batched input data. This is ideal if the input dataset is very large, or you have limited available main memory. However, for very large datasets, it is better to run batch prediction using the Google Cloud Machine Learning Engine services. Two output formats are supported: csv and json. The output may also be sharded. Another feature of batch prediction is the option to run evaluation--prediction on data that contains the target column. Like local_predict, the input data must match the schema used for training."
]
},
{
Expand Down Expand Up @@ -1139,7 +1135,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"As everything was written to LOCAL_ROOT, we can simply remove this folder. If you want to delete those files, uncomment and run the next cell. If you want to run any Service notebook, don't delete LOCAL_ROOT."
"As everything was written to LOCAL_ROOT, we can simply remove this folder. If you want to delete those files, uncomment and run the next cell."
]
},
{
Expand Down