Skip to content

Commit

Permalink
Merge 67b0a59 into 4dfe0f3
Browse files Browse the repository at this point in the history
  • Loading branch information
boyuangong committed Dec 18, 2018
2 parents 4dfe0f3 + 67b0a59 commit fcfddf8
Show file tree
Hide file tree
Showing 4 changed files with 46 additions and 15 deletions.
2 changes: 1 addition & 1 deletion autokeras/__init__.py
@@ -1,4 +1,4 @@
from autokeras.image.image_supervised import ImageClassifier, ImageRegressor
from autokeras.image.image_supervised import ImageClassifier, ImageRegressor, PortableImageSupervised
from autokeras.text.text_supervised import TextClassifier, TextRegressor
from autokeras.tabular.tabular_supervised import TabularClassifier, TabularRegressor
from autokeras.net_module import CnnModule, MlpModule
8 changes: 4 additions & 4 deletions autokeras/utils.py
Expand Up @@ -18,7 +18,6 @@
from scipy.ndimage import zoom



class NoImprovementError(Exception):
def __init__(self, message):
self.message = message
Expand Down Expand Up @@ -49,12 +48,11 @@ def pickle_to_file(obj, path):
"""Save the pickle file to the specified path."""
pickle.dump(obj, open(path, 'wb'))


# TODO cannot detect nvidia-smi in Windows normally. We need a fall back for windows
def get_device():
""" If CUDA is available, use CUDA device, else use CPU device.
When choosing from CUDA devices, this function will choose the one with max memory available.
Returns: string device name.
"""
# TODO: could use gputil in the future
Expand Down Expand Up @@ -140,7 +138,7 @@ def verbose_print(new_father_id, new_graph, new_model_id):
"""Print information about the operation performed on father model to obtain current model and father's id."""
cell_size = [24, 49]

logging.info('New Model Id - '+str(new_model_id))
logging.info('New Model Id - ' + str(new_model_id))
header = ['Father Model ID', 'Added Operation']
line = '|'.join(str(x).center(cell_size[i]) for i, x in enumerate(header))
logging.info('\n' + '+' + '-' * len(line) + '+')
Expand All @@ -155,6 +153,7 @@ def verbose_print(new_father_id, new_graph, new_model_id):
logging.info('|' + line + '|')
logging.info('+' + '-' * len(line) + '+')


def validate_xy(x_train, y_train):
"""Validate `x_train`'s type and the shape of `x_train`, `y_train`."""
try:
Expand Down Expand Up @@ -264,4 +263,5 @@ def get_system():
return Constant.SYS_LINUX
if os.name == 'nt':
return Constant.SYS_WINDOWS

raise EnvironmentError('Unsupported environment')
22 changes: 22 additions & 0 deletions examples/portable_load.py
@@ -0,0 +1,22 @@
import os

from keras.datasets import mnist

from autokeras import ImageClassifier
from autokeras.utils import pickle_from_file

# Customer temp dir by your own
TEMP_DIR = '/tmp/autokeras_U8KEOQ'
model_file_name = os.path.join(TEMP_DIR, 'test_autokeras_model.pkl')

if __name__ == '__main__':
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(x_train.shape + (1,))
x_test = x_test.reshape(x_test.shape + (1,))
clf = ImageClassifier(verbose=True, augment=False, path=TEMP_DIR, resume=True)
clf.fit(x_train, y_train, time_limit=30 * 60)
clf.final_fit(x_train, y_train, x_test, y_test)
clf.export_autokeras_model(model_file_name)
model = pickle_from_file(model_file_name)
results = model.evaluate(x_test, y_test)
print(results)
29 changes: 19 additions & 10 deletions mkdocs/docs/start.md
Expand Up @@ -17,7 +17,6 @@ You need to download the code from the GitHub repo and run the following command
pip install -r requirements.txt
python setup.py install

=======

#### How to visualize the best selected architecture ?

Expand Down Expand Up @@ -105,13 +104,25 @@ The returned values `x_train` and `y_train` are the numpy arrays,
which can be directly feed into the `fit` function of `ImageClassifier`.

#### How to export keras models?
clf.load_searcher().load_best_model().produce_keras_model().save('my_model.h5')
This uses the keras function model.save() to export a single HDF5 file containing the architecture of the model, the weights of the model, the training configuration, and the state of the optimizer. See https://keras.io/getting-started/faq/#how-can-i-save-a-keras-model

Note: This is being built into AutoKeras as ImageClassifier().export_keras_model()

#### how to export Portable model
from autokeras import ImageClassifier
clf = ImageClassifier(verbose=True, augment=False)
clf.load_searcher().load_best_model().produce_keras_model().save('my_model.h5')
clf.export_autokeras_model(model_file_name)
The model will be stored into the path `model_file_name`.

#### How to load exported Portable model?
from autokeras.utils import pickle_from_file
model = pickle_from_file(model_file_name)
results = model.evaluate(x_test, y_test)
print(results)

The model will be loaded from the path `model_file_name` and then you can use the functions listed in `PortableImageSupervised`.

This uses the keras function model.save() to export a single HDF5 file containing the architecture of the model, the weights of the model, the training configuration, and the state of the optimizer. See https://keras.io/getting-started/faq/#how-can-i-save-a-keras-model
Note: This is being built into AutoKeras as ImageClassifier().export_keras_model()

#### How to visualize keras models?

Expand All @@ -122,6 +133,7 @@ This is not specific to AutoKeras, however, the following will generate a .PNG v
from keras.utils import plot_model
plot_model(model, to_file='my_model.png')

# CnnGenerator tutorial

`CnnGenerator` in `net_module.py` is a child class of `Networkmodule`. It can generates neural architecture with basic cnn modules
Expand Down Expand Up @@ -208,7 +220,7 @@ Normally, there's two place to call the MlpGenerator, one is call `MlpGenerator.
For example, in a image classification class `ImageClassifier`, one can initialize the cnn module as:

```python
self.mlp = MlpGenerator(loss, metric, searcher_args, path, verbose)
mlpModule = MlpModule(loss, metric, searcher_args, path, verbose)
```
Where:
* `loss` and `metric` determines by the type of training model(classification or regression or others)
Expand All @@ -218,7 +230,7 @@ Where:

Then, for the searching part, one can call:
```python
self.mlp.fit(n_output_node, input_shape, train_data, test_data, time_limit=24 * 60 * 60)
mlpModule.fit(n_output_node, input_shape, train_data, test_data, time_limit=24 * 60 * 60)
```
where:
* n_output_node: A integer value represent the number of output node in the final layer.
Expand All @@ -230,13 +242,10 @@ where:

And for final testing(testing the best searched model), one can call:
```python
self.mlp.final_fit(train_data, test_data, trainer_args=None, retrain=False)
mlpModule.final_fit(train_data, test_data, trainer_args=None, retrain=False)
```
where:
* train_data: A DataLoader instance representing the training data.
* test_data: A DataLoader instance representing the testing data.
* trainer_args: A dictionary containing the parameters of the ModelTrainer constructor.
* retrain: A boolean of whether reinitialize the weights of the model.



0 comments on commit fcfddf8

Please sign in to comment.