Skip to content

Commit

Permalink
[HOPSWORKS-2077] Append: mock tf and pydoop modules if ModuleNotFound…
Browse files Browse the repository at this point in the history
…Error for documentation (#125)

Co-authored-by: davitbzh <davit@logicalclocks.com>
  • Loading branch information
davitbzh and davitbzh committed Oct 28, 2020
1 parent 0b27bdb commit 581fd15
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 6 deletions.
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,10 +86,12 @@ td.save(feature_join)

Feed the training dataset to a TensorFlow model:
```python
train_input_feeder = training_dataset.feed(target_name="label",
split="train",
is_training=True)
train_input = train_input_feeder.tf_record_dataset()
tf_data_object = training_dataset.tf_data(target_name="label",
split="train",
is_training=True)
train_input = tf_data_object.tf_record_dataset(batch_size=32,
num_epochs=5,
process=True)
```

You can find more examples on how to use the library in our [hops-examples](https://github.com/logicalclocks/hops-examples) repository.
Expand Down
5 changes: 3 additions & 2 deletions python/hsfs/core/tfdata_engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,17 +16,18 @@

import re
import itertools
import mock
from typing import Optional

try:
import tensorflow as tf
except ModuleNotFoundError:
pass
tf = mock.Mock()

try:
from pydoop import hdfs
except ModuleNotFoundError:
pass
hdfs = mock.Mock()

try:
import boto3
Expand Down
1 change: 1 addition & 0 deletions python/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ def read(fname):
"PyMySQL",
"pyjks",
"sqlalchemy",
"mock",
],
extras_require={
"dev": [
Expand Down

0 comments on commit 581fd15

Please sign in to comment.