Skip to content

Commit

Permalink
Merge pull request #604 from dhiganthrao/master
Browse files Browse the repository at this point in the history
Replaced commit() with flush() in documentation.
  • Loading branch information
mynameisvinn committed Feb 21, 2021
2 parents 7f0a614 + db717c6 commit f0bddd7
Show file tree
Hide file tree
Showing 12 changed files with 14 additions and 13 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ ds = Dataset(

ds["image"][:] = np.zeros((4, 512, 512))
ds["label"][:] = np.zeros((4, 512, 512))
ds.commit()
ds.flush()
```

3. Access it from anywhere else in the world, on any device having a command line:
Expand Down
4 changes: 2 additions & 2 deletions README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ ds = Dataset(
# filling the data containers with data (here - zeroes to initialize)
ds["image"][:] = np.zeros((4, 512, 512))
ds["label"][:] = np.zeros((4, 512, 512))
ds.commit() # executing the creation of the dataset
ds.flush() # executing the creation of the dataset
```

您也可以指明 `s3://bucket/path`, `gcs://bucket/path` 或 azure 路径。您可以在[这里](https://docs.activeloop.ai/en/latest/simple.html#data-storage)找到云储存的更多相关信息。
Expand Down Expand Up @@ -141,7 +141,7 @@ ds = Dataset(

ds["image"][:] = np.zeros((4, 512, 512))
ds["label"][:] = np.zeros((4, 512, 512))
ds.commit()
ds.flush()
```

3. 在任何地点,以任何机器,只要有命令行就可访问它:
Expand Down
2 changes: 1 addition & 1 deletion docs/source/concepts/dataset.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ To add data to the dataset:
```python
ds["image"][:] = np.ones((4, 512, 512))
ds["label"][:] = np.ones((4, 512, 512))
ds.commit()
ds.flush()
```

## Load the data
Expand Down
2 changes: 1 addition & 1 deletion docs/source/simple.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ This code creates dataset in *"./data/examples/new_api_intro"* folder with overw
After this we can loop over dataset and read/write from it.


### Why commit?
### Why flush?

Since caching is in place, you need to tell the program to push final changes to a permanent storage.

Expand Down
4 changes: 2 additions & 2 deletions docs/source/tutorials/new_api.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ for i in range(len(ds)):

print(ds["image", 5].numpy())
print(ds["label", 100:110].numpy())
ds.commit()
ds.flush()
```

4) Transferring from TFSDS
Expand All @@ -75,7 +75,7 @@ This code creates dataset in *"./data/examples/new_api_intro"* folder with overw
After this we can loop over dataset and read/write from it.


### **Why commit?**
### **Why flush?**

Since caching is in place, you need to tell program to push final changes to permanent storage.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2827,7 +2827,7 @@
"# Upload Data\n",
"ds[\"histograms\"][:] = histograms_combined\n",
"ds[\"yields\"][:] = yield_values_linked_hists\n",
"ds.commit()\n",
"ds.flush()\n",
"\n",
"# Load the data\n",
"print(\"Load data...\")\n",
Expand Down
2 changes: 1 addition & 1 deletion examples/basic.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ def main():
# Upload Data
ds["image"][:] = np.ones((4, 512, 512))
ds["label"][:] = np.ones((4, 512, 512))
ds.commit()
ds.flush()

# Load the data
ds = Dataset(tag)
Expand Down
2 changes: 1 addition & 1 deletion examples/big_image.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ def main():
(2048, 2048, 4), dtype="uint8"
) # single chunk read/write
print(ds._tensors["/image"].get_shape((3,)))
ds.commit()
ds.flush()


if __name__ == "__main__":
Expand Down
2 changes: 1 addition & 1 deletion examples/large_dataset_build.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ def create_large_dataset():

for i in range(len(ds) // 10):
ds["image", i * 10 : i * 10 + 10] = i * array
ds.commit()
ds.flush()

ds = hub.Dataset("./data/examples/large_dataset_build")
print(ds.keys, ds["image"].shape, ds["image"].dtype)
Expand Down
2 changes: 1 addition & 1 deletion examples/mnist_upload_speed_benchmark.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ def main():
# with Timer(f"Sample {i}"):
ds["image", i : i + step] = arr

ds.commit()
ds.flush()


if __name__ == "__main__":
Expand Down
2 changes: 1 addition & 1 deletion examples/new_api_intro.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ def main():
print(ds.shape)
print(ds["label", 100:110].numpy())
with Timer("Committing"):
ds.commit()
ds.flush()

ds = Dataset(path)
print(ds.schema)
Expand Down
1 change: 1 addition & 0 deletions hub/api/dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -656,6 +656,7 @@ def flush(self):

def commit(self):
""" Deprecated alias to flush()"""
logger.warning("commit() is deprecated. Use flush() instead")
self.flush()

def close(self):
Expand Down

0 comments on commit f0bddd7

Please sign in to comment.