Feature/hello objectron notebook #749

haiyangdeperci · 2021-04-06T09:32:10Z

This PR adds the requested filtering.

github-actions · 2021-04-06T09:32:40Z

Locust summary

Git references

Initial: 3e86342
Terminal: 76257f4

hub/api/dataset.py

Changes:

Name: Dataset
Type: class
Changed lines: 177
Total lines: 1087
Changes:
- Name: Dataset.__init__
  Type: function
  Changed lines: 1
  Total lines: 182
- Name: Dataset._get_dynamic_tensor_dtype
  Type: function
  Changed lines: 7
  Total lines: 7
- Name: Dataset._get_compressor
  Type: function
  Changed lines: 13
  Total lines: 13
- Name: Dataset._generate_storage_tensors
  Type: function
  Changed lines: 2
  Total lines: 25
- Name: Dataset.to_pytorch
  Type: function
  Changed lines: 2
  Total lines: 24
- Name: Dataset.to_tensorflow
  Type: function
  Changed lines: 2
  Total lines: 14
- Name: Dataset.from_directory
  Type: function
  Changed lines: 148
  Total lines: 149
  Changes:

benchmarks/benchmark_dataset_comparison.py

Changes:

Name: HubAdapter
Type: class
Changed lines: 18
Total lines: 18
Changes:
- Name: HubAdapter.__init__
  Type: function
  Changed lines: 2
  Total lines: 2
- Name: HubAdapter.__len__
  Type: function
  Changed lines: 2
  Total lines: 2
- Name: HubAdapter.shape
  Type: function
  Changed lines: 2
  Total lines: 2
- Name: HubAdapter.__iter__
  Type: function
  Changed lines: 3
  Total lines: 3
- Name: HubAdapter.__getitem__
  Type: function
  Changed lines: 3
  Total lines: 3
Name: Timer
Type: class
Changed lines: 4
Total lines: 4
Changes:
- Name: Timer.__init__
  Type: function
  Changed lines: 3
  Total lines: 3
Name: prepare_torch_dataset
Type: function
Changed lines: 25
Total lines: 25
Name: time_iter_hub_local_pytorch
Type: function
Changed lines: 23
Total lines: 23
Name: time_iter_hub_wasabi_pytorch
Type: function
Changed lines: 19
Total lines: 19
Name: time_iter_hub_s3_pytorch
Type: function
Changed lines: 24
Total lines: 24
Name: time_iter_pytorch
Type: function
Changed lines: 20
Total lines: 20
Name: time_iter_hub_local_tensorflow
Type: function
Changed lines: 15
Total lines: 15
Name: time_iter_hub_wasabi_tensorflow
Type: function
Changed lines: 12
Total lines: 12
Name: time_iter_hub_s3_tensorflow
Type: function
Changed lines: 17
Total lines: 17
Name: time_iter_tensorflow
Type: function
Changed lines: 19
Total lines: 19

hub/api/integrations.py

Changes:

Name: _to_tensorflow
Type: function
Changed lines: 13
Total lines: 110
Changes:
- Name: _to_tensorflow._get_active_item
  Type: function
  Changed lines: 1
  Total lines: 13
- Name: _to_tensorflow.dict_to_tf
  Type: function
  Changed lines: 2
  Total lines: 5
- Name: _to_tensorflow.dtype_to_tf
  Type: function
  Changed lines: 2
  Total lines: 11
- Name: _to_tensorflow.get_output_shapes
  Type: function
  Changed lines: 2
  Total lines: 7
- Name: _to_tensorflow.output_shapes_from_dict
  Type: function
  Changed lines: 2
  Total lines: 5
Name: TorchDataset
Type: class
Changed lines: 4
Total lines: 81
Changes:
- Name: TorchDataset.__init__
  Type: function
  Changed lines: 1
  Total lines: 11
- Name: TorchDataset._init_ds
  Type: function
  Changed lines: 2
  Total lines: 14
- Name: TorchDataset._get_active_item
  Type: function
  Changed lines: 1
  Total lines: 13
Name: _to_pytorch
Type: function
Changed lines: 7
Total lines: 33

hub/api/tests/test_dataset.py

Changes:

Name: test_dataset_from_directory
Type: function
Changed lines: 49
Total lines: 49
Changes:
- Name: test_dataset_from_directory.create_image
  Type: function
  Changed lines: 8
  Total lines: 8
- Name: test_dataset_from_directory.data_in_dir
  Type: function
  Changed lines: 6
  Total lines: 6
- Name: test_dataset_from_directory.root_dir_image
  Type: function
  Changed lines: 9
  Total lines: 9
- Name: test_dataset_from_directory.del_data
  Type: function
  Changed lines: 5
  Total lines: 5
Name: test_tensorview_slicing
Type: function
Changed lines: 2
Total lines: 9
Name: test_dataset_copy_s3_local
Type: function
Changed lines: 2
Total lines: 14
Name: test_dataset_copy_gcs_local
Type: function
Changed lines: 2
Total lines: 14
Name: test_dataset_copy_azure_local
Type: function
Changed lines: 5
Total lines: 21
Name: test_dataset_copy_hub_local
Type: function
Changed lines: 2
Total lines: 14
Name: test_dataset_copy_gcs_s3
Type: function
Changed lines: 2
Total lines: 14
Name: test_dataset_copy_exception
Type: function
Changed lines: 2
Total lines: 12
Name: test_dataset_filter_3
Type: function
Changed lines: 2
Total lines: 19

hub/store/dynamic_tensor.py

Changes:

Name: DynamicTensor
Type: class
Changed lines: 18
Total lines: 484
Changes:
- Name: DynamicTensor.__init__
  Type: function
  Changed lines: 1
  Total lines: 114
- Name: DynamicTensor.__getitem__
  Type: function
  Changed lines: 14
  Total lines: 24
- Name: DynamicTensor.get_shape
  Type: function
  Changed lines: 3
  Total lines: 16

hub/api/datasetview.py

Changes:

Name: DatasetView
Type: class
Changed lines: 8
Total lines: 320
Changes:
- Name: DatasetView.to_tensorflow
  Type: function
  Changed lines: 2
  Total lines: 13
- Name: DatasetView.to_pytorch
  Type: function
  Changed lines: 6
  Total lines: 23

hub/store/metastore.py

Changes:

Name: MetaStorage
Type: class
Changed lines: 3
Total lines: 133
Changes:
- Name: MetaStorage.__setitem__
  Type: function
  Changed lines: 1
  Total lines: 17
- Name: MetaStorage.__delitem__
  Type: function
  Changed lines: 2
  Total lines: 16

hub/api/tensorview.py

Changes:

Name: TensorView
Type: class
Changed lines: 7
Total lines: 335
Changes:
- Name: TensorView.numpy
  Type: function
  Changed lines: 7
  Total lines: 56

hub/client/base.py

Changes:

Name: HubHttpClient
Type: class
Changed lines: 5
Total lines: 104
Changes:
- Name: HubHttpClient.request
  Type: function
  Changed lines: 5
  Total lines: 52

hub/compute/transform.py

Changes:

Name: Transform
Type: class
Changed lines: 10
Total lines: 399
Changes:
- Name: Transform.upload
  Type: function
  Changed lines: 10
  Total lines: 56
  Changes:

hub/exceptions.py

Changes:

Name: AuthenticationException
Type: class
Changed lines: 1
Total lines: 4
Changes:
- Name: AuthenticationException.__init__
  Type: function
  Changed lines: 1
  Total lines: 3
Name: AuthorizationException
Type: class
Changed lines: 1
Total lines: 8
Changes:
- Name: AuthorizationException.__init__
  Type: function
  Changed lines: 1
  Total lines: 7
Name: NotFoundException
Type: class
Changed lines: 1
Total lines: 7
Changes:
- Name: NotFoundException.__init__
  Type: function
  Changed lines: 1
  Total lines: 6
Name: BadRequestException
Type: class
Changed lines: 1
Total lines: 12
Changes:
- Name: BadRequestException.__init__
  Type: function
  Changed lines: 1
  Total lines: 11
Name: OverLimitException
Type: class
Changed lines: 1
Total lines: 7
Changes:
- Name: OverLimitException.__init__
  Type: function
  Changed lines: 1
  Total lines: 6
Name: ServerException
Type: class
Changed lines: 1
Total lines: 4
Changes:
- Name: ServerException.__init__
  Type: function
  Changed lines: 1
  Total lines: 3
Name: BadGatewayException
Type: class
Changed lines: 1
Total lines: 4
Changes:
- Name: BadGatewayException.__init__
  Type: function
  Changed lines: 1
  Total lines: 3
Name: GatewayTimeoutException
Type: class
Changed lines: 1
Total lines: 4
Changes:
- Name: GatewayTimeoutException.__init__
  Type: function
  Changed lines: 1
  Total lines: 3
Name: WaitTimeoutException
Type: class
Changed lines: 1
Total lines: 4
Changes:
- Name: WaitTimeoutException.__init__
  Type: function
  Changed lines: 1
  Total lines: 3
Name: LockedException
Type: class
Changed lines: 1
Total lines: 4
Changes:
- Name: LockedException.__init__
  Type: function
  Changed lines: 1
  Total lines: 3
Name: HubDatasetNotFoundException
Type: class
Changed lines: 1
Total lines: 5
Changes:
- Name: HubDatasetNotFoundException.__init__
  Type: function
  Changed lines: 1
  Total lines: 4
Name: PermissionException
Type: class
Changed lines: 1
Total lines: 5
Changes:
- Name: PermissionException.__init__
  Type: function
  Changed lines: 1
  Total lines: 4
Name: ShapeArgumentNotFoundException
Type: class
Changed lines: 1
Total lines: 5
Changes:
- Name: ShapeArgumentNotFoundException.__init__
  Type: function
  Changed lines: 1
  Total lines: 4
Name: DirectoryNotEmptyException
Type: class
Changed lines: 1
Total lines: 5
Changes:
- Name: DirectoryNotEmptyException.__init__
  Type: function
  Changed lines: 1
  Total lines: 4
Name: SchemaArgumentNotFoundException
Type: class
Changed lines: 1
Total lines: 5
Changes:
- Name: SchemaArgumentNotFoundException.__init__
  Type: function
  Changed lines: 1
  Total lines: 4
Name: ValueShapeError
Type: class
Changed lines: 1
Total lines: 5
Changes:
- Name: ValueShapeError.__init__
  Type: function
  Changed lines: 1
  Total lines: 4
Name: NoneValueException
Type: class
Changed lines: 1
Total lines: 5
Changes:
- Name: NoneValueException.__init__
  Type: function
  Changed lines: 1
  Total lines: 4
Name: ShapeLengthException
Type: class
Changed lines: 1
Total lines: 5
Changes:
- Name: ShapeLengthException.__init__
  Type: function
  Changed lines: 1
  Total lines: 4
Name: ModuleNotInstalledException
Type: class
Changed lines: 1
Total lines: 5
Changes:
- Name: ModuleNotInstalledException.__init__
  Type: function
  Changed lines: 1
  Total lines: 4
Name: DaskModuleNotInstalledException
Type: class
Changed lines: 1
Total lines: 5
Changes:
- Name: DaskModuleNotInstalledException.__init__
  Type: function
  Changed lines: 1
  Total lines: 4
Name: WrongUsernameException
Type: class
Changed lines: 1
Total lines: 8
Changes:
- Name: WrongUsernameException.__init__
  Type: function
  Changed lines: 1
  Total lines: 7
Name: NotHubDatasetToOverwriteException
Type: class
Changed lines: 1
Total lines: 11
Changes:
- Name: NotHubDatasetToOverwriteException.__init__
  Type: function
  Changed lines: 1
  Total lines: 10
Name: NotHubDatasetToAppendException
Type: class
Changed lines: 1
Total lines: 8
Changes:
- Name: NotHubDatasetToAppendException.__init__
  Type: function
  Changed lines: 1
  Total lines: 7
Name: DynamicTensorNotFoundException
Type: class
Changed lines: 1
Total lines: 5
Changes:
- Name: DynamicTensorNotFoundException.__init__
  Type: function
  Changed lines: 1
  Total lines: 4
Name: DynamicTensorShapeException
Type: class
Changed lines: 1
Total lines: 12
Changes:
- Name: DynamicTensorShapeException.__init__
  Type: function
  Changed lines: 1
  Total lines: 11
Name: NotIterable
Type: class
Changed lines: 1
Total lines: 5
Changes:
- Name: NotIterable.__init__
  Type: function
  Changed lines: 1
  Total lines: 4
Name: AddressNotFound
Type: class
Changed lines: 1
Total lines: 5
Changes:
- Name: AddressNotFound.__init__
  Type: function
  Changed lines: 1
  Total lines: 4
Name: OutOfBoundsError
Type: class
Changed lines: 1
Total lines: 4
Name: AlignmentError
Type: class
Changed lines: 1
Total lines: 4
Name: IncompatibleShapes
Type: class
Changed lines: 1
Total lines: 4
Name: IncompatibleBroadcasting
Type: class
Changed lines: 1
Total lines: 4
Name: IncompatibleTypes
Type: class
Changed lines: 1
Total lines: 4
Name: WrongTypeError
Type: class
Changed lines: 1
Total lines: 4
Name: NotAuthorized
Type: class
Changed lines: 1
Total lines: 4
Name: NotFound
Type: class
Changed lines: 1
Total lines: 4
Name: FileSystemException
Type: class
Changed lines: 1
Total lines: 4
Name: S3Exception
Type: class
Changed lines: 1
Total lines: 4
Name: S3CredsParseException
Type: class
Changed lines: 1
Total lines: 4
Name: NotZarrFolderException
Type: class
Changed lines: 1
Total lines: 2
Name: StorageTensorNotFoundException
Type: class
Changed lines: 1
Total lines: 2

hub/report.py

Changes:

Name: ExceptionWithReporting
Type: class
Changed lines: 4
Total lines: 4
Changes:
- Name: ExceptionWithReporting.__init__
  Type: function
  Changed lines: 3
  Total lines: 3

hub/schema/class_label.py

Changes:

Name: ClassLabel
Type: class
Changed lines: 15
Total lines: 161
Changes:
- Name: ClassLabel.__init__
  Type: function
  Changed lines: 14
  Total lines: 84

hub/schema/image.py

Changes:

Name: Image
Type: class
Changed lines: 6
Total lines: 75
Changes:
- Name: Image.__init__
  Type: function
  Changed lines: 1
  Total lines: 48

benchmarks/benchmark_compress_time.py

Changes:

Name: bench_pil_compression
Type: function
Changed lines: 5
Total lines: 5
Name: bench_hub_compression
Type: function
Changed lines: 15
Total lines: 15

benchmarks/benchmark_dataset_iter.py

Changes:

Name: time_iter_pytorch
Type: function
Changed lines: 19
Total lines: 19
Name: time_iter_tensorflow
Type: function
Changed lines: 16
Total lines: 16

benchmarks/benchmark_random_access.py

Changes:

Name: time_random_access
Type: function
Changed lines: 6
Total lines: 6

benchmarks/benchmark_sequential_access.py

Changes:

Name: time_batches
Type: function
Changed lines: 23
Total lines: 23
Name: time_tiledb
Type: function
Changed lines: 56
Total lines: 56
Name: time_zarr
Type: function
Changed lines: 36
Total lines: 36
Name: time_hub
Type: function
Changed lines: 11
Total lines: 11

benchmarks/benchmark_sequential_write.py

Changes:

Name: time_batches
Type: function
Changed lines: 25
Total lines: 25
Name: time_tiledb
Type: function
Changed lines: 25
Total lines: 25
Name: time_zarr
Type: function
Changed lines: 12
Total lines: 12
Name: time_hub
Type: function
Changed lines: 22
Total lines: 22

hub/api/tests/test_converters.py

Changes:

Name: test_to_tensorflow
Type: function
Changed lines: 1
Total lines: 10

hub/cli/auth.py

Changes:

Name: reporting
Type: function
Changed lines: 1
Total lines: 11

hub/schema/bbox.py

Changes:

Name: BBox
Type: class
Changed lines: 1
Total lines: 57

hub/schema/deserialize.py

Changes:

Name: deserialize
Type: function
Changed lines: 2
Total lines: 123

hub/schema/tests/test_features.py

Changes:

Name: test_class_label_2
Type: function
Changed lines: 1
Total lines: 52
Name: test_classlabel_repr
Type: function
Changed lines: 2
Total lines: 8

hub/schema/text.py

Changes:

Name: Text
Type: class
Changed lines: 1
Total lines: 49

hub/store/tests/test_dynamic_tensor.py

Changes:

Name: test_dynamic_tensor_2
Type: function
Changed lines: 1
Total lines: 15

hub/utils.py

Changes:

Name: batchify
Type: function
Changed lines: 2
Total lines: 9

codecov · 2021-04-06T09:37:02Z

Codecov Report

Merging #749 (a210eea) into master (3e86342) will increase coverage by 0.11%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #749      +/-   ##
==========================================
+ Coverage   89.54%   89.65%   +0.11%     
==========================================
  Files          65       65              
  Lines        4418     4418              
==========================================
+ Hits         3956     3961       +5     
+ Misses        462      457       -5

Impacted Files	Coverage Δ
hub/client/base.py	`72.00% <0.00%> (+6.66%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3e86342...a210eea. Read the comment docs.

Diveafall

Can we separate this into a notebook with a different name so that this doesn't get override the notebook created by @mynameisvinn. Feel free to rename Vinn's notebook if it makes more sense.

haiyangdeperci added 6 commits March 22, 2021 10:23

update tensors from generic types to hub schema representations

0621b9e

represent annotations in the notebook

0d334c5

fix typos and timings

455b650

pass element as an argument and clear some output

1f89bbd

change an example

aadef32

add filtering of frames

76257f4

merge with master

a210eea

haiyangdeperci requested a review from Diveafall April 6, 2021 15:31

Diveafall reviewed Apr 8, 2021

View reviewed changes

Diveafall merged commit 4d587b9 into master Apr 8, 2021

Diveafall deleted the feature/hello_objectron_notebook branch April 8, 2021 15:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/hello objectron notebook #749

Feature/hello objectron notebook #749

haiyangdeperci commented Apr 6, 2021

github-actions bot commented Apr 6, 2021

codecov bot commented Apr 6, 2021 •

edited

Diveafall left a comment

Feature/hello objectron notebook #749

Feature/hello objectron notebook #749

Conversation

haiyangdeperci commented Apr 6, 2021

github-actions bot commented Apr 6, 2021

Locust summary

Git references

codecov bot commented Apr 6, 2021 • edited

Codecov Report

Diveafall left a comment

Choose a reason for hiding this comment

codecov bot commented Apr 6, 2021 •

edited