Support GCS #3919

Marishka17 · 2021-11-18T07:14:28Z

Motivation and context

Resolve #3760

How has this been tested?

Manually + extended jest tests

Checklist

I submit my changes into the develop branch
I have added description of my changes into CHANGELOG file
I have updated the documentation accordingly
I have added tests to cover my changes
I have linked related issues (read github docs)
I have increased versions of npm packages if it is necessary (cvat-canvas,
cvat-core, cvat-data and cvat-ui)

License

I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.
~~- [ ] I have updated the license header for each file (see an example below)~~

# Copyright (C) 2021 Intel Corporation
#
# SPDX-License-Identifier: MIT

Marishka17 · 2021-11-23T09:07:57Z

cvat/apps/engine/models.py

    provider_type = models.CharField(max_length=20, choices=CloudProviderChoice.choices())
-    resource = models.CharField(max_length=63)
+    resource = models.CharField(max_length=222)


@nmanovic, I've increased restrictions for bucket names length because google cloud storage has 2 restriction variants - 63 like other providers and 222 with dots in bucket name. Are we going to support the second case?

@Marishka17 , you can even have 255 (just in case). I really don't think that we have to be very strict here.

cvat-core/src/cloud-storage.js

cvat-core/src/server-proxy.js

cvat-ui/src/assets/google-cloud.svg

cvat-ui/src/components/create-cloud-storage-page/cloud-storage-form.tsx

bsekachev · 2021-11-24T12:10:46Z

cvat-ui/src/components/create-cloud-storage-page/cloud-storage-form.tsx

@@ -170,15 +198,37 @@ export default function CreateCloudStorageForm(props: Props): JSX.Element {
                duration: 15,
            });
        }
-    }, [cloudStorage]);
+    }, [cloudStorageId]);


effect callback also depends on cloudStorage

cvat-ui/src/components/create-cloud-storage-page/cloud-storage-form.tsx

cvat-ui/src/components/create-cloud-storage-page/location.tsx

bsekachev · 2021-11-24T12:27:50Z

cvat-ui/src/components/create-cloud-storage-page/location.tsx

+                className: 'cvat-incorrect-add-region-notification',
+            });
+        } else {
+            const regionsCopy = regions;


I hardly think it is a copy, that is probably link to the same object. Check if regions contains newRegionKey after it was inserted to regionsCopy.

If you need exactly copy, maybe better to use Object and create a shallow copy this way: const copy = { ...source }

…-form.tsx Co-authored-by: Boris Sekachev <boris.sekachev@intel.com>

bsekachev · 2021-11-24T17:16:09Z

Looks like the patch works for me. But maybe one reccomendation.
After I select a key file, the field is empty anyway

Probably better to show there file name.

Marishka17 · 2021-11-26T08:44:44Z

@nmanovic, Could you please take a look at the server part of this PR?

nmanovic · 2021-11-26T12:20:44Z

cvat/apps/engine/serializers.py

@@ -809,6 +811,7 @@ class CloudStorageSerializer(serializers.ModelSerializer):
    key = serializers.CharField(max_length=20, allow_blank=True, required=False)
    secret_key = serializers.CharField(max_length=40, allow_blank=True, required=False)
    key_file_path = serializers.CharField(max_length=64, allow_blank=True, required=False)


@Marishka17 , I believe key_file_path has to be removed. What do you think? Probably you can do that in a separate PR.

Hmm, I added the option to upload a key file for a case when CVAT instance is deployed remotely. But now I've looked at it from a different side. Ok, I'll prepare PR for this.

…o openvinotoolkit-develop * 'develop' of https://github.com/openvinotoolkit/cvat: (181 commits) Cypress. Update case 105. Add check "Google cloud storage" provider fields. (cvat-ai#3980) Cypress. Fix case 108 for Firefox. (cvat-ai#3981) update project section (cvat-ai#3979) Cypress. Test "Rotated bounding boxes" (cvat-ai#3961) Issue deleting (cvat-ai#3952) Remove key_file_path field (cvat-ai#3959) update the doc for change default hostname or port (cvat-ai#3915) CVAT_server. Test for "Project updated time". (cvat-ai#3953) fix: cvat/requirements/base.txt to reduce vulnerabilities (cvat-ai#3970) Fix notification (cvat-ai#3960) Update documentation for China users (cvat-ai#3946) Support GCS (cvat-ai#3919) Cypress. Update the test for check issue 3810. (cvat-ai#3900) Update the documentation about the smooth image option (cvat-ai#3947) Project tasks pagination (cvat-ai#3910) Update documentation about review mode (cvat-ai#3944) Preserve the order of the label attributes in the object item details. (cvat-ai#3945) Added smooth image option (cvat-ai#3933) Fixed issue: autoborder points are visible for invisible shapes (cvat-ai#3931) Add FiftyOne to partners list (cvat-ai#3943) ...

* openvinotoolkit-develop: (181 commits) Cypress. Update case 105. Add check "Google cloud storage" provider fields. (cvat-ai#3980) Cypress. Fix case 108 for Firefox. (cvat-ai#3981) update project section (cvat-ai#3979) Cypress. Test "Rotated bounding boxes" (cvat-ai#3961) Issue deleting (cvat-ai#3952) Remove key_file_path field (cvat-ai#3959) update the doc for change default hostname or port (cvat-ai#3915) CVAT_server. Test for "Project updated time". (cvat-ai#3953) fix: cvat/requirements/base.txt to reduce vulnerabilities (cvat-ai#3970) Fix notification (cvat-ai#3960) Update documentation for China users (cvat-ai#3946) Support GCS (cvat-ai#3919) Cypress. Update the test for check issue 3810. (cvat-ai#3900) Update the documentation about the smooth image option (cvat-ai#3947) Project tasks pagination (cvat-ai#3910) Update documentation about review mode (cvat-ai#3944) Preserve the order of the label attributes in the object item details. (cvat-ai#3945) Added smooth image option (cvat-ai#3933) Fixed issue: autoborder points are visible for invisible shapes (cvat-ai#3931) Add FiftyOne to partners list (cvat-ai#3943) ...

Marishka17 added 15 commits October 11, 2021 22:58

Tmp

ab3d2f9

Merge develop

f327b9e

tmp

59c7df1

Add key file upload

179f4b4

Remove api key support

fe4e0b5

fix case when key file was removed

91095b1

Add test

48c5b06

Update search

a9e11bb

styles

6a3c6a9

Fix updating

ddf949d

Fix extra notification

ae74855

Fix reset values

15e6354

Fix file closing

0abbd33

Change restrictions

1aee872

Styles

b3a72b8

Marishka17 requested review from nmanovic and bsekachev November 23, 2021 08:56

Marishka17 marked this pull request as ready for review November 23, 2021 08:57

Marishka17 linked an issue Nov 23, 2021 that may be closed by this pull request

Support Google Cloud Storage in UI #3760

Closed

2 tasks

Marishka17 commented Nov 23, 2021

View reviewed changes

Marishka17 added 2 commits November 23, 2021 12:11

Merge develop

b93358e

Update migration dependency

f17e8a0

bsekachev reviewed Nov 24, 2021

View reviewed changes

Marishka17 and others added 2 commits November 24, 2021 16:01

Update cvat-ui/src/components/create-cloud-storage-page/cloud-storage…

a730250

…-form.tsx Co-authored-by: Boris Sekachev <boris.sekachev@intel.com>

Update cvat-ui/src/components/create-cloud-storage-page/cloud-storage…

7741671

…-form.tsx Co-authored-by: Boris Sekachev <boris.sekachev@intel.com>

Marishka17 changed the title ~~Support GCS~~ [WIP] Support GCS Nov 24, 2021

Fix comments

62f5402

Marishka17 added 2 commits November 25, 2021 11:53

Fix dependency

0a2db6a

Display key file name

9003d9d

Marishka17 added 6 commits November 26, 2021 10:34

f

e5eef51

styles

0388ac2

Merge develop

422bb3c

Increase versions

0c9940d

Update changelog

2a05379

Remove excess

b1dd1a1

Marishka17 changed the title ~~[WIP] Support GCS~~ Support GCS Nov 26, 2021

Marishka17 requested a review from bsekachev November 26, 2021 08:51

nmanovic reviewed Nov 26, 2021

View reviewed changes

Merge branch 'develop' into mk/support_gcs

c92b141

nmanovic approved these changes Nov 26, 2021

View reviewed changes

nmanovic merged commit f59d1f5 into develop Nov 26, 2021

nmanovic deleted the mk/support_gcs branch November 26, 2021 12:41

Marishka17 mentioned this pull request Nov 29, 2021

Remove key_file_path field #3959

Merged

5 tasks

dvkruchinin mentioned this pull request Dec 2, 2021

Cypress. Update case 105. Add check "Google cloud storage" provider fields. #3980

Merged

8 tasks

nmanovic mentioned this pull request Mar 4, 2022

Release v2.0.0 #4422

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support GCS #3919

Support GCS #3919

Marishka17 commented Nov 18, 2021 •

edited

Loading

Marishka17 Nov 23, 2021

nmanovic Nov 26, 2021

bsekachev Nov 24, 2021

bsekachev Nov 24, 2021

bsekachev Nov 24, 2021

bsekachev commented Nov 24, 2021

Marishka17 commented Nov 26, 2021

nmanovic Nov 26, 2021

Marishka17 Nov 26, 2021

Support GCS #3919

Support GCS #3919

Conversation

Marishka17 commented Nov 18, 2021 • edited Loading

Motivation and context

How has this been tested?

Checklist

License

Marishka17 Nov 23, 2021

Choose a reason for hiding this comment

nmanovic Nov 26, 2021

Choose a reason for hiding this comment

bsekachev Nov 24, 2021

Choose a reason for hiding this comment

bsekachev Nov 24, 2021

Choose a reason for hiding this comment

bsekachev Nov 24, 2021

Choose a reason for hiding this comment

bsekachev commented Nov 24, 2021

Marishka17 commented Nov 26, 2021

nmanovic Nov 26, 2021

Choose a reason for hiding this comment

Marishka17 Nov 26, 2021

Choose a reason for hiding this comment

Marishka17 commented Nov 18, 2021 •

edited

Loading