Enforce space quota on uploads

[refs]

Acceptance Criteria

• Space members can not add files to a Space if they would exceed the quota limit
• If users try to do it and hit the limit, they'll see a meaningful error message (API-level)
• acceptance tests cover the implemented functionality

TODO

• Check with nested folders within a storage space

Regression found

The create storage space response returns an invalid webdav url. Needs to be fixed.

Questions

What is the behavior if:

• the space quota is 99% used and it is reduced to 50%? Can we still delete? (i.e move items to the trash bin)
• can a space have a larger quota than the root of the FS?
  i.e: space quota: 50 bytes, root FS quota: 20 bytes
• Should the metadata be taken into consideration? Are we allowed to go over the total quota?

Technical Debt

• Should we guard against setting the space quota to a number lower than the number of user bytes?
• Size calculations are done ignoring the .space folder. This folder needs to only host the description and image of the storage, to prevent any leaks.

Testing

1. create a storage space with quota set to 20 bytes

curl -k -X POST 'https://localhost:9200/graph/v1.0/drives' -u admin:admin -d '{"name":"marketing", "quota": {"total": 20}}' -v

The resulting node will use up 39 bytes (because of the metadata)

2. attempt to upload a file with 10 bytes. It will fail, because the first upload ate up all the available space:

curl -k -X PUT https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157\!d36bfa36-9441-4ec4-9d32-f2e8880bd301/test.txt -d "1234567890" -u admin:admin -v

[C0rby]

I haven't tested it but I think this method will not work correctly when you start at a sub-sub-folder.
/<space-root>/f1/f2 f2.Parent() = f1 and f1 has a name so that will return f1, if I'm not mistaken.

[refs]

reva/pkg/storage/utils/decomposedfs/spaces.go

Lines 92 to 94 in 1b66c13

	if err := n.SetMetadata(xattrs.SpaceNameAttr, req.Name); err != nil {
	return nil, err
	}

AFAIK we only set the name attribute at the root of the space 🤔

[refs]

If what you meant to say is that if a file does not contain a name the function will early return, then you're right :p. This is the tests I'm running:

Test with a nested folder:

1. create test folders under space's root: <root>/f1/f2

curl -k -X MKCOL https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157\!492dc039-70c3-44e2-8a71-bdcf690a9dbe/f1 -u admin:admin

curl -k -X MKCOL https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157\!492dc039-70c3-44e2-8a71-bdcf690a9dbe/f1 -u admin:admin

2. upload in f2

curl -k -X PUT https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157\!492dc039-70c3-44e2-8a71-bdcf690a9dbe/f1/f2/test.txt -d "12345678901" -u admin:admin -v

3. update the space quota

xattr -w user.ocis.quota 10 /var/tmp/ocis/storage/users/spaces/project/492dc039-70c3-44e2-8a71-bdcf690a9dbe

4. upload again

curl -k -X PUT https://localhost:9200/dav/spaces/1284d238-aa92-42ce-bdc4-0b0000009157\!492dc039-70c3-44e2-8a71-bdcf690a9dbe/f1/f2/test3.txt -d "12345678901" -u admin:admin -v

Should and must fail with insufficient storage.

I will push the new patch.

[C0rby]

Ah now I found my mistake. I was thinking of the attribute name not spaceName. Every node has a name but not every node has a spaceName. So this is fine. 👍

[refs]

DO NOT MERGE just yet. This reports the wrong real disk usage of the blobs, as filepath.Walk does not follow symbolic links.