Skip to content
This repository has been archived by the owner on May 14, 2024. It is now read-only.

Web app can delete file in S3 #29

Closed
robert-clegg-tessella opened this issue Nov 6, 2018 · 2 comments
Closed

Web app can delete file in S3 #29

robert-clegg-tessella opened this issue Nov 6, 2018 · 2 comments
Assignees
Milestone

Comments

@robert-clegg-tessella
Copy link
Collaborator

robert-clegg-tessella commented Nov 6, 2018

Story points: 2

A DELETE request to the Web API (route "deletefile") deletes a single file at the S3 end

Acceptance criteria:
GET request to: http://172.28.129.160:8888/deletefile/{bucket-name}/{relative-path-to-file} (e.g. http://172.28.129.160:8888/deletefile/test-bucket/testdir/testfile.txt)

Should return a warning message if the bucket or the file does not exist at the S3 end

Otherwise, deletes the file and returns a message confirming the deletion

@robert-clegg-tessella robert-clegg-tessella added this to the Sprint 2 milestone Nov 7, 2018
oliver-tarrant-tessella added a commit that referenced this issue Nov 12, 2018
robert-clegg-tessella added a commit that referenced this issue Nov 13, 2018
…-a-file-#29

Testing/add unit tests for deleting a file #29
@daniel-hunt-tessella
Copy link
Contributor

daniel-hunt-tessella commented Nov 30, 2018

Up-to-date Acceptance Criteria
Authorised DELETE request to: http://172.28.129.160:8888/deletefile/
With body of form {"bucket": "<bucket-name>", "file": "<file-path-and-name>"}.
Eg: {"bucket": "ditto-anything-i-want", "file": "/2test2/33testing/444testing/555testing/test.txt"}

Should return a warning message if the bucket or the file does not exist at the S3 end

Otherwise, deletes the file and returns a message confirming the deletion


Testing
Note that as part of #71 status and status code errors will be fixed. ✔️
Works for the happy path and if the bucket or file does not exist at the S3 end ✔️

❌ Problem caused when a file path is given that goes outside the bucket - eg {"bucket": "ditto-anything-i-want", "file": ".."}. Also applies when "file": "" is given
This causes a 500 Internal Server Error
This actually happens for all API calls that require a file or directory

2018-11-30 09:47:57,514 ditto-web-api DEBUG    Called delete-file handler
DEBUG:ditto-web-api:Called delete-file handler
2018-11-30 09:47:57,514 ditto-web-api DEBUG    About to check for warning to do with bucket name ditto-anything-i-want
DEBUG:ditto-web-api:About to check for warning to do with bucket name ditto-anything-i-want
2018-11-30 09:47:57,517 ditto-web-api DEBUG    Bucket "ditto-anything-i-want" does exist
DEBUG:ditto-web-api:Bucket "ditto-anything-i-want" does exist
2018-11-30 09:47:57,517 ditto-web-api DEBUG    No bucket related warnings found
DEBUG:ditto-web-api:No bucket related warnings found
ERROR:tornado.application:Uncaught exception DELETE /deletefile/ (172.28.129.1)
HTTPServerRequest(protocol='http', host='172.28.129.160:8888', method='DELETE', uri='/deletefile/', version='HTTP/1.1', remote_ip='172.28.129.1')
Traceback (most recent call last):
  File "/home/vagrant/ditto_web_api/venv/lib64/python3.6/site-packages/tornado/web.py", line 1592, in _execute
    result = yield result
  File "/home/vagrant/ditto_web_api/venv/lib64/python3.6/site-packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/home/vagrant/ditto_web_api/venv/lib64/python3.6/site-packages/tornado/gen.py", line 326, in wrapper
    yielded = next(result)
  File "/home/vagrant/ditto_web_api/venv/lib64/python3.6/site-packages/tornado_json/schema.py", line 160, in_wrapper
    output = rh_method(self, *args, **kwargs)
  File "/home/vagrant/ditto_web_api/DittoWebApi/src/handlers/delete_file.py", line 25, in delete
    result = self._data_replication_service.try_delete_file(bucket_name, file_rel_path)
  File "/home/vagrant/ditto_web_api/DittoWebApi/src/services/data_replication/data_replication_service.py", line 92, in try_delete_file
    if not self._external_data_service.does_object_exist(bucket_name, file_rel_path):
  File "/home/vagrant/ditto_web_api/DittoWebApi/src/services/external/external_data_service.py", line 79, indoes_object_exist
    key = bucket.get_key(file_name)
  File "/home/vagrant/ditto_web_api/DittoWebApi/src/services/external/storage_adapters/boto_bucket.py", line13, in get_key
    key = self._bucket.get_key(*args, **kwargs)
  File "/home/vagrant/ditto_web_api/venv/lib64/python3.6/site-packages/boto/s3/bucket.py", line 193, in get_key
    key, resp = self._get_key_internal(key_name, headers, query_args_l)
  File "/home/vagrant/ditto_web_api/venv/lib64/python3.6/site-packages/boto/s3/bucket.py", line 232, in _get_key_internal
    response.status, response.reason, '')
boto.exception.S3ResponseError: S3ResponseError: 400 Bad Request

ERROR:tornado.access:500 DELETE /deletefile/ (172.28.129.1) 61.91ms

oliver-tarrant-tessella added a commit that referenced this issue Dec 7, 2018
…ile-or-directory-outside-of-root/#29

Bugfix/error when trying to acess file or directory outside of root/#29
oliver-tarrant-tessella referenced this issue Dec 12, 2018
…-root-#29

Identical dir paths without final backslash are accepted as equal
@daniel-hunt-tessella
Copy link
Contributor

Problem caused by attempting to delete a file outside of the root directory has been resolved. This now returns a 403 Forbidden error instead of the previous 500 error

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants