Skip to content

7 implement tus protocol#12

Merged
Cadons merged 41 commits intomainfrom
7-implement-tus-protocol
Sep 28, 2024
Merged

7 implement tus protocol#12
Cadons merged 41 commits intomainfrom
7-implement-tus-protocol

Conversation

@Cadons
Copy link
Copy Markdown
Owner

@Cadons Cadons commented Sep 25, 2024

No description provided.

Cadons and others added 30 commits September 13, 2024 22:52
…ndling in chunk uploads

fix: Improve URL validation in HttpClient and enhance success/error logging in request execution

TUS protool implementation began
…d enhance upload logic with success callback
…ogress tracking, and modify progress return type
Updated TusClient and tests to use std::filesystem::path for
file paths instead of std::string. Modified TusClient.h and
TusClient.cpp to reflect the new constructor parameter type.
Adjusted generateTestFile in TusClientTest.cpp to improve
readability and correctness with std::filesystem::path.
Included necessary headers in relevant files.

WINDOWS Update
This commit adds the ICacheManager interface, which provides methods for storing, retrieving, and removing TUS files from the cache. The interface includes the following methods:
- `store`: Stores a TUS file in the cache.
- `retrieve`: Retrieves a TUS file from the cache.
- `remove`: Removes a TUS file from the cache.

These changes are necessary to support the implementation of the TUSFile class for managing file upload records in the cache.

Refactor the TusClient class to use std::filesystem::path for file paths

This commit refactors the TusClient class to use the `std::filesystem::path` type for file paths. This change improves the handling of file paths and ensures compatibility with the TUS protocol.

fix: Correct chunk upload logic and improve chunk file removal handling

This commit fixes issues with the chunk upload logic in the TusClient class. It corrects the calculation of the chunk upload offset and improves the handling of chunk file removal.

Update TusClient constructor to include appName parameter

This commit updates the TusClient constructor to include an additional `appName` parameter. This parameter is used to associate the uploaded file with a specific application.

Update TUSFile constructor to remove unnecessary parameters

This commit updates the TUSFile constructor to remove the `uploadedBytes` and `fileSize` parameters, as they are no longer needed. The constructor now only requires the `filePath`, `uploadUrl`, and `appName` parameters.

These changes are necessary to ensure proper initialization of the TUSFile object.
…ient

- Update conanfile.py to include nlohmann_json library as a dependency
- Update CMakeLists.txt to find and link nlohmann_json library
- Update libtusclient.h and TusClient.h to remove redundant TEMP_DIR definition
- Update TusClient.cpp to remove unnecessary comments
- Update TUSFile.cpp and TUSFile.h to add getIdentificationHash() and getResumeFrom() methods
- Add IRepository.h and CacheRepository.h for managing cache operations
- Add CacheRepositoryTest.cpp for testing cache repository operations
…headers for TUSClient

Refactor: Add ICacheManager interface for managing cache operations in CacheRepository.h
Refactor: Update TusClientTest.cpp to use "testapp" as the application name in the TusClient constructor
feat: Add TUSFile class for managing file upload records in cache
…headers for TUSClient

Reasoning:
- The headers TusClient.h and libtusclient.h were updated to include necessary headers for TUSClient.

Refactor: Update dependencies and include necessary headers for TUSClient

Reasoning:
- Dependencies were updated and necessary headers were included for TUSClient.

Refactor: Update TusClientTest.cpp to use "testapp" as the application name in the TusClient constructor

Reasoning:
- TusClientTest.cpp was modified to use "testapp" as the application name in the TusClient constructor.

Refactor: Add ICacheManager interface for managing cache operations

Reasoning:
- An ICacheManager interface was added for managing cache operations.

feat: Add ICacheManager interface for managing cache operations

Reasoning:
- An ICacheManager interface was added for managing cache operations.
This commit adds a new class called TUSChunk, which represents a chunk of a TUS file. A chunk is a part of a file that is uploaded to the server in a single request. The TUSChunk class has two member functions: getData() to retrieve the data of the chunk, and getChunkSize() to get the offset of the chunk in the file.

The addition of the TUSChunk class will enable more efficient handling of file uploads in the TUSClient library.
…lient

The isLastRequestCompleted() function in the IHttpClient interface is no longer used and has been removed. This function was previously used to check if the last request was completed, but it is no longer necessary for the current implementation.

This commit removes the isLastRequestCompleted() function from the IHttpClient interface to clean up the code and remove unused functionality.

Refactor: Remove unused isLastRequestCompleted() function from HttpClient

The isLastRequestCompleted() function in the HttpClient class is no longer used and has been removed. This function was previously used to check if the last request was completed, but it is no longer necessary for the current implementation.

This commit removes the isLastRequestCompleted() function from the HttpClient class to clean up the code and remove unused functionality.
The resume() method in TusClient.cpp has been updated to fix a bug where the m_status was not properly set to READY before calling getUploadInfo(). This caused issues with the upload process. The code has been refactored to ensure that m_status is set correctly before calling getUploadInfo(). This commit resolves the issue and improves the reliability of the resume() method.
This commit refactors the code to properly clean up CURL handles in the RequestTask and HttpClient classes. The RequestTask class now has a destructor that calls `curl_easy_cleanup` to release the CURL handle. Similarly, the HttpClient class now cleans up the CURL handle in the `execute` method before removing the request from the queue. This ensures that CURL handles are properly cleaned up and prevents potential resource leaks.
…nism in TusClient.cpp, RequestTask.cpp, and HttpClient.cpp

Remove debug feature responsible for producing verbose output in setupCURLRequest() function in HttpClient.cpp
Modify TusClientTest.cpp to resume() upload operation in a detached thread to avoid blocking the main thread
Cadons and others added 8 commits September 23, 2024 22:58
…ttpClient.cpp, and TusClientTest.cpp

TODO: fix resume
Add a constant TUS_PROTOCOL_VERSION in libtusclient.h to define the version of the TUS protocol being used. This constant will be used throughout the codebase to ensure compatibility with the TUS server.

Refactor: Add copy constructor for TUSFile

Add a copy constructor for the TUSFile class in TUSFile.h and TUSFile.cpp. This constructor allows creating a new TUSFile object by copying the properties of an existing TUSFile object. This will be useful for certain operations in the codebase.

Refactor: Update IRepository interface

Update the IRepository interface in IRepository.h to use shared pointers for the add and remove methods. This change allows using shared pointers for the TUSFile objects in the repository, ensuring proper memory management and avoiding potential memory leaks.

Refactor: Update CacheRepository implementation

Update the CacheRepository implementation in CacheRepository.h and CacheRepository.cpp to use shared pointers for the add and remove methods. This change aligns the implementation with the updated IRepository interface.

Refactor: Add getTusServerInformation method in TusClient.h

Add a new method getTusServerInformation in TusClient.h to retrieve the server information for the TUS server. This method returns a map containing the server's supported extensions, resumable version, and TUS version.

Test: Add unit tests for getTusServerInformation, cancelUpload, and retryUpload

Add unit tests in TusClientTest.cpp to test the newly added methods getTusServerInformation, cancelUpload, and retryUpload. These tests ensure the correct functionality of these methods and verify that they return the expected results.

Test: Add helper function generateSimpleFile

Add a helper function generateSimpleFile in TusClientTest.cpp to generate a simple test file for the unit tests. This function creates a text file with the content "Hello World" and returns its path.

Test: Update clientCreationTest in TusClientTest.cpp

Update the clientCreationTest in TusClientTest.cpp to use the generateSimpleFile helper function for creating the test file. This change ensures consistency and improves the readability of the test.

Test: Update uploadTest in TusClientTest.cpp

Update the uploadTest in TusClientTest.cpp to include a sleep call before canceling the upload. This change allows simulating a longer upload process before canceling it, ensuring proper testing of the cancelUpload method.

Test: Add unit tests for cancelUpload and retryUpload

Add unit tests in TusClientTest.cpp to test the newly added methods cancelUpload and retryUpload. These tests verify that the cancelUpload method cancels the upload process and that the retryUpload method resumes the upload from where it was canceled.

Test: Update cancelUpload and retryUpload tests

Update the cancelUpload and retryUpload tests in TusClientTest.cpp to include assertions for the status of the TusClient object after canceling and retrying the upload. These assertions ensure that the status is correctly updated after these operations.

Test: Remove unnecessary code in TusClientTest.cpp

Remove unnecessary code in TusClientTest.cpp, including the creation of a test file and the sleep calls. These changes improve the readability and maintainability of the test code.
Refactor the TusClient::upload() method to include a check for an empty file before creating the TusFile object. This ensures that the upload process is only initiated when there is a valid file to upload.

Refactor: Remove GTEST_SKIP() in TusClientTest

Remove the GTEST_SKIP() statement in the TusClientTest::retryUpload() test case. This statement was used for skipping the test temporarily and is no longer needed.

Refactor: Add random header generation in CacheRepositoryTest

Add a random header generation using the <random> library in the CacheRepositoryTest.cpp file. This will enhance the testing of the CacheRepository class by introducing randomization in the header generation process.
- Update TusClient::uploadChunks() to properly handle the status check using atomic load() function.
- Update TusClient::uploadChunk() to check if the status is not UPLOADING before uploading the chunk.
- Add check for empty file in TusClient::cancel() to prevent cancellation of non-existing upload.
- Update TusClient::status() to return the status using atomic load() function.
- Update TusClient::retry() to reset the necessary variables before retrying the upload.

Refactor: Remove unused code in HttpClient.cpp

- Remove unused destructor in HttpClient.cpp.
@Cadons Cadons self-assigned this Sep 25, 2024
@Cadons Cadons linked an issue Sep 25, 2024 that may be closed by this pull request
- Added ChunkUtility class in utility/ChunkUtility.h to provide utility methods for working with chunks.
- Implemented getChunkSizeFromGB(), getChunkSizeFromMB(), and getChunkSizeFromKB() methods in ChunkUtility class to calculate chunk sizes in bytes based on the given size in GB, MB, or KB.
- Added ChunkUtility.cpp file to define the implementation of the ChunkUtility class methods.

Related to: #issue-number
@Cadons Cadons merged commit f18ab18 into main Sep 28, 2024
@Cadons Cadons deleted the 7-implement-tus-protocol branch August 2, 2025 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Implement tus protocol

2 participants