Files are saved as folders containing xl.meta files

[TheSharpOwl]

Describe the bug

When uploading a file (image png/jpg or json file) using a string stream, let's say sample.png I get a folder called sample.png and inside of it there is xl.meta file.

I tried to save the image locally and it was working totally okay so the encoder code is working. Also, I remember one year ago it was saving on minio container fine but now I cameback to the same exact code one year ago but still the issue arises.

Conditions:

• Docker image for minio to connect to

Where it was tried:

• Ubuntu 18
• Windows 10

How it can be seen

• Using a terminal and trying to cd the image name
• Using volumes mounting

Expected Behavior

Get an image/json file not a folder

Current Behavior

I get a folder called sample.png and inside of it there is xl.meta file.

Reproduction Steps

// connect to a bucket using the examples then using this method:
bool saveData(const char *rawData,
                           size_t dataSize,
                           const std::string &filename) {
  Aws::S3::Model::PutObjectRequest putObjectRequest;
  putObjectRequest.WithBucket(bucketName_.c_str()).WithKey(filename.c_str());

  auto data = Aws::MakeShared<Aws::StringStream>(
      "PutObjectInputStream",
      std::stringstream::in | std::stringstream::out
          | std::stringstream::binary);

  data->write(rawData, dataSize);

  putObjectRequest.SetBody(data);
   // these lines were tried too and they didn't fix it
  // putObjectRequest.SetContentType("image/jpeg");
  // putObjectRequest.SetContentLength(dataSize);
  auto result = client_->PutObject(putObjectRequest);
  if (result.IsSuccess()) {
    return true;
  }
  return false;
}

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

1.11.65 (vcpkg)

Compiler and Version used

Visual Studio 2017 15.9.48 Community

Operating System and version

Windows 10 19044.3086

[yasminetalby]

Hello @TheSharpOwl ,

Thank you very much for your submission. I have attempted to reproduce your use case and was unsuccessful to observe the behavior when attempting to upload a jpg file.

Here is my sample :

Aws::Client::ClientConfiguration clientConfig;
        clientConfig.endpointOverride = Aws::String("S3-API endpoint");
        clientConfig.scheme = Aws::Http::Scheme::HTTP;
        Aws::S3::S3Client client( Aws::Auth::AWSCredentials("minioadmin", "miniokey"), clientConfig,
                                 Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never, false);

        Aws::S3::Model::PutObjectRequest putObjectRequest;
        putObjectRequest.SetContentType("image/jpeg");
        Aws::String bucketName = "myBucketName";
        Aws::String fileName = "myFile.jpg";
        putObjectRequest.SetBucket(bucketName.c_str());
        putObjectRequest.WithKey(fileName.c_str());

        auto inputData = Aws::MakeShared<Aws::StringStream>(
                "PutObjectInputStream",
                fileName.c_str(),
                std::stringstream::in
                | std::stringstream::binary);

        putObjectRequest.SetBody(inputData);
        auto result = client.PutObject(putObjectRequest);
        if (result.IsSuccess())
        {
            std::cout << "PutObject completed" << std::endl;
        }
        else
        {
            std::cout << "PutObject error: " <<
                      result.GetError().GetExceptionName() << " " <<
                      result.GetError().GetMessage() << std::endl;

        }
 }

    Aws::ShutdownAPI(options);

    return 0;
}

My sample does cover how to upload a jpg file, however your usecase/request might be different. If that is the case please let me know what you would like to achieve in detail so I can either investigate further or provide guidance depending on your use case.

Thank you very much for your time and feedback.

Sincerely,

Yasmine

[TheSharpOwl]

Thank you @yasminetalby for the reply
I have written a complete example in this repo:

https://github.com/TheSharpOwl/aws_test

It produces the same problem for me and with a json file (just to make it easier and remove encoding/decoding part)

In the end of my code I didn't call Aws::ShutdownAPI(options) because seems when it goes out of the main scope it already does that so calling it manually causes a crash.

[yasminetalby]

Hello @TheSharpOwl ,

Thank you very much for providing a complete example. I have run this and the behavior on my end is that the upload is successful and that the myFile.json is of Content-Type text/json.

Could you share the logs associated with your request (make sure to remove all sensitive information if any is present). You can retrieve the logs following the developer guide Logging section. On my end, the put request has content-type: text/json which matches the content type provided through putObjectRequest.SetContentType("text/json");

As a side note, it is best practice to follow the proper Initialization and ShutDown of the SDK as outlined in the Basic Use section of the developer guide. Applications that use the AWS SDK for C++ must initialize it. Similarly, before the application terminates, the SDK must be shut down.

Thank you very much for your time and collaboration.
Sincerely,

Yasmine

[TheSharpOwl]

@yasminetalby Please find my log file here: (I enabled Trace mode and removed my directories information)

aws_sdk_2023-07-04-16.log

[yasminetalby]

Hello @TheSharpOwl ,

I have investigate this a little more. The SDK is behaving as expected. If you look at the MinIo console you will see your file with the set content-type. It seems that MinIo store the content of an uploaded file in a file called xl.meta in a directory with the name of the file uploaded. The SDK is working as designed. It seems that the change you are experiencing are coming from a change of behavior from MinIo (example see discussion: minio/minio#17017).

Best regards,

Yasmine

[yasminetalby]

I would recommend opening a discussion within the mini repository if you would like to discuss this with the minio team : https://github.com/minio/minio/discussions

Best regards,

Yasmine