WithProxy is ignored

[chrisbecke]

Issue

We upgraded our own project to the net6.0 framework, and that included upgrading the minio-dotnet SDK to version 4.0.4 As a consequence of this update the Minio SDK now seems to be ignoring .WithProxy().

To get minio-dotnet to route traffic via a proxy it now seems necessary to use WithHttpClient.

To Reproduce

• Create a new .net6 console application using Visual Studio.
• Enable Docker Compose Support, and add a minio entry as per the docker-compose below.
• Create a reverse proxy using express.js, and add that to the compose.
• Run the project and see that .WithProxy is never used, but .WithHttpClient() does route through the proxy.

docker-compose.yml

version: '3.4'

services:
  copytest:
    image: ${DOCKER_REGISTRY-}copytest
    build:
      context: .
      dockerfile: CopyTest/Dockerfile
    depends_on:
    - proxy

  proxy:
    image: ${DOCKER_REGISTRY-}proxy
    build:
      context: proxy
      target: dev
    volumes:
    - ./proxy:/src
    - /src/node_modules
    depends_on:
    - minio

  minio:
    image: minio/minio:latest
    command: server /data --console-address :9001
    volumes:
    - /data
    ports:
    - 8080:9001

  mc:
    image: minio/mc:latest
    profiles: [ "tools" ]
    working_dir: /work
    volumes:
    - ./.mc:/root/.mc
    - ./:/work

Program.cs

using Minio;
using Minio.DataModel;
using System.Net;

var minio = new MinioClient()
                        .WithCredentials("minioadmin","minioadmin")
                        .WithEndpoint("minio:9000")
//                        .WithHttpClient( new HttpClient(new HttpClientHandler(){ Proxy = new WebProxy("proxy", 3000) }))
                        .WithProxy(new WebProxy("proxy", 3000))
                        .Build();

string fromBucketName = "uploads";
string fromObjectName = "test.txt";
string destBucketName = "staging";
string destObjectName = "test.txt";
ServerSideEncryption sseSrc = null;
ServerSideEncryption sseDest = null;

var cpSrcArgs = new CopySourceObjectArgs()
    .WithBucket(fromBucketName)
    .WithObject(fromObjectName)
    .WithServerSideEncryption(sseSrc);
var cpArgs = new CopyObjectArgs()
    .WithBucket(destBucketName)
    .WithObject(destObjectName)
    .WithCopyObjectSource(cpSrcArgs)
    .WithServerSideEncryption(sseDest);

await minio.CopyObjectAsync(cpArgs);

proxy/index.js

var express  = require('express');
var app      = express();
var httpProxy = require('http-proxy');
var apiProxy = httpProxy.createProxyServer();
var server = 'http://minio:9000';
 
app.all("/*", function(req, res) {
    console.log('redirecting to minio');
    apiProxy.web(req, res, {target: server});
});

app.listen(3000);

console.log('proxy listening');

proxy/Dockerfile

FROM node:14-alpine as base

WORKDIR /src
COPY package*.json /
EXPOSE 3000

FROM base as dev
ENV NODE_ENV=development
RUN npm install -g nodemon && npm install
COPY . /
CMD ["nodemon", "bin/www"]

In summary these were the steps I used to create the projects and run the test scenarios.

dotnet new CopyTest
dotnet add CopyTest package minio-dotnet
# Edit the generated Program.cs and paste in the above code now
mkdir proxy
cd proxy
npm init
npm install express
npm install http-proxy
# Add the index.js and Dockerfile now
cd ..
docker compose build proxy
docker compose up proxy
# Switch to a second terminal
docker compose run --rm mc alias set minio http://minio9000 minioadmin minioadmin
docker compose run --rm mc mb minio/uploads
docker compose run --rm mc mb minio/staging
docker compose run --rm mc cp docker-compose.yml minio/uploads/test.txt
docker compose build copytest
docker compose run copytest
# edit Program.cs to switch from WithProxy to WithHttpclient
docker compose build copytest
docker compose run copytest

[ebozduman]

@chrisbecke ,

This issue must have been fixed by PR #641 (Reuse HttpClient and fix the wrong TLS and Proxy settings during MinioClient creation).
Please let us know if you still see the issue, or close it if fixed.

[alexwlsnr]

I'm pretty sure this PR doesn't fix the issue @chrisbecke reported. I also experienced this following the documentation. For example:

 var client = new MinioClient()
    .WithEndpoint(this.Settings.S3Endpoint)
    .WithCredentials(
        this.Settings.S3AccessKey,
        this.Settings.S3SecretKey)
    .WithProxy(new WebProxy(this.Settings.HttpProxyUri))
    .WithSSL();

Doesn't apply the proxy settings. Directly supplying a HttpClient with a proxy set does appear to work:

var httpClientHandler = new HttpClientHandler()
{
    Proxy = new WebProxy(this.Settings.HttpProxyUri)
};

var client = new MinioClient()
    .WithEndpoint(this.Settings.S3Endpoint)
    .WithCredentials(
        this.Settings.S3AccessKey,
        this.Settings.S3SecretKey)
    .WithProxy(new WebProxy(this.Settings.HttpProxyUri))
    .WithHttpClient(new HttpClient(httpClientHandler));
    .WithSSL();

Which is a valid workaround, but the docs suggest setting the proxy alone should be effective.

[ebozduman]

Thank you @alexwlsnr
I'll look into it.

[ebozduman]

@chrisbecke ,
Could you provide the name of the docker image to reproduce the issue and test the fix I have?
or @alexwlsnr
Could you provide the information about your proxy server, what kind it is, its config information etc to reproduce the issue?
The issue cannot reproduced with a regular nginx reverse proxy as it takes care of traffic in both directions.

[harshavardhana]

Fixed by #683