proxyUrl option doesn't work behind a corporate proxy

[rasulsafa]

Core Library

MSAL Node (@azure/msal-node)

Core Library Version

1.18.3

Wrapper Library

Not Applicable

Wrapper Library Version

N/A

Public or Confidential Client?

Confidential

Description

For the past year or so, I have been using msal-node behind a corporate proxy with the workaround posted by another user here (overriding the networkClient with proxy-compatible versions of sendGetRequestAsync and sendPostRequestAsync). This has been working well, and now that msal-node natively supports proxies I wanted to try it out. Unfortunately, passing the proxyUrl parameter and disabling the custom networkClient yields an Endpoints Resolution Error

ClientAuthError: endpoints_resolution_error: Error: could not resolve endpoints. Please check network and try again. Detail: ClientAuthError: openid_config_error: Could not retrieve endpoints. Check your authority and verify the .well-known/openid-configuration endpoint returns the required endpoints. Attempted to retrieve endpoints from: https://login.microsoftonline.com/my tenant/v2.0/.well-known/openid-configuration

I investigated msal-nodes implementation of proxy compatibility to see if I could figure out the issue. Connection to the corporate proxy seemed to succeed. Log statements I put in the networkRequestViaProxy function showed that connection to the proxy was succeeding with 200 Connection established However, accessing microsoft's login servers seemed to be yielding a 400 Bad Request.

{
  headers: {
    'Content-Type': 'text/html',
    'Cache-Control': 'no-cache',
    'Content-Length': '2488',
    'X-Frame-Options': 'deny',
    'Proxy-Connection': 'Close'
  },
  body: {
    error: 'client_error',
    error_description: 'A client error occured.\n' +
      'Http status code: 400\n' +
      'Http status message: badrequest\n' +
      'Headers: {"Content-Type":"text/html","Cache-Control":"no-cache","Content-Length":"2488","X-Frame-Options":"deny","Proxy-Connection":"Close"}'
  },
  status: 400
}

And these are the contents of the payload variable in that function:

GET https://login.microsoftonline.com/my tenant/v2.0/.well-known/openid-configuration HTTP/1.1
Host: login.microsoftonline.com
Connection: close

The existing proxy implementation using http.request and manually writing http requests through the socket seems overly complex and dated - it would be great if it were rewritten to use fetch and https-proxy-agent. They're far newer and cleaner, and it's how the MS Graph Client implements proxy support. I've never had a single issue with the Graph Clients proxy support.

Error Message

ClientAuthError: endpoints_resolution_error: Error: could not resolve endpoints. Please check network and try again. Detail: ClientAuthError: openid_config_error: Could not retrieve endpoints. Check your authority and verify the .well-known/openid-configuration endpoint returns the required endpoints. Attempted to retrieve endpoints from: https://login.microsoftonline.com/my tenant/v2.0/.well-known/openid-configuration

Msal Logs

No response

MSAL Configuration

const clientConfig = {
  auth: {
    clientId: vault.azure_ad_client_id,
    authority: 'https://login.microsoftonline.com/my tenant/',
    clientCertificate: {
      thumbprint: vault.azure_ad_cert_thumbprint,
      privateKey: Buffer.from(vault.azure_ad_decrypted_private_key_base64, 'base64').toString(),
    },
  },
  system: {
     //networkClient: proxyClient,
     proxyUrl: 'http://host:port'
  },
};

Relevant Code Snippets

//This is the implementation of the networkClient that works through the proxy without any issues

import fetch from 'node-fetch';
import { HttpsProxyAgent } from 'https-proxy-agent';

export const proxyUrl = 'http://host:port';
export const proxyAgent = new HttpsProxyAgent(proxyUrl);

export const proxyClient = {
  sendGetRequestAsync: async (url, options) => {
    const response = await fetch(url, { agent: proxyAgent, ...options });
    const json = await response.json();
    const headers = response.headers.raw();
    const obj = {
      headers: Object.create(Object.prototype, headers),
      body: json,
      status: response.status,
    };
    return obj;
  },
  sendPostRequestAsync: async (url, options) => {
    const sendingOptions = options || {};
    sendingOptions.method = 'post';
    const response = await fetch(url, { agent: proxyAgent, ...sendingOptions });
    const json = await response.json();
    const headers = response.headers.raw();
    const obj = {
      headers: Object.create(Object.prototype, headers),
      body: json,
      status: response.status,
    };
    return obj;
  },
};

Reproduction Steps

1. Setup a proxy
2. Attempt to use msal-node with proxyUrl

Expected Behavior

The expected behavior is the 200 Ok response and msal-node being able to fetch the token

Identity Provider

Azure AD / MSA

Browsers Affected (Select all that apply)

None (Server), Other

Regression

No response

Source

External (Customer)

[sameerag]

cc @Robbie-Microsoft @bgavrilMS Can you folks check this?

[bgavrilMS]

Hi @rasulsafa - if you just use https://github.com/AzureAD/microsoft-authentication-library-for-js/blob/dev/samples/msal-node-samples/custom-INetworkModule-and-network-tracing/HttpClientAxios.ts object (no MSAL), can you use it to make a call to https://login.microsoftonline.com/b458eb0a-178f-4197-8da4-514c0fe6f17b/v2.0/.well-known/openid-configuration ?

This extensibility point gives you full control over the HTTP stack, so it's up to you to set it in such a way as to take into account the proxy.

[bgavrilMS]

Similar issue #6483

[rasulsafa]

Hi @bgavrilMS, I was not able to get it working with an Axios-based custom networkClient. It appears proxy support on Axios is also currently bugged as well. I was however able to make it work with node-fetch and https-proxy-agent as outlined in my original message

[bgavrilMS]

Thanks for the summary @rasulsafa. Much appreciated that you provided the implementation that works for you. Can you link the Graph SDK API on this please?

I will mark this as a bug then, to update the implementation to use to use fetch and https-proxy-agent as what we have is clearly not working.

[Robbie-Microsoft]

Hi @bgavrilMS, I was not able to get it working with an Axios-based custom networkClient. It appears proxy support on Axios is also currently bugged as well. I was however able to make it work with node-fetch and https-proxy-agent as outlined in my original message

What version of axios did you use when you tried this sample? It appears our samples use either 1.4.0 or 1.3.4. Other users have been able to get the axios implementation working before.

For insight, we switched to manually writing http requests through the socket because we didn't want to rely on 3rd party libraries. If it supports proxies, we would like to switch to node's native fetch as soon that moves out of the experimental phase.

[rasulsafa]

@Robbie-Microsoft I had initially tried it with the latest version, 1.5.1. I just retested it with 1.4.0 and still get the endpoints resolution error. I understand not wanting to rely on third party libraries, but msal-node and the MS Graph Client having such disparate proxy support is pretty frustrating. The graph client as it is currently uses a fetch polyfill and is compatible with https-proxy-agent.

If there's unlikely to be any major change until Node's native fetch supports proxies, then could I suggest a happy medium in the meantime? Currently, msal-node doesn't support https-proxy-agent passed into customAgentOptions because in networkRequestViaHttps, the contents of customAgentOptions are presumed to be the options parameter of the native http(s).Agent class and passed into it's constructor, rather than being treated as an Agent object itself. https.request as you are using it, when used with https-proxy-agent, has complete proxy support. If I simply change customOptions.agent = new https.Agent(agentOptions); to customOptions.agent = agentOptions in HttpClient, and then in my config pass an https-proxy-agent object as such:

system: {
    customAgentOptions: new HttpsProxyAgent('http://proxy.XXXX.com:8080')
}

I'm able to successfully connect through the proxy. To preserve backwards compatibility, something like this can be done:

if(agentOptions.constructor && (agentOptions.constructor.prototype instanceof https.Agent || agentOptions.constructor.prorotype instanceof http.Agent))
    customOptions.agent = agentOptions;
else 
    customOptions.agent = https.Agent(agentOptions);

This way, we check if the end user passed in their own custom agent object, and if they did, we use that. If they just passed in a dictionary of options, we create the Agent object for them and use that. We check if it's either an http or https agent, because both are valid in https.request (https-proxy-agent for some reason is a child of http.Agent.) This ensures that anyone can use their own proxy agent, backwards compatibility is preserved, and msal-node would have relatively easy proxy support with no extra third parties. The burden of using a third party proxy agent library falls on us end users. Simply doing npm i https-proxy-agent and passing the object into customAgentOptions is much easier than the status quo.

[bgavrilMS]

Agreed, let's analyze the stance of Azure SDK and Microsoft Graph SDK and consolidate.

[danielspatacean94]

Hello,

Any news regarding this issue?
We're facing it too.

Behind a corporate proxy, there's no way to connect, currently.

[Tcharl]

Also, the current implementation does not supports authenticated pxy

[Robbie-Microsoft]

@bgavrilMS and I are still discussing this, but have other priorities right now. In the meantime, you all can implement your own custom network modules. See this sample.

Note: Node's native fetch api is now stable in node v21
Time permitting, I would like to rewrite the msal-node HttpClient to use this when Node v22 is released in April. I'm still waiting for the Node folks to add docs for fetch so I can learn about how proxies are implemented in the api.

Additionally, we are not actively supporting msal-node v1 anymore unless there is a critical bug that needs to be addressed.

[bgavrilMS]

We will revisit when we move to use fetch with a newer version of nodejs / a major version bump of this lib. Keeping this as feature request. It's too difficult to test all these scenarios otherwise.

Solution for folks wanting to use a proxy is to provide their own INetworkModule.

[AlexVFornazieri]

Working INetworkModule implementation with Proxy

TROUBLESHOOTING

• Duble check if any other lib or own fuction is requesting direct to login.microsoftonline.com, then change it to use same aproach in code bellow with "https-proxy-agent" and "node-fetch";
• Getting NS LOOKUP or ENOTFOUND errors? Duble check if you correctly import the 'node-fetch', the native fetch will not work with https-proxy-agent;

import type { INetworkModule, NetworkRequestOptions, NetworkResponse } from "@azure/msal-node";
import { HttpsProxyAgent } from "https-proxy-agent";
import fetch from 'node-fetch';

const proxyUrl = process.env['HTTPS_PROXY'] || process.env['HTTP_PROXY'];
if (!proxyUrl) {
    throw new Error('Missing HTTP/S_PROXY env');
}

export const proxyAgent = new HttpsProxyAgent(proxyUrl);

export class CustomHttpClient implements INetworkModule {
    sendGetRequestAsync<T>(url: string, options?: NetworkRequestOptions): Promise<NetworkResponse<T>> {
        return this.sendRequestAsync(url, 'GET', options);
    }
    sendPostRequestAsync<T>(url: string, options?: NetworkRequestOptions): Promise<NetworkResponse<T>> {
        return this.sendRequestAsync(url, 'POST', options);
    }

    private async sendRequestAsync<T>(
        url: string,
        method: 'GET' | 'POST',
        options: NetworkRequestOptions = {},
    ): Promise<NetworkResponse<T>> {
        try {
            const requestOptions = {
                method: method,
                headers: options.headers,
                body: method === 'POST' ? options.body : undefined,
                agent: proxyAgent,
            };

            console.log('>>> url', url, requestOptions);

            const response = await fetch(url, requestOptions);
            const data = await response.json() as any;

            const headersObj: Record<string, string> = {};
            response.headers.forEach((value, key) => {
                headersObj[key] = value;
            });

            return {
                headers: headersObj,
                body: data,
                status: response.status,
            };
        } catch (err) {
            console.error('CustomRequest', err);
            throw new Error('Custom request error');
        }
    }
}

[rasulsafa]

Duble check if you correctly import the 'node-fetch', the native fetch will not work with https-proxy-agent;

On a related note, if you're using Node 18+, native fetch breaks node-fetch and necessitates the usage of --no-experimental-fetch