How should I use timeouts/retries to avoid ETIMEDOUT

[rfennell]

Problem

I have been seeing a problem with an Aure DevOps task when it is making a large number of API calls i.e. when generating release notes where there are hundreds of associated items to get details for.

I am guessing it is a throttling issue.

I see the error, but not at a consistent point in the logic

Error: connect ETIMEDOUT 13.107.42.18:443

The strange thing is this only occurs for hosted agent, a private agent is fine.

My assumption is that this is down to the increased latency from an agent hosted on my local network as opposed to one in an Azure data centre.

Environment

Node version: 12.16.1
Npm version: 6.13
OS and version: Problem seen on MS Hosted Agent VS2017-win2016
azure-devops-node-api version: 10.1.0

What I have tried

I have tried adding retry/timeout options when I initialise my WebApi, but it does not appears to have had any effect.

A code snippet of the initialiser for the WebApi is

import * as vstsInterfaces from "azure-devops-node-api/interfaces/common/VsoBaseInterfaces";
// load creds handler from helper function
let credentialHandler: vstsInterfaces.IRequestHandler = util.getCredentialHandler();
const options = {
                allowRetries: true,
                maxRetries: 20,
                socketTimeout: 30000,
} as vstsInterfaces.IRequestOptions;
let vsts = new webApi.WebApi(tpcUri, credentialHandler, options);

Question

However, this does not seem to have helped.

1. Have I missed something else I need to do?
2. Or must I wrapper each API call in my own retry code?

[rfennell]

Is there any news on this issue?

I have been trying to do more testing

• If I run an Azure DevOps pipeline on a hosted agent that calls the Node SDK a few hundred times I get the timeout issue. You can see the failing test run here
• If I run exactly the same pipeline on a private agent (on my development PC) I do not, it works fine.

So, I have built a test harness to try to force a timeout locally. This test harness run the same basic logic my task, the one that is timing out. However, not matter how fast/repeatedly I call the API from my development PC I cannot force a timeout. The basic test harness is

async function run (): Promise<string> {
    var promise = new Promise<string>(async (resolve, reject) => {

        let token: string = "<A PAT>";
        var teamProject = "GitHub"
        let authHandler = webApi.getPersonalAccessTokenHandler(token); 
        let vsts = new webApi.WebApi("https://dev.azure.com/richardfennell", authHandler);
        var buildApi: IBuildApi = await vsts.getBuildApi();

        let builds = await buildApi.getBuilds( teamProject , [53]);
        console.log(`Found ${builds.length} builds`);
        for (var build of builds) {
            console.log(`Getting the details of ${build.id}`);
            var buildCommits = await buildApi.getBuildChanges(teamProject, build.id);
            var buildWorkitems = await buildApi.getBuildWorkItemsRefs(teamProject, build.id);
        }

        resolve (`Called API {1+(2*builds.length)} times`);
   });
   return promise;
}

I guess the question is what difference between a private agent and a hosted one that could cause this issue. It is not as if the hosted agents are that fast, but they do have very hast connection to the internet

[rfennell]

I have now also run my test harness node app in an Azure hosted VM - this did not work show the error, so it seems specific to the hosted build agent

[rfennell]

Any progress on this?

[github-actions]

This issue has had no activity in 90 days. Please comment if it is not actually stale

[rfennell]

I am still seeing this issue