Download jobs from version 3.5.0.0 does not work with F&O 10.0.29

[AeyPKA]

Describe the bug
Testing RIS version 3.5.0.0 with F&O 10.0.29 (10.0.1326.46), download jobs can't download file.

To Reproduce
With Download jobs, debug at the Download.cs > ProcessDownloadQueue

Expected behavior
Should be able to download file as previous platform

Screenshots

Download location: https://(D365FO)/api/connector/dequeue/4a052843-4250-4157-a372-6d936c253582}

response {StatusCode: 500, ReasonPhrase: 'Internal Server Error', Version: 1.1, Content: System.Net.Http.StreamContent, Headers:
{
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Server-Timing: total;dur=580.0954
ms-dyn-aid: 2b2cc6c0-b827-0006-a762-652b27b8d801
Content-Security-Policy: frame-ancestors 'self' https://org56cdb880.crm5.dynamics.com/
Request-Context: appId=cid-v1:47532108-1d6f-4e03-9925-5a529abc43fe
Date: Mon, 05 Sep 2022 09:40:33 GMT
Content-Length: 3014
Content-Type: application/json; charset=utf-8
}} System.Net.Http.HttpResponseMessage

Recurring Integration Scheduler version (please complete the following information):

• Version 3.5.0.0

Dynamics 365 version (please complete the following information):

• Application version : 10.0.29 (10.0.1326.46)
• Platform version : Update53 (7.0.6545.43)

Additional context

If orginal code of "HttpClientHelper.GetRequestAsync", it will keep hanging at the line "return await _httpClient.GetAsync(uri);".

    /// <summary>
    /// Http Get request
    /// </summary>
    /// <param name="uri">Request Uri</param>
    /// <param name="addAuthorization">Add authorization header</param>
    /// <returns>Http response</returns>
    public async Task<HttpResponseMessage> GetRequestAsync(Uri uri, bool addAuthorization = true)
    {
        _httpClient.DefaultRequestHeaders.Clear();
        if(addAuthorization)
        {
            _httpClient.DefaultRequestHeaders.Add("Authorization", await _authenticationHelper.GetValidAuthenticationHeader());
        }
        if (_settings.LogVerbose || Log.IsDebugEnabled)
        {
            Log.Debug($@"Job: {_settings.JobKey}. HttpClientHelper.GetRequestAsync is being called.
                     Uri: {uri.AbsoluteUri},
                    Parameters:
                    addAuthorization: {addAuthorization}");
        }
        return await _httpClient.GetAsync(uri);
    }

I change a bit of code to use "HttpResponseMessage responseMessage;", then I can see error.

    public async Task<HttpResponseMessage> GetRequestAsync(Uri uri, bool addAuthorization = true)
    {
        HttpResponseMessage responseMessage;

        _httpClient.DefaultRequestHeaders.Clear();
        if(addAuthorization)
        {
            _httpClient.DefaultRequestHeaders.Add("Authorization", await _authenticationHelper.GetValidAuthenticationHeader());
        }
        if (_settings.LogVerbose || Log.IsDebugEnabled)
        {
            Log.Debug($@"Job: {_settings.JobKey}. HttpClientHelper.GetRequestAsync is being called.
                Uri: {uri.AbsoluteUri},
                Parameters:
                addAuthorization: {addAuthorization}");
        }
        responseMessage = await _httpClient.GetAsync(uri).ConfigureAwait(false);
        return responseMessage;
    }

Not sure if 10.0.29 changes download structure or not.

[AeyPKA]

I tried to recreate a new recurring data job. It's ok when dequeue: https://(D365FO)/api/connector/dequeue/4a052843-4250-4157-a372-6d936c253582}

But once, in download,

https://(D365FO)/api/connector/download/6b21c27b-2847-48f1-95b3-76739caf131f?correlation-id=d95888b8-09c2-4807-8732-63aeba6bc8ee&blob=98d6253c-1ada-4744-8fda-178156081ab5,

At the commad "await _httpClient.GetAsync(uri).ConfigureAwait(false);", it keep hanging in "Exectuting" status

[Jerome2506]

We experience this also after upgrading to 10.0.29. We have created a MS support request: 2209060050001958.

[AeyPKA]

We experience this also after upgrading to 10.0.29. We have created a MS support request: 2209060050001958.

Mirosoft informed this is a known issue about Dequeue API. Planed to be fixed in later PU53.

Details for issue 676100 (dynamics.com)

To work around, must request MS support to clean Azure Blob queue.

[AeyPKA]

After clearing blob queue.

When calling Dequeue API. https://(D365FO)/api/connector/dequeue/4a052843-4250-4157-a372-6d936c253582. it’s faster response, less than 1 minute.

But when having data to download and call Download API, no return response. https://(D365FO)/api/connector/download/4a052843-4250-4157-a372-6d936c253582?correlation-id=adc1f450-9dca-415c-bd6b-5cbfa33285b5&blob=ad5e5dd6-e9a4-4e5b-96f0-e6172c99afc9

I set internal to every 10 mins. But the Download API doesn’t return response yet until next round of callling Dequeue API.

Then, I think no choice. I can’t set longer interval than 10 minutes, customer can’t wait that long. So, must hope BUG 676100 will be released in GA version.

[keithchanhaha]

I experience the same error after upgrade to 10.0.29. Is there any workaround?

Quartz.Core.JobRunShell - Job Download jobs.EDI_Export threw a JobExecutionException:
Parameters: refire = False, unscheduleFiringTrigger = False, unscheduleAllTriggers = False
Quartz.JobExecutionException: Download job: Download jobs.EDI_Export failed ---> System.Exception: Job: Download jobs.EDI_Export. Download failure. InternalServerError
at RecurringIntegrationsScheduler.Job.Download.d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at RecurringIntegrationsScheduler.Job.Download.d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at RecurringIntegrationsScheduler.Job.Download.d__12.MoveNext()
--- End of inner exception stack trace ---
at RecurringIntegrationsScheduler.Job.Download.d__12.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Quartz.Core.JobRunShell.d__9.MoveNext() [See nested exception: System.Exception: Job: Download jobs.EDI_Export. Download failure. InternalServerError
at RecurringIntegrationsScheduler.Job.Download.d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at RecurringIntegrationsScheduler.Job.Download.d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at RecurringIntegrationsScheduler.Job.Download.d__12.MoveNext()]

[Jerome2506]

Keith, we advised the following to our customers:

o For 10.0.29, platform version 7.0.6545.58 or higher should be used. This means that the GA version should not be used.
o For 10.0.30, platform version 7.0.6592.33 or higher (GA version) should be used.

I experience the same error after upgrade to 10.0.29. Is there any workaround?

Quartz.Core.JobRunShell - Job Download jobs.EDI_Export threw a JobExecutionException: Parameters: refire = False, unscheduleFiringTrigger = False, unscheduleAllTriggers = False Quartz.JobExecutionException: Download job: Download jobs.EDI_Export failed ---> System.Exception: Job: Download jobs.EDI_Export. Download failure. InternalServerError at RecurringIntegrationsScheduler.Job.Download.d__14.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at RecurringIntegrationsScheduler.Job.Download.d__13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at RecurringIntegrationsScheduler.Job.Download.d__12.MoveNext() --- End of inner exception stack trace --- at RecurringIntegrationsScheduler.Job.Download.d__12.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Quartz.Core.JobRunShell.d__9.MoveNext() [See nested exception: System.Exception: Job: Download jobs.EDI_Export. Download failure. InternalServerError at RecurringIntegrationsScheduler.Job.Download.d__14.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at RecurringIntegrationsScheduler.Job.Download.d__13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at RecurringIntegrationsScheduler.Job.Download.d__12.MoveNext()]

[keithchanhaha]

Submitted the case to MS support and replied it is under investigation, no walkaround yet.

How do you know 10.0.30 can fix this issue?

Keith, we advised the following to our customers:

o For 10.0.29, platform version 7.0.6545.58 or higher should be used. This means that the GA version should not be used. o For 10.0.30, platform version 7.0.6592.33 or higher (GA version) should be used.

I experience the same error after upgrade to 10.0.29. Is there any workaround?
Quartz.Core.JobRunShell - Job Download jobs.EDI_Export threw a JobExecutionException: Parameters: refire = False, unscheduleFiringTrigger = False, unscheduleAllTriggers = False Quartz.JobExecutionException: Download job: Download jobs.EDI_Export failed ---> System.Exception: Job: Download jobs.EDI_Export. Download failure. InternalServerError at RecurringIntegrationsScheduler.Job.Download.d__14.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at RecurringIntegrationsScheduler.Job.Download.d__13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at RecurringIntegrationsScheduler.Job.Download.d__12.MoveNext() --- End of inner exception stack trace --- at RecurringIntegrationsScheduler.Job.Download.d__12.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Quartz.Core.JobRunShell.d__9.MoveNext() [See nested exception: System.Exception: Job: Download jobs.EDI_Export. Download failure. InternalServerError at RecurringIntegrationsScheduler.Job.Download.d__14.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at RecurringIntegrationsScheduler.Job.Download.d__13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at RecurringIntegrationsScheduler.Job.Download.d__12.MoveNext()]

[Jerome2506]

Submitted the case to MS support and replied it is under investigation, no walkaround yet.

How do you know 10.0.30 can fix this issue?

Keith, we advised the following to our customers:
o For 10.0.29, platform version 7.0.6545.58 or higher should be used. This means that the GA version should not be used. o For 10.0.30, platform version 7.0.6592.33 or higher (GA version) should be used.

I experience the same error after upgrade to 10.0.29. Is there any workaround?
Quartz.Core.JobRunShell - Job Download jobs.EDI_Export threw a JobExecutionException: Parameters: refire = False, unscheduleFiringTrigger = False, unscheduleAllTriggers = False Quartz.JobExecutionException: Download job: Download jobs.EDI_Export failed ---> System.Exception: Job: Download jobs.EDI_Export. Download failure. InternalServerError at RecurringIntegrationsScheduler.Job.Download.d__14.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at RecurringIntegrationsScheduler.Job.Download.d__13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at RecurringIntegrationsScheduler.Job.Download.d__12.MoveNext() --- End of inner exception stack trace --- at RecurringIntegrationsScheduler.Job.Download.d__12.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Quartz.Core.JobRunShell.d__9.MoveNext() [See nested exception: System.Exception: Job: Download jobs.EDI_Export. Download failure. InternalServerError at RecurringIntegrationsScheduler.Job.Download.d__14.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at RecurringIntegrationsScheduler.Job.Download.d__13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at RecurringIntegrationsScheduler.Job.Download.d__12.MoveNext()]

You said you experienced the same issue, so I took that for granted. I mentioned the platform versions that the issue was fixed for us. Potentially you are having a slightly different issue then.