Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebJob gets aborted. No exceptions are logged. #19686

Closed
AkshathaSP opened this issue Nov 29, 2018 · 15 comments
Closed

WebJob gets aborted. No exceptions are logged. #19686

AkshathaSP opened this issue Nov 29, 2018 · 15 comments

Comments

@AkshathaSP
Copy link

AkshathaSP commented Nov 29, 2018


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

@AkshathaSP
Copy link
Author

I have created a WebJob of Triggered type with Diagonostic logging. It basically fetches data from Azure SQL. I triggered the Web job at 11/28/2018 08:16:23 and the last log of fetching data is 11/29/2018 4:53:13.
The status suddenly shows aborted and there is no exception / errors logged.

@AkshathaSP
Copy link
Author

Is there any limitation on the time that a WebJob can run ?

@AkshathaSP
Copy link
Author

Note: Always On is enabled for this WebJob

@AjayKumar-MSFT
Copy link
Contributor

@AkshathaSP , Could you please provide the Azure document that you are referring to? And, what App Service Plan are you using?
You may review and make changes to WEBJOBS_IDLE_TIMEOUT and stopping_wait_time parameters appropriately based on your requirement and see if that helps. Checkout the document for more details on this configuration.

@AkshathaSP
Copy link
Author

Hi,
I was going through this document:
https://docs.microsoft.com/en-us/azure/app-service/web-sites-create-web-jobs.
We are using 'Basic: 1 Small' App service plan.

@AkshathaSP
Copy link
Author

What is the default time out if WEBJOBS_IDLE_TIMEOUT and stopping_wait_time are not explicitly specified ?

@AjayKumar-MSFT
Copy link
Contributor

@AkshathaSP , Thanks for the update.
For Continuous jobs, there is a default period of 5 seconds waiting for the job process to shutdown before getting killed.
For Triggered jobs. When a shutdown request is detected there is a 30 seconds default waiting period for the job process to stop.
You can change the grace period of a job by specifying it (in seconds) in the settings.job file where the name of the setting is stopping_wait_time like so:
{ "stopping_wait_time": 60 }

If you need 30 minutes of execution time before azure aborts it use:
"stopping_wait_time": 1800
As an aside, note that App Service has a 230 second limit for http requests that don't return data.

@AjayKumar-MSFT
Copy link
Contributor

@AkshathaSP ,Since we have not heard back from you we will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly reopen the issue and continue the discussion.

@AkshathaSP
Copy link
Author

Hi @AjayKumar-MSFT ,
I could simulate the issue:
Start WebJob. Make sure the status is running.
Go to App service -> Diagnostic logs
Add/Modify any logging settings and save.
Refresh WebJob and check logs. The Job is aborted.
Is there any background process of App Service causing this abort?

@AkshathaSP
Copy link
Author

What happens to the App Service in the background when a Diagnostic log settings are changed while the WebJobs are running ?

Is it better to run as Worker role instead to avoid background process killing the WebJob ?

@AjayKumar-MSFT
Copy link
Contributor

@AkshathaSP , As mentioned in the document here “Unlike changing the web.config file, enabling Application diagnostics or changing diagnostic log levels does not recycle the app domain that the application runs within.”
Typically, you will want to build your webjob to handle any sort of webjob restarts ect as those can happen at any time, especially if you have a long running job. Let me review on this further and post back.

@AkshathaSP
Copy link
Author

@AjayKumar-MSFT , It is used only on need basis, so we trigger it manually, it is not a scheduled WebJob. What is the recommended way to add retry functionality in code for this case ? Since there is no retry policy settings available for triggered type unlike in scheduled type.

@AjayKumar-MSFT
Copy link
Contributor

@AkshathaSP , You have mentioned that there are no error logs, if you haven’t done this already, try adding Try/Catch blocks to capture any specific error message/error code.
Just to highlight, whatever is written to console output and console error will go to a log file for the specific triggered webjob run. You can see it on the WebJobs dashboard but the file itself is located under d:\home\data\jobs\triggered{jobname}{jobrunid}.

  1. If your requirement fits, configure the WebJob as a manual trigger, then use the Azure Scheduler to trigger the WebJob. When you have configured your WebJob, click on its properties in the Azure Portal and you should see a web hook url.
  2. Now create a new scheduler Job.
  3. Select the method as Post and paste in the webhook url. Once you've completed this configuration you can then configure the schedule
  4. Using the scheduler allows you to configure retry policies and also error actions.

If the issue still persists, could you please send an email to AzCommunity at Microsoft dot com with the below details we would like to work closer with you on this matter.
This GitHub URL:
Subscription ID:

@AkshathaSP
Copy link
Author

Azure Scheduler is retiring soon as per the documentation: https://docs.microsoft.com/en-us/azure/scheduler/scheduler-get-started-portal . And LogicApps is not feasible for current requirements.
For now, I will be optimizing the code to reduce its execution time and implement retry logic to handle any internal exceptions.
Thank you for all the leads.

@AjayKumar-MSFT
Copy link
Contributor

@AkshathaSP, Yes, Azure Scheduler is retiring, and Azure Logic Apps would be replacing it. Since you have mentioned that Logic Apps does not fit your requirement or scenario, please do optimize the code as highlighted and see how that goes. If you need further assistance on this issue as mentioned in my previous post, do send an email we would be more than happy to assist you further on this matter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants