-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ExecuteScriptAsync blocking forever after several successful runs. #1348
Comments
Hey @AnQueth - can you give some more context about when you are running this? Are you running this many times in succession, or does it block when you call it from a specific event or stack? |
so it's a scraping situation. string selectOption = " function modelChange(){ try{ var el = document.getElementById("" + el + "");" +
|
@AnQueth I don't want to suggest that there isn't an issue, just wanted to confirm that it doesn't always fail with multiple calls. I've just successfully called ExecuteScriptAsync to modify the DOM, 1032 times in 1.7s, without any failures. |
yes, i don't know why it will not let the await finish but the js runs. |
Where are you calling this from, are you inadvertently blocking it? I've just tried some adhoc tests, to call 100 times async, then 100 times sync, then 100 times async again, with no issues, using this page: https://www.w3.org/WAI/UA/2002/06/thead-test Script to style border of table:
Then on the .Net side
|
@AnQueth I wonder if the javascript itself is hanging or blocking? Is there anything in your javascript that could possibly block? When you run into this issue, if you open the devtools on that page, can you manually run your script in the console successfully? Also, thanks @ukandrewc for the extra testing here and confirming it isn't a wider problem (potentially)! |
@champnic No problem. We use ExecutescriptAsync a lot, so it's in our interest to see if it can be broken ;-) |
here's the whole code. it's a messy mess because it's just hacky thing. i did have this before, and it would block to. there is nothing there should block. string selectOption = "document.documentElement.outerHTML"; ` using Azure.Messaging.ServiceBus;
using Microsoft.Web.WebView2.Core;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Text;
using System.Text.Json;
using System.Text.RegularExpressions;
using System.Threading;
using System.Threading.Tasks;
using System.Web;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
namespace autoastat
{
/// <summary>
/// Interaction logic for MainWindow.xaml
/// </summary>
public partial class MainWindow : Window
{
ServiceBusClient serviceBusClient = new ServiceBusClient("E");
private const string Dir = "f:\\scrape";
private int _currentMake = 1;
private int _currentModel = 1;
private string _currentUri = "";
private bool _loggedin;
private Uri _referer;
private string _username = "login";
private string _password = "7Fpassword";
private ServiceBusSender _sbc;
private int CMake
{
get => _currentMake;
set
{
_currentMake = value;
Save();
}
}
private int CModel
{
get => _currentModel;
set
{
_currentModel = value;
Save();
}
}
private void Save()
{
var ob = new
{
CModel = _currentModel,
CMake = _currentMake,
Page = _currentUri
};
var d = JsonConvert.SerializeObject(ob);
File.WriteAllText("data.json", d);
}
private void Load()
{
if (File.Exists("data.json"))
{
var d = JsonConvert.DeserializeObject<dynamic>(File.ReadAllText("data.json"));
_currentMake = (int)d["CMake"].Value;
_currentModel = (int)d["CModel"].Value;
_currentUri = d["Page"].Value;
}
}
public MainWindow()
{
AppDomain.CurrentDomain.UnhandledException += CurrentDomain_UnhandledException;
InitializeComponent();
}
private void CurrentDomain_UnhandledException(object sender, UnhandledExceptionEventArgs e)
{
}
private async void Grid_Loaded(object sender, RoutedEventArgs e)
{
_sbc = serviceBusClient.CreateSender("scrape");
Load();
var env = await CoreWebView2Environment.CreateAsync();
await web.EnsureCoreWebView2Async(env);
web.CoreWebView2.ProcessFailed += CoreWebView2_ProcessFailed;
web.CoreWebView2.OpenDevToolsWindow();
//web.CoreWebView2.AddWebResourceRequestedFilter("https://autoastat.com/build/vehicle/main.1e19bf9f.js",
// Microsoft.Web.WebView2.Core.CoreWebView2WebResourceContext.All);
web.CoreWebView2.AddWebResourceRequestedFilter("*", Microsoft.Web.WebView2.Core.CoreWebView2WebResourceContext.All);
web.CoreWebView2.WebResourceRequested += this.CoreWebView2_WebResourceRequested;
web.CoreWebView2.DOMContentLoaded += this.CoreWebView2_DOMContentLoaded;
web.CoreWebView2.NavigationStarting += this.CoreWebView2_NavigationStarting;
web.CoreWebView2.WebResourceResponseReceived += this.CoreWebView2_WebResourceResponseReceived;
web.CoreWebView2.NavigationCompleted += this.CoreWebView2_NavigationCompleted;
web.CoreWebView2.Navigate("https://autoastat.com/en/login");
}
private void CoreWebView2_ProcessFailed(object sender, Microsoft.Web.WebView2.Core.CoreWebView2ProcessFailedEventArgs e)
{
}
private void CoreWebView2_NavigationStarting(object sender, Microsoft.Web.WebView2.Core.CoreWebView2NavigationStartingEventArgs e)
{
}
private async void CoreWebView2_WebResourceRequested(object sender, Microsoft.Web.WebView2.Core.CoreWebView2WebResourceRequestedEventArgs e)
{
if (_referer != null)
e.Request.Headers.SetHeader("Referer", _referer.ToString());
if (e.Request.Uri.Contains("main") && e.Request.Uri.Contains("js"))
{
HttpClient cl = new HttpClient();
var d = e.GetDeferral();
var data = await cl.GetStringAsync(e.Request.Uri);
const string s = "(function(e){n(\"8h6d\");";
data = data.Insert(data.IndexOf(s) + s.Length, "window.$=e;");
Stream st = new MemoryStream(Encoding.UTF8.GetBytes(data));
e.Response = web.CoreWebView2.Environment.CreateWebResourceResponse(st, 200, "OK", $"Content-Type: application/javascript");
d.Complete();
}
}
private async void CoreWebView2_DOMContentLoaded(object sender, Microsoft.Web.WebView2.Core.CoreWebView2DOMContentLoadedEventArgs e)
{
if (web.Source.PathAndQuery.EndsWith("en/"))
{
}
}
private async void CoreWebView2_WebResourceResponseReceived(object sender, Microsoft.Web.WebView2.Core.CoreWebView2WebResourceResponseReceivedEventArgs e)
{
if (e.Request.Uri.Contains("en/form?"))
{
try
{
var s = await e.Response.GetContentAsync();
var buf = new byte[s.Length];
s.Read(buf, 0, buf.Length);
var html = Encoding.UTF8.GetString(buf);
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(html);
if (!Directory.Exists(Dir))
Directory.CreateDirectory(Dir);
foreach (var node in doc.DocumentNode.SelectNodes("//div[@class='block']"))
{
//await File.WriteAllTextAsync(
// $"{Dir}\\{CMake}_{CModel}_{DateTimeOffset.UtcNow.Ticks}", node.OuterHtml);
var a = node.SelectSingleNode(".//a").GetAttributeValue("href", "");
var al = "https://autoastat.com" + a;
var fn = GetFileName(al);
await _sbc.SendMessageAsync(new ServiceBusMessage(al));
using (var fs = File.Open(Dir + "\\" + fn, FileMode.Append, FileAccess.Write, FileShare.ReadWrite))
{
using (var sr = new StreamWriter(fs))
{
sr.WriteLine(al);
}
}
//File.AppendAllText($"{Dir}\\{fn}", al + "\r\n");
}
HtmlAgilityPack.HtmlNode? paging = null;
var pn = doc.DocumentNode.SelectNodes("//a[@class='page-link']");
if (pn != null)
{
paging = pn.LastOrDefault();
}
if (paging == null)
{
await Task.Delay(new Random().Next(60000, 120000));
_currentUri = "";
Save();
CModel++;
_referer = null;
web.Source = new Uri("https://autoastat.com/en/");
}
else
{
_referer = web.Source;
string u = "https://autoastat.com" +
paging.GetAttributeValue("href", "").Replace("&", "&");
var src = HttpUtility.ParseQueryString(_referer.Query).Get("page");
var ns = HttpUtility.ParseQueryString(new Uri(u).Query).Get("page");
if (!int.TryParse(src, out var old))
{
old = 0;
}
if (!int.TryParse(ns, out var news))
{
}
if (news > old)
{
await Task.Delay(new Random().Next(60000, 120000));
web.Source = new Uri(u);
_currentUri = u;
Save();
}
else
{
await Task.Delay(new Random().Next(60000, 120000));
CModel++;
_referer = null;
_currentUri = "";
Save();
web.Source = new Uri("https://autoastat.com/en/");
}
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.ToString());
web.Reload();
}
}
if (e.Request.Uri.Contains("main") && e.Request.Uri.EndsWith(".js"))
{
//var s = await e.Response.GetContentAsync();
//var buf = new byte[s.Length];
//s.Read(buf, 0, buf.Length);
//string g = Encoding.UTF8.GetString(buf);
}
if (e.Request.Uri.Contains("/en/make"))
{
try
{
var s = await e.Response.GetContentAsync();
// _makes = await JsonSerializer.DeserializeAsync<Make>(s);
var buf = new byte[s.Length];
s.Read(buf, 0, buf.Length);
var _makes = JsonConvert.DeserializeObject<Make>(Encoding.UTF8.GetString(buf));
Task.Delay(1000).ContinueWith(async (t) =>
{
Application.Current.Dispatcher.Invoke(async () =>
{
string el = "select-make";
string selectOption = " try{ var el = document.getElementById(\"" + el + "\");" +
"var op = el.options[" + CMake + "]; " +
"$('#" + el + "').val(op.value);" +
"$('#" + el + "').trigger('change');" +
"} catch(e){alert(e);}";
var r = await web.CoreWebView2.ExecuteScriptAsync(selectOption);
});
});
}
catch (Exception ex)
{
Debug.WriteLine(ex.ToString());
}
}
if (e.Request.Uri.Contains("en/model"))
{
Task.Delay(500).ContinueWith(async (t) =>
{
Application.Current.Dispatcher.Invoke(async () =>
{
string el = "select-model";
string selectOption = " function modelChange(){ try{ var el = document.getElementById(\"" + el + "\");" +
"var op = el.options[" + CModel + "]; " +
"$('#" + el + "').val(op.value);" +
"$('#" + el + "').trigger('change');" +
"return 'good';" +
"} catch(e){ return 'bad'; }}\r\n" +
"modelChange();";
var r = await web.CoreWebView2.ExecuteScriptAsync(selectOption);
if (r == "\"bad\"")
{
CMake++;
CModel = 1;
web.Reload();
//web.Source = new Uri("https://autoastat.com/en/");
}
});
});
}
if (e.Request.Uri.Contains("en/date"))
{
var s = await e.Response.GetContentAsync();
// _makes = await JsonSerializer.DeserializeAsync<Make>(s);
var buf = new byte[s.Length];
s.Read(buf, 0, buf.Length);
string ds = "";
try
{
ds = Encoding.UTF8.GetString(buf);
var dates = JsonConvert.DeserializeObject<Dates>(ds);
var dts = dates.results.Select(z => DateTime.Parse(z)).Where(z =>
z >= DateTime.Today.AddDays(-14)).OrderByDescending(p => p).ToArray();
if (dts.Count() == 0)
{
dts = dates.results.Select(z => DateTime.Parse(z)).OrderByDescending(p => p).ToArray();
}
var old = dts.LastOrDefault();
var n = dts.FirstOrDefault();
if (old == default)
old = n;
if (old == default && n == default)
{
return;
}
Task.Delay(20000).ContinueWith(async (t) =>
{
Application.Current.Dispatcher.Invoke(async () =>
{
string selectOption = " try{ $('#select-date-from').val('" + old.ToString("yyyy-MM-dd") + "') ; " +
"$('#select-date-to').val('" + n.ToString("yyyy-MM-dd") + "') ; " +
"$('#search').click(); " +
"} catch(e){console.log(e);}";
var r = await web.CoreWebView2.ExecuteScriptAsync(selectOption);
});
});
}
catch
{
Debug.WriteLine(ds);
CModel++;
web.Reload();
}
}
}
private async void CoreWebView2_NavigationCompleted(object sender, Microsoft.Web.WebView2.Core.CoreWebView2NavigationCompletedEventArgs e)
{
if (_loggedin)
{
_loggedin = false;
if (_currentUri != string.Empty)
{
web.Source = new Uri(_currentUri);
return;
}
else
{
_currentUri = web.Source.ToString();
Save();
}
}
if (web.Source.PathAndQuery.EndsWith("login"))
{
string login = "document.getElementById('_username').value = '" + _username + "';" +
"document.getElementById('_password').value = '" + _password + "';" +
" document.querySelectorAll(\"button[type='submit']\")[2].click();";
await web.CoreWebView2.ExecuteScriptAsync(login);
_loggedin = true;
}
}
private string GetFileName(string al)
{
int c = 0;
foreach (byte x in al)
{
c += x;
}
int r = c % 1;
return r.ToString() + ".urls";
}
}
} |
@AnQueth Unfortunately, you've got too much going on there, to be able to offer anything constructive. |
Hi, I am encountering a similar issue. I just checked my code with your comment in mind re: deadlock with Navigation completed. These events happen very early on as the page loads. The issue happens much later during the 50th refresh. I reduced my code to multiple refresh clicks (using ExecuteScriptAsync) on a web page. This async function sets a flag indicating await has completed. The refreshes stop happening after about 50 of them. For 2 second intervals at tick number 50, for 6s at tick number 48. The interesting thing is this piece of code worked for over 6 hours last Friday, May 28th. Do not know if the webView2 uses whatever the latest Edge browser is on this Windows 10 machine. Here is the function Here is the caller: This is what seems to be the sequence of events;
This makes me think that this is an issue with webView2. Thanks a bunch! |
yes, it ran fine 2 weeks ago for me to. |
@hmurty Do you get the same if your JS calls |
ukandrewc thank you for suggestion, will try and get back to you in a day or two. |
@ukandrewc location.reload() takes a longer time. It would be hard to get new data every two seconds. There are other parts of the code where loading a new page takes over 10 seconds. Why I prefer to use minor changes on a page for quick updates. |
@hmurty Sorry, I was only asking because I've tried a timer test, like yours, but using |
@ukandrewc location.reload() does not activate embedded JavaScript code but having .NET code click on a refresh button initiates the embedded handler for the click. The webView2 browser object has to manage JavaScript from ExecuteScriptAsync and also the embedded JavaScript. There may be an issue there if the external JS comes on top of the embedded JS that has not yet finished. This is why I increased the time between two timer ticks but results did not change. |
@hmurty are you able to share the url you are using? |
@ukandrewc it is an account I log into. Will try to find something similar that is public in a day or two. It is stock quotation data. Probably Yahoo Finance has something similar. Even if it is delayed data it will be changing regularly. If there is a hang up after so many refreshes this would point to .NET. Thank you for your interest! |
Everything worked great before memorial day weekend. After is when I noticed the problem so the timing does match the upgrade. |
Thanks for the info both! @hmurty That's the version of the SDK package. For the runtime, you can go to "Apps and Features" and search for "webview2 runtime" in the list to get the version: |
Doing this work on two PCs. Both have a date of 6/5/2021 for Microsoft.Edge.WebView2.Runtime. |
Have tried to reproduce the issue without success.
.NET
@AnQueth @hmurty - do you have any suggestions how to tweak this code to see ExecuteScriptAsync blocked?
|
@vbryh-msft thank you for taking the time to do this test. I will try to get to your suggestions 1-3 during the coming week. |
@hmurty Thanks for your suggestion. Used XMLHttpRequest with next code and .NET part with
Requested timer.txt file is updated separately every second with current time. |
@vbryh-msft Thanks for doing this. My code does pretty much the same thing. Next will try to implement your suggestions. Will set aside a day during this coming weekend. |
The issue should be fixed in Edge Canary 93.0.950.0+. Please let me know if you still able to repro it. |
Is it fixed in WebView2 runtime? The latest version of WebView2 runtime is 92.0.902.78 and I still having this problem there. The code is very simple -- several times per hour the webview component navigates different pages and I am getting the page DOM by calling ExecuteScriptAsync:
randomly after 10-20 hours of working ExecuteScriptAsync blocks and the code never executes past the line with ExecuteScriptAsync. The webview itself stays responsible, is able to do further navigations, but all new calls to ExecuteScriptAsync are blocked, the only way to make it working is to restart the app. |
@gribunin this fix is not available in release yet. You would need to download a Canary channel to test it. Please let me know if you have more questions. |
long long long coding.... |
@vbryh-msft When can we expect this fix be in the release version? We also face this problem, hope it can be fixed soon. |
@CenturionYun the issue was fixed and available in releases since 93.0.961.0 version. |
@vbryh-msft Does it require a Runtime fix or SDK fix? I read the SDK release note from here https://docs.microsoft.com/en-us/microsoft-edge/webview2/release-notes, is says it had been fixed in 1.0.955-prerelease, but no prod release mentions it. We still see this issue on our users' side with latest Runtime and SDK, but we cannot reproduce it. |
@CenturionYun - as per our release process all bugfixes from the prerelease are available in the next release also(we will see if we can make it more clear in the notes). Also I have checked - the fix for this issues requires Runtime update. What exactly you see on your user's side? |
@vbryh-msft We call ExecuteScript in an Win32 App to get time(just for test) every 2 seconds, we find that it does not return after a few calls, it only happen on some computers, not all. |
@CenturionYun - WebView2 will not auto sleep by itself when app is inactive - you can do it using TrySuspend API |
this script and other scripts have a tendency to block forever on the await for the script to finish.
When this happens the webview2 is responsive in the ui, but the ExecuteScruptAsync will break again after refreshing it.
string selectOption = "document.documentElement.outerHTML";
var r = await web.CoreWebView2.ExecuteScriptAsync(selectOption);
var html = Regex.Unescape(r);
Version
SDK: latest nuget
Runtime: i have prod and dev builds installed for edge and the webview2 runtime
Framework: wpf
OS: win 10 20h2
AB#33665218
The text was updated successfully, but these errors were encountered: