Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Different default stack size across platforms #33622

Closed
markusmobius opened this issue Mar 15, 2020 · 3 comments
Closed

Different default stack size across platforms #33622

markusmobius opened this issue Mar 15, 2020 · 3 comments
Labels
question Answer questions and provide assistance, not an issue with source code or documentation. untriaged New issue has not been triaged by the area owner

Comments

@markusmobius
Copy link

I am using HtmlAgilityPack within a heavily threaded application to process html files. Occassionally there are badly misformed HTML files with many nested child objects which cause a stack overflow error.

HtmlAgilityPack has an option to limit child nodes:

                            //get the title
                            try
                            {
                                var htmlDoc = new HtmlDocument();
                                htmlDoc.OptionMaxNestedChildNodes = 200; //this prevents stack overflow
                                htmlDoc.LoadHtml(page.html);
                                var t = htmlDoc.DocumentNode.Descendants("title").FirstOrDefault();
                                if (t != null)
                                {
                                    title = System.Net.WebUtility.HtmlDecode(t.InnerText);
                                    //get rid of tabs, linebreaks
                                    title = title.Replace('\r', ' ').Replace('\t', ' ').Trim().Replace("\n", "###");
                                }
                            }
                            catch (Exception e)
                            {
                                //ignore title
                            }

[This code is running in one of 16 threads. The entire program is running on a 16-core Ubuntu or Azure VM.]

This fixed the error on Ubuntu. However, on Windows I keep getting stack overflow error. Are there differences in stack management between Ubuntu and Windows that could cause this?

Update: I fixed it by manually increasing maxStackSize in the Thread constructor to 16MB. My understanding is that it's 1MB for 32bit and 4MB for 64 bit processes. It it possible that the (a) the stack is used differently on Linux and Window netcore runtime or (b) the default stack size values are different on Linux and Windows?

@scalablecory scalablecory changed the title Stack overflow on Windows but not on Ubuntu Different default stack size across platforms Mar 16, 2020
@scalablecory scalablecory transferred this issue from dotnet/core Mar 16, 2020
@scalablecory scalablecory added the question Answer questions and provide assistance, not an issue with source code or documentation. label Mar 16, 2020
@Dotnet-GitSync-Bot
Copy link
Collaborator

I couldn't add an area label to this Issue.

Checkout this page to find out which area owner to ping, or please add exactly one area label to help train me in the future.

@Dotnet-GitSync-Bot Dotnet-GitSync-Bot added the untriaged New issue has not been triaged by the area owner label Mar 16, 2020
@scalablecory
Copy link
Contributor

You will probably want to file a bug with the Html Agility Pack project to see if a recursionless algorithm is possible, but we should be able to answer the default stack size question.

CC @jkotas

@janvorli
Copy link
Member

janvorli commented Mar 16, 2020

The default stack size for a thread is platform specific and it also differs between primary thread and the secondary threads. The primary thread size limit is set when an application by the OS / OS specific runtime libraries. E.g. on Linux, it can be set by "ulimit -s" command, on many Linux distros it is 8MB by default. We can set the default stack size for the secondary threads though and we do that. If a stack size was specified in the Thread constructor, we use that value (after applying minimum stack limit imposed by the OS / OS specific runtime). If it was not specified, we use a default value that was set by the COMPlus_DefaultStackSize env variable in case it was set (please note the env variable is set to a hexadecimal value without any prefix / suffix, so e.g. 10000 means 1MB). If it was not set, we use whatever default the OS / OS specific runtime sets. There is also DEFAULT_STACK_SIZE host configuration property that can be used by custom hosts.
The only special case is Linux distros based on MUSL library which sets the default secondary stack size to a ridiculously low value of 80kB, so we override it to 1.5MB instead). The default value on glibc based linux distros is based on the primary thread stack size limit. On OSX, the default secondary stack size is 512KB. On Windows, we set the default stack size using a compiler option when building the host executables to 1.5MB.

@dotnet dotnet locked as resolved and limited conversation to collaborators Dec 10, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
question Answer questions and provide assistance, not an issue with source code or documentation. untriaged New issue has not been triaged by the area owner
Projects
None yet
Development

No branches or pull requests

5 participants