Skip to content

Commit

Permalink
[5574] Enable large pdbs by default
Browse files Browse the repository at this point in the history
Windows PDBs default to using a page size of 4 KiB and have a maximum of
2^20 pages. This gives them a maximum size of 4 GiB which Chrome's test
binaries have already started exceeding in some configuration, and
chrome.dll is rapidly approaching. This problem was anticipated in 2020
(https://developercommunity.visualstudio.com/t/pdb-limit-of-4-gib-is-likely-to-be-a-problem-in-a/904784)
and windbg, the Visual Studio debugger, the Visual Studio linker, the
lld linker, Windows Performance Analyzer, Windbg Preview, and other
tools were updated to support larger page sizes and therefore larger
PDBs.

Separately, another limitation of 4 KiB PDB pages was found when doing
ARM64 Windows builds of Chrome, documented here:

https://bugs.chromium.org/p/chromium/issues/detail?id=1406510#c32

Large PDBs require an updated version of dbghelp.dll, but we have long
been prevented from updating dbghelp.dll because newer versions of the
debugger tools don't work on Windows 7. Now that Windows 7 is no longer
supported we are finally able to upgrade dbghelp.dll.

Therefore this change:

- changes the default value of use_large_pdbs to true, switching us to
an 8 KiB page size, allowing 8 GiB PDBs. This page size can be trivially
increased if needed in the future.
- changes the toolchain hash to one with the new Debuggers directory
(details below)
- Adds $root_out_dir/dbghelp.dll and $root_out_dir/dbgcore.dll to
runtime_libs so that we deploy these DLLs to all test bots.
- Copies the x64 version of msdia140.dll to the output directory
so that symupload and dump_syms (which are both always x64 binaries)
will get a known-good version that can handle our large-page PDBs.

Because large PDBs fail with the version of dbghelp.dll that we have in
the toolchain package this change also includes the hash of a new
toolchain package. This was created by copying over the five main
directories from the Debuggers directory from the Windows 11 SDK
(version 10.0.22621.755) to the current packaged toolchain and
repackaging. The commands to do this are:

  cd "third_party\depot_tools\win_toolchain\vs_files\1023ce2e82\Windows Kits\10\Debuggers"
  rmdir . /s/q
  xcopy "c:\Program Files (x86)\Windows Kits\10\Debuggers" . /s/v >nul
  rmdir arm /s/q
  rmdir ddk /s/q
  rmdir redist /s/q
  cd ..\..\..\..
  call python3 ..\package_from_installed.py --repackage 1023ce2e82

Bug: 1273169, 1245726, 1406510
Change-Id: Ic1cd02c6b38bb1b8eb2b6a6ec542a7ff29f644a6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4222275
Commit-Queue: Wanda Mora <morawand@google.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Cr-Commit-Position: refs/branch-heads/5574@{#3}
Cr-Branched-From: 3d2478f-refs/heads/main@{#1100136}
  • Loading branch information
randomascii authored and Chromium LUCI CQ committed Feb 4, 2023
1 parent 487446e commit c2d640a
Show file tree
Hide file tree
Showing 4 changed files with 24 additions and 15 deletions.
4 changes: 1 addition & 3 deletions build/config/win/BUILD.gn
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,7 @@ declare_args() {
use_clang_diagnostics_format = false

# Indicates whether to use /pdbpagesize:8192 to allow PDBs larger than 4 GiB.
# This requires updated debugging and profiling tools which are not widely
# distributed yet which is why it is currently opt-in.
use_large_pdbs = false
use_large_pdbs = true
}

# This is included by reference in the //build/config/compiler config that
Expand Down
20 changes: 15 additions & 5 deletions build/vs_toolchain.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@

from gn_helpers import ToGNString

# VS 2019 16.61 with 10.0.20348.0 SDK, 10.0.19041 version of Debuggers
# VS 2019 16.61 with 10.0.20348.0 SDK, 10.0.22621.755 version of Debuggers,
# with ARM64 libraries and UWP support.
# See go/chromium-msvc-toolchain for instructions about how to update the
# toolchain.
Expand All @@ -32,7 +32,7 @@
# Affects the availability of APIs in the toolchain headers.
# * //docs/windows_build_instructions.md mentions of VS or Windows SDK.
# Keeps the document consistent with the toolchain version.
TOOLCHAIN_HASH = '1023ce2e82'
TOOLCHAIN_HASH = '0b5ee4d2b1'

script_dir = os.path.dirname(os.path.realpath(__file__))
json_data_file = os.path.join(script_dir, 'win_toolchain.json')
Expand Down Expand Up @@ -412,17 +412,21 @@ def CopyDlls(target_dir, configuration, target_cpu):


def _CopyDebugger(target_dir, target_cpu):
"""Copy dbghelp.dll and dbgcore.dll into the requested directory as needed.
"""Copy dbghelp.dll, dbgcore.dll, and msdia140.dll into the requested
directory.
target_cpu is one of 'x86', 'x64' or 'arm64'.
dbghelp.dll is used when Chrome needs to symbolize stacks. Copying this file
from the SDK directory avoids using the system copy of dbghelp.dll which then
ensures compatibility with recent debug information formats, such as VS
2017 /debug:fastlink PDBs.
ensures compatibility with recent debug information formats, such as
large-page PDBs. Note that for these DLLs to be deployed to swarming bots they
also need to be listed in group("runtime_libs").
dbgcore.dll is needed when using some functions from dbghelp.dll (like
MinidumpWriteDump).
msdia140.dll is needed for tools like symupload.exe and dump_syms.exe.
"""
win_sdk_dir = SetEnvironmentAndGetSDKDir()
if not win_sdk_dir:
Expand All @@ -448,6 +452,12 @@ def _CopyDebugger(target_dir, target_cpu):
target_path = os.path.join(target_dir, debug_file)
_CopyRuntimeImpl(target_path, full_path)

# The x64 version of msdia140.dll is always used because symupload and
# dump_syms are always built as x64 binaries.
dia_path = os.path.join(NormalizePath(os.environ['GYP_MSVS_OVERRIDE_PATH']),
'DIA SDK', 'bin', 'amd64', 'msdia140.dll')
_CopyRuntimeImpl(os.path.join(target_dir, 'msdia140.dll'), dia_path)


def _GetDesiredVsToolchainHashes():
"""Load a list of SHA1s corresponding to the toolchains that we want installed
Expand Down
6 changes: 5 additions & 1 deletion build/win/BUILD.gn
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,11 @@ if (is_win) {
}

group("runtime_libs") {
data = []
# These are needed for any tests that need to decode stacks.
data = [
"$root_out_dir/dbghelp.dll",
"$root_out_dir/dbgcore.dll",
]
if (is_component_build) {
# Copy the VS runtime DLLs into the isolate so that they don't have to be
# preinstalled on the target machine. The debug runtimes have a "d" at
Expand Down
9 changes: 3 additions & 6 deletions docs/windows_build_instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,9 @@ $ PATH_TO_INSTALLER.EXE ^
installed. This can be installed separately or by checking the appropriate box
in the Visual Studio Installer.

The SDK Debugging Tools must also be installed. If the Windows 10 SDK was
installed via the Visual Studio installer, then they can be installed by going
to: Control Panel → Programs → Programs and Features → Select the "Windows
Software Development Kit" → Change → Change → Check "Debugging Tools For
Windows" → Change. Or, you can download the standalone SDK installer and use it
to install the Debugging Tools.
The 10.0.22621.755 (Windows 11) SDK Debugging Tools must also be installed. This
version of the Debugging tools is needed in order to support reading the
large-page PDBs that Chrome uses to allow greater-than 4 GiB PDBs.

## Install `depot_tools`

Expand Down

0 comments on commit c2d640a

Please sign in to comment.