Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Elide allocation of uninit static #62655

Open
wants to merge 12 commits into
base: master
from

Conversation

@HeroicKatora
Copy link
Contributor

commented Jul 13, 2019

Prepares eliding fully undefined allocations as long as they are not
written to. Reads of such uninitialized bytes should not occur or be
replaced by special handling at the call site.

@oli-obk @RalfJung

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Jul 13, 2019

r? @estebank

(rust_highfive has picked a reviewer for you, use r? to override)

src/librustc/mir/interpret/allocation.rs Outdated Show resolved Hide resolved
src/librustc/mir/interpret/allocation.rs Outdated Show resolved Hide resolved
src/librustc/mir/interpret/allocation.rs Outdated Show resolved Hide resolved
src/librustc/ty/print/pretty.rs Outdated Show resolved Hide resolved
src/librustc_mir/interpret/memory.rs Show resolved Hide resolved
src/librustc_mir/interpret/memory.rs Outdated Show resolved Hide resolved
@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Jul 14, 2019

The job mingw-check of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-07-14T16:23:03.7007773Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-07-14T16:23:03.7193190Z ##[command]git config gc.auto 0
2019-07-14T16:23:03.7278895Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-07-14T16:23:03.7337988Z ##[command]git config --get-all http.proxy
2019-07-14T16:23:03.7470418Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/62655/merge:refs/remotes/pull/62655/merge
---
2019-07-14T16:23:37.3875334Z do so (now or later) by using -b with the checkout command again. Example:
2019-07-14T16:23:37.3876578Z 
2019-07-14T16:23:37.3877980Z   git checkout -b <new-branch-name>
2019-07-14T16:23:37.3879445Z 
2019-07-14T16:23:37.3880150Z HEAD is now at 45fe28d0e Merge 4ad12fb8806a9366ddba46a51dc42fc3c14e8f15 into 7d41ebf768faca490addc7c616b3a9274621f0e9
2019-07-14T16:23:37.4010998Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-07-14T16:23:37.4013246Z ==============================================================================
2019-07-14T16:23:37.4013311Z Task         : Bash
2019-07-14T16:23:37.4013348Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-07-14T16:25:24.3027305Z Attempting with retry: curl -y 30 -Y 10 --connect-timeout 30 -f -L -C - -o /tmp/rustci_docker_cache https://.s3.amazonaws.com/docker/a4940e6914a5e1f6360ebc241d0d850aff083be5c43d268fd716ce48f63cb24e84238becb662f27f8c5d7740413ca17da24bb9cdc003ef32d7b03d2c9052b94d
2019-07-14T16:25:24.3091704Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2019-07-14T16:25:24.3092185Z                                  Dload  Upload   Total   Spent    Left  Speed
2019-07-14T16:25:24.3092277Z 
2019-07-14T16:25:24.3093194Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: .s3.amazonaws.com
2019-07-14T16:25:25.3180471Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2019-07-14T16:25:25.3181012Z                                  Dload  Upload   Total   Spent    Left  Speed
2019-07-14T16:25:25.3181261Z 
2019-07-14T16:25:25.3181261Z 
2019-07-14T16:25:25.3183149Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: .s3.amazonaws.com
2019-07-14T16:25:27.3259097Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2019-07-14T16:25:27.3259174Z                                  Dload  Upload   Total   Spent    Left  Speed
2019-07-14T16:25:27.3259205Z 
2019-07-14T16:25:27.3259205Z 
2019-07-14T16:25:27.3294716Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: .s3.amazonaws.com
2019-07-14T16:25:30.3382123Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2019-07-14T16:25:30.3382805Z                                  Dload  Upload   Total   Spent    Left  Speed
2019-07-14T16:25:30.3383104Z 
2019-07-14T16:25:30.3383104Z 
2019-07-14T16:25:30.3384027Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: .s3.amazonaws.com
2019-07-14T16:25:34.3460838Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2019-07-14T16:25:34.3461361Z                                  Dload  Upload   Total   Spent    Left  Speed
2019-07-14T16:25:34.3461589Z 
2019-07-14T16:25:34.3461589Z 
2019-07-14T16:25:34.3502439Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: .s3.amazonaws.com
2019-07-14T16:25:34.3503684Z The command has failed after 5 attempts.
2019-07-14T16:25:34.3863170Z open /tmp/rustci_docker_cache: no such file or directory
2019-07-14T16:25:34.3884193Z Attempting with retry: docker build --rm -t rust-ci -f /home/vsts/work/1/s/src/ci/docker/mingw-check/Dockerfile /home/vsts/work/1/s/src/ci/docker
2019-07-14T16:25:34.6046250Z Sending build context to Docker daemon  521.7kB
2019-07-14T16:25:34.6047008Z 
2019-07-14T16:25:34.6062837Z Step 1/6 : FROM ubuntu:16.04
---
2019-07-14T16:25:50.3991591Z Reading package lists...
2019-07-14T16:25:51.2257561Z Reading package lists...
2019-07-14T16:25:51.3663827Z Building dependency tree...
2019-07-14T16:25:51.3663996Z Reading state information...
2019-07-14T16:25:51.4639889Z The following additional packages will be installed:
2019-07-14T16:25:51.4641212Z   binutils binutils-mingw-w64-i686 binutils-mingw-w64-x86-64 bzip2 cmake-data
2019-07-14T16:25:51.4641507Z   cpp cpp-5 dpkg-dev g++-5 g++-mingw-w64 g++-mingw-w64-i686
2019-07-14T16:25:51.4641886Z   g++-mingw-w64-x86-64 gcc gcc-5 gcc-mingw-w64 gcc-mingw-w64-base
2019-07-14T16:25:51.4642360Z   gcc-mingw-w64-i686 gcc-mingw-w64-x86-64 git-man libarchive13 libasan2
2019-07-14T16:25:51.4642964Z   libasn1-8-heimdal libatomic1 libbabeltrace-ctf1 libbabeltrace1 libbz2-1.0
2019-07-14T16:25:51.4644096Z   libdpkg-perl liberror-perl libexpat1 libffi6 libgcc-5-dev libgdbm3
2019-07-14T16:25:51.4644522Z   libglib2.0-0 libgmp10 libgnutls30 libgomp1 libgssapi-krb5-2
2019-07-14T16:25:51.4644890Z   libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal
2019-07-14T16:25:51.4645257Z   libheimntlm0-heimdal libhogweed4 libhx509-5-heimdal libicu55 libidn11
2019-07-14T16:25:51.4645257Z   libheimntlm0-heimdal libhogweed4 libhx509-5-heimdal libicu55 libidn11
2019-07-14T16:25:51.4645869Z   libisl15 libitm1 libjsoncpp1 libk5crypto3 libkeyutils1 libkrb5-26-heimdal
2019-07-14T16:25:51.4667332Z   libkrb5-3 libkrb5support0 libldap-2.4-2 liblsan0 liblzo2-2 libmagic1 libmpc3
2019-07-14T16:25:51.4668154Z   libmpdec2 libmpfr4 libmpx0 libnettle6 libp11-kit0 libperl5.22
2019-07-14T16:25:51.4668555Z   libpython2.7-minimal libpython2.7-stdlib libpython3.5 libpython3.5-minimal
2019-07-14T16:25:51.4669113Z   libpython3.5-stdlib libquadmath0 libroken18-heimdal librtmp1 libsasl2-2
2019-07-14T16:25:51.4670101Z   libsasl2-modules-db libsqlite3-0 libssl1.0.0 libstdc++-5-dev libtasn1-6
2019-07-14T16:25:51.4670544Z   libtsan0 libubsan0 libwind0-heimdal libxml2 linux-libc-dev mime-support
2019-07-14T16:25:51.4671069Z   mingw-w64-common mingw-w64-i686-dev mingw-w64-x86-64-dev openssl patch perl
2019-07-14T16:25:51.4671513Z   perl-modules-5.22 python2.7-minimal zlib1g-dev
2019-07-14T16:25:51.4671704Z Suggested packages:
2019-07-14T16:25:51.4672232Z   binutils-doc bzip2-doc codeblocks eclipse ninja-build cpp-doc gcc-5-locales
2019-07-14T16:25:51.4672508Z   debian-keyring g++-multilib g++-5-multilib gcc-5-doc libstdc++6-5-dbg
2019-07-14T16:25:51.4672777Z   gcc-multilib manpages-dev autoconf automake libtool flex bison gcc-doc
2019-07-14T16:25:51.4673524Z   libasan2-dbg liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg
2019-07-14T16:25:51.4673747Z   libmpx0-dbg libquadmath0-dbg gdb-doc gettext-base git-daemon-run
2019-07-14T16:25:51.4673747Z   libmpx0-dbg libquadmath0-dbg gdb-doc gettext-base git-daemon-run
2019-07-14T16:25:51.4674027Z   | git-daemon-sysvinit git-doc git-el git-email git-gui gitk gitweb git-arch
2019-07-14T16:25:51.4674257Z   git-cvs git-mediawiki git-svn lrzip glibc-doc gnutls-bin krb5-doc krb5-user
2019-07-14T16:25:51.4674490Z   libstdc++-5-doc make-doc wine wine64 ed diffutils-doc perl-doc
2019-07-14T16:25:51.4674768Z   libterm-readline-gnu-perl | libterm-readline-perl-perl python2.7-doc
2019-07-14T16:25:51.4674950Z   binfmt-support
2019-07-14T16:25:51.4674991Z Recommended packages:
2019-07-14T16:25:51.4675261Z   build-essential fakeroot libalgorithm-merge-perl gfortran-mingw-w64
2019-07-14T16:25:51.4675644Z   gnat-mingw-w64 libc-dbg gdbserver less rsync ssh-client manpages
2019-07-14T16:25:51.4675871Z   manpages-dev libfile-fcntllock-perl libglib2.0-data shared-mime-info
2019-07-14T16:25:51.4676139Z   xdg-user-dirs krb5-locales libsasl2-modules libssl-doc xml-core netbase
2019-07-14T16:25:51.4676180Z   rename
2019-07-14T16:25:51.4676217Z The following NEW packages will be installed:
2019-07-14T16:25:51.4676480Z   binutils binutils-mingw-w64-i686 binutils-mingw-w64-x86-64 bzip2
2019-07-14T16:25:51.4676700Z   ca-certificates cmake cmake-data cpp cpp-5 curl dpkg-dev file g++ g++-5
2019-07-14T16:25:51.4677034Z   g++-mingw-w64 g++-mingw-w64-i686 g++-mingw-w64-x86-64 gcc gcc-5
2019-07-14T16:25:51.4677344Z   gcc-mingw-w64 gcc-mingw-w64-base gcc-mingw-w64-i686 gcc-mingw-w64-x86-64 gdb
2019-07-14T16:25:51.4677569Z   git git-man libarchive13 libasan2 libasn1-8-heimdal libatomic1
2019-07-14T16:25:51.4677945Z   libbabeltrace-ctf1 libbabeltrace1 libc-dev-bin libc6-dev libcc1-0
2019-07-14T16:25:51.4678438Z   libffi6 libgcc-5-dev libgdbm3 libglib2.0-0 libgmp10 libgnutls30 libgomp1
2019-07-14T16:25:51.4678656Z   libgssapi-krb5-2 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal
2019-07-14T16:25:51.4678912Z   libheimntlm0-heimdal libhogweed4 libhx509-5-heimdal libicu55 libidn11
2019-07-14T16:25:51.4679563Z   libisl15 libitm1 libjsoncpp1 libk5crypto3 libkeyutils1 libkrb5-26-heimdal
2019-07-14T16:25:51.4679563Z   libisl15 libitm1 libjsoncpp1 libk5crypto3 libkeyutils1 libkrb5-26-heimdal
2019-07-14T16:25:51.4679855Z   libkrb5-3 libkrb5support0 libldap-2.4-2 liblsan0 liblzo2-2 libmagic1 libmpc3
2019-07-14T16:25:51.4680185Z   libmpdec2 libmpfr4 libmpx0 libnettle6 libp11-kit0 libperl5.22
2019-07-14T16:25:51.4680480Z   libpython2.7-minimal libpython2.7-stdlib libpython3.5 libpython3.5-minimal
2019-07-14T16:25:51.4680755Z   libpython3.5-stdlib libquadmath0 libroken18-heimdal librtmp1 libsasl2-2
2019-07-14T16:25:51.4681074Z   libsasl2-modules-db libsqlite3-0 libssl-dev libssl1.0.0 libstdc++-5-dev
2019-07-14T16:25:51.4681358Z   libtasn1-6 libtsan0 libubsan0 libwind0-heimdal libxml2 linux-libc-dev make
2019-07-14T16:25:51.4681743Z   mime-support mingw-w64 mingw-w64-common mingw-w64-i686-dev
2019-07-14T16:25:51.4682062Z   mingw-w64-x86-64-dev openssl patch perl perl-modules-5.22 pkg-config
2019-07-14T16:25:51.4683024Z   python2.7 python2.7-minimal sudo xz-utils zlib1g-dev
2019-07-14T16:25:51.4683196Z The following packages will be upgraded:
2019-07-14T16:25:51.7726207Z 1 upgraded, 112 newly installed, 0 to remove and 4 not upgraded.
2019-07-14T16:25:51.7726923Z Need to get 187 MB of archives.
2019-07-14T16:25:51.7727435Z After this operation, 968 MB of additional disk space will be used.
2019-07-14T16:25:51.7728197Z Get:1 http://archive.ubuntu.com/ubuntu xenial/main amd64 libgdbm3 amd64 1.8.3-13.1 [16.9 kB]
---
2019-07-14T16:25:53.7391298Z Get:97 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 zlib1g-dev amd64 1:1.2.8.dfsg-2ubuntu4.1 [168 kB]
2019-07-14T16:25:53.7493044Z Get:98 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libssl-dev amd64 1.0.2g-1ubuntu4.15 [1344 kB]
2019-07-14T16:25:53.7666036Z Get:99 http://archive.ubuntu.com/ubuntu xenial/main amd64 pkg-config amd64 0.29.1-0ubuntu1 [45.0 kB]
2019-07-14T16:25:53.7673637Z Get:100 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python2.7 amd64 2.7.12-1ubuntu0~16.04.4 [224 kB]
2019-07-14T16:25:53.7706619Z Get:101 http://archive.ubuntu.com/ubuntu xenial/universe amd64 binutils-mingw-w64-i686 amd64 2.26-3ubuntu1+6.6 [1782 kB]
2019-07-14T16:25:53.9011766Z Get:102 http://archive.ubuntu.com/ubuntu xenial/universe amd64 binutils-mingw-w64-x86-64 amd64 2.26-3ubuntu1+6.6 [2029 kB]
2019-07-14T16:25:54.3112614Z Get:103 http://archive.ubuntu.com/ubuntu xenial/universe amd64 mingw-w64-common all 4.0.4-2 [4787 kB]
2019-07-14T16:25:54.4015308Z Get:104 http://archive.ubuntu.com/ubuntu xenial/universe amd64 mingw-w64-i686-dev all 4.0.4-2 [2059 kB]
2019-07-14T16:25:54.4392924Z Get:105 http://archive.ubuntu.com/ubuntu xenial/universe amd64 gcc-mingw-w64-base amd64 5.3.1-8ubuntu3+17 [11.2 kB]
2019-07-14T16:25:54.4393346Z Get:106 http://archive.ubuntu.com/ubuntu xenial/universe amd64 gcc-mingw-w64-i686 amd64 5.3.1-8ubuntu3+17 [27.3 MB]
2019-07-14T16:25:54.9738184Z Get:107 http://archive.ubuntu.com/ubuntu xenial/universe amd64 g++-mingw-w64-i686 amd64 5.3.1-8ubuntu3+17 [19.8 MB]
2019-07-14T16:25:55.2982889Z Get:108 http://archive.ubuntu.com/ubuntu xenial/universe amd64 mingw-w64-x86-64-dev all 4.0.4-2 [3238 kB]
2019-07-14T16:25:55.3472029Z Get:109 http://archive.ubuntu.com/ubuntu xenial/universe amd64 gcc-mingw-w64-x86-64 amd64 5.3.1-8ubuntu3+17 [27.4 MB]
2019-07-14T16:25:55.8041326Z Get:110 http://archive.ubuntu.com/ubuntu xenial/universe amd64 g++-mingw-w64-x86-64 amd64 5.3.1-8ubuntu3+17 [20.4 MB]
2019-07-14T16:25:56.1361477Z Get:111 http://archive.ubuntu.com/ubuntu xenial/universe amd64 g++-mingw-w64 all 5.3.1-8ubuntu3+17 [10.7 kB]
2019-07-14T16:25:56.1361875Z Get:112 http://archive.ubuntu.com/ubuntu xenial/universe amd64 gcc-mingw-w64 all 5.3.1-8ubuntu3+17 [10.7 kB]
2019-07-14T16:25:56.1362971Z Get:113 http://archive.ubuntu.com/ubuntu xenial/universe amd64 mingw-w64 all 4.0.4-2 [9274 B]
2019-07-14T16:25:58.6677720Z debconf: delaying package configuration, since apt-utils is not installed
2019-07-14T16:25:58.6678691Z Fetched 187 MB in 4s (41.9 MB/s)
2019-07-14T16:25:58.6678873Z (Reading database ... 
2019-07-14T16:25:58.6678959Z (Reading database ... 5%
2019-07-14T16:25:58.6679020Z (Reading database ... 10%
2019-07-14T16:25:58.6679054Z (Reading database ... 15%
---
2019-07-14T16:26:20.5718798Z Unpacking git (1:2.7.4-0ubuntu1.6) ...
2019-07-14T16:26:21.2245404Z Selecting previously unselected package libpython2.7-stdlib:amd64.
2019-07-14T16:26:21.2263284Z Preparing to unpack .../libpython2.7-stdlib_2.7.12-1ubuntu0~16.04.4_amd64.deb ...
2019-07-14T16:26:21.2378387Z Unpacking libpython2.7-stdlib:amd64 (2.7.12-1ubuntu0~16.04.4) ...
2019-07-14T16:26:22.0246650Z Selecting previously unselected package zlib1g-dev:amd64.
2019-07-14T16:26:22.0263908Z Preparing to unpack .../zlib1g-dev_1%3a1.2.8.dfsg-2ubuntu4.1_amd64.deb ...
2019-07-14T16:26:22.0376234Z Unpacking zlib1g-dev:amd64 (1:1.2.8.dfsg-2ubuntu4.1) ...
2019-07-14T16:26:22.1290057Z Preparing to unpack .../libssl-dev_1.0.2g-1ubuntu4.15_amd64.deb ...
2019-07-14T16:26:22.1405009Z Unpacking libssl-dev:amd64 (1.0.2g-1ubuntu4.15) ...
2019-07-14T16:26:22.4355509Z Selecting previously unselected package pkg-config.
2019-07-14T16:26:22.4372349Z Preparing to unpack .../pkg-config_0.29.1-0ubuntu1_amd64.deb ...
2019-07-14T16:26:22.4372349Z Preparing to unpack .../pkg-config_0.29.1-0ubuntu1_amd64.deb ...
2019-07-14T16:26:22.4485951Z Unpacking pkg-config (0.29.1-0ubuntu1) ...
2019-07-14T16:26:22.5362882Z Selecting previously unselected package python2.7.
2019-07-14T16:26:22.5384014Z Preparing to unpack .../python2.7_2.7.12-1ubuntu0~16.04.4_amd64.deb ...
2019-07-14T16:26:22.5505933Z Unpacking python2.7 (2.7.12-1ubuntu0~16.04.4) ...
2019-07-14T16:26:22.6323730Z Selecting previously unselected package binutils-mingw-w64-i686.
2019-07-14T16:26:22.6341444Z Preparing to unpack .../binutils-mingw-w64-i686_2.26-3ubuntu1+6.6_amd64.deb ...
2019-07-14T16:26:22.6451013Z Unpacking binutils-mingw-w64-i686 (2.26-3ubuntu1+6.6) ...
2019-07-14T16:26:23.1690543Z Selecting previously unselected package binutils-mingw-w64-x86-64.
2019-07-14T16:26:23.1718168Z Preparing to unpack .../binutils-mingw-w64-x86-64_2.26-3ubuntu1+6.6_amd64.deb ...
2019-07-14T16:26:23.1832967Z Unpacking binutils-mingw-w64-x86-64 (2.26-3ubuntu1+6.6) ...
2019-07-14T16:26:23.7623697Z Selecting previously unselected package mingw-w64-common.
2019-07-14T16:26:23.7637859Z Preparing to unpack .../mingw-w64-common_4.0.4-2_all.deb ...
2019-07-14T16:26:23.7757815Z Unpacking mingw-w64-common (4.0.4-2) ...
2019-07-14T16:26:25.2254999Z Selecting previously unselected package mingw-w64-i686-dev.
2019-07-14T16:26:25.2274797Z Preparing to unpack .../mingw-w64-i686-dev_4.0.4-2_all.deb ...
2019-07-14T16:26:25.2380919Z Unpacking mingw-w64-i686-dev (4.0.4-2) ...
2019-07-14T16:26:26.3670841Z Selecting previously unselected package gcc-mingw-w64-base.
2019-07-14T16:26:26.3694276Z Preparing to unpack .../gcc-mingw-w64-base_5.3.1-8ubuntu3+17_amd64.deb ...
2019-07-14T16:26:26.3870219Z Unpacking gcc-mingw-w64-base (5.3.1-8ubuntu3+17) ...
2019-07-14T16:26:26.4680011Z Selecting previously unselected package gcc-mingw-w64-i686.
2019-07-14T16:26:26.4696705Z Preparing to unpack .../gcc-mingw-w64-i686_5.3.1-8ubuntu3+17_amd64.deb ...
2019-07-14T16:26:26.4810339Z Unpacking gcc-mingw-w64-i686 (5.3.1-8ubuntu3+17) ...
2019-07-14T16:26:30.3581033Z Selecting previously unselected package g++-mingw-w64-i686.
2019-07-14T16:26:30.3605264Z Preparing to unpack .../g++-mingw-w64-i686_5.3.1-8ubuntu3+17_amd64.deb ...
2019-07-14T16:26:30.3762821Z Unpacking g++-mingw-w64-i686 (5.3.1-8ubuntu3+17) ...
2019-07-14T16:26:34.1461443Z Selecting previously unselected package mingw-w64-x86-64-dev.
2019-07-14T16:26:34.1485532Z Preparing to unpack .../mingw-w64-x86-64-dev_4.0.4-2_all.deb ...
2019-07-14T16:26:34.1597433Z Unpacking mingw-w64-x86-64-dev (4.0.4-2) ...
2019-07-14T16:26:36.1591086Z Selecting previously unselected package gcc-mingw-w64-x86-64.
2019-07-14T16:26:36.1617751Z Preparing to unpack .../gcc-mingw-w64-x86-64_5.3.1-8ubuntu3+17_amd64.deb ...
2019-07-14T16:26:36.1728352Z Unpacking gcc-mingw-w64-x86-64 (5.3.1-8ubuntu3+17) ...
2019-07-14T16:26:40.0726917Z Selecting previously unselected package g++-mingw-w64-x86-64.
2019-07-14T16:26:40.0748607Z Preparing to unpack .../g++-mingw-w64-x86-64_5.3.1-8ubuntu3+17_amd64.deb ...
2019-07-14T16:26:40.0856229Z Unpacking g++-mingw-w64-x86-64 (5.3.1-8ubuntu3+17) ...
2019-07-14T16:26:43.9415405Z Selecting previously unselected package g++-mingw-w64.
2019-07-14T16:26:43.9445211Z Preparing to unpack .../g++-mingw-w64_5.3.1-8ubuntu3+17_all.deb ...
2019-07-14T16:26:43.9599614Z Unpacking g++-mingw-w64 (5.3.1-8ubuntu3+17) ...
2019-07-14T16:26:44.6826282Z Selecting previously unselected package gcc-mingw-w64.
2019-07-14T16:26:44.6828887Z Preparing to unpack .../gcc-mingw-w64_5.3.1-8ubuntu3+17_all.deb ...
2019-07-14T16:26:44.6829428Z Unpacking gcc-mingw-w64 (5.3.1-8ubuntu3+17) ...
2019-07-14T16:26:44.6829899Z Selecting previously unselected package mingw-w64.
2019-07-14T16:26:44.6830323Z Preparing to unpack .../mingw-w64_4.0.4-2_all.deb ...
2019-07-14T16:26:44.6830730Z Unpacking mingw-w64 (4.0.4-2) ...
2019-07-14T16:26:44.6831583Z Setting up libgdbm3:amd64 (1.8.3-13.1) ...
2019-07-14T16:26:44.6832038Z Setting up libffi6:amd64 (3.2.1-4) ...
2019-07-14T16:26:44.6832469Z Setting up libglib2.0-0:amd64 (2.48.2-0ubuntu4.3) ...
2019-07-14T16:26:44.6832665Z No schema files found: doing nothing.
---
2019-07-14T16:26:50.0803074Z Setting up zlib1g-dev:amd64 (1:1.2.8.dfsg-2ubuntu4.1) ...
2019-07-14T16:26:50.1166399Z Setting up libssl-dev:amd64 (1.0.2g-1ubuntu4.15) ...
2019-07-14T16:26:50.1592716Z Setting up pkg-config (0.29.1-0ubuntu1) ...
2019-07-14T16:26:50.2342861Z Setting up python2.7 (2.7.12-1ubuntu0~16.04.4) ...
2019-07-14T16:26:50.9540995Z Setting up binutils-mingw-w64-i686 (2.26-3ubuntu1+6.6) ...
2019-07-14T16:26:50.9974380Z Setting up binutils-mingw-w64-x86-64 (2.26-3ubuntu1+6.6) ...
2019-07-14T16:26:51.0455842Z Setting up mingw-w64-common (4.0.4-2) ...
2019-07-14T16:26:51.1342837Z Setting up mingw-w64-i686-dev (4.0.4-2) ...
2019-07-14T16:26:51.1717839Z Setting up gcc-mingw-w64-base (5.3.1-8ubuntu3+17) ...
2019-07-14T16:26:51.2123312Z Setting up gcc-mingw-w64-i686 (5.3.1-8ubuntu3+17) ...
2019-07-14T16:26:51.2394265Z update-alternatives: using /usr/bin/i686-w64-mingw32-gcc-posix to provide /usr/bin/i686-w64-mingw32-gcc (i686-w64-mingw32-gcc) in auto mode
2019-07-14T16:26:51.2394887Z update-alternatives: warning: skip creation of /usr/bin/i686-w64-mingw32-gcc-5 because associated file /usr/bin/i686-w64-mingw32-gcc-5-posix (of link group i686-w64-mingw32-gcc) doesn't exist
2019-07-14T16:26:51.2501906Z update-alternatives: using /usr/bin/i686-w64-mingw32-gcc-win32 to provide /usr/bin/i686-w64-mingw32-gcc (i686-w64-mingw32-gcc) in auto mode
2019-07-14T16:26:51.2504245Z update-alternatives: warning: skip creation of /usr/bin/i686-w64-mingw32-gcc-5 because associated file /usr/bin/i686-w64-mingw32-gcc-5-win32 (of link group i686-w64-mingw32-gcc) doesn't exist
2019-07-14T16:26:51.2901588Z Setting up g++-mingw-w64-i686 (5.3.1-8ubuntu3+17) ...
2019-07-14T16:26:51.3227050Z update-alternatives: using /usr/bin/i686-w64-mingw32-g++-posix to provide /usr/bin/i686-w64-mingw32-g++ (i686-w64-mingw32-g++) in auto mode
2019-07-14T16:26:51.3329324Z update-alternatives: using /usr/bin/i686-w64-mingw32-g++-win32 to provide /usr/bin/i686-w64-mingw32-g++ (i686-w64-mingw32-g++) in auto mode
2019-07-14T16:26:51.3594171Z Setting up mingw-w64-x86-64-dev (4.0.4-2) ...
2019-07-14T16:26:51.3998726Z Setting up gcc-mingw-w64-x86-64 (5.3.1-8ubuntu3+17) ...
2019-07-14T16:26:51.4272299Z update-alternatives: using /usr/bin/x86_64-w64-mingw32-gcc-posix to provide /usr/bin/x86_64-w64-mingw32-gcc (x86_64-w64-mingw32-gcc) in auto mode
2019-07-14T16:26:51.4275866Z update-alternatives: warning: skip creation of /usr/bin/x86_64-w64-mingw32-gcc-5 because associated file /usr/bin/x86_64-w64-mingw32-gcc-5-posix (of link group x86_64-w64-mingw32-gcc) doesn't exist
2019-07-14T16:26:51.4361440Z update-alternatives: using /usr/bin/x86_64-w64-mingw32-gcc-win32 to provide /usr/bin/x86_64-w64-mingw32-gcc (x86_64-w64-mingw32-gcc) in auto mode
2019-07-14T16:26:51.4363034Z update-alternatives: warning: skip creation of /usr/bin/x86_64-w64-mingw32-gcc-5 because associated file /usr/bin/x86_64-w64-mingw32-gcc-5-win32 (of link group x86_64-w64-mingw32-gcc) doesn't exist
2019-07-14T16:26:51.4600833Z Setting up g++-mingw-w64-x86-64 (5.3.1-8ubuntu3+17) ...
2019-07-14T16:26:51.4863736Z update-alternatives: using /usr/bin/x86_64-w64-mingw32-g++-posix to provide /usr/bin/x86_64-w64-mingw32-g++ (x86_64-w64-mingw32-g++) in auto mode
2019-07-14T16:26:51.4951577Z update-alternatives: using /usr/bin/x86_64-w64-mingw32-g++-win32 to provide /usr/bin/x86_64-w64-mingw32-g++ (x86_64-w64-mingw32-g++) in auto mode
2019-07-14T16:26:51.5174156Z Setting up g++-mingw-w64 (5.3.1-8ubuntu3+17) ...
2019-07-14T16:26:51.5605359Z Setting up gcc-mingw-w64 (5.3.1-8ubuntu3+17) ...
2019-07-14T16:26:51.6073177Z Setting up mingw-w64 (4.0.4-2) ...
2019-07-14T16:26:51.6958699Z Processing triggers for ca-certificates (20170717~16.04.2) ...
2019-07-14T16:26:51.7106910Z Updating certificates in /etc/ssl/certs...
2019-07-14T16:26:53.0777072Z 148 added, 0 removed; done.
2019-07-14T16:26:53.0778508Z Running hooks in /etc/ca-certificates/update.d...
---
2019-07-14T16:27:30.3989607Z Removing intermediate container 00448ceaff53
2019-07-14T16:27:30.3990962Z  ---> 707c1ec03720
2019-07-14T16:27:30.4034525Z Successfully built 707c1ec03720
2019-07-14T16:27:30.5264548Z Successfully tagged rust-ci:latest
2019-07-14T16:27:30.6017511Z Built container sha256:707c1ec03720255934dc2d72d6d1df85cf1dba7a8472c14fa9fabfbd119df0f9
2019-07-14T16:27:30.6034399Z Uploading finished image to https://.s3.amazonaws.com/docker/a4940e6914a5e1f6360ebc241d0d850aff083be5c43d268fd716ce48f63cb24e84238becb662f27f8c5d7740413ca17da24bb9cdc003ef32d7b03d2c9052b94d
2019-07-14T16:28:52.3393694Z upload failed: - to s3:///docker/a4940e6914a5e1f6360ebc241d0d850aff083be5c43d268fd716ce48f63cb24e84238becb662f27f8c5d7740413ca17da24bb9cdc003ef32d7b03d2c9052b94d Parameter validation failed:
2019-07-14T16:28:52.3395213Z Invalid bucket name "": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$"
2019-07-14T16:28:53.2416553Z [CI_JOB_NAME=mingw-check]
2019-07-14T16:28:53.2464499Z Starting sccache server...
2019-07-14T16:28:53.2970386Z configure: processing command line
2019-07-14T16:28:53.2970785Z configure: 
---
2019-07-14T16:33:20.7184886Z     Checking syntax_ext v0.0.0 (/checkout/src/libsyntax_ext)
2019-07-14T16:34:05.3262851Z     Checking rustc_allocator v0.0.0 (/checkout/src/librustc_allocator)
2019-07-14T16:34:05.5811329Z     Checking rustc_mir v0.0.0 (/checkout/src/librustc_mir)
2019-07-14T16:34:06.7612472Z     Checking rustc_typeck v0.0.0 (/checkout/src/librustc_typeck)
2019-07-14T16:34:13.9185862Z error[E0027]: pattern does not mention field `size`
2019-07-14T16:34:13.9189057Z    --> src/librustc_mir/interpret/snapshot.rs:289:13
2019-07-14T16:34:13.9189792Z     |
2019-07-14T16:34:13.9190584Z 289 |         let Allocation { alloc, relocations, align, mutability, extra: () } = self;
2019-07-14T16:34:13.9191420Z     |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ missing field `size`
2019-07-14T16:34:13.9199638Z error[E0308]: mismatched types
2019-07-14T16:34:13.9200419Z    --> src/librustc_mir/interpret/snapshot.rs:290:13
2019-07-14T16:34:13.9201094Z     |
2019-07-14T16:34:13.9201094Z     |
2019-07-14T16:34:13.9201811Z 290 |         let AllocationBytes { bytes, undef_mask } = alloc;
2019-07-14T16:34:13.9203007Z     |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected enum `std::option::Option`, found struct `rustc::mir::interpret::AllocationBytes`
2019-07-14T16:34:13.9203569Z     |
2019-07-14T16:34:13.9204247Z     = note: expected type `std::option::Option<rustc::mir::interpret::AllocationBytes>`
2019-07-14T16:34:13.9204825Z                found type `rustc::mir::interpret::AllocationBytes`
2019-07-14T16:34:14.1352568Z     Checking rustc_lint v0.0.0 (/checkout/src/librustc_lint)
2019-07-14T16:34:15.2558203Z error: aborting due to 2 previous errors
2019-07-14T16:34:15.2559751Z 
2019-07-14T16:34:15.2560470Z Some errors have detailed explanations: E0027, E0308.
2019-07-14T16:34:15.2560470Z Some errors have detailed explanations: E0027, E0308.
2019-07-14T16:34:15.2564726Z For more information about an error, try `rustc --explain E0027`.
2019-07-14T16:34:15.3202516Z error: Could not compile `rustc_mir`.
2019-07-14T16:34:15.3203050Z warning: build failed, waiting for other jobs to finish...
2019-07-14T16:34:15.5603752Z error: build failed
2019-07-14T16:34:15.5625217Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "check" "--target" "x86_64-unknown-linux-gnu" "-j" "2" "--release" "--color" "always" "--features" "" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "--message-format" "json"
2019-07-14T16:34:15.5632602Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap check
2019-07-14T16:34:15.5632911Z Build completed unsuccessfully in 0:05:22
2019-07-14T16:34:15.5632911Z Build completed unsuccessfully in 0:05:22
2019-07-14T16:34:16.6832284Z ##[error]Bash exited with code '1'.
2019-07-14T16:34:16.6859888Z ##[section]Starting: Checkout
2019-07-14T16:34:16.6861519Z ==============================================================================
2019-07-14T16:34:16.6861575Z Task         : Get sources
2019-07-14T16:34:16.6861623Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@estebank

This comment has been minimized.

Copy link
Contributor

commented Jul 14, 2019

@rust-highfive rust-highfive assigned oli-obk and unassigned estebank Jul 14, 2019

/// Note that the bytes of a pointer represent the offset of the pointer
pub bytes: Vec<u8>,
/// The byte allocation and definedness.
/// Bytes of fully undefined allocations are allocated lazily on potentially mutable changes.

This comment has been minimized.

Copy link
@RalfJung

RalfJung Jul 16, 2019

Member

"potentially mutable changes"?

This comment has been minimized.

Copy link
@HeroicKatora

HeroicKatora Jul 16, 2019

Author Contributor

It's always allocated within get_bytes_mut but I hadn't realized that the method also marks the bytes as defined when I had written that comment. Good catch.

/// The actual bytes of the allocation.
/// Note that the bytes of a pointer represent the offset of the pointer
pub bytes: Vec<u8>,
/// Denotes undefined memory. Reading from undefined memory is forbidden in miri

This comment has been minimized.

Copy link
@RalfJung

RalfJung Jul 16, 2019

Member

This is not true, memcpy of undefined memory is allowed. Also the important part is that it's forbidden in Rust.

I think we should state this positively: "Denotes which part of this allocation is initialized". Maybe it should even be renamed to InitMask?

This comment has been minimized.

Copy link
@HeroicKatora

HeroicKatora Jul 16, 2019

Author Contributor

This was meant as 'interpreting undefined memory as a value of some type' is forbidden, i.e. not even integral values such as u32 are allowed to be uninitialized. Can you propose a clearer alternative documentation?

This comment has been minimized.

Copy link
@RalfJung

RalfJung Jul 16, 2019

Member

I did: "Denotes which part of this allocation is initialized". I wouldn't talk about what exact consequences that has as those rules are changing.

If you really want you can link to https://www.ralfj.de/blog/2019/07/14/uninit.html for an explanation why explicit tracking is needed. ;)

This comment has been minimized.

Copy link
@HeroicKatora

HeroicKatora Jul 16, 2019

Author Contributor

Ah, so you would simply leave out mentioning the effects in miri as one should be able to infer that from the state. That sounds well, it just escaped me that this was your proposed replacement. I also read your blogpost, a (slightly unfortunate) naming difference of undef in llvm and here with undef_mask.

This comment has been minimized.

Copy link
@RalfJung

RalfJung Jul 16, 2019

Member

Actually I think it is an unfortunate naming alignment because LLVM's undef is different in subtle ways from what we are doing. Which is why I propose to talk about the "initialized mask" instead.

///
/// Interpreter computation should not depend on the content of these bytes and caller has the
/// ensure the range is valid.
pub fn get_raw_bytes(&self, range: Range<usize>) -> BytesOrElidedUndef<'_> {

This comment has been minimized.

Copy link
@RalfJung

RalfJung Jul 16, 2019

Member

What is "the logical part of the allocation"? Inf act I don't understand any part of this doc comment.

This comment has been minimized.

Copy link
@HeroicKatora

HeroicKatora Jul 18, 2019

Author Contributor

I wanted to say that this ignores the UndefMask, relocations, and range checks. It probably should have said exactly that:

    /// Directly access the underlying bytes of an allocation.
    ///
    /// This ignores the undef mask, relocations, and requires the caller to have made proper
    /// bounds checks.
@estebank

This comment has been minimized.

Copy link
Contributor

commented Jul 16, 2019

@rust-highfive rust-highfive assigned RalfJung and unassigned oli-obk Jul 16, 2019

@RalfJung

This comment has been minimized.

Copy link
Member

commented Jul 16, 2019

I think (hope, really) that @oli-obk and me will both review this.^^ Not sure why you switched us around here.

I also still think it should be investigated why the test program that triggered this consumes 20GB as opposed to 2. There seems to be something deeply wrong somewhere and we shouldn't just paper over that.

@HeroicKatora

This comment has been minimized.

Copy link
Contributor Author

commented Jul 16, 2019

I also still think it should be investigated why the test program that triggered this consumes 20GB as opposed to 2. There seems to be something deeply wrong somewhere and we shouldn't just paper over that.

Yes, this might not fix it entirely. The resulting constant in the program is not a MaybeUninit alone but a struct containing a AtomicUsize and a MaybeUninit where the former is initialized and thus the value of the struct type is surely not affected by the optimization (but the intermediate value of a MaybeUninit::uninit() should be, and drop overall memory usage).

@RalfJung

This comment has been minimized.

Copy link
Member

commented Jul 16, 2019

Yes, this will definitely not fix it entirely as the resulting constant in it is not a MaybeUninit alone but a struct containing a AtomicUsize and a MaybeUninit where the former is initialized and thus the value of the struct type is surely not affected by the optimization.

To land this we should definitely have benchmarks that it helps, at least. Otherwise it seems pointless.

/// Raw accessors. Provide compatibility through lazy allocation of undef.
impl<Tag, Extra> Allocation<Tag, Extra> {
pub fn len(&self) -> usize {
self.alloc.as_ref()

This comment has been minimized.

Copy link
@oli-obk

oli-obk Jul 18, 2019

Contributor

you can just make this function body self.size.bytes() as usize, the alloc.bytes debug check is just a sanity check after all

@oli-obk

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2019

I don't think we need a benchmark. Simply having a compile-pass test with static FOO: MaybeUninit<[u8; 1<<50]> = MaybeUninit::uninit(); should be enough. If we ever break it, tests will fail, barring someone having a gazillion bytes of RAM in 20 years

@RalfJung

This comment has been minimized.

Copy link
Member

commented Jul 18, 2019

a compile-pass test

build-pass, you mean. :)

@HeroicKatora

This comment has been minimized.

Copy link
Contributor Author

commented Jul 18, 2019

I'm going to rename a few parts based on feedback above:

  • AllocationContent instead of AllocationBytes
  • Allocation::alloc_content instead of Allocation::alloc
  • AllocationBytes<'a> instead of BytesOrElidedUndef (which would still be applicable for constant bytes)
  • AllocationBytes::Uninit instead of ElidedUndef
  • get_or_allocate_content instead of force_allocation
@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Jul 18, 2019

The job mingw-check of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-07-18T23:11:31.9850321Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-07-18T23:11:32.0028838Z ##[command]git config gc.auto 0
2019-07-18T23:11:32.0116765Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-07-18T23:11:32.0168985Z ##[command]git config --get-all http.proxy
2019-07-18T23:11:32.0295810Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/62655/merge:refs/remotes/pull/62655/merge
---
2019-07-18T23:12:06.4690489Z do so (now or later) by using -b with the checkout command again. Example:
2019-07-18T23:12:06.4690534Z 
2019-07-18T23:12:06.4690718Z   git checkout -b <new-branch-name>
2019-07-18T23:12:06.4690744Z 
2019-07-18T23:12:06.4690786Z HEAD is now at d036a3f38 Merge 3a94e1a3faf0aea0cb2cd79ace82e60b5bb5805b into 311376d30dc1cfa622142a9f50317b1e0cb4608a
2019-07-18T23:12:06.4828962Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-07-18T23:12:06.4831765Z ==============================================================================
2019-07-18T23:12:06.4831812Z Task         : Bash
2019-07-18T23:12:06.4831850Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-07-18T23:20:56.4060964Z    Compiling rand_chacha v0.1.0
2019-07-18T23:20:56.7306233Z    Compiling rand v0.6.1
2019-07-18T23:20:56.8169733Z    Compiling rustc_llvm v0.0.0 (/checkout/src/librustc_llvm)
2019-07-18T23:20:59.2046546Z     Checking tempfile v3.0.5
2019-07-18T23:21:00.7172997Z error[E0609]: no field `alloc` on type `&rustc::mir::interpret::Allocation`
2019-07-18T23:21:00.7177899Z    |
2019-07-18T23:21:00.7177899Z    |
2019-07-18T23:21:00.7178746Z 38 |             llvals.push(cx.const_bytes(&alloc.alloc.bytes[next_offset..offset]));
2019-07-18T23:21:00.7179537Z 
2019-07-18T23:21:00.7179537Z 
2019-07-18T23:21:00.7180210Z error[E0609]: no field `alloc` on type `&rustc::mir::interpret::Allocation`
2019-07-18T23:21:00.7181400Z    |
2019-07-18T23:21:00.7181400Z    |
2019-07-18T23:21:00.7181859Z 42 |             &alloc.alloc.bytes[offset..(offset + pointer_size)],
2019-07-18T23:21:00.7182506Z 
2019-07-18T23:21:00.7182506Z 
2019-07-18T23:21:00.7194504Z error[E0609]: no field `alloc` on type `&rustc::mir::interpret::Allocation`
2019-07-18T23:21:00.7196600Z    |
2019-07-18T23:21:00.7196600Z    |
2019-07-18T23:21:00.7197155Z 54 |     if alloc.alloc.bytes.len() >= next_offset {
2019-07-18T23:21:00.7197996Z 
2019-07-18T23:21:00.7197996Z 
2019-07-18T23:21:00.7204785Z error[E0609]: no field `alloc` on type `&rustc::mir::interpret::Allocation`
2019-07-18T23:21:00.7206685Z    |
2019-07-18T23:21:00.7206685Z    |
2019-07-18T23:21:00.7207276Z 55 |         llvals.push(cx.const_bytes(&alloc.alloc.bytes[next_offset ..]));
2019-07-18T23:21:00.7208071Z 
2019-07-18T23:21:00.7208071Z 
2019-07-18T23:21:00.7483351Z error[E0609]: no field `alloc` on type `&rustc::mir::interpret::Allocation`
2019-07-18T23:21:00.7484743Z     |
2019-07-18T23:21:00.7484743Z     |
2019-07-18T23:21:00.7485752Z 440 |                     let sect_name = if alloc.alloc.bytes.iter().all(|b| *b == 0) {
2019-07-18T23:21:00.7487142Z 
2019-07-18T23:21:00.7487142Z 
2019-07-18T23:21:00.7500257Z error[E0609]: no field `alloc` on type `&rustc::mir::interpret::Allocation`
2019-07-18T23:21:00.7501386Z     |
2019-07-18T23:21:00.7501386Z     |
2019-07-18T23:21:00.7502197Z 461 |                         alloc.alloc.bytes.as_ptr() as *const _,
2019-07-18T23:21:00.7506032Z 
2019-07-18T23:21:00.7506032Z 
2019-07-18T23:21:00.7506742Z error[E0609]: no field `alloc` on type `&rustc::mir::interpret::Allocation`
2019-07-18T23:21:00.7508018Z     |
2019-07-18T23:21:00.7508018Z     |
2019-07-18T23:21:00.7508515Z 462 |                         alloc.alloc.bytes.len() as c_uint,
2019-07-18T23:21:00.7509590Z 
2019-07-18T23:21:02.0518642Z error: aborting due to 7 previous errors
2019-07-18T23:21:02.0518830Z 
2019-07-18T23:21:02.0519432Z For more information about this error, try `rustc --explain E0609`.
2019-07-18T23:21:02.0519432Z For more information about this error, try `rustc --explain E0609`.
2019-07-18T23:21:02.0802094Z error: Could not compile `rustc_codegen_llvm`.
2019-07-18T23:21:02.0802197Z 
2019-07-18T23:21:02.0802433Z To learn more, run the command again with --verbose.
2019-07-18T23:21:02.0822827Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "check" "--target" "x86_64-unknown-linux-gnu" "-j" "2" "--release" "--color" "always" "--manifest-path" "/checkout/src/librustc_codegen_llvm/Cargo.toml" "--message-format" "json"
2019-07-18T23:21:02.0837840Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap check
2019-07-18T23:21:02.0839177Z Build completed unsuccessfully in 0:05:46
2019-07-18T23:21:02.0839177Z Build completed unsuccessfully in 0:05:46
2019-07-18T23:21:03.8561241Z ##[error]Bash exited with code '1'.
2019-07-18T23:21:03.8593942Z ##[section]Starting: Checkout
2019-07-18T23:21:03.8595962Z ==============================================================================
2019-07-18T23:21:03.8596020Z Task         : Get sources
2019-07-18T23:21:03.8596069Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Jul 19, 2019

The job mingw-check of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-07-19T15:10:26.2267721Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-07-19T15:10:26.2465544Z ##[command]git config gc.auto 0
2019-07-19T15:10:26.2514559Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-07-19T15:10:26.2572076Z ##[command]git config --get-all http.proxy
2019-07-19T15:10:26.2702929Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/62655/merge:refs/remotes/pull/62655/merge
---
2019-07-19T15:11:00.6313254Z do so (now or later) by using -b with the checkout command again. Example:
2019-07-19T15:11:00.6313294Z 
2019-07-19T15:11:00.6313462Z   git checkout -b <new-branch-name>
2019-07-19T15:11:00.6313483Z 
2019-07-19T15:11:00.6313519Z HEAD is now at 54c665fbc Merge b1159d72ee78d1abed86eef8589fb61de27696e1 into 527dce7137f7a3c7bf47d9a503abf25f88ea22de
2019-07-19T15:11:00.6442404Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-07-19T15:11:00.6445159Z ==============================================================================
2019-07-19T15:11:00.6445222Z Task         : Bash
2019-07-19T15:11:00.6445287Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-07-19T15:19:47.4063604Z    Compiling rand_pcg v0.1.1
2019-07-19T15:19:47.8115367Z    Compiling rand v0.6.1
2019-07-19T15:19:49.2477925Z    Compiling rustc_llvm v0.0.0 (/checkout/src/librustc_llvm)
2019-07-19T15:19:50.4229458Z     Checking tempfile v3.0.5
2019-07-19T15:19:52.6329369Z error[E0615]: attempted to take value of method `relocations` on type `&rustc::mir::interpret::Allocation`
2019-07-19T15:19:52.6329881Z    |
2019-07-19T15:19:52.6329881Z    |
2019-07-19T15:19:52.6330153Z 28 |     let mut llvals = Vec::with_capacity(alloc.relocations.len() + 1);
2019-07-19T15:19:52.6330847Z    |                                               ^^^^^^^^^^^ help: use parentheses to call the method: `relocations(...)`
2019-07-19T15:19:52.6330893Z 
2019-07-19T15:19:52.6331190Z error[E0615]: attempted to take value of method `relocations` on type `&rustc::mir::interpret::Allocation`
2019-07-19T15:19:52.6331649Z    |
2019-07-19T15:19:52.6331649Z    |
2019-07-19T15:19:52.6331922Z 33 |     for &(offset, ((), alloc_id)) in alloc.relocations.iter() {
2019-07-19T15:19:52.6332245Z    |                                            ^^^^^^^^^^^ help: use parentheses to call the method: `relocations(...)`
2019-07-19T15:19:52.6332311Z 
2019-07-19T15:19:52.6876269Z error[E0609]: no field `alloc` on type `&rustc::mir::interpret::Allocation`
2019-07-19T15:19:52.6877901Z    |
2019-07-19T15:19:52.6877901Z    |
2019-07-19T15:19:52.6878575Z 38 |             llvals.push(cx.const_bytes(&alloc.alloc.bytes[next_offset..offset]));
2019-07-19T15:19:52.6879594Z 
2019-07-19T15:19:52.6879594Z 
2019-07-19T15:19:52.6885908Z error[E0609]: no field `alloc` on type `&rustc::mir::interpret::Allocation`
2019-07-19T15:19:52.6886997Z    |
2019-07-19T15:19:52.6886997Z    |
2019-07-19T15:19:52.6887519Z 42 |             &alloc.alloc.bytes[offset..(offset + pointer_size)],
2019-07-19T15:19:52.6888533Z 
2019-07-19T15:19:52.6888533Z 
2019-07-19T15:19:52.6906478Z error[E0609]: no field `alloc` on type `&rustc::mir::interpret::Allocation`
2019-07-19T15:19:52.6907637Z    |
2019-07-19T15:19:52.6907637Z    |
2019-07-19T15:19:52.6908417Z 54 |     if alloc.alloc.bytes.len() >= next_offset {
2019-07-19T15:19:52.6908952Z 
2019-07-19T15:19:52.6908952Z 
2019-07-19T15:19:52.6915970Z error[E0609]: no field `alloc` on type `&rustc::mir::interpret::Allocation`
2019-07-19T15:19:52.6917058Z    |
2019-07-19T15:19:52.6917058Z    |
2019-07-19T15:19:52.6917622Z 55 |         llvals.push(cx.const_bytes(&alloc.alloc.bytes[next_offset ..]));
2019-07-19T15:19:52.6918460Z 
2019-07-19T15:19:52.6918460Z 
2019-07-19T15:19:52.7190025Z error[E0609]: no field `alloc` on type `&rustc::mir::interpret::Allocation`
2019-07-19T15:19:52.7191455Z     |
2019-07-19T15:19:52.7191455Z     |
2019-07-19T15:19:52.7191933Z 440 |                     let sect_name = if alloc.alloc.bytes.iter().all(|b| *b == 0) {
2019-07-19T15:19:52.7192792Z 
2019-07-19T15:19:52.7192792Z 
2019-07-19T15:19:52.7205003Z error[E0609]: no field `alloc` on type `&rustc::mir::interpret::Allocation`
2019-07-19T15:19:52.7206165Z     |
2019-07-19T15:19:52.7206165Z     |
2019-07-19T15:19:52.7206684Z 461 |                         alloc.alloc.bytes.as_ptr() as *const _,
2019-07-19T15:19:52.7207343Z 
2019-07-19T15:19:52.7207343Z 
2019-07-19T15:19:52.7211241Z error[E0609]: no field `alloc` on type `&rustc::mir::interpret::Allocation`
2019-07-19T15:19:52.7212495Z     |
2019-07-19T15:19:52.7212495Z     |
2019-07-19T15:19:52.7212921Z 462 |                         alloc.alloc.bytes.len() as c_uint,
2019-07-19T15:19:52.7214102Z 
2019-07-19T15:19:54.0718804Z error: aborting due to 9 previous errors
2019-07-19T15:19:54.0718951Z 
2019-07-19T15:19:54.0719204Z Some errors have detailed explanations: E0609, E0615.
2019-07-19T15:19:54.0719204Z Some errors have detailed explanations: E0609, E0615.
2019-07-19T15:19:54.0719433Z For more information about an error, try `rustc --explain E0609`.
2019-07-19T15:19:54.0998693Z error: Could not compile `rustc_codegen_llvm`.
2019-07-19T15:19:54.0998790Z 
2019-07-19T15:19:54.0999025Z To learn more, run the command again with --verbose.
2019-07-19T15:19:54.1022570Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "check" "--target" "x86_64-unknown-linux-gnu" "-j" "2" "--release" "--color" "always" "--manifest-path" "/checkout/src/librustc_codegen_llvm/Cargo.toml" "--message-format" "json"
2019-07-19T15:19:54.1037381Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap check
2019-07-19T15:19:54.1037455Z Build completed unsuccessfully in 0:05:47
2019-07-19T15:19:54.1037455Z Build completed unsuccessfully in 0:05:47
2019-07-19T15:19:55.7290461Z ##[error]Bash exited with code '1'.
2019-07-19T15:19:55.7320916Z ##[section]Starting: Checkout
2019-07-19T15:19:55.7322928Z ==============================================================================
2019-07-19T15:19:55.7322975Z Task         : Get sources
2019-07-19T15:19:55.7323017Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@oli-obk

This comment has been minimized.

Copy link
Contributor

commented Jul 24, 2019

one general note: @eddyb asked whether we could just grow the bytes-vector and defined bitmask as needed. I believe now that we have the explicit size field, we could do that. So instead of having an Option wrapper around these fields, we'd grow the fields as needed and start out with them being empty. All accesses need to go through the size field first to check whether we're accessing something that is in the Allocation, but has no backing memory

@HeroicKatora

This comment has been minimized.

Copy link
Contributor Author

commented Jul 24, 2019

I've wondered the same thing but somewhat unintuitively it seemed much more involved, even though it would not require an AllocationContent. But the Option<_> wrapping does a good job of tracking the initialization state even where other code tries to access the bytes of the allocation directly without going through one of the accessors—be it mutable or immutable. The above .bss-segment question is a very good example for this. It would require a good lot of care to avoid mistakenly accessing the bytes outside the allocation size. This also affects interaction with relocations and I feel like it it's not straightforward at all and might considerably widen the scope of the PR.

RalfJung referenced this pull request Jul 24, 2019

@RalfJung

This comment has been minimized.

Copy link
Member

commented Jul 24, 2019

But the Option<_> wrapping does a good job of tracking the initialization state even where other code tries to access the bytes of the allocation directly without going through one of the accessors—be it mutable or immutable. The above .bss-segment question is a very good example for this. It would require a good lot of care to avoid mistakenly accessing the bytes outside the allocation size.

I agree. But either way, eventually bytes and some other fields should become private, and then such a "prefix" representation should work well.

It would probably be easier to first make the field private and then refactor the internal representation, but now that you already did the refactoring -- maybe it is worth doing the "privatization" at the same time?

@HeroicKatora

This comment has been minimized.

Copy link
Contributor Author

commented Jul 24, 2019

It would probably be easier to first make the field private and then refactor the internal representation, but now that you already did the refactoring -- maybe it is worth doing the "privatization" at the same time?

With a completely private field and better accessors the vector of dynamic size seems more reasonable than the Option<_> though, since @oli-obk is completely right that the size-field does a good enough job of tracking the initialization state as well. So I'd remove the option (but definitely keep AllocationBytes.) At least the option made it more obvious how and where to handle the uninitialized cases 😅

I may have been on the pessimistic side with the assessment of the work required for that. Maybe.

@bors

This comment has been minimized.

Copy link
Contributor

commented Jul 25, 2019

☔️ The latest upstream changes (presumably #62961) made this pull request unmergeable. Please resolve the merge conflicts.

@HeroicKatora HeroicKatora force-pushed the HeroicKatora:elide-uninit-static branch from aed2116 to 71e89e4 Jul 26, 2019

@bors

This comment has been minimized.

Copy link
Contributor

commented Jul 27, 2019

☔️ The latest upstream changes (presumably #63043) made this pull request unmergeable. Please resolve the merge conflicts.

HeroicKatora added some commits Jul 12, 2019

Manage byte allocation and definedness separately
Prepares eliding fully undefined allocations as long as they are not
written to. Reads of such uninitialized bytes should not occur or be
replaced by special handling at the call site.
Move actual allocation into an option
Part of a rework to elide allocating undefined bytes: Allocation is able
to avoid allocating memory (for bytes and the UndefMap) while exactly
all bytes are in an initial undefined state. Any mutable access to
either component will force an allocation. Various parts of interpret
and codegen bypass the accessors so a few extra utilities are added to
ease the transition to consideration for the new logical state.
Restore previous src-dest order in memory
Since this no longer requires the mutable access and thus allocation of
the destination undef bytes this change also avoids writing the undef
mask if the destination is also fully undef.
Give better names to new types and members
Keep names as globally unique as possible. Also avoids naming
implementation details.
Move relocations into AllocationContent
Relocations only make sense if at least some part is considered
initialized.
Codegen for undef statics: create undef bytes
Specialize the code path for fully undef allocations. The existing const
initialized now works precisely with AllocationContent instead of the
full Allocation after having moved relocations. When there is not
allocation content then there are no bytes to consider and instead
create a const array of undef bytes.
Cleanup tidy checks
Only myself to blame for forgetting to run these in the first place.

@HeroicKatora HeroicKatora force-pushed the HeroicKatora:elide-uninit-static branch from 71e89e4 to 8f91c98 Jul 29, 2019

@bors

This comment has been minimized.

Copy link
Contributor

commented Aug 2, 2019

☔️ The latest upstream changes (presumably #63214) made this pull request unmergeable. Please resolve the merge conflicts.

@JohnCSimon

This comment has been minimized.

Copy link

commented Aug 10, 2019

Ping from triage @oli-obk @HeroicKatora this PR has sat idle for over a week. Looks like there are merge conflicts to be resolved. Thanks!

@oli-obk
Copy link
Contributor

left a comment

Please comment when you force push, I don't get a notification on force pushes

alloc_content.clear_relocations(cx, ptr, size);
}

Ok(())

This comment has been minimized.

Copy link
@oli-obk

oli-obk Aug 10, 2019

Contributor

Remove the OK and make the function not have a return type

@@ -437,7 +451,7 @@ impl StaticMethods for CodegenCx<'ll, 'tcx> {
//
// We could remove this hack whenever we decide to drop macOS 10.10 support.
if self.tcx.sess.target.target.options.is_like_osx {
let sect_name = if alloc.bytes.iter().all(|b| *b == 0) {
let sect_name = if alloc.alloc_content.is_none() {

This comment has been minimized.

Copy link
@oli-obk

oli-obk Aug 10, 2019

Contributor

You need to check for all zeros in the defined case, too

@@ -451,15 +465,20 @@ impl StaticMethods for CodegenCx<'ll, 'tcx> {
// go into custom sections of the wasm executable.
if self.tcx.sess.opts.target_triple.triple().starts_with("wasm32") {
if let Some(section) = attrs.link_section {
let alloc_content = match &alloc.alloc_content {
// FIXME: This is not really a bug but makes it testable except on wasm
None => bug!(),

This comment has been minimized.

Copy link
@oli-obk

oli-obk Aug 10, 2019

Contributor

You need to impl the fallback

@HeroicKatora

This comment has been minimized.

Copy link
Contributor Author

commented Aug 10, 2019

I put this off a bit, sorry. I'm now skeptical if this is the best way of achieving the goal. Since it broke a few times it seems to be far more intrusive of a change than (I had) originally expected. I also wonder if it would be desirable to first make bytes private without changing their allocation behaviour and provide the actual optimization in a second PR. This might satisfy @eddyb 's request of partial allocation better as well and reduce overhead by eliminating the need for AllocationContent completely.. Not sure?

@RalfJung

This comment has been minimized.

Copy link
Member

commented Aug 10, 2019

I also wonder if it would be desirable to first make bytes private without changing their allocation behaviour and provide the actual optimization in a second PR.

Sounds reasonable to me.

@chocol4te

This comment has been minimized.

Copy link
Contributor

commented Aug 20, 2019

Ping from triage! @HeroicKatora @oli-obk

Label is waiting-on-review, but perhaps more discussion is needed?

@HeroicKatora

This comment has been minimized.

Copy link
Contributor Author

commented Aug 20, 2019

The changes here have evolved into a larger undertaking and one main portion of it was split into another PR (#63561). This one should maybe be marked as blocked since rebasing the changes here depends on landing the other first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
9 participants
You can’t perform that action at this time.