Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v8: load_objects_from_file() using utf-8 #30218

Closed

Conversation

@cclauss
Copy link
Contributor

cclauss commented Nov 2, 2019

This PR uses io.open(encoding="utf-8") to resolve a UnicodeDecodeError reported at nodejs/code-and-learn#99 (comment)

This can be replicated by running docker build . using the following:

Dockerfile
FROM ubuntu:bionic

# Install tools
RUN apt-get update
RUN apt-get install -y build-essential python3 python3-pip ninja-build make g++ vim git sudo

# Expose ports used by net tests
ENV NODE_COMMON_PORT=12346
EXPOSE 12346
# debug port used by v8 debugger
EXPOSE 5858
# debug port used by v8 insepctor
EXPOSE 9229

# Set up user node-dev: the tests fail when run by the root
RUN adduser --disabled-password --gecos '' node-dev
RUN adduser node-dev sudo
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER node-dev
WORKDIR /home/node-dev
ENV HOME /home/node-dev

# Clone the code and build
RUN git clone --depth=1 https://github.com/nodejs/node.git
WORKDIR /home/node-dev/node
RUN python3 -m pip install setuptools
RUN python3 ./configure --ninja
ENV BUILD_WITH ninja
RUN make -j2 test
Checklist
@cclauss cclauss mentioned this pull request Nov 2, 2019
3 of 3 tasks complete
@nodejs-github-bot

This comment has been minimized.

@targos

This comment has been minimized.

Copy link
Member

targos commented Nov 2, 2019

do you want me to upstream the change?

@cclauss

This comment has been minimized.

Copy link
Contributor Author

cclauss commented Nov 2, 2019

Yes, please. Should we wait for our Jenkins tests to pass first? Your call.

It works on the Dockerfile when I do a COPY gen-postmortem-metadata.py ./deps/v8/tools/ with these mods.

@targos

This comment has been minimized.

Copy link
Member

targos commented Nov 2, 2019

@cclauss

This comment has been minimized.

Copy link
Contributor Author

cclauss commented Nov 3, 2019

Nice! Any hints on how to get that arm-fanned to pass?

@targos

This comment has been minimized.

Copy link
Member

targos commented Nov 3, 2019

CL landed. I'll update this to be a cherry pick

@cjihrig
cjihrig approved these changes Nov 3, 2019
Copy link
Member

legendecas left a comment

LGTM

deps/v8/tools/gen-postmortem-metadata.py Outdated Show resolved Hide resolved
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd
@targos targos force-pushed the cclauss:fix-gen-postmortem-metadata.py branch from f4122ca to c21d28e Nov 3, 2019
@nodejs-github-bot

This comment has been minimized.

@cclauss

This comment has been minimized.

Copy link
Contributor Author

cclauss commented Nov 4, 2019

Landed in c80771d

@cclauss cclauss closed this Nov 4, 2019
@cclauss cclauss deleted the cclauss:fix-gen-postmortem-metadata.py branch Nov 4, 2019
cclauss added a commit that referenced this pull request Nov 4, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

PR-URL: #30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos added a commit to targos/node that referenced this pull request Nov 4, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

PR-URL: nodejs#30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos added a commit that referenced this pull request Nov 5, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

PR-URL: #30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
@targos targos mentioned this pull request Nov 5, 2019
addaleax added a commit to targos/node that referenced this pull request Nov 5, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

PR-URL: nodejs#30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos added a commit that referenced this pull request Nov 8, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

PR-URL: #30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos added a commit to targos/node that referenced this pull request Nov 8, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

PR-URL: nodejs#30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos added a commit to targos/node that referenced this pull request Nov 8, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

PR-URL: nodejs#30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos added a commit that referenced this pull request Nov 10, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

PR-URL: #30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos added a commit that referenced this pull request Nov 10, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

PR-URL: #30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos added a commit that referenced this pull request Nov 11, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

PR-URL: #30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos added a commit to targos/node that referenced this pull request Nov 17, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

PR-URL: nodejs#30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
MylesBorins added a commit that referenced this pull request Nov 21, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

Backport-PR-URL: #30513
PR-URL: #30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
@BridgeAR BridgeAR mentioned this pull request Nov 21, 2019
targos added a commit to targos/node that referenced this pull request Dec 5, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

PR-URL: nodejs#30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos added a commit to targos/node that referenced this pull request Dec 5, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

PR-URL: nodejs#30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
targos added a commit to targos/node that referenced this pull request Dec 6, 2019
Original commit message:

    [postmortem] Load files using utf-8 to support Python 3

    Change-Id: I174d38cc33210c07d1a7596627e1b2d21bb06313
    Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1895560
    Reviewed-by: Mathias Bynens <mathias@chromium.org>
    Commit-Queue: Michaël Zasso <mic.besace@gmail.com>
    Cr-Commit-Position: refs/heads/master@{#64717}

Refs: v8/v8@a7dffcd

PR-URL: nodejs#30218
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.