Dev env setup script #5717

BenWilson2 · 2022-04-18T19:49:53Z

What changes are proposed in this pull request?

Add a development environment setup script for Python dev

How is this patch tested?

Manually

Does this PR change the documentation?

No. You can skip the rest of this section.
Yes. Make sure the changed pages / sections render correctly by following the steps below.

Check the status of the ci/circleci: build_doc check. If it's successful, proceed to the
next step, otherwise fix it.
Click Details on the right to open the job page of CircleCI.
Click the Artifacts tab.
Click docs/build/html/index.html.
Find the changed pages / sections and make sure they render correctly.

Release Notes

Is this a user-facing change?

No. You can skip the rest of this section.
Yes. Give a description of this change to be included in the release notes for MLflow users.

Add a Development Environment setup script for automated construction of a CI-friendly Python virtual environment.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

Interface

area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
area/windows: Windows support

Language

language/r: R APIs and clients
language/java: Java APIs and clients
language/new: Proposals for new client languages

Integrations

integrations/azure: Azure and Azure ML integrations
integrations/sagemaker: SageMaker integrations
integrations/databricks: Databricks integrations

How should the PR be classified in the release notes? Choose one:

rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
rn/feature - A new user-facing feature worth mentioning in the release notes
rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
rn/documentation - A user-facing documentation change worth mentioning in the release notes

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

…v-setup-script Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

…v-setup-script Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

harupy · 2022-04-18T23:44:45Z

dev/dev-env-setup.sh

+    brew update
+    brew install pyenv


Does this script only support macOS?

great point. I'll add the pyenv setup for a few OS's.

Can we just provide the pyenv installation instructions and let contributors install pyenv by themselves?

I'm concerned about maintenance costs. If we support a few OS, I think we need to test the script on the supported operating systems.

harupy · 2022-04-19T00:30:01Z

dev/dev-env-setup.sh

+MLFLOW_HOME=$(pwd)
+
+# Get the minimum supported version from MLflow to ensure any feature development adheres to legacy Python versions
+min_py_version=$(grep "python_requires=" "$MLFLOW_HOME/setup.py" | grep -E -o "([0-9]{1,}\.)+[0-9]{1,}")


We might want to add a command to print out the minimum supported python version in setup.py.

Sorry I wasn't clear. I meant a command like ListDependencies:

mlflow/setup.py

Line 90 in f7a42be

class ListDependencies(distutils.cmd.Command):

added (but not pushed yet)

dev/dev-env-setup.sh

harupy · 2022-04-19T01:36:19Z

dev/dev-env-setup.sh

+rm -rf "$tmp_dir"
+
+# Install dev requirements and test plugin
+pip install "$( (( $quiet == 1 && $verbose == 0 )) && printf %s '-q' )" -r "$MLFLOW_HOME/requirements/dev-requirements.txt"


Core maintainers like us need all the extra dependencies, but most contributors don't, and installing all of them is difficult to succeed and takes a long time. Can we install a minimum set of dev dependencies by default?

Should we make the [extras] optional? Or:

Options:
--packages
"light" - skinny, small, lint
"full" - light + large, doc, extra-ml
"all" - dev-requirements + [extras]

WDYT?

harupy · 2022-04-19T03:13:34Z

dev/dev-env-setup.sh

@@ -0,0 +1,158 @@
+#!/usr/bin/env bash


Can we run this script in one of the github action workflows to ensure it works?

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

harupy · 2022-04-21T03:50:58Z

dev/dev-env-setup.sh

+  pip install $( (( quiet == 1 && verbose == 0 )) && printf %s '-q' ) -e "$MLFLOW_HOME/tests/resources//mlflow-test-plugin"
+  echo "Finished installing pip dependencies."
+else
+  pip install $( (( quiet == 1 && verbose == 0 )) && printf %s '-q' ) -r "$MLFLOW_HOME/requirements/skinny-requirements.txt"


I think we can remove this line because skinny-requirements.txt is a subset of small-requirements.txt.

harupy · 2022-04-21T03:50:58Z

dev/dev-env-setup.sh

+  pip install $( (( quiet == 1 && verbose == 0 )) && printf %s '-q' ) -e "$MLFLOW_HOME/tests/resources//mlflow-test-plugin"
+  echo "Finished installing pip dependencies."
+else
+  pip install $( (( quiet == 1 && verbose == 0 )) && printf %s '-q' ) -r "$MLFLOW_HOME/requirements/skinny-requirements.txt"


I think we can remove this line because skinny-requirements.txt is a subset of small-requirements.txt.

harupy · 2022-04-21T03:51:37Z

.github/workflows/dev-setup.yml

@@ -0,0 +1,90 @@
+name: Dev environment setup


Do we need to run this workflow in any PRs? or only when we change dev/dev-env-setup.sh?

I think we'd likely be running this on a weekly schedule. I really don't want to run this on PRs or during nightly builds since it's going to impact our PR feedback heavily for something that really shouldn't be changing very often at all.

I think we can do something similar to the cross-version test. Run this job weekly and when we update dev/dev-env-setup.sh.

harupy · 2022-04-21T04:02:40Z

.github/workflows/dev-setup.yml

+  run:
+    shell: bash --noprofile --norc -exo pipefail {0}
+
+jobs:


Let's use strategy.matrix to DRY the code:

mlflow/.github/workflows/js.yml

Lines 23 to 24 in 8b0bfc5

strategy:

matrix:

great idea!

harupy · 2022-04-21T04:03:04Z

.github/workflows/dev-setup.yml

+
+
+


Suggested change

harupy · 2022-04-21T04:05:49Z

dev/dev-env-setup.sh

+    deactivate
+    rm -rf "$directory"
+    echo "Virtual environment removed from '$directory'. Installing new instance."
+    pyenv exec virtualenv "$directory"


Does virtualenv automatically pick up the current python version?

it goes through a discovery process that, if the version is found on the local system, will by default create a symlink. Do you think we should disable this and do a copy mode to force a replica?

harupy · 2022-04-21T04:07:16Z

dev/dev-env-setup.sh

+    rm -rf "$directory"
+    echo "Virtual environment removed from '$directory'. Installing new instance."


Can we use could use --clear flag:

$ virtualenv --help ... --clear remove the destination directory if exist before starting (will overwrite files otherwise) (default: False)

harupy · 2022-04-21T04:10:28Z

dev/dev-env-setup.sh

+      echo "$PYENV_BIN" >> "$GITHUB_PATH"
+      echo "PYENV_ROOT=$PYENV_ROOT" >> "$GITHUB_ENV"


I think these two lines should be executed only in GitHub Actions workflows.

harupy · 2022-04-23T00:53:43Z

.github/workflows/dev-setup.yml

+        os: [ubuntu-latest, windows-latest, macos-latest]
+        include:
+          - os: ubuntu-latest
+            shell: bash


I think we can remove this because we set the workflow-level default shell (bash --noprofile --norc -exo pipefail {0}).

harupy · 2022-04-23T00:54:46Z

dev/dev-env-setup.sh

+      git clone --depth 1 https://github.com/pyenv/pyenv.git "$HOME/.pyenv"
+      PYENV_ROOT="$HOME/.pyenv"
+      PYENV_BIN="$PYENV_ROOT/bin"
+      if [ "$MLFLOW_DEV_ENV_CI_RUN" == 1 ]; then


Suggested change

if [ "$MLFLOW_DEV_ENV_CI_RUN" == 1 ]; then

if [ ! -z "$GITHUB_ACTIONS" ]; then

Can we use the GITHUB_ACTIONS environment variable instead?

harupy · 2022-04-23T00:55:40Z

and shall we support installing "node.js" in the env setup script ? node.js is used in mflow web server.

I'd say no because the number of contributions to MLflow UI is not large. We can add this later if there is a request.

harupy · 2022-04-23T01:06:18Z

dev/dev-env-setup.sh

+        wget -O ~/brew_install.sh https://raw.githubusercontent.com/Homebrew/install/master/install.sh
+        bash ~/brew_install.sh


Suggested change

wget -O ~/brew_install.sh https://raw.githubusercontent.com/Homebrew/install/master/install.sh

bash ~/brew_install.sh

bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

to avoid creating brew_install.sh. I took this command from https://brew.sh/.

thanks! Added!

dev/dev-env-setup.sh

WeichenXu123 · 2022-04-23T12:03:03Z

dev/dev-env-setup.sh

+esac
+
+# Install the Python version if it cannot be found
+pyenv install -s $PY_INSTALL_VERSION


I suggest use pyenv install -s -v $PY_INSTALL_VERSION
the -v make pyenv print compilation status to stdout which helps debugging when error happens.

and shall we add a "pyenv_root_dir" argument to allow user to specify where pyenv download the python tarballs to ?

the -v make pyenv print compilation status to stdout which helps debugging when error happens.

Have you tried running pyenv install -v? It's too verbose.

shall we add a "pyenv_root_dir" argument to allow user to specify where pyenv download the python tarballs to ?

I don't see a need for that argument . In what situation would that argument be useful?

Have you tried running pyenv install -v? It's too verbose

Yes it might be too verbose but it helps provides more clues when compiling failed.

I don't see a need for that argument . In what situation would that argument be useful?

E.g the case the system disk remaining capacity is not sufficient, we can specify the path to be on other disk. Just a minor suggestion.

Yes it might be too verbose but it helps provides more clues when compiling failed.

See https://github.com/pyenv/pyenv/blob/fab0082bd5cdda07f0bfdd69a9c676bc2d2906b3/plugins/python-build/bin/python-build#L135. pyenv stores build logs in a temporary file and prints out its path when build fails.

E.g the case the system disk remaining capacity is not sufficient, we can specify the path to be on other disk. Just a minor suggestion.

Can we just remove some files to free disk space?

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

…v-setup-script Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

harupy · 2022-05-02T02:20:33Z

dev/dev-env-setup.sh

+
+  Example usage:
+
+  From root of MLflow repository on local with a destination virtualenv path of <MLFLOW_HOME>/.venvs/mlflow-dev:


Suggested change

From root of MLflow repository on local with a destination virtualenv path of <MLFLOW_HOME>/.venvs/mlflow-dev:

From root of MLflow repository on local with a destination virtualenv path of <MLFLOW_HOME>/.venv:

Can we use .venv instead of .venvs/mlflow-dev-env?

https://docs.python.org/3/library/venv.html#creating-virtual-environments says:

... (a common name for the target directory is .venv).

I saw https://github.com/mlflow/mlflow/pull/5717/files#diff-76e7ca2f2843ac5f659b45382768c1b64bf8704e8d3c6189c4e3623f5d465bc2R162-R170. Now the current approach makes sense.

harupy · 2022-05-02T02:28:28Z

.github/workflows/dev-setup.yml

+          git config --global user.email "test@mlflow.org"
+      - name: Run Environment tests
+        run: |
+          TERM=xterm bash ./dev/test-dev-env-setup.sh


Is TERM=xterm only required on github actions?

yes. In a local terminal client there will be some defined terminal value to allow for syntactic formatting. I took this tip directly from a Github dev on an issue that they responded on that recommended specifying this if using any sort of tput commands within the script.

harupy · 2022-05-02T02:33:46Z

dev/dev-env-setup.sh

+      # Check if brew is installed and install it if it isn't present
+      # Note: if xcode isn't installed, this will fail.
+      if [ -z "$(command -v brew)" ]; then
+        echo "Brew is required to install pyenv on MacOS. Installing in your home directory."


Suggested change

echo "Brew is required to install pyenv on MacOS. Installing in your home directory."

echo "Homebrew is required to install pyenv on MacOS. Installing in your home directory."

nit

harupy · 2022-05-02T02:35:52Z

dev/dev-env-setup.sh

+      PYENV_ROOT="$HOME/.pyenv"
+      PYENV_BIN="$PYENV_ROOT/bin"


Suggested change

PYENV_ROOT="$HOME/.pyenv"

PYENV_BIN="$PYENV_ROOT/bin"

PYENV_ROOT="$HOME/.pyenv"

PYENV_BIN="$PYENV_ROOT/bin"

PATH="$PYENV_BIN:$PATH"

I think we need to add PYENV_BIN to PATH, otherwise pyenv ... doesn't work.

harupy · 2022-05-02T02:40:44Z

dev/dev-env-setup.sh

+rd="$MLFLOW_HOME/requirements"
+
+# Get the minimum supported version from MLflow to ensure any feature development adheres to legacy Python versions
+min_py_version=$(grep "python_requires=" "$MLFLOW_HOME/setup.py" | grep -E -o "([0-9]{1,}\.)+[0-9]{1,}")


Suggested change

min_py_version=$(grep "python_requires=" "$MLFLOW_HOME/setup.py" | grep -E -o "([0-9]{1,}\.)+[0-9]{1,}")

min_py_version=$(python setup.py -q min_python_version)

Let's use the min_python_version command.

harupy · 2022-05-02T02:46:50Z

dev/dev-env-setup.sh

+
+-h, -help,        --help        Display help
+
+-d, -directory    --directory   The path to install the virtual environment into


Does ./dev/dev-env-setup.sh -directory or ./dev/dev-env-setup.sh --directory work?

it does now!

btw do we need -directory?

removed those redundant arg keys

harupy · 2022-05-02T02:50:15Z

dev/test-dev-env-setup.sh

+err=0
+trap 'err=1' ERR


Can we fail-fast?

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

harupy · 2022-05-10T07:03:49Z

dev/dev-env-setup.sh

+if [[ -n "$override_py_ver" ]]; then
+  echo "$(tput bold; tput setaf 1)You are overriding the recommended version of Python for MLflow development: $min_py_version. $(tput sgr0)"
+  min_py_version="$(grep -o "^[0-9]*\.[0-9]*" <<< "$override_py_ver")"
+fi
+
+# Resolve a minor version to the latest micro version
+case $min_py_version in
+  "3.7") PY_INSTALL_VERSION="3.7.13" ;;
+  "3.8") PY_INSTALL_VERSION="3.8.13" ;;
+  "3.9") PY_INSTALL_VERSION="3.9.11" ;;
+  "3.10") PY_INSTALL_VERSION="3.10.3" ;;
+esac
+
+microver=$(grep -o '\.' <<< "$override_py_ver" | wc -l)
+if [[ $microver -gt 1 ]]; then
+  PY_INSTALL_VERSION=$override_py_ver
+fi


This block is a bit difficult to understand for me. Can we do something like this?

minor_to_micro() { case $1 in "3.7") echo "3.7.13" ;; "3.8") echo "3.8.13" ;; "3.9") echo "3.9.11" ;; "3.10") echo "3.10.3" ;; esac } if override_py_ver is specified if override_py_ver looks like 3.x PY_INSTALL_VERSION=$(minor_to_micro $override_py_ver) elif override_py_ver looks like 3.x.y PY_INSTALL_VERSION=$override_py_ver else echo("Invalid version ...") exit 1 fi else PY_INSTALL_VERSION=$(minor_to_micro $min_py_version) fi

great idea! This logic is much easier to follow.

harupy · 2022-05-10T07:20:24Z

.github/workflows/dev-setup.yml

+        with:
+          repository: ${{ github.event.inputs.repository }}
+          ref: ${{ github.event.inputs.ref }}
+      - uses: ./.github/actions/setup-pyenv


Suggested change

- uses: ./.github/actions/setup-pyenv

Can we remove this line and test pyenv installation?

definitely! Removing and validating CI build.

harupy · 2022-05-10T07:51:01Z

CONTRIBUTING.rst

+
+.. code-block:: bash
+
+    dev/dev-env-setup.sh -d ~/.venvs/mlflow-dev -q


Suggested change

dev/dev-env-setup.sh -d ~/.venvs/mlflow-dev -q

dev/dev-env-setup.sh -d .venvs/mlflow-dev -q

Can we create an environment in the current working directory?

great catch. Simplifies local env activation for the repo.

harupy

LGTM once the remaining comments are addressed!

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

BenWilson2 added 8 commits April 5, 2022 20:41

WIP

6514c07

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

wip

b2d3c7c

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

wip

0877979

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

wip script

825a6b2

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

Merge branch 'master' of https://github.com/mlflow/mlflow into dev-en…

d3e8760

…v-setup-script Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

wip

3e3ffbf

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

Merge branch 'master' of https://github.com/mlflow/mlflow into dev-en…

6135f10

…v-setup-script Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

Finish logic and update contributing guide

a993d85

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

github-actions bot added area/build Build and test infrastructure for MLflow area/docs Documentation issues rn/feature Mention under Features in Changelogs. labels Apr 18, 2022

BenWilson2 requested review from dbczumar, harupy and WeichenXu123 April 18, 2022 19:50

Finish a bash code block example

364d7e2

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

harupy reviewed Apr 18, 2022

View reviewed changes

harupy reviewed Apr 19, 2022

View reviewed changes

dev/dev-env-setup.sh Outdated Show resolved Hide resolved

harupy reviewed Apr 19, 2022

View reviewed changes

BenWilson2 added 2 commits April 19, 2022 17:26

WIP

20eb335

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

wip testing

f2f1fb5

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

harupy reviewed Apr 21, 2022

View reviewed changes

.github/workflows/dev-setup.yml Outdated

Comment on lines 88 to 90

Copy link

Member

harupy Apr 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

harupy reviewed Apr 21, 2022

View reviewed changes

harupy reviewed Apr 23, 2022

View reviewed changes

dev/dev-env-setup.sh Show resolved Hide resolved

WeichenXu123 reviewed Apr 23, 2022

View reviewed changes

BenWilson2 added 2 commits April 27, 2022 20:31

Remove windows support and cleanup

75b9d16

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

Merge branch 'master' of https://github.com/mlflow/mlflow into dev-en…

954a16d

…v-setup-script Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

harupy reviewed May 2, 2022

View reviewed changes

dev/test-dev-env-setup.sh Outdated

Comment on lines 6 to 7

err=0

trap 'err=1' ERR

Copy link

Member

harupy May 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we fail-fast?

BenWilson2 added 7 commits May 2, 2022 16:39

fail fast testing script and trigger CI when file modified

e35032c

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

fix test script

9c321cc

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

update actions rules

56dd405

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

testing

41cd787

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

remove debug

2d6839f

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

streamlining and corrections

2818a89

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

change for modified arg key

1acb19f

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

harupy reviewed May 10, 2022

View reviewed changes

harupy approved these changes May 10, 2022

View reviewed changes

BenWilson2 added 2 commits May 10, 2022 12:18

PR feedback

7ede0b6

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

remove redundant arg keys

d2b1c23

Signed-off-by: Ben Wilson <benjamin.wilson@databricks.com>

BenWilson2 merged commit d9058fc into master May 10, 2022

harupy mentioned this pull request Sep 15, 2022

[BUG] pos_label param in mlflow.sklearn.autolog not working properly #6797

Open

21 tasks

		rm -rf "$directory"
		echo "Virtual environment removed from '$directory'. Installing new instance."

		echo "$PYENV_BIN" >> "$GITHUB_PATH"
		echo "PYENV_ROOT=$PYENV_ROOT" >> "$GITHUB_ENV"

	if [ "$MLFLOW_DEV_ENV_CI_RUN" == 1 ]; then
	if [ ! -z "$GITHUB_ACTIONS" ]; then

		wget -O ~/brew_install.sh https://raw.githubusercontent.com/Homebrew/install/master/install.sh
		bash ~/brew_install.sh

	wget -O ~/brew_install.sh https://raw.githubusercontent.com/Homebrew/install/master/install.sh
	bash ~/brew_install.sh
	bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"


		Example usage:

		From root of MLflow repository on local with a destination virtualenv path of <MLFLOW_HOME>/.venvs/mlflow-dev:

	echo "Brew is required to install pyenv on MacOS. Installing in your home directory."
	echo "Homebrew is required to install pyenv on MacOS. Installing in your home directory."

	min_py_version=$(grep "python_requires=" "$MLFLOW_HOME/setup.py" \| grep -E -o "([0-9]{1,}\.)+[0-9]{1,}")
	min_py_version=$(python setup.py -q min_python_version)


		-h, -help, --help Display help

		-d, -directory --directory The path to install the virtual environment into


		.. code-block:: bash

		dev/dev-env-setup.sh -d ~/.venvs/mlflow-dev -q

	dev/dev-env-setup.sh -d ~/.venvs/mlflow-dev -q
	dev/dev-env-setup.sh -d .venvs/mlflow-dev -q

Dev env setup script #5717

Dev env setup script #5717

Conversation

BenWilson2 commented Apr 18, 2022

What changes are proposed in this pull request?

How is this patch tested?

Does this PR change the documentation?

Release Notes

Is this a user-facing change?

What component(s), interfaces, languages, and integrations does this PR affect?

How should the PR be classified in the release notes? Choose one:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harupy Apr 19, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harupy Apr 19, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harupy Apr 21, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harupy Apr 23, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harupy commented Apr 23, 2022 • edited Loading

harupy Apr 23, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harupy Apr 23, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WeichenXu123 Apr 23, 2022 • edited Loading

Choose a reason for hiding this comment

harupy Apr 23, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harupy May 2, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harupy May 2, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harupy May 2, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harupy May 10, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harupy left a comment

Choose a reason for hiding this comment

harupy Apr 19, 2022 •

edited

Loading

harupy Apr 19, 2022 •

edited

Loading

harupy Apr 21, 2022 •

edited

Loading

harupy Apr 23, 2022 •

edited

Loading

harupy commented Apr 23, 2022 •

edited

Loading

harupy Apr 23, 2022 •

edited

Loading

harupy Apr 23, 2022 •

edited

Loading

WeichenXu123 Apr 23, 2022 •

edited

Loading

harupy Apr 23, 2022 •

edited

Loading

harupy May 2, 2022 •

edited

Loading

harupy May 2, 2022 •

edited

Loading

harupy May 2, 2022 •

edited

Loading

harupy May 10, 2022 •

edited

Loading