Easy Install and MISC Fix #108

cindytsai · 2024-05-02T05:30:35Z

Easy Install and MISC Fix

Enhancements

MISC

Reorganize CMake and add minimum version requirement for Python.
Fix the yt_rma_particle printing error msg when failing to read particle data from the particle_data dictionary even though the particle data isn't loaded. (These kind of particle data will be generated through user-defined C function)

Easy Install

Make libyt fetches dependencies itself when -DJUPYTER_KERNEL=ON.
The dependencies will be stored under libyt/vendor for reuse and linkage purpose.

APIs

Remove uint (unsigned int), ulong (unsigned long). This is useless and unnecessary.
Add velocity_unit to yt_param_yt. This is important in cosmology simulation. libyt now enforces user to set this parameter manually.
- Inform Fish.
- Need yt_libyt 0.0.8 releases

Test

cmake-build-test.yml for making sure each option can compile successfully and independently.
example-test-run.yml for running example/amr-example and make sure it can run.

Doc

Add quick start page and example.
Update document to make install page easier to follow (I hope)
Add setting OMPI_MCA_osc=sm,pt2pt when running libyt in parallel.

Example

Add quick start example.

Bug

Fix the logic of identifying user not-done-yet error and real error. It now follows the rule:
1. If the last line has valid '' -> not-done-yet
2. If err msg matches those from compound keywords (ex: if) and colon exist at last line and error lineno is at the last line -> not-done-yet
3. If err msg matches those from bracket (ex: (, [, """) -> not-done-yet
4. Else, it's a true error.

It turns out the the identifying multi-line statement or so called distinguish what is not-done-yet code is more complicated then I initially thought, since each python version prints out different error messages and even defines error lineno differently. And some API (PyErr_Fetch and PyErr_Restore) are deprecated since python 3.12. These are the API that is deprecated, the others are stable API.

~~Make sure exception hook is set before entering interactive mode or jupyter mode.~~
- The exception hook to avoid MPI abort if one of the process gets error can be overwritten by yt.enable_parallelism() or other stuff. I'm not sure if this is a bug or not. (Probably not, should ask Matt)
Fix yt_libyt when using mpi4py in Serial Mode.
- Push to PyPI

GitHub Action

Runner image reference: https://github.com/actions/runner-images/tree/main/images/macos
Some Python version is not supported in some macOS, check the reference.

TODOs

Publish jupyter_libyt on PyPI

Check

amr-example run
Enzo
Gamer

updates: - [github.com/pre-commit/mirrors-clang-format: v17.0.6 → v18.1.1](pre-commit/mirrors-clang-format@v17.0.6...v18.1.1)

Though it can build example, it cannot install libyt and linked properly. it also install vendor library which is not we want.

…lling libyt.

…ratch.

1. If nlohmann_json_DIR is not provided, xeus/xeus-zmq will also count as not found. 2. Since nlohmann_json/xeus/xeus-zmq does not have XXXTargets.cmake, it gets errors when populating the path. --- Don't know why is this happening, fix it later.

Not yet tested on eureka.

User can try to provided READLINE_PATH to help find, but this is not mandatory

Can build, but it seems to link to system readline when READLINE_PATH is provided.

Readline does not use cmake to build, it is better to install by user. Normally, every computer would have one (?), and it is easy to install.

cindytsai · 2024-05-02T05:33:13Z

@hyschive Can you try compiling libyt on MacOS?
Because macos-latest and macos-14 image in GitHub Actions failed to set up Python < 3.11. (Don't know why)
Maybe follow the Install page generated by the PR, it wouldn't take long.

Still thinking whether should I add admonition.

cindytsai · 2024-05-10T16:36:45Z

Thank you @ChunYen-Chen for going through the documents, and giving feedbacks.

macos-latest which is now macos-14 only supports python 3.11 and 3.12

I think I accidentally removed index_offset in the example.

cindytsai · 2024-05-31T01:50:40Z

Hi @hyschive,

I'm enforcing setting velocity_unit in libyt. Please update libyt and use yt_libyt >= 0.0.8.

pre-commit-ci bot and others added 30 commits March 18, 2024 17:35

[pre-commit.ci] pre-commit autoupdate

59f7ec3

updates: - [github.com/pre-commit/mirrors-clang-format: v17.0.6 → v18.1.1](pre-commit/mirrors-clang-format@v17.0.6...v18.1.1)

[pre-commit.ci] auto fixes from pre-commit.com hooks

864bc62

Vendoring all dependencies and build example successfully.

a75dbbc

Though it can build example, it cannot install libyt and linked properly. it also install vendor library which is not we want.

Ignore vendor

5fa4a44

Fix don't install vendor's target.

b85efda

Add comments, and reset redefining install and export.

7853243

Can link to shared library dependencies install via cmake after insta…

1a5b932

…lling libyt.

Move setting rpath to INSTALL_RPATH

33d0a70

Add CMakeLIsts.txt for example, and fix the indentation.

c1f9b7b

Add RPATH if DJUPYTER_KERNEL=ON and use vendoring.

15ceebf

Accept user pass in dependency dir (find_package) or download from sc…

7363140

…ratch.

Populate right after checking dependency

9e84787

Reset disable export.

b2528f9

Accept user provide library when compiling

4ea543b

Link based on vendor or provided path to dependency

fa77031

Remove

c789515

Get ZeroMQ dependency

7f97827

Not yet tested on eureka.

Add zeromq to rpath if vendoring

1be7f0e

Don't need to populate, only make sure zeromq exist.

363e450

libzmq (ZeroMQ) build shared library at libzmq-build/lib

bc9f230

Single out fetched version, neglecting "v"

722e9c1

Find readline library, and link to it.

3827df1

User can try to provided READLINE_PATH to help find, but this is not mandatory

Fix linking to readline.

1c596f5

Can build, but it seems to link to system readline when READLINE_PATH is provided.

Fix ) for formatting

15a75fe

Remove target_link_directories.

ecbc0da

Do shallow clone.

c13cd9a

Print fatal error when cannot find readline.

dfc6105

Readline does not use cmake to build, it is better to install by user. Normally, every computer would have one (?), and it is easy to install.

Move things related to share folder to share/CMakeLists.txt

ae10c39

Add comments to cmake.

532e01c

cindytsai added 19 commits May 2, 2024 00:44

Merge remote-tracking branch 'yt-project/main' into EasyInstall

0daf16d

Get yt_libyt from PyPI.

df1713e

Add horizontal lines in quick start and example.

1a4f71c

Try setup python v5

7355a55

Try macOS-latest.

9ac3ac3

Create FAQs page to collect all the FAQs.

ee242a9

Separate how to install page to how to install and details.

fd05808

Refactor after moving how-to-install to a folder.

ba94453

Fix the anchor after splitting details and how-to-install.

73719a5

Rename install section to libyt and add admontion

7a8db74

cont. last commit.

9a965ba

Single out required versions to a column.

7b597e3

Move FAQs to a single page and them link them.

2577094

Update python dependency notes and admonition.

8722941

Still thinking whether should I add admonition.

Add minimum required version of Python.

f52d3d9

Note --enable-shared if compile Python from source, and add FAQs.

9a32526

Add pip.

a15ebc4

format requirements and remove label meanings at home page.

a74453d

Update content.

16b8012

cindytsai added 4 commits May 10, 2024 11:54

use runner image macos-12, it supports python3.7~3.12.

97dee95

macos-latest which is now macos-14 only supports python 3.11 and 3.12

set velocity_unit in yt_param_yt.

eb27c96

Remove uint, ulong typedef.

848eeac

typo

a8c45c7

I think I accidentally removed index_offset in the example.

cindytsai merged commit 49ec75e into yt-project:main May 31, 2024
26 checks passed

cindytsai mentioned this pull request May 31, 2024

Dependencies Too Hard to Install #106

Closed

cindytsai deleted the EasyInstall branch June 5, 2024 21:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Easy Install and MISC Fix #108

Easy Install and MISC Fix #108

cindytsai commented May 2, 2024 •

edited

Loading

cindytsai commented May 2, 2024 •

edited

Loading

cindytsai commented May 10, 2024

cindytsai commented May 31, 2024

Easy Install and MISC Fix #108

Easy Install and MISC Fix #108

Conversation

cindytsai commented May 2, 2024 • edited Loading