Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Prepare for Pypi installable Ledger Python package #2246

Draft
wants to merge 5 commits into
base: feature/pypi
Choose a base branch
from
Draft

Prepare for Pypi installable Ledger Python package #2246

wants to merge 5 commits into from

Conversation

afh
Copy link
Member

@afh afh commented Apr 20, 2023

This PR adds support for building the Ledger Python package via pip and possibly other Python installation methods supporting pyproject.toml using scikit-build-core build backends. 馃悕馃コ

Please note, that this PR focuses on the build setup for Python and non-Python builds. There is another conversation to be had about how to best structure the Ledger Python package, i.e. where the Ledger (binary) Python module should reside, and how to deal with backwards compatibility.

Here are the steps in which this PR has been tested:

  • Install the main dependencies: python, boost (including boost-python), mpfr, gmp, gpgme
  • Create a virtual environment: python -m venv venv
  • Activate virtual environment: source venv/bin/activate
  • Update to the latest version of pip: pip install -U pip (optional)
  • Install the Ledger Python package: /usr/bin/env CMAKE_BUILD_PARALLEL_LEVEL=$(sysctl -n hw.ncpu) pip install --verbose --no-dependencies git+https://github.com/afh/ledger.git@pypi
  • If all goes well you can test the functionality using: python ${PATH_TO}/ledger.git/python/demo.py

Folks wanting to test are invited to follow the instructions above and try out any alternative way to install the Ledger Python package. If you do, please your experience, especially if you ran into problems.

I left a few notes as comments below for the reviewers.

鈩癸笍 Given the potential impact of this PR I've set it up as a draft to have a conversation about the future of the Ledger Python Package.

Looking forward to feedback 馃檪

Closes #1934

@afh afh added enhancement New feature or request python Related to the Python API labels Apr 20, 2023
@afh afh added this to the 3.4 milestone Apr 20, 2023
@afh afh requested review from a team April 20, 2023 21:40
@afh afh self-assigned this Apr 20, 2023
@afh afh mentioned this pull request Apr 20, 2023
Open
afh
afh commented Apr 20, 2023
View reviewed changes
CMakeLists.txt
${SKBUILD_PROJECT_NAME}
VERSION ${SKBUILD_PROJECT_VERSION}
LANGUAGES C CXX
)

list(APPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/doc")
include(LedgerVersion)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handling of version numbers could use a clean-up, I'll leave this for future me.

afh
afh commented Apr 20, 2023
View reviewed changes
CMakeLists.txt
if (HAVE_EDIT)
target_link_libraries(${_target} ${EDIT_LIB})
target_link_libraries(${_target} PUBLIC ${EDIT_LIB})
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When using Python_add_library (see below) CMake complained about use of keyword vs. plain arguments; adding the PUBLIC and PRIVATE scope keywords here fixed the error.

鈩癸笍 ${EDIT_LIB} needs to be public as the ${LEDGER_CLI_SOURCES} use it and the linker will fail otherwise.

afh
afh commented Apr 20, 2023
View reviewed changes
CMakeLists.txt
set_target_properties(${_target} PROPERTIES LINK_FLAGS "-undefined dynamic_lookup")
else()
target_link_libraries(${_target} ${Boost_LIBRARIES} ${Python_LIBRARIES})
target_link_libraries(${_target} PUBLIC ${Boost_LIBRARIES} ${Python_LIBRARIES})
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

鈩癸笍 The ${Boost_LIBRARIES} need to be public as the Util and Math tests use it and the linker will fail otherwise.

afh
afh commented Apr 20, 2023
View reviewed changes
CMakeLists.txt
if (NOT SKBUILD)
set(SKBUILD_PROJECT_NAME ledger)
set(SKBUILD_PROJECT_VERSION 3.3.2)
endif()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SKBUILD variables are set when building using the scikit-build-core build backend with pip

afh
afh commented Apr 20, 2023
View reviewed changes
CMakeLists.txt
add_subdirectory(doc)
add_subdirectory(contrib)
add_subdirectory(test)
endif()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to build the supporting stuff for the Python package.

afh
afh commented Apr 20, 2023
View reviewed changes
pyproject.toml
[project.urls]
homepage = "https://ledger-cli.org"
documentation = "https://ledger-cli.org/docs.html"
repository = "https://git.ledger-cli.org/ledger"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do the above project metadata seem reasonable to folks?

afh
afh commented Apr 20, 2023
View reviewed changes
pyproject.toml

[tool.scikit-build.cmake.define]
USE_PYTHON = "ON"
USE_GPGME = "ON"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be most useful I think the Ledger Python Package should provide support for encrypted journals.
If many folks disagree I can look into possibly making it a install-time option, if scikit-build-core supports that.

afh
afh commented Apr 20, 2023
View reviewed changes
src/CMakeLists.txt
# FIXME: symlink would be sufficient:
# maybe using install(CODE "...") and
# execute_process(COMMAND "${CMAKE_COMMAND}" -E create_symlink ...).
# Windows will need a special case due to not supporting symlinks.
add_custom_command(
TARGET libledger POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
$<TARGET_FILE:libledger> "${CMAKE_BINARY_DIR}/${_ledger_python_module_name}")
$<TARGET_FILE:libledger>
"${CMAKE_BINARY_DIR}/python/${SKBUILD_PROJECT_NAME}/_${_ledger_python_module_name}")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

鈩癸笍 This somewhat unusual destination is necessary in order to make the python related tests pass from CMake's build directory.

@afh afh changed the base branch from master to feature/pypi April 25, 2023 05:09
@leszekhanusz
Copy link

I'm on Ubuntu 23.04
I tried to install the Python package on conda envs with Python version 3.9, 3.10 and 3.11 and it fails every time.

For Python 3.9 and 3.10, because apparently my system has only libboost Python binding for Python 3.11
For Python 3.11, with the following error:

    Could not find a package configuration file provided by "Gpgmepp"
    (requested version 1.13.1) with any of the following names:

      GpgmeppConfig.cmake
      gpgmepp-config.cmake

See complete logs:

@leszekhanusz leszekhanusz mentioned this pull request Jul 19, 2023
Open
@gudzpoz gudzpoz mentioned this pull request Feb 4, 2024
Open
@Rudd-O
Copy link
Contributor

Rudd-O commented Feb 23, 2024

Also note #2332 may just render this effort futile.

@afh
Copy link
Member Author

afh commented Feb 23, 2024

Can you elaborate on this a bit, @Rudd-O? In my mind #2332 is an issue that can be more or less easily be fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@afh afh

Labels
enhancement New feature or request python Related to the Python API
Projects
None yet
Milestone
3.4
Development

Successfully merging this pull request may close these issues.

PyPI package for ledger.so
3 participants
@afh @leszekhanusz @Rudd-O