Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for python3.11 #1

Closed
wants to merge 38 commits into from
Closed

Add support for python3.11 #1

wants to merge 38 commits into from

Conversation

OlivierLDff
Copy link
Owner

  • Update the linux/mac/windows CI to handle python 3.11

Type

  • Bug fix (non-breaking change which fixes an issue): Fixes #
  • New feature (non-breaking change which adds functionality). Resolves #
  • Breaking change (fix or feature that would cause existing functionality to not work as expected) Resolves #

Motivation and Context

Checklist:

  • I have run python util/check_style.py --apply to apply Open3D code style
    to my code.
  • This PR changes Open3D behavior or adds new functionality.
    • Both C++ (Doxygen) and Python (Sphinx / Google style) documentation is
      updated accordingly.
    • I have added or updated C++ and / or Python unit tests OR included test
      results
      (e.g. screenshots or numbers) here.
  • I will follow up and update the code if CI fails.
  • For fork PRs, I have selected Allow edits from maintainers.

Description

cielavenir and others added 9 commits August 9, 2023 08:39
msgpack 6.0 changed the cmake config. This PR adds that support.
* infra for tensor index-reduction and interface for voxeldownsample

* basic sum reduction

* temp fix for workload nums

* temp fix with contiguous input

* separate index tensor

* add CPU counterpart

* clean up cpp part

* add unit test for index_add_

* fix point attribute shape

* fix unit tests

* fix doc, also fix several doc issues in t.pointcloud

* fix benchmark
* SimplifyQuadricDecimation: Check for flipped triangles connected to both vertices of the removed edge

The previous code checks if any triangles connected to the removed vertex are flipped in the process of collapsing the edge. However, since the remaining vertex on the collapsed edge can be moved during the operation, triangles connected to this vertex can flip as well. This commit puts the flip-checking code into a loop so we can check both sets of triangles.

* SimplifyQuadricDecimation: Disallow creating very small triangles

This sets a threshold on how much any triangle can shrink in area during an edge collapse which should avoid creating degenerate triangles. The threshold is somewhat arbitrary, I found 0.001 worked well but it may make sense for this to become a function parameter.

* SimplifyQuadricDecimation: Prevent 'pinching' of triangles

When a vertex is connected to exactly 3 triangles and an edge of one of these triangles not connected to the vertex is collapsed, the remaining 2 triangles end up sharing the same vertices, with opposite normals. This creates a non-manifold location which is nearly always undesirable.
This commit adds a check to see if any of the triangles attached to the collapsing edge share an edge that is not connected to `vidx0` or `vidx1`

* SimplifyQuadricDecimation: Save some memory by clearing some vectors after they are no longer needed

* Update Changelog with SimplifyQuadricDecimation fixes

* SimplifyQuadricDecimation: Reduce memory requirements by removing need for vbars & costs maps

Reduces the memory footprint by 27% in my tests runs ~15% faster as we remove a lot of allocations.

* SimplifyQuadricDecimation: Simplify checking for pinched triangles

Changing from tracking edges (`unordered_set<Vector2i>`) to vertices (`unordered_map<int, int>) gives a good performance increase
* fix 6291

* fix 6236

* address review comments
* Add install with upgrade option when installing latest development package.
* util/install-deps-python.sh was removed in isl-org#2146
* Grammar fix.
* Move open3d_ml and arm to after docker.
* organize TOC - Move C++ and Contribution to end and keep python section together.
* Fix sphinx warnings (Title underline too short).
* Fix extra heading drop when going one level down.
* Fix cross reference links
* Add files/directories to ignore for sphinx, reduces inconsistency warnings.
* Fix `Unknown directive type "code_block"` warning.
* Add uvmaps to official docs, fix - `WARNING: document isn't included in any toctree`.
* Fix yaml lexer name - See https://pygments.org/languages/
* Fix warnings - `Inline substitution_reference start-string without end-string.`
* Only copy open3d_tutorial.py once
* Move list of classes to a global variable
* Use f-strings as requested in PR
* Add python_api_in as ignored directory.
* Fix pybind docs with literal block for tree output
* Fix doxygen docs with code block for tree outpu
* Correct class names in doxygen docs for RedWood dataset
- Update the linux/mac/windows CI to handle python 3.11
@OlivierLDff OlivierLDff force-pushed the support-python-311 branch 2 times, most recently from 02fd1bc to 36531b4 Compare August 17, 2023 07:15
@OlivierLDff OlivierLDff force-pushed the support-python-311 branch 8 times, most recently from e55988b to 41ab5a0 Compare August 17, 2023 12:07
ssheorey and others added 7 commits August 17, 2023 22:04
…e voxels (isl-org#6325)

Since the triangle can span across multiple voxels, we should not break when we find the first relevant voxel.

Co-authored-by: Sharon Berezalsky <sharon@buildots.com>
Co-authored-by: Rishabh Singh <rishabh.17iitkgp@gmail.com>
…elGrid::RemoveVoxel (isl-org#6023)

Co-authored-by: Rishabh Singh <rishabh.17iitkgp@gmail.com>
…cene (isl-org#6051)

Co-authored-by: Seb Mascha <sebastien@summerrobotics.ai>
Co-authored-by: Benjamin Ummenhofer <benjaminum@gmail.com>
Co-authored-by: Benjamin Ummenhofer <benjamin.ummenhofer@intel.com>
@atharvjairath-entrupy
Copy link

Hi, @OlivierLDff I wanted to extend my gratitude for your efforts in working on the Python 3.11 support for Open3D. I was trying the repo out, but faced error while building.

How to replicate:

sudo apt-get install libosmesa6-dev
sudo snap install cmake --classic
sudo apt install libglew-dev

git clone --recursive https://github.com/OlivierLDff/Open3D.git

cd Open3D
./util/install_deps_ubuntu.sh
git submodule update --init --recursive
mkdir build && cd build
cmake -DENABLE_HEADLESS_RENDERING=ON \
                 -DBUILD_GUI=OFF \
                 -DUSE_SYSTEM_GLEW=OFF \
                 -DUSE_SYSTEM_GLFW=OFF \
                 ..

Worked Fine Till here.

make -j$(nproc)

Error ouccred:

[ 85%] Building CXX object examples/cpp/CMakeFiles/PCDFileFormat.dir/PCDFileFormat.cpp.o
In file included from /home/atharv/entrupy/Open3D/build/_deps/ext_pybind11-src/include/pybind11/attr.h:13,
                 from /home/atharv/entrupy/Open3D/build/_deps/ext_pybind11-src/include/pybind11/pybind11.h:45,
                 from /home/atharv/entrupy/Open3D/build/_deps/ext_pybind11-src/include/pybind11/numpy.h:12,
                 from /home/atharv/entrupy/Open3D/build/_deps/ext_pybind11-src/include/pybind11/eigen.h:12,
                 from /home/atharv/entrupy/Open3D/cpp/pybind/../pybind/open3d_pybind.h:27,
                 from /home/atharv/entrupy/Open3D/cpp/pybind/../pybind/camera/camera.h:10,
                 from /home/atharv/entrupy/Open3D/cpp/pybind/camera/camera.cpp:8:
/home/atharv/entrupy/Open3D/build/_deps/ext_pybind11-src/include/pybind11/cast.h: In function ‘std::string pybind11::detail::error_string()’:
/home/atharv/entrupy/Open3D/build/_deps/ext_pybind11-src/include/pybind11/cast.h:446:36: error: invalid use of incomplete type ‘PyFrameObject’ {aka ‘struct _frame’}
  446 |                 "  " + handle(frame->f_code->co_filename).cast<std::string>() +
      |                                    ^~
In file included from /home/atharv/anaconda3/envs/py11/include/python3.11/Python.h:42,
                 from /home/atharv/entrupy/Open3D/build/_deps/ext_pybind11-src/include/pybind11/detail/common.h:124,
                 from /home/atharv/entrupy/Open3D/cpp/pybind/../pybind/open3d_pybind.h:25,
                 from /home/atharv/entrupy/Open3D/cpp/pybind/../pybind/camera/camera.h:10,
                 from /home/atharv/entrupy/Open3D/cpp/pybind/camera/camera.cpp:8:
/home/atharv/anaconda3/envs/py11/include/python3.11/pytypedefs.h:22:16: note: forward declaration of ‘PyFrameObject’ {aka ‘struct _frame’}
   22 | typedef struct _frame PyFrameObject;
      |                ^~~~~~
In file included from /home/atharv/entrupy/Open3D/build/_deps/ext_pybind11-src/include/pybind11/attr.h:13,
                 from /home/atharv/entrupy/Open3D/build/_deps/ext_pybind11-src/include/pybind11/pybind11.h:45,
                 from /home/atharv/entrupy/Open3D/build/_deps/ext_pybind11-src/include/pybind11/numpy.h:12,
                 from /home/atharv/entrupy/Open3D/build/_deps/ext_pybind11-src/include/pybind11/eigen.h:12,
                 from /home/atharv/entrupy/Open3D/cpp/pybind/../pybind/open3d_pybind.h:27,
                 from /home/atharv/entrupy/Open3D/cpp/pybind/../pybind/camera/camera.h:10,
                 from /home/atharv/entrupy/Open3D/cpp/pybind/camera/camera.cpp:8:
/home/atharv/entrupy/Open3D/build/_deps/ext_pybind11-src/include/pybind11/cast.h:446:75: error: expected primary-expression before ‘>’ token
  446 |                 "  " + handle(frame->f_code->co_filename).cast<std::string>() +
      |                                                                           ^
/home/atharv/entrupy/Open3D/build/_deps/ext_pybind11-src/include/pybind11/cast.h:446:77: error: expected primary-expression before ‘)’ token
  446 |                 "  " + handle(frame->f_code->co_filename).cast<std::string>() +


....
make[2]: *** [cpp/pybind/CMakeFiles/pybind.dir/build.make:104: cpp/pybind/CMakeFiles/pybind.dir/core/core.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[ 85%] Building CXX object examples/cpp/CMakeFiles/PointCloud.dir/PointCloud.cpp.o
make[2]: *** [cpp/pybind/CMakeFiles/pybind.dir/build.make:90: cpp/pybind/CMakeFiles/pybind.dir/core/blob.cpp.o] Error 1
[ 85%] Building CXX object examples/cpp/CMakeFiles/PoseGraph.dir/PoseGraph.cpp.o
make[2]: *** [cpp/pybind/CMakeFiles/pybind.dir/build.make:76: cpp/pybind/CMakeFiles/pybind.dir/camera/camera.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:3319: cpp/pybind/CMakeFiles/pybind.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

OS: Linux

I understand that this pull request is a work in progress, and I wanted to clarify if these issues are expected behavior at this stage of development.
Once Again thank you for your contribution!

@OlivierLDff
Copy link
Owner Author

I didn't run into these pybind11 issue. Maybe updating it again would fix it?

ssheorey and others added 7 commits September 7, 2023 08:06
Windows MSVC revert to fmt 6
gcc-12 maybe-uninitialized warning
set VTK compatible version
documentation for draw
Build Open3D viewer Debian package in CI and make available as Github artifact.
Viewer CMake project file from @errissa
Update to C++ 17
TensorFlow 2.13
Tensorboard 2.13
scipy 1.11.2
Move PyTorch compile warning to where PyTorch version is available.
Update pip, scipy versions.
Use requirements_test.txt in CI
tensorflow-gpu -> tensorflow for 2.13+
remove CUDA_VERSION setting in ci_utils.sh. THis happens in docker/build_docker.sh now.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet