Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] [unitree/cross] Fix unitree system build #1793

Closed

Conversation

tkmtnt7000
Copy link
Member

@tkmtnt7000 tkmtnt7000 commented Apr 26, 2023

Currently, unitree system build (build_ros1_dependencies.sh and build_ros1.sh) fails at several points.

This PR includes

  • Fix app_manager url and version (772dc9a)
  • Install python-typing to avoid import typing error (772dc9a)
  • Add libfcl-dev and libccd-dev to dependencies because geometric_shapes package needs them (0ec465c, 929ae6e)
  • Avoid setuptools version error caused by spidev==3.6 (af35715)

But, this PR do not fix the following issue

  • Unable to install python-twisted in build_ros1_dependencies.sh because of the following error.
Installed /opt/jsk/System/ros1_dependencies/lib/python2.7/site-packages/Twisted-17.9.0-py2.7-linux-aarch64.egg                                                                                                                                                                                                                                                                               
Processing dependencies for Twisted==17.9.0                                                                                                                                                                                                                                                                                                                                                  
Searching for zope.interface>=3.6.0                                                                                                                                                                                                                                                                                                                                                          
Reading https://pypi.python.org/simple/zope.interface/                                                                                                                                                                                                                                                                                                                                       
Downloading https://files.pythonhosted.org/packages/1f/ef/cae21b6295cb2960429f463e6e99daf3623a0c4ced7242a809a0c94c9605/zope.interface-6.1a2.tar.gz#sha256=167ea1a5c28aad2fa7fb0ca8761229bf54766ab6e696c80e592ee8c936ab1776                                                                                                                                                                   
Best match: zope.interface 6.1a2                                                                                                                                                                                                                                                                                                                                                             
Processing zope.interface-6.1a2.tar.gz                                                                                                                                                                                                                                                                                                                                                       
Writing /tmp/easy_install-ur5jc9/zope.interface-6.1a2/setup.cfg                                                                                                                                                                                                                                                                                                                              
Running zope.interface-6.1a2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-ur5jc9/zope.interface-6.1a2/egg-dist-tmp-3dtby3                                                                                                                                                                                                                                                              
warning: no files found matching '*.txt' under directory 'docs'                                                                                                                                                                                                                                                                                                                              
src/zope/interface/_zope_interface_coptimizations.c:765:5: error: unknown type name ‘Py_hash_t’                                                                                                                                                                                                                                                                                              
     Py_hash_t _v_cached_hash;                                                                                                                                                                                                                                                                                                                                                               
     ^~~~~~~~~                                                                                                                                                                                                                                                                                                                                                                               
src/zope/interface/_zope_interface_coptimizations.c:907:8: error: unknown type name ‘Py_hash_t’                                                                                                                                                                                                                                                                                              
 static Py_hash_t                                                                                                                                                                                                                                                                                                                                                                            
        ^~~~~~~~~                                                                                                                                                                                                                                                                                                                                                                            
********************************************************************************                                                                                                                                                                                                                                                                                                             
WARNING:                                                                                                                                                                                                                                                                                                                                                                                     
                                                                                                                                                                                                                                                                                                                                                                                             
        An optional code optimization (C extension) could not be compiled.                               
                                                                                                         
        Optimizations for this package will not be available!                                                                                                                                                      
                                                                                                                                                                                                                                                                                                                                                                                             
command 'aarch64-linux-gnu-gcc' failed with exit status 1                                                
********************************************************************************                         
  File "build/bdist.linux-aarch64/egg/zope/interface/tests/test_advice.py", line 117                                                                                                                               
    class Meta(type, metaclass=Metameta):                                                                                                                                                                          
                              ^                                                                          
SyntaxError: invalid syntax                                                                              
                                                                                                         
  File "build/bdist.linux-aarch64/egg/zope/interface/adapter.py", line 336                               
    yield from cls._allKeys(v, i - 1, new_parent_k)                                                                                                                                                                
             ^                                                                                           
SyntaxError: invalid syntax                                                                                                                                                                                        
                                                                                                         
  File "build/bdist.linux-aarch64/egg/zope/interface/registry.py", line 294                                                                                                                                        
    yield from self.utilities.lookupAll((), interface)                                                   
             ^                                                                                           
SyntaxError: invalid syntax                                                                              
                                                                                                                                                                                                                   
creating /opt/jsk/System/ros1_dependencies/lib/python2.7/site-packages/zope.interface-6.1a2-py2.7-linux-aarch64.egg                                                                                                
Extracting zope.interface-6.1a2-py2.7-linux-aarch64.egg to /opt/jsk/System/ros1_dependencies/lib/python2.7/site-packages                                                                                           
  File "/opt/jsk/System/ros1_dependencies/lib/python2.7/site-packages/zope.interface-6.1a2-py2.7-linux-aarch64.egg/zope/interface/adapter.py", line 336                                                            
    yield from cls._allKeys(v, i - 1, new_parent_k)                                                                                                                                                                
             ^                                                                                           
SyntaxError: invalid syntax                                                                              
                                                                                                         
  File "/opt/jsk/System/ros1_dependencies/lib/python2.7/site-packages/zope.interface-6.1a2-py2.7-linux-aarch64.egg/zope/interface/registry.py", line 294                                                                                                                        
    yield from self.utilities.lookupAll((), interface)                                                                                                                                                             
             ^                                                                                           
SyntaxError: invalid syntax                                                                              
                                                                                                         
  File "/opt/jsk/System/ros1_dependencies/lib/python2.7/site-packages/zope.interface-6.1a2-py2.7-linux-aarch64.egg/zope/interface/tests/test_advice.py", line 117                                                  
    class Meta(type, metaclass=Metameta):                                                                                                                                                                          
                              ^                                                                          
SyntaxError: invalid syntax                                                                              
                                                                                                                                                                                                                   
Adding zope.interface 6.1a2 to easy-install.pth file                                                     
                                                                                                         
Installed /opt/jsk/System/ros1_dependencies/lib/python2.7/site-packages/zope.interface-6.1a2-py2.7-linux-aarch64.egg                                                                                                                                                            
Searching for constantly>=15.1                                                                                                                                                                                     
Reading https://pypi.python.org/simple/constantly/                                                                                                                                                                 
Downloading https://files.pythonhosted.org/packages/b9/65/48c1909d0c0aeae6c10213340ce682db01b48ea900a7d9fce7a7910ff318/constantly-15.1.0-py2.py3-none-any.whl#sha256=dd2fa9d6b1a51a83f0d7dd76293d734046aa176e384bf6e33b7e44880eb37c5d                                           
Best match: constantly 15.1.0                                                                                                                                                                                      
Processing constantly-15.1.0-py2.py3-none-any.whl                                                        
Installing constantly-15.1.0-py2.py3-none-any.whl to /opt/jsk/System/ros1_dependencies/lib/python2.7/site-packages                                                                                                                                                              
Adding constantly 15.1.0 to easy-install.pth file                                                        
                                                                                                                                                                                                                   
Installed /opt/jsk/System/ros1_dependencies/lib/python2.7/site-packages/constantly-15.1.0-py2.7.egg                                     
Searching for Automat>=0.3.0                                                                             
Reading https://pypi.python.org/simple/Automat/                                                          
Downloading https://files.pythonhosted.org/packages/29/90/64aabce6c1b820395452cc5472b8f11cd98320f40941795b8069aef4e0e0/Automat-22.10.0-py2.py3-none-any.whl#sha256=c3164f8742b9dc440f3682482d32aaff7bb53f71740dd018533f9de286b64180                                             
Best match: Automat 22.10.0                                                                                                                                                                   
Processing Automat-22.10.0-py2.py3-none-any.whl                     
Installing Automat-22.10.0-py2.py3-none-any.whl to /opt/jsk/System/ros1_dependencies/lib/python2.7/site-packages                                                                                                                                                                
writing requirements to /opt/jsk/System/ros1_dependencies/lib/python2.7/site-packages/Automat-22.10.0-py2.7.egg/EGG-INFO/requires.txt                                                                                                                                           
Adding Automat 22.10.0 to easy-install.pth file                     
Installing automat-visualize script to /opt/jsk/System/ros1_dependencies/bin                                                            
                                                                                                                                                                                              
Installed /opt/jsk/System/ros1_dependencies/lib/python2.7/site-packages/Automat-22.10.0-py2.7.egg                                       
Searching for attrs>=19.2.0                                                                                                                                                                   
Reading https://pypi.python.org/simple/attrs/                                                                                                                                                 
Downloading https://files.pythonhosted.org/packages/97/90/81f95d5f705be17872843536b1868f351805acf6971251ff07c1b8334dbb/attrs-23.1.0.tar.gz#sha256=6279836d581513a26f1bf235f9acd333bc9115683f14f7e8fae46c98fc50e015                                                              
Best match: attrs 23.1.0                                                                                                                                                                      
Processing attrs-23.1.0.tar.gz                                                                                                                                                                
error: Couldn't find a setup script in /tmp/easy_install-_r9Nfn/attrs-23.1.0.tar.gz    

I installed attrs<23.0 manually and then tried to install python-twisted, but almost same error occurs. I think easy_install cannnot find attrs<23.0. in local.

@k-okada
Copy link
Member

k-okada commented Apr 26, 2023 via email

@tkmtnt7000
Copy link
Member Author

It seems that zope.interface==4.3.2 is installed manually by ros1_dependencies_build_scripts

@tkmtnt7000
Copy link
Member Author

tkmtnt7000 commented Apr 26, 2023

Memo:

Twisted==17.9.0's requirements
https://github.com/twisted/twisted/blob/667758e33d808ba730ccbdaa5741b64c2fe15399/src/twisted/python/_setup.py#L223-L232

Python libraries in arm64v8_System (昨年までビルドに成功していたもの)

tsukamoto@tsukamoto-desktop-ryzen ~/ros/unitree_ws/src/jsk-ros-pkg/jsk_robot/jsk_unitree_robot/cross/arm64v8_System/ros1_dependencies/lib/python2.7/site-packages (PR-fix-unitree-system-build *%=) 
$ ls
Automat-20.2.0-py2.7.egg                constantly-15.1.0-py2.7.egg  incremental-16.10.1-py2.7.egg          sip.pyi          tornado-4.5.3-py2.7-linux-aarch64.egg
PyAudio-0.2.11-py2.7-linux-aarch64.egg  cv2.so                       lcm                                    sip.so           trollius-2.0.1-py2.7.egg
Twisted-17.9.0-py2.7-linux-aarch64.egg  easy-install.pth             mailutils                              sipconfig.py     txaio-2.8.1-py2.7.egg
attrs-22.1.0-py2.7.egg                  future-0.15.2-py2.7.egg      pymongo-3.6.1-py2.7-linux-aarch64.egg  sipdistutils.py  websocket_client-0.44.0-py2.7.egg
autobahn-17.10.1-py2.7.egg              futures-3.2.0-py2.7.egg      setuptools.pth                         site.py          zope.interface-4.3.2-py2.7-linux-aarch64.egg
backports_abc-0.5-py2.7.egg             futures-3.3.0-py2.7.egg      singledispatch-3.4.0.3-py2.7.egg       site.pyc         zope.interface-5.5.0.dev0-py2.7.egg

成功していないものの途中経過

user@2aee8519342c:/opt/jsk/System/ros1_dependencies/lib/python2.7/site-packages$ ls
autobahn-17.10.1-py2.7.egg   future-0.15.2-py2.7.egg        PyAudio-0.2.11-py2.7-linux-aarch64.egg  sip.pyi                                Twisted-17.9.0-py2.7-linux-aarch64.egg
Automat-22.10.0-py2.7.egg    futures-3.2.0-py2.7.egg        pymongo-3.6.1-py2.7-linux-aarch64.egg   sip.so                                 txaio-2.8.1-py2.7.egg
backports_abc-0.5-py2.7.egg  futures-3.4.0-py2.7.egg        setuptools.pth                          site.py                                websocket_client-0.44.0-py2.7.egg
constantly-15.1.0-py2.7.egg  incremental-16.10.1-py2.7.egg  singledispatch-3.4.0.3-py2.7.egg        site.pyc                               zope.interface-4.3.2-py2.7-linux-aarch64.egg
cv2.so                       lcm                            sipconfig.py                            tornado-4.5.3-py2.7-linux-aarch64.egg  zope.interface-6.1a2-py2.7-linux-aarch64.egg
easy-install.pth             mailutils                      sipdistutils.py                         trollius-2.0.1-py2.7.egg

@@ -45,5 +45,6 @@ docker run -it --rm \
pip install -U --user pip && \
export PYTHONPATH=\"/opt/jsk/System/ros1_dependencies/lib/python2.7/site-packages\" && \
export PKG_CONFIG_PATH=\"/opt/jsk/${INSTALL_ROOT}/ros1_dependencies/lib/pkgconfig\" && \
pip install spidev==3.5 pixel-ring==0.1.0 && \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We try to separate ‘data’ and ‘processing’, so we will write which modules to install in go_requirements.txt and try to build_ros_dependencies.sh without depending on installed modules or even unitree robot. This way, we can integrate this file with Pepper cross-environment near future.

Copy link
Member

@k-okada k-okada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tkmtnt7000 good try, but I think it is too hard to keep update with latest python modules. Since we had able to run the system before, we try to install the software version of that time.

@@ -29,7 +29,9 @@ RUN apt install -y --no-install-recommends $(cat ros-packages.txt | grep -v ^#)
RUN apt install -y python-pip
RUN git clone https://github.com/k-okada/rosinstall_generator /tmp/rosinstall_generator -b add_depend_type
RUN pip install /tmp/rosinstall_generator
RUN apt install -y python3-vcstool
RUN apt install -y python3-vcstool python-typing
# python-typing is necessary to avoid import error in build_ros1_dependencies.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The basic idea of this Docker files is to build same environment as a real Unitree robot, except for development tools that are only needed to build theenvronment, such as rosintall or vcstool. Therefore, installing python-typing here will not work on a real environment unless they alredy have one.

@@ -230,3 +230,17 @@ repositories:
postfix:
type: tar
url: http://mirror.postfix.jp/postfix-release/official/postfix-3.4.3.tar.gz
libfcl-dev:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I adopted this idea, but on the ohter hand, you can install an old version of geometric_shape, Currently we do not have clearner way of doing this, because the list of repositories are generated with rosinstall_generator (

[ ${UPDATE_SOURCE_ROOT} -eq 0 ] || ROS_PACKAGE_PATH=src:\$ROS_PACKAGE_PATH rosinstall_generator ${EUSCOLLADA_DEPENDS} ${ROSEUS_DEPENDS} ${ROSEUS_MONGO_DEPENDS} ${ROSEUS_SMACH_DEPENDS} ${JSK_ROBOT_STARTUP_DEPENDS} ${DIAGNOSTIC_AGGREGATOR} ${PR2EUS} --verbose --deps --rosdistro melodic --exclude RPP --depend-type buildtool buildtool_export build run | tee unitree_ros1_system.repos && \
),

may be we added something like https://github.com/jsk-ros-pkg/jsk_robot/blob/master/jsk_unitree_robot/cross/build_ros1.sh#L34


mkdir -p /home/user/ros1_dependencies_sources/build/libfcl-dev
cd /home/user/ros1_dependencies_sources/build/libfcl-dev
LD_LIBRARY_PATH=/opt/jsk/${INSTALL_ROOT}/ros1_dependencies/lib:${LD_LIBRARY_PATH}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it did not find libccd , when building test code, so I just skip it. I haven't tested if fcl works in running environmnet (in a real robot)

@tkmtnt7000
Copy link
Member Author

Thank you very much for reviewing. Sorry for late reply.

good try, but I think it is too hard to keep update with latest python modules. Since we had able to run the system before, we try to install the software version of that time.

I agree. When I worked my way through the process of trying to keep update with the latest python version, I found it much more difficult, too.

@tkmtnt7000
Copy link
Member Author

Moved to #1794

@tkmtnt7000 tkmtnt7000 closed this May 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants