Getting stuck at yt_dlp.extractor.lazy_extrac

[programmeddeath1]

I am running nuitka as follows -

python3 -m nuitka --standalone --onefile \
    --enable-plugin=pyqt5 \
    --enable-plugin=numpy \
    --include-package=roslib \
    --include-package=rospy \
    --include-package=std_msgs \
    --include-package=geometry_msgs \
    --include-package=sensor_msgs \
    --include-data-file=../delta_description/config/delta_robot.rviz=config/delta_robot.rviz \
    --include-data-file=/usr/bin/neato=graphviz/bin/neato \
    --include-data-file=/usr/bin/dot=graphviz/bin/dot \
    --include-data-file=/usr/bin/fdp=graphviz/bin/fdp \
    --include-data-file=/usr/bin/sfdp=graphviz/bin/sfdp \
    --include-data-file=/usr/bin/twopi=graphviz/bin/twopi \
    --include-data-file=/usr/bin/circo=graphviz/bin/circo \
    --output-dir=output \
    main.py

It gets stuck at the yt_dlp.extractor.lazy_extrac step. It's usually around 99.2 to 99.6 % of PASS 1:/

Nuitka-Options: Used command line options: --standalone --onefile
Nuitka-Options: --enable-plugin=pyqt5 --enable-plugin=numpy
Nuitka-Options: --include-package=roslib --include-package=rospy
Nuitka-Options: --include-package=std_msgs --include-package=geometry_msgs
Nuitka-Options: --include-package=sensor_msgs
Nuitka-Options: --include-data-file=../delta_description/config/delta_robot.rviz=config/delta_robot.rviz
Nuitka-Options: --include-data-file=/usr/bin/neato=graphviz/bin/neato
Nuitka-Options: --include-data-file=/usr/bin/dot=graphviz/bin/dot
Nuitka-Options: --include-data-file=/usr/bin/fdp=graphviz/bin/fdp
Nuitka-Options: --include-data-file=/usr/bin/sfdp=graphviz/bin/sfdp
Nuitka-Options: --include-data-file=/usr/bin/twopi=graphviz/bin/twopi
Nuitka-Options: --include-data-file=/usr/bin/circo=graphviz/bin/circo
Nuitka-Options: --output-dir=output main.py
Nuitka-Plugins:WARNING: numpy: This plugin has been deprecated, do not enable it
Nuitka-Plugins:WARNING: anymore.
Nuitka-Plugins:WARNING: pyqt5: For the obsolete PyQt5 the Nuitka support is
Nuitka-Plugins:WARNING: incomplete. Threading, callbacks to compiled functions,
Nuitka-Plugins:WARNING: etc. may not be working.
Nuitka-Plugins:WARNING:     Complex topic! More information can be found at
Nuitka-Plugins:WARNING: https://nuitka.net/info/pyqt5.html
Nuitka: Starting Python compilation with Nuitka '2.2.3' on Python '3.10'
Nuitka: commercial grade 'not installed'.
Nuitka-Plugins:WARNING: options-nanny: Using module 'attr' (version 21.2.0) with
Nuitka-Plugins:WARNING: incomplete support due to condition 'version("attrs") <
Nuitka-Plugins:WARNING: (23,1)': 'attrs' is not fully supported before version
Nuitka-Plugins:WARNING: 23.1.0
Nuitka-Plugins:pyqt5: Unwanted import of 'PySide2' that conflicts with 'PyQt5'  
Nuitka-Plugins:pyqt5: encountered, preventing its inclusion. As a result an
Nuitka-Plugins:pyqt5: "ImportError" might be given at run time. Uninstall the
Nuitka-Plugins:pyqt5: module it for fully compatible behavior with the
Nuitka-Plugins:pyqt5: uncompiled code.
Nuitka-Plugins:WARNING: options-nanny: Module has parameter: Torch JIT is       
Nuitka-Plugins:WARNING: disabled by default in standalone mode, make a choice
Nuitka-Plugins:WARNING: explicit with
Nuitka-Plugins:WARNING: '--module-parameter=torch-disable-jit=yes|no'
PASS 1: 99.6%|████████████████████████▉| 5741/5765, yt_dlp.extractor.lazy_extrac

python -m nuitka --version

2.2.3
Commercial: None
Python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
Flavor: Debian Python
Executable: /usr/bin/python3
OS: Linux
Arch: aarch64
Distribution: Ubuntu (based on Debian) 22.04.4
Version C compiler: /usr/bin/gcc (gcc 11).

nuitka was installed with

python3 -m pip install nuitka

python -m pip freeze

yt-dlp==2022.4.8

I tried upgrading it too it seems to be stuck at - PASS 1: 99.3%|████████████████████████▊| 5838/5880, yt_dlp.extractor.lazy_extrac

I am not using yt-dlp but seems one of the used packages has that as a dependency.

My imports are as follows -

import sys, os, shutil,zipfile
import typing
from PyQt5 import QtCore, QtGui, QtWidgets
from PyQt5.QtWidgets import *
from PyQt5.QtGui import *
from PyQt5.QtCore import *
from collections import defaultdict

from PyQt5.uic import loadUi
from datetime import datetime
import os,subprocess, json
from time import sleep
import time
import atexit
import depthai as dai
from libraries import *
import math
import pandas as pd
import numpy as np
import socket
import roslib #; roslib.load_manifest('rviz_python_tutorial')
import sys

from rviz import bindings as rviz
# ROS Imports
import rospy
from std_msgs.msg import String, Empty, Int32, UInt8
from trajectory_msgs.msg import JointTrajectoryPoint
from geometry_msgs.msg import Point
from custom_msg.srv import Trajectory,TrajectoryResponse,TrajectoryRequest
from custom_msg.msg import ObjectPose,CameraStreamMsg,JoggingMsg
import cv2

from ultralytics import YOLO
from threading import Thread
import yaml,torch

Is there a way to debug or print more in detail during packaging?

[kayhayen]

You might want to use no --include-package and instead report the missing dependencies so they can be fixed, and at least use only --include-module as it otherwise includes a lot of code, and you probably run out of memory.

[programmeddeath1]

It remains stuck at that point with no increase in ram or CPU usage.
when i run with just

python3 -m nuitka main.py

It creates a .bin file which I can execute from my application path which contains other folders like images, configs, libraries, models etc.

Ill try adding those missing dependency folders one by one to the path.

But I want to create a single executable which will run the app standalone. If i run with just python3 -m nuitka --follow-imports main.py the follow imports also gets stuck at that point where yt-dlib gets stuck at extract. How should i proceed if i want to add the python and other dependencies also into the standalone folder package?

[programmeddeath1]

Thank you for your help!
When I simply run

python3 -m nuitka main.py

It creates a main.bin file, and if i run that file from my folder everything in that PyQT5 application works perfectly. I had read there were issues with callback and threading in your pyqt5 support doc, but even threading using background processes and event callbacks are working properly.

I am only facing 1 major issue with the camera stream callback which happens through a slot signal mechanis,, where an image is updated from the background stream. The process runs in the background as per the logs but the image, does'nt seem to render, but if the process is killed, a single image is rendered after the event end.

Other threaded screen updates using signals and slots and events are working properly. Seems like this one issue could also be workedaround which would save an entire migration to pyqt6 due to many dependencies to be handled.

The doc had suggested very limiting workarounds, do you think If i try, this single threaded signal for images could also be fixed through a workaround? You have better understanding as to guide if I should put my effort in this or look for another solution.

[kayhayen]

How about you report the output of --standalone --noinclude-custom-mode=yt_dlp:error and we remove as bloat, if that's unused. For distribution compiling the top level file only won't do much.