qh3 is having a major version

[Ousret]

Hello there,

As pymobiledevice3 was a early adopter of qh3 and now a "long" dependent of it,
I though to give a heads up about the forthcoming major of qh3.

The idea would to promote qh3 v1 to stable, but before that the least we can do is to ensure the compatibility.
Normally it should work as is.

The main change to be concerned about is: "qh3 no longer depend on anything. not even cryptography". Wrote a dedicated bridge using PyO3 and aws-lc-rs. The idea behind this is to ease the http3 adoption through niquests

I can see that you have an explicit dependency on cryptography.

Would you assess the v1.0.0b1 so that I can move forward? See jawah/qh3#25

Regards,

[doronz88]

I just verified with latest version and it does break.
Do I need to make some API changes?
Anyhow at least for now I'll add qh3<1.0.0 to avoid API break

That was the full traceback, if it gives you any hints:

2024-04-17 10:56:40 users-MacBook-Pro.local CoreDeviceTunnelService[70209] DEBUG Connecting to fe80::fc1b:abff:febe:948f%en15:64471
Traceback (most recent call last):
  File "/Users/user/.local/bin/pymobiledevice3", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/user/dev/pymobiledevice3/pymobiledevice3/__main__.py", line 100, in main
    cli()
  File "/Users/user/Library/Application Support/pipx/venvs/pymobiledevice3/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/Library/Application Support/pipx/venvs/pymobiledevice3/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/user/Library/Application Support/pipx/venvs/pymobiledevice3/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/Library/Application Support/pipx/venvs/pymobiledevice3/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/Library/Application Support/pipx/venvs/pymobiledevice3/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/Library/Application Support/pipx/venvs/pymobiledevice3/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/dev/pymobiledevice3/pymobiledevice3/cli/cli_common.py", line 120, in wrapper
    func(*args, **kwargs)
  File "/Users/user/dev/pymobiledevice3/pymobiledevice3/cli/remote.py", line 193, in cli_start_tunnel
    asyncio.run(
  File "/opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/user/dev/pymobiledevice3/pymobiledevice3/cli/remote.py", line 170, in start_tunnel_task
    await tunnel_task(service, secrets=secrets, script_mode=script_mode, max_idle_timeout=max_idle_timeout,
  File "/Users/user/dev/pymobiledevice3/pymobiledevice3/cli/remote.py", line 113, in tunnel_task
    async with start_tunnel(
  File "/opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/dev/pymobiledevice3/pymobiledevice3/remote/tunnel_service.py", line 956, in start_tunnel
    async with start_tunnel_over_core_device(
  File "/opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/dev/pymobiledevice3/pymobiledevice3/remote/tunnel_service.py", line 940, in start_tunnel_over_core_device
    async with service_provider.start_quic_tunnel(
  File "/opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/dev/pymobiledevice3/pymobiledevice3/remote/tunnel_service.py", line 409, in start_quic_tunnel
    async with aioquic_connect(
  File "/opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/dev/qh3/qh3/asyncio/client.py", line 97, in connect
    await protocol.wait_connected()
  File "/Users/user/dev/qh3/qh3/asyncio/protocol.py", line 127, in wait_connected
    await asyncio.shield(self._connected_waiter)
ConnectionError

[Ousret]

Unfortunately the stack trace isn't helpful.
What cipher, signature and key type are you using to establish the tunnel?

[doronz88]

These lines should help provide all useful information about the QUIC connection establishment

pymobiledevice3/pymobiledevice3/remote/tunnel_service.py

Lines 387 to 417 in 9713fab

	async def start_quic_tunnel(
	self, secrets_log_file: Optional[TextIO] = None,
	max_idle_timeout: float = RemotePairingQuicTunnel.MAX_IDLE_TIMEOUT) -> AsyncGenerator[TunnelResult, None]:
	private_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
	parameters = await self.create_quic_listener(private_key)
	cert = make_cert(private_key, private_key.public_key())
	configuration = QuicConfiguration(
	alpn_protocols=['RemotePairingTunnelProtocol'],
	is_client=True,
	certificate=cert,
	private_key=private_key,
	verify_mode=VerifyMode.CERT_NONE,
	verify_hostname=False,
	max_datagram_frame_size=RemotePairingQuicTunnel.MAX_QUIC_DATAGRAM,
	idle_timeout=max_idle_timeout
	)
	configuration.secrets_log_file = secrets_log_file
	host = self.hostname
	port = parameters['port']
	self.logger.debug(f'Connecting to {host}:{port}')
	async with aioquic_connect(
	host,
	port,
	configuration=configuration,
	create_protocol=RemotePairingQuicTunnel,
	) as client:
	self.logger.debug('quic connected')
	client = cast(RemotePairingQuicTunnel, client)
	await client.wait_connected()

[Ousret]

Ah. I think I got what is wrong.

extract from changelog:

Breaking: You may no longer pass certificates (along with private keys) as object that comes from cryptography. You have to encode them into PEM format.

private_key = rsa.generate_private_key(public_exponent=65537, key_size=2048) 
parameters = await self.create_quic_listener(private_key) 
cert = make_cert(private_key, private_key.public_key()) 

I suppose private_key and cert are objects from cryptography.
Also, I should raise a clear exception showing that this is not doable anymore.

[doronz88]

I still need it as a PSK, so just changing the encoding of it?

[doronz88]

Also, I just released v4.1.3 with qh3>=0.11.5,<1.0.0 so we'll have a smooth migration when you'll release yours :)

Huge thanks for the heads up! 😊

[Ousret]

Of course, you may keep your private key and cert object for ext usage. Just convert it when establishing the quic tunnel.
It's the least we can do.

[doronz88]

I have a cryptography's Certificate object. I tried using the following to convert the encoding into PEM instead:

        configuration = QuicConfiguration(
            alpn_protocols=['RemotePairingTunnelProtocol'],
            is_client=True,
            certificate=cert.public_bytes(Encoding.PEM),
            private_key=private_key.private_bytes(Encoding.PEM, PrivateFormat.TraditionalOpenSSL,
                                                  NoEncryption()).decode(),
            verify_mode=VerifyMode.CERT_NONE,
            verify_hostname=False,
            max_datagram_frame_size=RemotePairingQuicTunnel.MAX_QUIC_DATAGRAM,
            idle_timeout=max_idle_timeout
        )

Still am getting the same errors. Can you guide me as to how exactly should I convert the data-structure?

[Ousret]

Try the following: dismiss the private_key and certificate kwargs in QuicConfiguration and use configuration.load_cert_chain(cert.public_bytes(Encoding.PEM), private_key.private_bytes(...))

[doronz88]

The new version seems to work great :)
Can you please update when it's officially released so I'll add a qh3>=1.0.0

[Ousret]

Perfect 👌
Will notify when it land.

[Ousret]

The release process is in progress. Expect the PyPI tag to be pushed in an hour or so.
Could you re-assess this new version using the "official" pre-built wheel and report back?

regards,

[doronz88]

Just tested and it works great :)