Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pyarmor in Docker: ERROR invalid license token, try to run pyarmor reg to register license again #1542

Closed
goraje opened this issue Oct 19, 2023 · 36 comments
Labels

Comments

@goraje
Copy link

goraje commented Oct 19, 2023

Hi, I've been trying to run Pyarmor from a Docker container, but even though the registration process was fixed with Pyarmor 8.4.0 when I run code obfuscation with pyarmor gen I'm getting the following error:

root@4b279a129f6b:/# pyarmor reg pyarmor-device-regfile-6005.7.zip
INFO     Python 3.11.6
INFO     Pyarmor 8.4.0 (trial), 000000, non-profits
INFO     Platform linux.aarch64
INFO     register "pyarmor-device-regfile-6005.7.zip"
INFO     machine id in group license: m98a1e7f27fc612f428043fb2edb27122
INFO     got machine id: m47f31be7ced2f9161c5221e56364fac2
INFO     got machine id: l47f31be7ced2f9161c5221e56364fac2
INFO     got machine id: i98444de277e1b3dd9b17d2181631a73c
INFO     got machine id: k1a298b2ce465500b355f6c9d777b845f
INFO     got machine id: g1a298b2ce465500b355f6c9d777b845f
INFO     got machine id: bb6889a725ecfcc7acd18b1dd249f4df9
INFO     no machine id matchs this group license
INFO     take this machine as docker container, and connect to docker host for authentication...
INFO     got docker host machine id: m98a1e7f27fc612f428043fb2edb27122
INFO     got docker host machine id: l98a1e7f27fc612f428043fb2edb27122
INFO     got docker host machine id: i98a1e7f27fc612f428043fb2edb27122
INFO     got docker host machine id: kc6eb2c01acd9320d96ee5992197d6a44
INFO     got docker host machine id: gc6eb2c01acd9320d96ee5992197d6a44
INFO     got docker host machine id: b28c5a05ba0fa8e759f3e172660d31c04
INFO     This license registration information:

License Type    : pyarmor-group
License No.     : pyarmor-vax-006005
License To      : *****
License Product : ******

BCC Mode        : Yes
RFT Mode        : Yes

Notes
* Offline obfuscation

root@4b279a129f6b:/# pyarmor gen foo.py
INFO     Python 3.11.6
INFO     Pyarmor 8.4.0 (group), 006005, ******
INFO     Platform linux.aarch64
INFO     search inputs ...
INFO     find script foo.py
INFO     find 1 top resources
ERROR    invalid license token, try to run `pyarmor reg` to register license again

and pyarmor.error.log still showing:

266 MainProcess 2023-10-18 06:58:09,818
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pyarmor/cli/__main__.py", line 712, in main
    main_entry(sys.argv[1:])
  File "/usr/local/lib/python3.11/site-packages/pyarmor/cli/__main__.py", line 700, in main_entry
    return args.func(ctx, args)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pyarmor/cli/__main__.py", line 230, in cmd_gen
    builder.process(options)
  File "/usr/local/lib/python3.11/site-packages/pyarmor/cli/generate.py", line 147, in process
    Pytransform3.pre_build(self.ctx)
  File "/usr/local/lib/python3.11/site-packages/pyarmor/cli/core/__init__.py", line 117, in pre_build
    m = Pytransform3.init(ctx)
        ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pyarmor/cli/core/__init__.py", line 97, in init
    m.init_ctx(ctx)
RuntimeError: invalid license token, try to run `pyarmor reg` to register license again

If I repeat the pyarmor reg registration procedure exactly the same thing happens. I'm working on an Apple silicon MacBook Pro.
One more thing that may or may not be of importance in this issue is that I can see the communication between the pyarmor-auth service and the container when I run pyarmor reg (I can see packets received and packets sent in the log of pyarmor-auth), whereas there is no indication of any communication when I run pyarmor gen.

@goraje goraje added the bug label Oct 19, 2023
@goraje goraje changed the title Pyarmor in docker: ERROR invalid license token, try to run pyarmor reg to register license again Pyarmor in Docker: ERROR invalid license token, try to run pyarmor reg to register license again Oct 19, 2023
@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

What's the output of pyarmor -v in docker container?

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

Try to start docker container with extra options --add-host=host.docker.internal:host-gateway

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

And is there /dev/disk in docker container?

@goraje
Copy link
Author

goraje commented Oct 19, 2023

Hi, I did what you instructed and this is the output. The problem still persists.

root@446598bcab51:/# pyarmor -v
Pyarmor 8.4.0 (group), 006005, ******

License Type    : pyarmor-group
License No.     : pyarmor-vax-006005
License To      : *****
License Product : ******

BCC Mode        : Yes
RFT Mode        : Yes

Notes
* Offline obfuscation

INFO     got docker host machine id: m98a1e7f27fc612f428043fb2edb27122
INFO     got docker host machine id: l98a1e7f27fc612f428043fb2edb27122
INFO     got docker host machine id: i98a1e7f27fc612f428043fb2edb27122
INFO     got docker host machine id: kc6eb2c01acd9320d96ee5992197d6a44
INFO     got docker host machine id: gc6eb2c01acd9320d96ee5992197d6a44
INFO     got docker host machine id: b28c5a05ba0fa8e759f3e172660d31c04
root@446598bcab51:/# echo "print('TEST')" > foo.py
root@446598bcab51:/# pyarmor gen foo.py
INFO     Python 3.11.6
INFO     Pyarmor 8.4.0 (group), 006005, RADKit
INFO     Platform linux.aarch64
INFO     search inputs ...
INFO     find script foo.py
INFO     find 1 top resources
ERROR    invalid license token, try to run `pyarmor reg` to register license again

@goraje
Copy link
Author

goraje commented Oct 19, 2023

No /dev/disk is not listed as one of the devices

root@7b51114ec541:/# ls /dev/
console  fd  full  mqueue  null  ptmx  pts  random  shm  stderr  stdin	stdout	tty  urandom  zero

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

How about $ tracert host.docker.internal?

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

It seems it should be traceroute not tracert. If not installed, try to install it by apt install traceroute in docker container

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

And what's the output of pyarmor-auth? It listens on IPv4 or IPv6 address, if it's IPv6, this will be problem.

@goraje
Copy link
Author

goraje commented Oct 19, 2023

Here' the traceroute output

root@0b5903d41cc2:/# traceroute host.docker.internal
traceroute to host.docker.internal (192.168.5.2), 30 hops max, 60 byte packets
 1  172.17.0.1 (172.17.0.1)  0.067 ms  0.012 ms  0.008 ms
 2  host.docker.internal (192.168.5.2)  0.727 ms  0.691 ms  0.666 ms
 3  * * *
 4  * * *
 5  * * *
 6  * * *
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *

and also maybe this could be useful:

root@0b5903d41cc2:/# telnet host.docker.internal 29092
Trying 192.168.5.2...
Connected to host.docker.internal.
Escape character is '^]'.

@goraje
Copy link
Author

goraje commented Oct 19, 2023

But the registration proceeds without fail so it seems that the connectivity to the auth service is there. There's just no call being made to it when I run pyarmor gen as far as I can tell.

@goraje
Copy link
Author

goraje commented Oct 19, 2023

Please correct me if I'm wrong but pyarmor gen should call the auth service for a runtime key and I was monitoring if the else part of the DockerAuthHandler process method was being called during pyarmor gen and it's not.

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

Try this solution, make sure host.docker.internal parsed to 172.17.0.1 in both of docker container and host

For example, adding one line in /etc/hosts

host.docker.internal 172.17.0.1

@goraje
Copy link
Author

goraje commented Oct 19, 2023

Hi, actually in my case on a Mac in the Docker container the host.docker.internal entry resolves to 192.168.5.2 and when I set it to the same value in /etc/hosts of the macOS host it doesn't change or fix the behavior with pyarmor gen (pyarmor reg still works). Setting the value to 172.17.0.1 is a no go because this network does not exist on the host.

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

Well, how about start docker with --add-host=host.docker.internal:172.17.0.1

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

If host.docker.internal isn't in same network of docker container, it will not sent auth request to docker host. This is the problem.

@goraje
Copy link
Author

goraje commented Oct 19, 2023

Then the registration fails

root@025a348e78d0:/# pyarmor reg pyarmor-device-regfile-6005.7.zip
INFO     Python 3.11.6
INFO     Pyarmor 8.4.0 (trial), 000000, non-profits
INFO     Platform linux.aarch64
INFO     register "pyarmor-device-regfile-6005.7.zip"
INFO     machine id in group license: m98a1e7f27fc612f428043fb2edb27122
INFO     got machine id: m64bf28cba3e2661680ccc54b3cd92b41
INFO     got machine id: l64bf28cba3e2661680ccc54b3cd92b41
INFO     got machine id: ia3e940b241521a711b8961de85aaedfb
INFO     got machine id: k6ff0851f0ca3db1f69c2eae5d43688e2
INFO     got machine id: g6ff0851f0ca3db1f69c2eae5d43688e2
INFO     got machine id: b7e263a34ce45701ac80be26b42806151
INFO     no machine id matchs this group license
INFO     take this machine as docker container, and connect to docker host for authentication...
INFO     could not get docker host machine id

if this machine is docker container, please run command `pyarmor-auth` in docker host, and try it again

otherwise please generate new group device license for this machine

more information please check section "using group license" in documentation "how-to register" guide

ERROR    this group device license is not for this machine

and there is no connectivity between the host and the container

@goraje
Copy link
Author

goraje commented Oct 19, 2023

If host.docker.internal isn't in same network of docker container, it will not sent auth request to docker host. This is the problem.

On macOS and Windows for that matter it won't be the case, because contrary to Linux you cannot run Docker natively. Instead you run it on the side in a Linux VM that resides within some kind of a private network (usually 192...). The Docker network 172... will exist only on the Linux VM but will not be present on your macOS/Windows host. Instead the docker.sock from the VM is passed to the host OS to allow for interaction via Docker's CLI and communication between the containers and the host is port forwarding-based.

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

Well I need research this to find new solution.

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

What's the output of ifconfig -a in docker host (MacOS)?

@goraje
Copy link
Author

goraje commented Oct 19, 2023

Well I need research this to find new solution.

Of course. I'm more than happy to offer my help. This is the output on the macOS side:

ifconfig -a
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
	options=1203<RXCSUM,TXCSUM,TXSTATUS,SW_TIMESTAMP>
	inet 127.0.0.1 netmask 0xff000000
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
	nd6 options=201<PERFORMNUD,DAD>
gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280
stf0: flags=0<> mtu 1280
anpi2: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=400<CHANNEL_IO>
	ether 36:d6:7f:85:5d:0c
	inet6 fe80::34d6:7fff:fe85:5d0c%anpi2 prefixlen 64 scopeid 0x4
	nd6 options=201<PERFORMNUD,DAD>
	media: none
	status: inactive
anpi1: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=400<CHANNEL_IO>
	ether 36:d6:7f:85:5d:0b
	inet6 fe80::34d6:7fff:fe85:5d0b%anpi1 prefixlen 64 scopeid 0x5
	nd6 options=201<PERFORMNUD,DAD>
	media: none
	status: inactive
anpi0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=400<CHANNEL_IO>
	ether 36:d6:7f:85:5d:0a
	inet6 fe80::34d6:7fff:fe85:5d0a%anpi0 prefixlen 64 scopeid 0x6
	nd6 options=201<PERFORMNUD,DAD>
	media: none
	status: inactive
en5: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=400<CHANNEL_IO>
	ether 36:d6:7f:85:5d:ea
	nd6 options=201<PERFORMNUD,DAD>
	media: none
	status: inactive
en6: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=400<CHANNEL_IO>
	ether 36:d6:7f:85:5d:eb
	nd6 options=201<PERFORMNUD,DAD>
	media: none
	status: inactive
en7: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=400<CHANNEL_IO>
	ether 36:d6:7f:85:5d:ec
	nd6 options=201<PERFORMNUD,DAD>
	media: none
	status: inactive
en1: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
	options=460<TSO4,TSO6,CHANNEL_IO>
	ether 36:ed:8b:e0:72:80
	media: autoselect <full-duplex>
	status: inactive
en2: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
	options=460<TSO4,TSO6,CHANNEL_IO>
	ether 36:ed:8b:e0:72:84
	media: autoselect <full-duplex>
	status: inactive
en3: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
	options=460<TSO4,TSO6,CHANNEL_IO>
	ether 36:ed:8b:e0:72:88
	media: autoselect <full-duplex>
	status: inactive
ap1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=6463<RXCSUM,TXCSUM,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
	ether ea:89:f3:af:e8:8e
	inet6 fe80::e889:f3ff:feaf:e88e%ap1 prefixlen 64 scopeid 0xe
	nd6 options=201<PERFORMNUD,DAD>
	media: autoselect
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=6463<RXCSUM,TXCSUM,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
	ether c8:89:f3:af:e8:8e
	inet 192.168.1.7 netmask 0xffffff00 broadcast 192.168.1.255
	inet6 fe80::10b8:8678:9d8b:46b2%en0 prefixlen 64 secured scopeid 0xf
	nd6 options=201<PERFORMNUD,DAD>
	media: autoselect
	status: active
bridge0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=63<RXCSUM,TXCSUM,TSO4,TSO6>
	ether 36:ed:8b:e0:72:80
	Configuration:
		id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0
		maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200
		root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0
		ipfilter disabled flags 0x0
	member: en1 flags=3<LEARNING,DISCOVER>
	        ifmaxaddr 0 port 10 priority 0 path cost 0
	member: en2 flags=3<LEARNING,DISCOVER>
	        ifmaxaddr 0 port 11 priority 0 path cost 0
	member: en3 flags=3<LEARNING,DISCOVER>
	        ifmaxaddr 0 port 12 priority 0 path cost 0
	nd6 options=201<PERFORMNUD,DAD>
	media: <unknown type>
	status: inactive
en4: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=6467<RXCSUM,TXCSUM,VLAN_MTU,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
	ether 50:a0:30:01:fc:32
	nd6 options=201<PERFORMNUD,DAD>
	media: autoselect (none)
	status: inactive
awdl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=6463<RXCSUM,TXCSUM,TSO4,TSO6,CHANNEL_IO,PARTIAL_CSUM,ZEROINVERT_CSUM>
	ether ca:b3:7f:36:3d:9d
	media: autoselect
	status: active
llw0: flags=8822<BROADCAST,SMART,SIMPLEX,MULTICAST> mtu 1500
	options=400<CHANNEL_IO>
	ether ca:b3:7f:36:3d:9d
	media: autoselect
	status: inactive

This is the Docker container

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.0.2  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:ac:11:00:02  txqueuelen 0  (Ethernet)
        RX packets 829  bytes 9479478 (9.0 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 506  bytes 35160 (34.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

and this is the man-in-between (the Linux VM I mentioned before):

ifconfig -a
docker0   Link encap:Ethernet  HWaddr 02:42:9B:18:7C:9C
          inet addr:172.17.0.1  Bcast:172.17.255.255  Mask:255.255.0.0
          inet6 addr: fe80::42:9bff:fe18:7c9c/64 Scope:Link
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:4579 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6304 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:278633 (272.1 KiB)  TX bytes:55119880 (52.5 MiB)

eth0      Link encap:Ethernet  HWaddr 52:55:55:DD:0D:8E
          inet addr:192.168.5.15  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: fe80::5055:55ff:fedd:d8e/64 Scope:Link
          inet6 addr: fec0::5055:55ff:fedd:d8e/64 Scope:Site
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5231 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3701 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:441361 (431.0 KiB)  TX bytes:511349 (499.3 KiB)

eth1      Link encap:Ethernet  HWaddr 5A:94:EF:38:97:3A
          inet addr:192.168.107.2  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: fe80::5894:efff:fe38:973a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:38783 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4202 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:57261212 (54.6 MiB)  TX bytes:316191 (308.7 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

@goraje
Copy link
Author

goraje commented Oct 19, 2023

And just to reiterate what I've said before. The container can send requests to host.docker.interal:29092 - the pyarmor-auth service and receive responses (pyarmor reg both sent and received a response).

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

and this is the man-in-between (the Linux VM I mentioned before)

Is it possible to run pyarmor-auth in this Linux VM? I have no more idea about docker in MacOs/Windows.

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

And just to reiterate what I've said before. The container can send
requests to host.docker.interal:29092 - the pyarmor-auth service
and receive responses ( pyarmor reg both sent and received a
response).

So what's the ip address of host.docker.internal in docker container?It's 172.x... or 192.x...

@goraje
Copy link
Author

goraje commented Oct 19, 2023

I've tried it out also setting host.docker.internal to 172.17.0.1 on both the Linux VM and the container. The problem still persists and this is the log from pyarmor-auth.

colima-pyarmor:/Users/jgora$ pyarmor-auth --debug pyarmor-device-regfile-6005.8.zip
2023-10-19 14:50:45,961: work path: /home/jgora.linux/.pyarmor/docker
2023-10-19 14:50:45,963: register "pyarmor-device-regfile-6005.8.zip"
2023-10-19 14:50:45,967: extracting license.lic
2023-10-19 14:50:45,967: extracting .pyarmor_capsule.zip
2023-10-19 14:50:45,968: machine id in group license: mec58608fff941488f5a72f812f3203c2
2023-10-19 14:50:45,968: got machine id: mec58608fff941488f5a72f812f3203c2
2023-10-19 14:50:45,968: this machine id matchs group license
2023-10-19 14:50:45,968: extracting tokens/mec58608fff941488f5a72f812f3203c2
2023-10-19 14:50:45,970: machine id: [b'mec58608fff941488f5a72f812f3203c2', b'lec58608fff941488f5a72f812f3203c2', b'i02dcbedfc6e0dd4f6b2d59a758fa9f15', b'kba0ba013eeb9a873be30a01d64da31e7', b'gba0ba013eeb9a873be30a01d64da31e7', b'b310417f33732ca2f9ce1e01e0b05c416']
2023-10-19 14:50:45,970: listen container auth request on 0.0.0.0:29092
2023-10-19 14:51:02,989: receive request from ('172.17.0.2', 53510)
2023-10-19 14:51:02,989: request data (64): b'PADHxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
2023-10-19 14:51:02,989: send auth result to ('172.17.0.2', 53510)
2023-10-19 14:51:02,989: response data (204): b'mec58608fff941488f5a72f812f3203c2\nlec58608fff941488f5a72f812f3203c2\ni02dcbedfc6e0dd4f6b2d59a758fa9f15\nkba0ba013eeb9a873be30a01d64da31e7\ngba0ba013eeb9a873be30a01d64da31e7\nb310417f33732ca2f9ce1e01e0b05c416\x00'
2023-10-19 14:51:09,439: receive request from ('172.17.0.2', 56422)
2023-10-19 14:51:09,439: request data (64): b'PADHxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
2023-10-19 14:51:09,439: send auth result to ('172.17.0.2', 56422)
2023-10-19 14:51:09,439: response data (204): b'mec58608fff941488f5a72f812f3203c2\nlec58608fff941488f5a72f812f3203c2\ni02dcbedfc6e0dd4f6b2d59a758fa9f15\nkba0ba013eeb9a873be30a01d64da31e7\ngba0ba013eeb9a873be30a01d64da31e7\nb310417f33732ca2f9ce1e01e0b05c416\x00'
2023-10-19 14:51:16,028: receive request from ('172.17.0.2', 56432)
2023-10-19 14:51:16,028: request data (32): b'PADK\x00\x00\x00\x00\x00\x00\x00\x00mlhdjeaemmbldfdfibgk'
Segmentation fault

Requests/responses:

  1. after pyarmor reg command
  2. after pyarmor -v command in the container
  3. after pyarmor gen in container - causes the segmentation fault

@goraje
Copy link
Author

goraje commented Oct 19, 2023

And just to reiterate what I've said before. The container can send
requests to host.docker.interal:29092 - the pyarmor-auth service
and receive responses ( pyarmor reg both sent and received a
response).

So what's the ip address of host.docker.internal in docker container?It's 172.x... or 192.x...

The address in the container was 192.x... when we discussed it previously.

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

But I think host.docker.internal could be configured to 172.17.0.1 in docker container by editing /etc/hosts in docker container.

Even it may not be changed in docker host.

@goraje
Copy link
Author

goraje commented Oct 19, 2023

But I think host.docker.internal could be configured to 172.17.0.1 in docker container by editing /etc/hosts in docker container.

Even it may not be changed in docker host.

When I worked directly on the Linux VM I've set it to 172.17.0.1 on both the container and the Linux VM

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

Or just for a test, patch /path/to/pyarmor/cli/register.py in docker container, replace host.docker.internal with 172.17.0.1, then run pyarmor reg xxx.

If docker host pyarmor-auth could receive request and send response, this will be one solution.

@goraje
Copy link
Author

goraje commented Oct 19, 2023

Or just for a test, patch /path/to/pyarmor/cli/register.py in docker container, replace host.docker.internal with 172.17.0.1, then run pyarmor reg xxx.

If docker host pyarmor-auth could receive request and send response, this will be one solution.

The result is exactly the same as here:

colima-pyarmor:/Users/jgora$ pyarmor-auth --debug pyarmor-device-regfile-6005.8.zip
2023-10-19 14:50:45,961: work path: /home/jgora.linux/.pyarmor/docker
2023-10-19 14:50:45,963: register "pyarmor-device-regfile-6005.8.zip"
2023-10-19 14:50:45,967: extracting license.lic
2023-10-19 14:50:45,967: extracting .pyarmor_capsule.zip
2023-10-19 14:50:45,968: machine id in group license: mec58608fff941488f5a72f812f3203c2
2023-10-19 14:50:45,968: got machine id: mec58608fff941488f5a72f812f3203c2
2023-10-19 14:50:45,968: this machine id matchs group license
2023-10-19 14:50:45,968: extracting tokens/mec58608fff941488f5a72f812f3203c2
2023-10-19 14:50:45,970: machine id: [b'mec58608fff941488f5a72f812f3203c2', b'lec58608fff941488f5a72f812f3203c2', b'i02dcbedfc6e0dd4f6b2d59a758fa9f15', b'kba0ba013eeb9a873be30a01d64da31e7', b'gba0ba013eeb9a873be30a01d64da31e7', b'b310417f33732ca2f9ce1e01e0b05c416']
2023-10-19 14:50:45,970: listen container auth request on 0.0.0.0:29092
2023-10-19 14:51:02,989: receive request from ('172.17.0.2', 53510)
2023-10-19 14:51:02,989: request data (64): b'PADHxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
2023-10-19 14:51:02,989: send auth result to ('172.17.0.2', 53510)
2023-10-19 14:51:02,989: response data (204): b'mec58608fff941488f5a72f812f3203c2\nlec58608fff941488f5a72f812f3203c2\ni02dcbedfc6e0dd4f6b2d59a758fa9f15\nkba0ba013eeb9a873be30a01d64da31e7\ngba0ba013eeb9a873be30a01d64da31e7\nb310417f33732ca2f9ce1e01e0b05c416\x00'
2023-10-19 14:51:09,439: receive request from ('172.17.0.2', 56422)
2023-10-19 14:51:09,439: request data (64): b'PADHxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
2023-10-19 14:51:09,439: send auth result to ('172.17.0.2', 56422)
2023-10-19 14:51:09,439: response data (204): b'mec58608fff941488f5a72f812f3203c2\nlec58608fff941488f5a72f812f3203c2\ni02dcbedfc6e0dd4f6b2d59a758fa9f15\nkba0ba013eeb9a873be30a01d64da31e7\ngba0ba013eeb9a873be30a01d64da31e7\nb310417f33732ca2f9ce1e01e0b05c416\x00'
2023-10-19 14:51:16,028: receive request from ('172.17.0.2', 56432)
2023-10-19 14:51:16,028: request data (32): b'PADK\x00\x00\x00\x00\x00\x00\x00\x00mlhdjeaemmbldfdfibgk'
Segmentation fault

and this should be the case actually since if I had host.docker.internal mapped to 172.17.0.1 in /etc/hosts previously than editing register.py didn't change much (we just skipped the lookup to /etc/hosts).
At the end of the log seen above is the request to generate the runtime key, but then we get a Segmentation fault and a response is not sent.

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

2023-10-19 14:51:16,028: receive request from ('172.17.0.2', 56432)
2023-10-19 14:51:16,028: request data (32):
b'PADK\x00\x00\x00\x00\x00\x00\x00\x00mlhdjeaemmbldfdfibgk'
Segmentation fault

Since pyarmor-auth could receive auth request from pyarmor gen ..., it seems network is fine.

Could you run pyarmor gen key --bind-data test -e 3 in docker host? Just test generate_runtime_key crash or not.

@goraje
Copy link
Author

goraje commented Oct 19, 2023

Could you run pyarmor gen key --bind-data test -e 3 in docker host? Just test generate_runtime_key crash or not.

Of course. It seems that it works just fine on the Docker host (Linux VM) itself.

colima-pyarmor:/Users/jgora$ pyarmor gen key --bind-data test -e 3
INFO     Python 3.10.13
INFO     Pyarmor 8.4.0 (group), 006005, ******
INFO     Platform linux.aarch64
INFO     start to generate outer runtime key "pyarmor.rkey"
INFO     write dist/pyarmor.rkey
INFO     generate outer runtime key OK

@goraje
Copy link
Author

goraje commented Oct 19, 2023

By editing pyarmor/cli/docker.py and pyarmor/cli/generate.py I've managed to establish that the Segmentation fault occurs in Builder's generate_runtime_key when it invokes Pytransform3.generate_runtime_key(self.ctx, outer). I don't know if this is helpful but when I execute pyarmor gen key --bind-data test -e 3 inside of the Docker container the ctx object received by the builder looks as follows:

{'alias_suffix': '{0}-{1}',
 'bootstrap_template': '# Pyarmor $rev, $timestamp\n'
                       'from $package import __pyarmor__\n'
                       '__pyarmor__(__name__, $path, $code)\n',
 'cfg': <configparser.ConfigParser object at 0xffff9df182e0>,
 'cmd_options': {'user_data': 'goejobmjndofhlbkknme'},
 'encoding': None,
 'extra_libs': {},
 'extra_resources': [],
 'global_path': '/home/jgora.linux/.pyarmor/docker/config',
 'home_path': '/home/jgora.linux/.pyarmor/docker',
 'input_paths': [],
 'local_path': '.pyarmor',
 'module_builtins': set(),
 'module_relations': {},
 'module_types': {},
 'obfuscated_modules': set(),
 'outputs': [],
 'plugins': [],
 'reg_path': '/home/jgora.linux/.pyarmor/docker',
 'resources': [],
 'rft_auto_excludes': {'super'},
 'rft_export_names': set(),
 'rft_transform_op': '?',
 'runtime_key': None,
 'runtime_keyfile': '.pyarmor.ikey',
 'variable_types': {}}

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

This patch may fix segmentfault issue

--- a/src/cli/docker.py
+++ b/src/cli/docker.py
@@ -53,7 +53,8 @@ class DockerAuthHandler(socketserver.BaseRequestHandler):
     def generate_runtime_key(self, userdata):
         ctx = CONFIG['ctx']
         ctx.cmd_options['user_data'] = userdata
-        return Builder(ctx).generate_runtime_key()
+        Pytransform3._pytransform3.init_ctx(ctx)
+        return Pytransform3.generate_runtime_key(ctx)

@jondy
Copy link
Contributor

jondy commented Oct 19, 2023

A quick fixed version 8.4.1 has been released for this issue

@goraje
Copy link
Author

goraje commented Oct 20, 2023

Hi, thank you for your assistance. It seems that after these changes if one starts the authentication service on the Linux VM that acts as the Docker host and starts the container with --add-host=host.docker.internal:172.17.0.1 then everything works as expected. Finally I have one last question. From what I understand from the documentation in order for a group license to remain valid on a VM the output of Pytransform3.get_hd_info(22) needs to remain the same between reboots. Can other MACHFLAGS like 16 and 11 in particular not remain constant. I've noticed that they sometimes change between reboots and I did my best to ensure that the drive's UUIDs and MAC addresses of the NICs remain the same.

@jondy
Copy link
Contributor

jondy commented Oct 20, 2023

Great!

It's better to use flag 22, for others (11 to 21), they may not work in some cases. For example, there are multiple physical network cards.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants