Python 3.9 support (and upcoming incompatibility with cpython 3.7 and 3.8)

[bdraco]

Brief description

With Python 3.9 entering more wider production it would be nice to get a 2.4.5 release to solve the following issue which has already been fixed:

46fa40f

Environment

• Scapy version: 2.4.4
• Python version: 3.9
• Operating System: Alpine Linux

[bdraco]

Also it looks like 3.8 and 3.7 will have the issue in the next release since https://bugs.python.org/issue41976 is backported

python/cpython#22598
python/cpython#22600
python/cpython#22601

[bdraco]

https://bugs.python.org/issue42580#msg387117

The issue was closed in the python bug tracker so its unlikely it will get resolved there.

[guedou]

This sounds like a fair request.

@gpotter2 @p-l- do you have some features that you want to implement before releasing 2.4.5?

[p-l-]

@guedou I don't, I wanted the new Net() / Net6() implementations, so that's all set for me!

[polybassa]

If it is possible, I would be very happy if I could get #3075 and #3078 in.
#3054, split in multiple smaller PRs, like #3099 would be nice to have.

[guedou]

@polybassa I think that these PR should be added to 2.5.0. @gpotter2 aims to make it the first release that includes typing.

[polybassa]

Alright. Thanks. Than I aim to make everything typed for 2.5

[gpotter2]

Fine by me. I have a few PRs that I like but they can wait for 2.5.

[gpotter2]

@p-l- @guedou I'm okay to do 2.4.5 whenever you're ready. We're starting to get more and more Python 3.9 requests

[guedou]

FTR, I started to manually test e9f0893 on *BSD using https://github.com/secdev/scapy/tree/master/doc/vagrant_ci.

[guedou]

Tests are OK on FreeBSD 12.1

[guedou]

Tests are now OK on OpenBSD 6.8. I will provide a simple fix.

[guedou]

@p-l- @gpotter2 FTR, recent versions of the cryptography module requires the Rust compile to build.

[gpotter2]

Actually please merge #3113 before 2.4.5

[guedou]

I am struggling with NetBSD 9. Scapy does not seem to work anymore. I am investigating.

[gpotter2]

Do you need help @guedou ? It would be cool to have 2.4.5 ASAP

[guedou]

I am still stuck with NetBSD. It seems that Scapy (2.4.4 and master) is not able to sniff on NetBSD8 and NetBSD9 with my virtual machines setup. Can you give it a try ?

[gpotter2]

I tried for quite some time but didn't manage to sniff on NetBSD9 using the native /dev/bpf sockets.. neither on master nor on any released version of scapy for that matter.. :/
It works if I you use libpcap (conf.use_pcap = True) though. I'd say that's a good-enough workaround for the time being.

We need to release a 3.9 compatible version asap. I'd say NetBSD support can wait since a workaround is available. Maybe scapy's NetBSD maintainer can help us patch it.

[guedou]

Let's set the deadline to April 11. After hours of comparing ktrace output and reading NetBSD documentation, I finally managed to make snif() and sr*() work! I will need more testing to confirm that this is a correct fix.

[gpotter2]

I would like to make sure libpcap (and #3143) work before 2.4.5. We can probably start doing RCs though

[guedou]

I just released v2.4.5rc1, see https://pypi.org/project/scapy/2.4.5rc1/

[gpotter2]

We can safely close this PR. I'd like to have a separate issue for 2.4.5 to start gathering the changelog

[half-duplex]

For those looking: Fixed in #3000