Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3 #162

Closed
Albertmoiseev opened this issue Dec 4, 2017 · 17 comments

Comments

@Albertmoiseev
Copy link

commented Dec 4, 2017

Hi everyone,

What do you think regarding implementing python 3.x support?

@stamparm

This comment has been minimized.

Copy link
Owner

commented Dec 4, 2017

Not a huge fan of that idea as lots of per-string operations are totally different. Why would you like it to be implemented in Python 3 in the first place?

@q2dg

This comment has been minimized.

Copy link

commented Mar 5, 2019

Because of this: https://pythonclock.org

@stamparm

This comment has been minimized.

Copy link
Owner

commented Mar 6, 2019

@q2dg will see. Currently I have a bigger priority project to work on with this "clock" on my mind, though, I have a strange feeling that the Python2 will be strong even after the set time

@mimugmail

This comment has been minimized.

Copy link

commented Jul 2, 2019

I'm also not a fan of the clock but maybe FreeBSD 12.1 will drop support when Python2 reaches EoL so I might get problem with OPNsense plugin in 2020. But .. no rush yet :)

uqs pushed a commit to freebsd/freebsd-ports that referenced this issue Jul 18, 2019
- Update to 0.14
- Add missing sqlite3 dependency
- Limit to python 2.7 because 3.x is not supported yet [1]

PR:		239110
Submitted by:	Michael Muenz <m.muenz@gmail.com> (maintainer)
See:		stamparm/maltrail#162 [1]


git-svn-id: svn+ssh://svn.freebsd.org/ports/head@506854 35697150-7ecd-e111-bb59-0022644237b5
uqs pushed a commit to freebsd/freebsd-ports that referenced this issue Jul 18, 2019
- Update to 0.14
- Add missing sqlite3 dependency
- Limit to python 2.7 because 3.x is not supported yet [1]

PR:		239110
Submitted by:	Michael Muenz <m.muenz@gmail.com> (maintainer)
See:		stamparm/maltrail#162 [1]
Jehops pushed a commit to Jehops/freebsd-ports that referenced this issue Jul 18, 2019
- Update to 0.14
- Add missing sqlite3 dependency
- Limit to python 2.7 because 3.x is not supported yet [1]

PR:		239110
Submitted by:	Michael Muenz <m.muenz@gmail.com> (maintainer)
See:		stamparm/maltrail#162 [1]


git-svn-id: svn+ssh://svn.freebsd.org/ports/head@506854 35697150-7ecd-e111-bb59-0022644237b5
mat813 pushed a commit to mat813/freebsd-ports that referenced this issue Jul 22, 2019
- Update to 0.14
- Add missing sqlite3 dependency
- Limit to python 2.7 because 3.x is not supported yet [1]

PR:		239110
Submitted by:	Michael Muenz <m.muenz@gmail.com> (maintainer)
See:		stamparm/maltrail#162 [1]


git-svn-id: https://svn.freebsd.org/ports/head@506854 35697150-7ecd-e111-bb59-0022644237b5
@MikhailKasimov

This comment has been minimized.

@mimugmail

This comment has been minimized.

Copy link

commented Aug 29, 2019

@stamparm

This comment has been minimized.

Copy link
Owner

commented Aug 29, 2019

Don't worry. Will do it when I catch time

stamparm added a commit that referenced this issue Sep 10, 2019
stamparm added a commit that referenced this issue Sep 10, 2019
stamparm added a commit that referenced this issue Sep 10, 2019
stamparm added a commit that referenced this issue Sep 10, 2019
stamparm added a commit that referenced this issue Sep 10, 2019
stamparm added a commit that referenced this issue Sep 11, 2019
@stamparm

This comment has been minimized.

Copy link
Owner

commented Sep 11, 2019

Stabilizing Python3 port. Now came across AssertionError: sorry, but this version only supports 100 named groups <- @MikhailKasimov, you know the reason behind it, although, I didn't know that Python (3?) made a hard cap limit here. Anyway, I have some more things to do here, but all in all, current prototype works (Note: environment variable MALTRAIL_DREI has to be set to 1 if somebody wants to take a sneak peek with Python3 and latest HEAD revision)

stamparm added a commit that referenced this issue Sep 11, 2019
stamparm added a commit that referenced this issue Sep 11, 2019
@MikhailKasimov

This comment has been minimized.

Copy link
Contributor

commented Sep 11, 2019

Stabilizing Python3 port. Now came across AssertionError: sorry, but this version only supports 100 named groups <- @MikhailKasimov, you know the reason behind it, although, I didn't know that Python (3?) made a hard cap limit here.

https://stackoverflow.com/questions/478458/python-regular-expressions-with-more-than-100-groups

@stamparm

This comment has been minimized.

Copy link
Owner

commented Sep 11, 2019

@MikhailKasimov I've stumbled upon that link, but it provides no solution to the problem. All those "replies" rely on third-party libraries (e.g. cregex). I'll deal with this

stamparm added a commit that referenced this issue Sep 11, 2019
stamparm added a commit that referenced this issue Sep 11, 2019
@stamparm

This comment has been minimized.

Copy link
Owner

commented Sep 11, 2019

Stabilized, patched lots of stuff, quick tested on a huge-pipe. It works pretty well :)

@stamparm stamparm closed this Sep 11, 2019
stamparm added a commit that referenced this issue Sep 11, 2019
@stamparm stamparm self-assigned this Sep 11, 2019
@stamparm

This comment has been minimized.

Copy link
Owner

commented Sep 11, 2019

@Godwottery to be honest, your first pull request (#3569) was a trigger for me to finish this up. Hence, I would like to thank you and put you on the https://github.com/stamparm/maltrail/#thank-you. Do you want me to use your real name (what is it?) or just the handle?

@Godwottery

This comment has been minimized.

Copy link

commented Sep 11, 2019

@Godwottery Do you want me to use your real name (what is it?) or just the handle?
I appreciate the thank-you. I would prefer if you use the handle.

@stamparm

This comment has been minimized.

Copy link
Owner

commented Sep 11, 2019

@Godwottery done :)

@stamparm

This comment has been minimized.

Copy link
Owner

commented Sep 12, 2019

p.s. There is somewhere a memory leak when running in multi-core/Python3 environment. Manifesting by a silent (kill) termination with an exit code 137 (after lots and lots of packets being processed - 10 minutes on a big pipe). Will try to find out why is it happening and patch it ASAP.

@stamparm

This comment has been minimized.

Copy link
Owner

commented Sep 12, 2019

pcapy leaks memory in Python3 environment :(. Seems that helpsystems/pcapy#37 has not been fixed in the latest PyPi version (0.11.4)

Even the simplest:

import time                                                                                                                                                                                                                                                                       
                                                                                                                                                                                                                                                                                  
cap = pcapy.open_live("any", 65536, True, 0)                                                                                                                                                                                                                                      
while True:                                                                                                                                                                                                                                                                       
    success = False                                                                                                                                                                                                                                                               
    try:                                                                                                                                                                                                                                                                          
        (header, packet) = cap.next()                                                                                                                                                                                                                                             
        if header is not None:                                                                                                                                                                                                                                                    
            success = True                                                                                                                                                                                                                                                        
    except (pcapy.PcapError, socket.timeout):                                                                                                                                                                                                                                     
        pass                                                                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                                                  
    if not success:                                                                                                                                                                                                                                                               
        time.sleep(0.05)  

leaks memory on Python3 + pcapy (0.11.4)

@stamparm

This comment has been minimized.

Copy link
Owner

commented Sep 12, 2019

With the latest commit/revision, Python3 leak should be gone. Patch(es) are dirty as hell, but even without pcapy fixing its memory leakage problems (Note: related to attribute access to wrapped objects), everything should work as expected

decke added a commit to decke/ports that referenced this issue Oct 13, 2019
- Update to 0.14
- Add missing sqlite3 dependency
- Limit to python 2.7 because 3.x is not supported yet [1]

PR:		239110
Submitted by:	Michael Muenz <m.muenz@gmail.com> (maintainer)
See:		stamparm/maltrail#162 [1]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.