Python 3 and pygobject port of Zenmap #2088

kulikjak · 2020-07-27T07:15:54Z

Hi; as part of our migration effort away from Python 2, I managed to port whole Zenmap to Python 3 and pygobject library. Except for some minor visual glitches (like windows being wider than in current version) and most likely several hidden bugs I wasn't able to find yet, it is fully functional and will most likely go into the next update of Solaris.

Its only big issue is that it is not Python 2 and PyGTK backward compatible. I don't think it's that big of a deal as both are obsolete today and many Linux distributions don't ship them anymore, but you might think otherwise. Python 2 compatibility can be most likely brought back with the addition of #1972, PyGTK compatibility would require much more effort.

This, together with #1807 and some setup.py and configure changes (I have those ready as well), should be enough to completely port Nmap to Python 3.

It still needs some work (mostly cleaning up), but I wanted to make this PR now so that you know that this exists and can make comments :).

EDIT: cleaning up is done

kulikjak · 2020-08-03T06:53:57Z

I cleaned and fixed all remaining issues I found, and I believe its ready for review. There are still some things that should be done, but its already a pretty big patch, and those are not necessary for this transition.

Done:

Migrate everything to Python 3
Migrate all PyGTK code to PyGObject
Migrate some deprecated Gtk stuff to newer alternatives (initializers, horizontal and vertical containers (e.g., HBox and VBox -> Box), Arrows, ...)
Remove no longer relevant code necessary for PyGTK backward compatibility

ToDo later:

Remove all deprecated Gtk code (Gtk.Alignment, Gtk.IconFactory & Gtk.IconSet, Gtk.Table, Gtk.UIManager, stock items and probably some other things as well)
move optparse to argparse
probably more stuff I am forgetting right now

vmkalbskopf · 2020-11-10T07:32:25Z

If I try compile the current state of the master branch, will I be able to use zenmap on Ubuntu 20.04?

chmelarp · 2020-11-14T09:20:07Z

I did some intensive tests of master-python3 branch on Ubuntu 20.04 and .10 both as root and ordinary user. It behaves like it did before :)
I give it a go to merge it into nmap master

Stunkymonkey · 2020-12-01T15:17:53Z

maybe also update the list of dependencies in /zenmap/README

baryluk · 2021-01-19T18:27:19Z

Just curious, any progress on finishing the review and merging this?

chmelarp · 2021-03-01T21:32:43Z

Hey,
after some time using the https://github.com/kulikjak/nmap I found one bug there. Also, the Readme might be updated, as it is confusing ... as Stunkymonkey noted. Also, an icon is missing there on Ubuntu :)
Thanks!

Version: 7.80SVN
Traceback (most recent call last):
File "/home/p3/Projects/Kali/zenmap3/zenmap/radialnet/gui/RadialNet.py", line 880, in draw
self.__draw(context)
File "/home/p3/Projects/Kali/zenmap3/zenmap/radialnet/gui/RadialNet.py", line 270, in check_graph_status
return function(*args)
File "/home/p3/Projects/Kali/zenmap3/zenmap/radialnet/gui/RadialNet.py", line 1023, in __draw
self.__draw_node(context, node)
File "/home/p3/Projects/Kali/zenmap3/zenmap/radialnet/gui/RadialNet.py", line 1171, in __draw_node
stride, data = get_pixels_for_cairo_image_surface(icon)
File "/home/p3/Projects/Kali/zenmap3/zenmap/radialnet/gui/Image.py", line 165, in get_pixels_for_cairo_image_surface
data[j:j + FORMAT_RGBA] = array.array('B', [r, g, b, a])
TypeError: an integer is required (got type str)

kulikjak · 2021-03-16T09:13:07Z

Hi, good to know that somebody is using it :).

Can you please let me know how to reproduce this? I am unable to do so, and looking at the code, I don't see any obvious issue there. Thanks!

Considering the Readme, you are right, but I don't really want to do that before somebody maintaining nmap expresses interest in this PR and the whole Python 3 migration.

surfzoid · 2021-05-04T13:20:52Z

Hi, i'm trying it for zenmap, mageia 8 have nmap for python 3 and no more py 2.
If i run zenmap directly from the git clone, it is okay except few warning :
https://paste.gnome.org/pnoaw5azd
trying to install on my system with sudo ./setup.py install
but :
https://paste.gnome.org/panvym1ln

surfzoid · 2021-05-04T13:41:48Z

Another thing, the zenmap README is obsolete, with python 2

kulikjak · 2021-05-04T15:29:20Z

Yes, setup.py is not changed in this PR because there is another one doing that (I am not sure if they are compatible now, we will have to merge all these changes once this gets pulled). README is also not changed - that one can wait, I guess ;).

surfzoid · 2021-05-04T15:48:32Z

Yes, setup.py is not changed in this PR because there is another one doing that (I am not sure if they are compatible now, we will have to merge all these changes once this gets pulled). README is also not changed - that one can wait, I guess ;).

since july 2020 did you get feed back from official nmap? do you know if it will be merged soon?

kulikjak · 2021-05-04T16:08:32Z

since july 2020 did you get feed back from official nmap? do you know if it will be merged soon?

Yes, it's discussed here (along with several other PRs for this transition): #1176. Just recently, they mentioned that it is a priority, but there are more pressing things to address first.

Borim7 · 2021-10-23T17:22:39Z

Thanks for the effort it is working fine.

I found 2 small points:

shebang line of zenmap file still points to python, which on most system is python2
When start zenmap the first time, the window size is a little bit to small to show all components. The list of scanned devices is only partly visible.
- After resizing everything is displayed. As the window size is memorized, it only happen on the first startup.

Borim7 · 2021-10-23T17:46:59Z

Your port crash when clicking on filter host below the host list with following traceback:

Version: 7.80SVN
Traceback (most recent call last):
  File "/Daten/code/nmap/zenmap/zenmapGUI/ScanInterface.py", line 283, in filter_toggle_toggled
    self.filter_hosts(self.filter_bar.get_filter_string())
  File "/Daten/code/nmap/zenmap/zenmapGUI/ScanInterface.py", line 300, in filter_hosts
    start = time.clock()
AttributeError: module 'time' has no attribute 'clock'

kulikjak · 2021-10-24T18:25:24Z

Thanks for the reports, I will look into those.

I migrated this using Python 3.7 and time.clock() was removed in 3.8, which I guess is why you are seeing the error I don't.

ferdnyc · 2021-11-26T23:07:00Z

@kulikjak

This is really promising! One thing I'd suggest is running your code with PYTHONWARNINGS=always set in the environment, to enable (in particular) DeprecationWarnings for future changes to Python or other libraries. (The breakage due to integer promotion being removed from Python 3.10 has shown how important that is.) Currently the branch has quite a lot of them:

$ PYTHONWARNINGS=always sudo python3 ./zenmap
[path]/nmap/zenmap/zenmapGUI/Icons.py:186: PyGTKDeprecationWarning: Gtk.IconSet(pixbuf) has been deprecated. Please use: Gtk.IconSet.new_from_pixbuf(pixbuf)
  iconset = Gtk.IconSet(pixbuf=pixbuf)
/usr/lib/python3.10/site-packages/gi/overrides/Gtk.py:1604: DeprecationWarning: Gtk.IconSet.new_from_pixbuf is deprecated
  iconset = Gtk.IconSet.new_from_pixbuf(pixbuf)
[path]/nmap/zenmap/zenmapGUI/Icons.py:187: DeprecationWarning: Gtk.IconFactory.add is deprecated
  iconfactory.add(key, iconset)
[path]/nmap/zenmap/zenmapGUI/Icons.py:189: DeprecationWarning: Gtk.IconFactory.add_default is deprecated
  iconfactory.add_default()
<frozen importlib._bootstrap>:671: ImportWarning: DynamicImporter.exec_module() not found; falling back to load_module()
[path]/nmap/zenmap/zenmapGUI/ScanToolbar.py:194: DeprecationWarning: Gtk.Container.set_focus_chain is deprecated
  self.target_entry.set_focus_chain((self.target_entry.get_child(),))
[path]/nmap/zenmap/zenmapGUI/ScanOpenPortsPage.py:204: DeprecationWarning: Gtk.ScrolledWindow.add_with_viewport is deprecated
  self.add_with_viewport(self.host)
[path]/nmap/zenmap/zenmapGUI/NmapOutputViewer.py:215: DeprecationWarning: Gdk.Color.to_string is deprecated
  "foreground", Gdk.Color(*text_color).to_string())
[path]/nmap/zenmap/radialnet/bestwidgets/labels.py:158: DeprecationWarning: Gtk.Misc.set_alignment is deprecated
  self.set_alignment(0, 0.50)
[path]/nmap/zenmap/radialnet/bestwidgets/expanders.py:146: DeprecationWarning: Gtk.Alignment.new is deprecated
  self.__alignment = Gtk.Alignment.new(0, 0, 1, 1)
[path]/nmap/zenmap/radialnet/bestwidgets/expanders.py:147: DeprecationWarning: Gtk.Alignment.set_padding is deprecated
  self.__alignment.set_padding(12, 0, 24, 0)
[path]/nmap/zenmap/radialnet/bestwidgets/boxes.py:219: DeprecationWarning: Gtk.Table.set_row_spacings is deprecated
  self.set_row_spacings(spacing)
[path]/nmap/zenmap/radialnet/bestwidgets/boxes.py:220: DeprecationWarning: Gtk.Table.set_col_spacings is deprecated
  self.set_col_spacings(spacing)
/usr/lib/python3.10/site-packages/gi/overrides/Gtk.py:1545: DeprecationWarning: Gtk.Table.attach is deprecated
  Gtk.Table.attach(self, child, left_attach, right_attach, top_attach, bottom_attach, xoptions, yoptions, xpadding, ypadding)
[path]/nmap/zenmap/radialnet/gui/ControlWidget.py:759: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "stock_id" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  self.__force = Gtk.ToolButton(Gtk.STOCK_REFRESH)
[path]/nmap/zenmap/radialnet/gui/ControlWidget.py:892: PyGIDeprecationWarning: GObject.timeout_add is deprecated; use GLib.timeout_add instead
  GObject.timeout_add(REFRESH_RATE, self.__update_options)
[path]/nmap/zenmap/radialnet/bestwidgets/buttons.py:145: DeprecationWarning: Gtk.Image.set_from_stock is deprecated
  self.__image.set_from_stock(stock, self.__size)
[path]/nmap/zenmap/zenmapGUI/MainWindow.py:234: DeprecationWarning: Gtk.ActionGroup.new is deprecated
  self.main_action_group = Gtk.ActionGroup.new('MainActionGroup')
/usr/lib/python3.10/site-packages/gi/overrides/Gtk.py:279: DeprecationWarning: Gtk.ActionGroup.add_action_with_accel is deprecated
  self.add_action_with_accel(action, accelerator)
[path]/nmap/zenmap/zenmapGUI/MainWindow.py:432: DeprecationWarning: Gtk.ActionGroup.list_actions is deprecated
  for action in self.main_action_group.list_actions():
[path]/nmap/zenmap/zenmapGUI/MainWindow.py:433: DeprecationWarning: Gtk.Action.set_accel_group is deprecated
  action.set_accel_group(self.main_accel_group)
[path]/nmap/zenmap/zenmapGUI/MainWindow.py:434: DeprecationWarning: Gtk.Action.connect_accelerator is deprecated
  action.connect_accelerator()
/usr/lib/python3.10/site-packages/gi/overrides/Gtk.py:406: DeprecationWarning: Gtk.UIManager.insert_action_group is deprecated
  return Gtk.UIManager.insert_action_group(self, buffer, length)
/usr/lib/python3.10/site-packages/gi/overrides/Gtk.py:403: DeprecationWarning: Gtk.UIManager.add_ui_from_string is deprecated
  return Gtk.UIManager.add_ui_from_string(self, buffer, length)
[path]/nmap/zenmap/zenmapGUI/MainWindow.py:507: DeprecationWarning: Gtk.UIManager.get_widget is deprecated
  menubar = self.ui_manager.get_widget('/menubar')

...That's all just between launching zenmap and the main window appearing.

ferdnyc · 2021-11-26T23:18:21Z

Speaking of window size, when you click "Details" in the upper-right corner of the output area (which is enabled once you've run at least one scan), the window that pops up is comically narrow, and its contents don't adjust if you widen it. (It unfortunately doesn't remember the new size, either, so it goes back to super-narrow every time.)

...Same crazy-narrow formatting in the "Host Details" tab of the main window, in fact.

kulikjak · 2022-01-15T15:59:50Z

Thanks @ferdnyc.

Going through the code with PYTHONWARNINGS=always is something I didn't consider; thanks for the suggestion! It's a pretty long time since I originally created this PR, so there might be some other things that need fixing as new Python runtimes are coming out. Also, there were several new additions to the Zenmap code which should be incorporated as well.

We also have GTK4 now, but I guess it's still to early to migrate directly to that.

I will look into that in the following days (including the weird default sizes).

ferdnyc · 2022-01-15T21:12:32Z

@kulikjak
I have some partial fixes for the layout stuff I can see about getting into shape to open a PR against your PR (one of my favorite git tricks), it might help some.

The larger issue is that the entire GUI is based on UIManager... which is already deprecated in its entirety, even for Gtk3. Ideally, everything would be (or, to borrow a tense from Douglas Adams, willan have been) ported to GtkApplication. But that means migrating all of the layouts, the Menus, the Actions... kind of "everything", gui-wise.

So I'm not sure a move to Gtk4 would be all that premature... it might actually make more sense than doing a migration within Gtk3, seeing as it's kind of a UI-design cul-de-sac at this stage.

That being said, I haven't really looked into how big a difference there is between them (I haven't really looked into Gtk4 at all), so I don't have a feel for the relative difficulty/effort each path would require.

kulikjak · 2022-01-16T09:08:01Z

@ferdnyc

@kulikjak I have some partial fixes for the layout stuff I can see about getting into shape to open a PR against your PR (one of my favorite git tricks), it might help some.

Thanks, that would be great!

The larger issue is that the entire GUI is based on UIManager... which is already deprecated in its entirety, even for Gtk3. Ideally, everything would be (or, to borrow a tense from Douglas Adams, willan have been) ported to GtkApplication. But that means migrating all of the layouts, the Menus, the Actions... kind of "everything", gui-wise.

Yeah, I know about that... When I started the migration, my goal was to do only changes necessary to run with gtk3 and python3, which is why I left many deprecation warnings behind. I didn't want to rewrite everything in one PR. But since this still isn't merged and GTK is moving fast, maybe it's time to look into that as well.

So I'm not sure a move to Gtk4 would be all that premature... it might actually make more sense than doing a migration within Gtk3, seeing as it's kind of a UI-design cul-de-sac at this stage.

That being said, I haven't really looked into how big a difference there is between them (I haven't really looked into Gtk4 at all), so I don't have a feel for the relative difficulty/effort each path would require.

I am more worried from the OS/distro point of view - GTK 4 is still pretty new and not everybody might have it available, especially considering that Nmap is used by a very diverse list of OSes/distros. And GTK 3 will most likely still be here for years to come.

I guess the best thing is to ask maintainers what they prefer.

@fyodor @bonsaiviking - when the time comes for this PR, would you rather prefer a smaller change that just makes Zenmap compatible with GKT3 and Python 3 (that said, it's still already 2000 lines of changes) or a bigger change where all deprecation warnings and legacy code are taken care of? And should we go directly to GTK4, or is it still too early from your point of view?

darkbasic · 2022-01-18T10:16:48Z

I think that most distros (like Ubuntu 22.04 LTS), even if they will stick to gtk3 apps when possible, they will still ship gtk4 alongside. By the time this gets released it will probably be better to have GTK4 IMHO.

jerji · 2022-01-22T05:06:56Z

Hey, I was running a scan and when I double clicked on a host zenmap crahsed.
Here's the error:

Traceback (most recent call last):
  File "/home/angel/Downloads/nmap/zenmap/radialnet/gui/RadialNet.py", line 812, in draw
    self.__draw(context)
  File "/home/angel/Downloads/nmap/zenmap/radialnet/gui/RadialNet.py", line 202, in check_graph_status
    return function(*args)
  File "/home/angel/Downloads/nmap/zenmap/radialnet/gui/RadialNet.py", line 955, in __draw
    self.__draw_node(context, node)
  File "/home/angel/Downloads/nmap/zenmap/radialnet/gui/RadialNet.py", line 1103, in __draw_node
    stride, data = get_pixels_for_cairo_image_surface(icon)
  File "/home/angel/Downloads/nmap/zenmap/radialnet/gui/Image.py", line 97, in get_pixels_for_cairo_image_surface
    data[j:j + FORMAT_RGBA] = array.array('B', [r, g, b, a])
TypeError: an integer is required (got type str)

Seems like changing line 93 of Image.py from a = '\xff' to a = 255 fixes the bug

kulikjak · 2022-08-25T15:41:08Z

So, I rebased everything onto the latest master incorporating other changes pushed since then and fixed minor regressions. If anybody from upstream is interested, it's ready again! ;)

@jerji thanks for the report - it's fixed as well.

surfzoid · 2022-11-03T19:36:32Z

Hi
i'm trying to build rpm, i cannot use zip download in spec file because is not versioned

surfzoid · 2022-11-04T09:21:40Z

Hi,
RPM is build and work
https://mageialinux-online.org/repository/8/x86_64/media/core/
For further update versionned zip should be nice.

kulikjak · 2022-11-06T11:10:34Z

I am not sure that is related to this Pull Request? But maybe I am just missing something...

Prove the Python syntax errors as discussed in * nmap#91 * nmap#342 * nmap#624 * nmap#665 * nmap#666 * nmap#1176 * nmap#1484 * nmap#1807 * nmap#1972 * nmap#2088 * nmap#2279 * nmap#2287 * nmap#2446 * nmap#2493 * nmap#2522 And many more...

kulikjak · 2022-12-08T13:25:48Z

Thank you!

Installation of translations hasn't worked since the conversion to Python 3 (nmap#2088). That's because os.walk is not a drop-in replacement for os.path.walk. This tweak to setup.py should address that (fixes nmap#2718).

Installation of translations hasn't worked since the conversion to Python 3 (nmap#2088), and was broken in a different way in the migration from setup.py to pyproject.toml since package data files are not found deep down in directory hierarachies. This should address that (fixes nmap#2718).

pyhalov mentioned this pull request Jul 30, 2020

nmap: update to 7.80 OpenIndiana/oi-userland#5961

Merged

kulikjak mentioned this pull request Aug 3, 2020

Zenmap and Ndiff are python2 only #1176

Closed

fabaff mentioned this pull request Feb 17, 2021

nmap-unfree: init at 7.91 NixOS/nixpkgs#113381

Merged

10 tasks

jtojnar mentioned this pull request Dec 5, 2021

A list of packages that grep for "python2" NixOS/nixpkgs#148779

Closed

kulikjak added 4 commits August 25, 2022 17:05

Initial Python 3 & GTK-3.0 changes

2e3c2ab

Additional changes to make it run

109820c

Mostly packing and several other fixes

0edaec2

Revert higboxes.py changes because of broken layout

81e3460

kulikjak added 9 commits August 25, 2022 17:05

Replace deprecated Gtk.HBox and Gtk.VBox

74fc6d9

Remove now obsolete code branches

5a227d9

Fix CrashReport window width

5abb5f1

Fix and update help functionality

d26cedd

Clean up the database interface

c488b5f

Fix NmapOutputProperties example update

ef948e5

Clean radialnet draw signal handlers

60e936f

Revert unnecessary property changes

ab3094c

Fix minor MainWindow regression

df54b73

kulikjak force-pushed the master-python3 branch from 9e06b7e to 1ebece3 Compare August 25, 2022 15:37

kulikjak added 5 commits August 25, 2022 17:55

Remove deprecated time.clock()

8609f2c

Change shebangs to python3

228d38d

Remove now unnecessary utf-8 source file encoding

876ff36

Rebase and fix minor issues

d0badb6

Fix TypeError reported on Nmap GitHub PR

8beebdd

kulikjak force-pushed the master-python3 branch from 1ebece3 to 8beebdd Compare August 25, 2022 15:55

cclauss mentioned this pull request Nov 7, 2022

GitHub Action to lint Python code #2562

Closed

nmap-bot closed this in 24b2631 Dec 7, 2022

dmiller-nmap mentioned this pull request Apr 17, 2023

Zenmap python3 #2607

Closed

pghmcfc mentioned this pull request Mar 1, 2024

Fix failure to find zenmap translations #2789

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python 3 and pygobject port of Zenmap #2088

Python 3 and pygobject port of Zenmap #2088

kulikjak commented Jul 27, 2020 •

edited

Loading

kulikjak commented Aug 3, 2020

vmkalbskopf commented Nov 10, 2020

chmelarp commented Nov 14, 2020

Stunkymonkey commented Dec 1, 2020

baryluk commented Jan 19, 2021

chmelarp commented Mar 1, 2021 •

edited

Loading

kulikjak commented Mar 16, 2021

surfzoid commented May 4, 2021

surfzoid commented May 4, 2021

kulikjak commented May 4, 2021

surfzoid commented May 4, 2021

kulikjak commented May 4, 2021

Borim7 commented Oct 23, 2021

Borim7 commented Oct 23, 2021

kulikjak commented Oct 24, 2021

ferdnyc commented Nov 26, 2021 •

edited

Loading

ferdnyc commented Nov 26, 2021 •

edited

Loading

kulikjak commented Jan 15, 2022

ferdnyc commented Jan 15, 2022

kulikjak commented Jan 16, 2022

darkbasic commented Jan 18, 2022 •

edited

Loading

jerji commented Jan 22, 2022 •

edited

Loading

kulikjak commented Aug 25, 2022 •

edited

Loading

surfzoid commented Nov 3, 2022 •

edited

Loading

surfzoid commented Nov 4, 2022

kulikjak commented Nov 6, 2022

kulikjak commented Dec 8, 2022

Python 3 and pygobject port of Zenmap #2088

Python 3 and pygobject port of Zenmap #2088

Conversation

kulikjak commented Jul 27, 2020 • edited Loading

kulikjak commented Aug 3, 2020

vmkalbskopf commented Nov 10, 2020

chmelarp commented Nov 14, 2020

Stunkymonkey commented Dec 1, 2020

baryluk commented Jan 19, 2021

chmelarp commented Mar 1, 2021 • edited Loading

kulikjak commented Mar 16, 2021

surfzoid commented May 4, 2021

surfzoid commented May 4, 2021

kulikjak commented May 4, 2021

surfzoid commented May 4, 2021

kulikjak commented May 4, 2021

Borim7 commented Oct 23, 2021

Borim7 commented Oct 23, 2021

kulikjak commented Oct 24, 2021

ferdnyc commented Nov 26, 2021 • edited Loading

ferdnyc commented Nov 26, 2021 • edited Loading

kulikjak commented Jan 15, 2022

ferdnyc commented Jan 15, 2022

kulikjak commented Jan 16, 2022

darkbasic commented Jan 18, 2022 • edited Loading

jerji commented Jan 22, 2022 • edited Loading

kulikjak commented Aug 25, 2022 • edited Loading

surfzoid commented Nov 3, 2022 • edited Loading

surfzoid commented Nov 4, 2022

kulikjak commented Nov 6, 2022

kulikjak commented Dec 8, 2022

kulikjak commented Jul 27, 2020 •

edited

Loading

chmelarp commented Mar 1, 2021 •

edited

Loading

ferdnyc commented Nov 26, 2021 •

edited

Loading

ferdnyc commented Nov 26, 2021 •

edited

Loading

darkbasic commented Jan 18, 2022 •

edited

Loading

jerji commented Jan 22, 2022 •

edited

Loading

kulikjak commented Aug 25, 2022 •

edited

Loading

surfzoid commented Nov 3, 2022 •

edited

Loading