Create modern python package for HaikuPorter #259

jurgenwigg · 2023-07-11T22:44:12Z

Main goal is to create proper python package which could be installed via pip install command. At this moment package is created in a such way that it is not possible to install package flawless on newer python versions.

Creating unit tests, doing cleanup and removing all python2.7 leftovers, enabling lint, types and other static code analysis will be done under separate PR.

scottmc

Why the rename? I really hate having_under_scores_in_files_names
It's been called haikuporter since the day it was created.

jurgenwigg · 2023-07-12T17:30:09Z

@scottmc I had to rename file due to issue I had on my computer. MacOS is case insensitive and it caused that while cloning the repository I had a big problem. File names will be as in the original repository - haikuporter. But file itself will be .py file instead of HaikuPorter or I can create directory for the source code of haikuporter itself. Both options are good but it depends on our decision.

In terms of future changes, for Python PEP8 suggests pretty clearly how to style names for classes, functions, etc. Enabling checking formatting for the python code would require setting custom RC file for pylint or changing the names to comply with PEP8 standard. This is open point for future PR.

pulkomandy · 2023-07-12T19:05:23Z

with some reorganization of the sources it should be possible to keep the name as "haikuporter", lowercase, for both the file and the directory. The file is meant to be installed in some /bin directory while the directory is meant to be a python module.

I think there will be a lot of people running haikuporter directly from the git repository, but they can probably handle some reorganization.

I agree that renaming everything to pep8 is a good idea, but we can still keep the "haikuporter" name :)

jurgenwigg · 2023-07-13T05:34:54Z

We're all on the same page :) my idea was to preserve whole functionality as it was before but in modern python package with all advantages of newer version and with possibility to release it on pypi :)

jurgenwigg · 2023-07-14T08:56:58Z

After migration HaikuPorter seems to work as before.

Importing module in the python itself:

» python
Python 3.11.4 (main, Jun  7 2023, 12:45:49) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import HaikuPorter
>>> import haikuporter
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'haikuporter'
>>> import haiku_porter
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'haiku_porter'
>>>

Executing haikuporter script from CLI:

» ./haikuporter mesa -j4
Error: Unable to find haikuports.conf in known search paths.
Error: See haikuports-sample.conf for more information

Migration to src based structure instead of flat one requires installation of HaikuPorter package with pip install . or pip install -e ..

waddlesplash · 2023-07-14T20:45:11Z

Haiku itself must be built on a case-sensitive system. You can make a case-sensitive disk in MacOS to check the repository out on and work on it that way.

waddlesplash · 2023-07-14T20:45:47Z

.pre-commit-config.yaml

@@ -0,0 +1,10 @@
+repos:


What's all this about?

It came by default with cookie cutter template which I used to create/update this python package. It depends on if we would like to have git pre-commit hooks with formatting or not. Also basic checks are included in this step. In theory it should help to keep good-looking python code in terms of style before committing it to the repo.

waddlesplash · 2023-07-14T20:46:13Z

CONTRIBUTING.rst

@@ -0,0 +1,151 @@
+============


I think changes to README files should be done separately from such a refactor like this.

Good point - this file should be added in separate PR. I think it would be really good to have such guide how to contribute to this project.

waddlesplash · 2023-07-14T20:46:40Z

Dockerfile

@@ -0,0 +1,21 @@
+# syntax=docker/dockerfile:1


This file doesn't seem to have much purpose, perhaps it can be removed? BuildMaster has its own docker setup.

At this moment I agree that this file can be removed. Maybe in the future it would be helpful for developing HaikuPorter, i.e. some integration tests.

jurgenwigg · 2023-07-16T15:49:03Z

The question is if HaikuPorter should be created in such way that it require case sensitive system? Proposed solution should work as previous package also in terms of case sensitive/insensitive. I don't see any obstacles in having haikuporter script 'as is' in root directory and HaikuPorter directory in src place.

jurgenwigg · 2023-07-16T19:13:59Z

Ok, I've decided that this PR should cover only updating existing file structure to be able to install and create proper modern python package. Everything should work like before the update. Everything related to updating documentation (use mkdocs or sphinx), CI and pre-commit hooks should be done under separate PRs.

waddlesplash · 2023-07-16T21:09:53Z

The question is if HaikuPorter should be created in such way that it require case sensitive system?

Using HaikuPorter for cross-compilation requires a Haiku build tree, which needs a case-sensitive filesystem. Using HaikuPorter on Haiku will always have a case-sensitive filesystem. So there's no reason to support case-insensitive ones.

pulkomandy · 2023-07-16T21:13:32Z

But it's just moving a few files to an src/ subdirectory, so there is not really any problem with doing it anyways. And being able to checkout the git repository on a case insensitive filesystem sounds useful for many reasons (archiving, for example).

waddlesplash · 2023-07-16T21:19:12Z

It's not just moving "a few files" but the entire sources, and furthermore due to the changes the renames aren't even tracked so we are losing all the history here.

I would prefer the haikuporter command/script be moved elsewhere (1 file) than all the others.

jurgenwigg · 2023-07-16T22:02:14Z

It's not just moving "a few files" but the entire sources, and furthermore due to the changes the renames aren't even tracked so we are losing all the history here.

I would prefer the haikuporter command/script be moved elsewhere (1 file) than all the others.

The easiest way is just to add .py extension to existing haikuporter script. This is the easiest way to keep original directory for source files (HaikuPorter) and keep script file for it in the root directory. If this proposition would be a good deal for now? :)

waddlesplash · 2023-07-16T22:05:26Z

Sounds fine.

jurgenwigg · 2023-07-17T13:41:10Z

Now everything should work correctly and package itself should allow to install it through pip install . or pip install -e . command.

Begasus · 2023-07-17T19:00:46Z

Did a few check builds with these changes (only renamed the symlink in ~/config/non-packaged/bin to haikuporter instead of haikuporter.py (didn't want to change all the other calls to it or restart Terminal to be able to use it).
Things works fine for the buildprocess (not mingling with the internals). :)

jurgenwigg · 2023-07-18T05:57:04Z

@waddlesplash @scottmc please check if your comments and findings were solved and if this PR can be merged :)

pulkomandy · 2023-07-18T16:20:00Z

.gitignore

+
+# From https://raw.githubusercontent.com/github/gitignore/main/Python.gitignore
+
+ Byte-compiled / optimized / DLL files


I think this is missing a #

Also probably some parts of this file are not needed.

This the base for further improvements. I've corrected missing #

pulkomandy · 2023-07-18T16:20:37Z

Makefile

@@ -0,0 +1,56 @@
+.PHONY: install
+install: ## Install the poetry environment and install the pre-commit hooks
+	@echo "🚀 Creating virtual environment using pyenv and poetry"


I think emojis are not well handled by Haiku terminal currently, maybe they should be avoided

The question is if we need this file anyway - maybe it can be removed? It came with the template.

Seems like not, it can be removed?

README.md

poetry.lock

waddlesplash · 2023-07-19T16:47:12Z

.gitignore

+
+.history*
+.vscode*
+poetry.lock
 *.pyc
 *.pyo
 recipeCache/


probably would be good to keep HaikuPorter-specific items at the top

waddlesplash · 2023-07-19T16:48:26Z

setup.py

@@ -1,29 +0,0 @@
-"""Setup script for HaikuPorter


I think we actually use this to package HaikuPorter? Not sure...

setup.py is not needed and can be safely replaced by newer solution which is using pyproject.toml to install the package. Installing package with pip works flawlessly. I'll link article/document describing why setup.py is deprecated and how should it be replaced.

waddlesplash · 2023-07-19T16:48:35Z

pyproject.toml

@@ -0,0 +1,102 @@
+[tool.poetry]


Is this (and poetry.toml) really needed?

Poetry is like cargo in rust. You don't have to use it, but I treat it like a "touch of the future" where python has better package manager than pip ;)

jurgenwigg and others added 5 commits July 11, 2023 22:13

Rename haikuporter to haiku_porter.py

ad853ff

initial commit

215deb0

removed .history directory

91dc0ee

formatted code with black and isort

fcc9dc1

added few tools

66f2135

scottmc requested changes Jul 12, 2023

View reviewed changes

jurgenwigg added 2 commits July 13, 2023 07:43

fix files names

9923db8

updated README.md

41f7545

fixes related to building package

987ec95

jurgenwigg changed the title ~~Draft: Create modern python package for haiku porter~~ Create modern python package for HaikuPorter Jul 14, 2023

waddlesplash reviewed Jul 14, 2023

View reviewed changes

removed files that should be added in separate PRs

26a1615

jurgenwigg added 6 commits July 16, 2023 21:04

added better import handling to haikuporter

815fadb

removed unused (at this moment) tests directory

c2a8e68

restoreg .gitattributes and .dockerignore

a30929a

added ignoring .vscode directory

fb49f2a

resoterd doc directory and removed new docs

8d70bf4

removed CI related configurations

588747d

jurgenwigg added 3 commits July 17, 2023 08:18

restored original source directory

79ec269

reverted some changes in haikuporter.py

21e3e56

updated .gitignore file and pyproject.toml

3eb544b

pulkomandy reviewed Jul 18, 2023

View reviewed changes

jurgenwigg added 6 commits July 18, 2023 22:01

Update README.md

ca798b1

Update README.md

218c89f

Update README.md

08c5a50

Delete poetry.lock

710bce3

Update .gitignore

a761486

Update .gitignore

d54d1f7

waddlesplash reviewed Jul 19, 2023

View reviewed changes

Create modern python package for HaikuPorter #259

Create modern python package for HaikuPorter #259

jurgenwigg commented Jul 11, 2023 •

edited

scottmc left a comment

jurgenwigg commented Jul 12, 2023

pulkomandy commented Jul 12, 2023

jurgenwigg commented Jul 13, 2023

jurgenwigg commented Jul 14, 2023

waddlesplash commented Jul 14, 2023

waddlesplash Jul 14, 2023

jurgenwigg Jul 16, 2023

waddlesplash Jul 14, 2023

jurgenwigg Jul 16, 2023

waddlesplash Jul 14, 2023

jurgenwigg Jul 16, 2023

jurgenwigg commented Jul 16, 2023

jurgenwigg commented Jul 16, 2023

waddlesplash commented Jul 16, 2023

pulkomandy commented Jul 16, 2023

waddlesplash commented Jul 16, 2023

jurgenwigg commented Jul 16, 2023

waddlesplash commented Jul 16, 2023

jurgenwigg commented Jul 17, 2023

Begasus commented Jul 17, 2023

jurgenwigg commented Jul 18, 2023

pulkomandy Jul 18, 2023

jurgenwigg Jul 18, 2023

pulkomandy Jul 18, 2023

jurgenwigg Jul 18, 2023

waddlesplash Jul 19, 2023

waddlesplash Jul 19, 2023

waddlesplash Jul 19, 2023

jurgenwigg Jul 20, 2023

waddlesplash Jul 19, 2023

jurgenwigg Jul 20, 2023


		# From https://raw.githubusercontent.com/github/gitignore/main/Python.gitignore

		Byte-compiled / optimized / DLL files

Create modern python package for HaikuPorter #259

Are you sure you want to change the base?

Create modern python package for HaikuPorter #259

Conversation

jurgenwigg commented Jul 11, 2023 • edited

scottmc left a comment

Choose a reason for hiding this comment

jurgenwigg commented Jul 12, 2023

pulkomandy commented Jul 12, 2023

jurgenwigg commented Jul 13, 2023

jurgenwigg commented Jul 14, 2023

waddlesplash commented Jul 14, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jurgenwigg commented Jul 16, 2023

jurgenwigg commented Jul 16, 2023

waddlesplash commented Jul 16, 2023

pulkomandy commented Jul 16, 2023

waddlesplash commented Jul 16, 2023

jurgenwigg commented Jul 16, 2023

waddlesplash commented Jul 16, 2023

jurgenwigg commented Jul 17, 2023

Begasus commented Jul 17, 2023

jurgenwigg commented Jul 18, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jurgenwigg commented Jul 11, 2023 •

edited