Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sbe: python3 build support #16

Closed
wants to merge 6 commits into from

Conversation

@hanetzer
Copy link
Contributor

hanetzer commented May 10, 2019

Greetings,

Been doing some tinkering with the build scripts for sbe. I lack any
hardware to test it with, so I can't say that these actually 'work', but
it has been build-tested on witherspoon_defconfig in a debian strech
chroot (I'm a gentoo user, which is part of why I've been doing this
work), using both python 2.7.13 and 3.5.3.

I think I caught all the compat issues, but I could be mistaken. I'd
really appreciate it if someone could give this a test run.

hanetzer added 5 commits May 10, 2019
This enables better compat between python2.7 and python3.x

Without this change, building with python3.x (python3.5 tested) as
/usr/bin/python will result in the following error:

File: "sbe/src/build/security/securityRegListGen.py", line 64
    -v, --verbose            enable verbose traces"
                                                  ^
SyntaxError: Missing parentheses in call to 'print'

Signed-off-by: Marty E. Plummer <hanetzer@startmail.com>
io.open is compatible with python3.x's open builtin. Open in text mode
(no 'b') as this is a plain csv file, and specify the encoding as utf8.

Without this change, building with python3.x (python3.5 tested) as
/usr/bin/python will result in the following error:

File "sbe/src/build/security/securityRegListGen.py", line 695, in <module>
  main(sys.argv)
File "sbe/src/build/security/securityRegListGen.py", line 592, in main
  for idx, row in enumerate(reader):
File "/usr/lib/python3.5/csv.py", line 109, in __next__
  self.fieldnames
File "/usr/lib/python3.5/csv.py", line 96, in fieldnames
  self._fieldnames = next(self.reader)
_csv.Error: iterator should return strings, not bytes
(did you open the file in text mode?)

Signed-off-by: Marty E. Plummer <hanetzer@startmail.com>
Without this change, building with python3.x (python3.5 tested) as
/usr/bin/python will result in the following error:

Traceback (most recent call last):
  File "./parsAndCutElf.py", line 79, in <module>
    parserElf(sys.argv)
  File "./parsAndCutElf.py", line 54, in parserElf
    if( (line.find(firstSection) != -1) ):
TypeError: a bytes-like object is required, not 'str'

Signed-off-by: Marty E. Plummer <hanetzer@startmail.com>
python2 returns an integer when both division operands are integers.
python3 will return a float in this situation. Use the floor division
operator (//) which returns an integer on both versions.

Without this change, building with python3.x (python3.5 tested) as
/usr/bin/python will result in the following error:

Traceback (most recent call last):
  File "sbe/src/boot/sbeCompression.py", line 198, in <module>
    main( sys.argv )
  File "sbe/src/boot/sbeCompression.py", line 181, in main
    compress(imagePath + "/" + image + ".base", imagePath + "/" + image + ".base.compressed")
  File "sbe/src/boot/sbeCompression.py", line 58, in compress
    for i in range(0, os.stat(inputFile).st_size / 4 ):
TypeError: 'float' object cannot be interpreted as an integer

Signed-off-by: Marty E. Plummer <hanetzer@startmail.com>
In python3.x dict.items() does (more or less) what dict.iteritems() did
in python2.x, but dict.iteritems() was removed in python3.x entirely.
Further, dict.items() in python2.7 is less efficient than dict.iteritems(),
so we attempt to use dict.iteritems(), and if that throws an AttributeError,
(which would happen on python3.x), fall back to dict.items().

Without this change, building with python3.x (python3.5 tested) as
/usr/bin/python will result in the following error:

Traceback (most recent call last):
  File "sbe/src/boot/sbeCompress
ion.py", line 198, in <module>
    main( sys.argv )
  File "sbe/src/boot/sbeCompression.py", line 181, in main
    compress(imagePath + "/" + image + ".base", imagePath + "/" + image + ".base.compressed")
  File "sbe/src/boot/sbeCompression.py", line 72, in compress
    sortedList =  sorted(instDict.iteritems(), key=operator.itemgetter(1), reverse = True)
AttributeError: 'dict' object has no attribute 'iteritems'

Signed-off-by: Marty E. Plummer <hanetzer@startmail.com>
@hanetzer hanetzer force-pushed the hanetzer:python3-for-master branch from 6447dbf to 3f1224c May 10, 2019
In python2.7, proc.stdout.read() returns a str type. On python3.x, it
returns bytes.

Without this change, building with python3.x (python3.5 tested) as
/usr/bin/python will result in the following error:

Traceback (most recent call last):
  File "./updateBuildTag.py", line 76, in <module>
    updateBuildTag(sys.argv)
  File "./updateBuildTag.py", line 62, in updateBuildTag
    cmd_build_tag = cmd_build_tag+" "+commit_id
TypeError: Can't convert 'bytes' object to str implicitly

Signed-off-by: Marty E. Plummer <hanetzer@startmail.com>
@hanetzer hanetzer force-pushed the hanetzer:python3-for-master branch from 3f1224c to 7506346 May 10, 2019
@sharkcz

This comment has been minimized.

Copy link

sharkcz commented Jan 10, 2020

and it has been merged, thanks to all involved

@hanetzer

This comment has been minimized.

Copy link
Contributor Author

hanetzer commented Jan 11, 2020

and it has been merged, thanks to all involved

Any reason the print function stuff has not been merged?

@srikantha-m

This comment has been minimized.

Copy link

srikantha-m commented Jan 13, 2020

We had done the changes and will be merged with a day all print function stuff.

@rajadas2

This comment has been minimized.

Copy link
Contributor

rajadas2 commented Jan 14, 2020

Hi Marty, I apologise for the delayed response here.
Most of the SBE team were busy working on P10 SBE stuff.

We have merged most of the stuff. It should show up in open-power/SBE now.

(We have pulled in your commit into SBE-master repo which we maintain for the product and push out the same into open-power/SBE also to keep up with the sync)

@rajadas2

This comment has been minimized.

Copy link
Contributor

rajadas2 commented Jan 16, 2020

All commits in this pull request are merged into master, it should replicate into openpower/sbe master branch.

@rajadas2 rajadas2 closed this Jan 16, 2020
@rajadas2

This comment has been minimized.

Copy link
Contributor

rajadas2 commented Jan 16, 2020

Marty, if you see any issue, feel free to open it again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.