Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

all: port to AIX / ppc64 #25893

Open
trex58 opened this issue Jun 14, 2018 · 47 comments

Comments

@trex58
Copy link

commented Jun 14, 2018

We have started the port of golang v1.10/v1.11 to AIX / ppc64.
It will be available on AIX 7.2 first, maybe on AIX 7.1 too (probably with recent TLs).
Required hardware is Power8 or 9.
64bit only.

Bootstrap with GCC Go 8.1 has been done.
golang tests failures are now being fixed.

Port of golang.org/x has also being started.
First: sys/unix, It will work for both GCC Go and golang on AIX.

We plan to finalize the port in August/September. Available first as RPMs on BullFreeware.

This was referenced Jun 14, 2018

@bradfitz bradfitz changed the title Port on AIX all: port to AIX Jun 14, 2018

@andybons andybons added the NeedsFix label Jun 14, 2018

@andybons andybons added this to the Unplanned milestone Jun 14, 2018

@morya

This comment has been minimized.

Copy link

commented Jun 27, 2018

Wonderful! Will you guys consider AIX 6.1 ?

@gopherbot

This comment has been minimized.

Copy link

commented Sep 27, 2018

Change https://golang.org/cl/138115 mentions this issue: go/build, runtime/internal/sys: add GOOS=aix

@trex58 trex58 changed the title all: port to AIX all: port to AIX / ppc64 Sep 28, 2018

@trex58

This comment has been minimized.

Copy link
Author

commented Sep 28, 2018

Wonderful! Will you guys consider AIX 6.1 ?

No. Sorry. AIX 6.1 does not provide all the features required by Go (either GCC Go or golang).

gopherbot pushed a commit that referenced this issue Sep 28, 2018
go/build, runtime/internal/sys: add GOOS=aix
This is the first commit of a series that will add AIX as an
operating system target for ppc64 architecture.

Updates #25893

Change-Id: I865b67a9c98277c11c1a56107be404ac5253277d
Reviewed-on: https://go-review.googlesource.com/138115
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138725 mentions this issue: os: add AIX operating system

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138719 mentions this issue: syscall: change solaris files to libc files

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138731 mentions this issue: cmd/cgo: add AIX operating system

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138720 mentions this issue: syscall: add AIX operating system

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138718 mentions this issue: internal/syscall: add AIX operating system

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138723 mentions this issue: mime: add AIX operating system

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138733 mentions this issue: all: change stack layout for AIX

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138728 mentions this issue: cmd/link: add new loading package for XCOFF files

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138721 mentions this issue: time: add AIX operating system

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138716 mentions this issue: runtime: add AIX operating system

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138717 mentions this issue: internal/poll: add AIX operating system

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138724 mentions this issue: net: add AIX operating system

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138722 mentions this issue: crypto: add AIX operating system

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138727 mentions this issue: debug/xcoff: Add new debug package

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138730 mentions this issue: cmd/link: add AIX operating system

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138729 mentions this issue: cmd: add DWARF64 support for aix port

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138726 mentions this issue: path: add AIX operating system

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138732 mentions this issue: cmd/go: add AIX operating system

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138734 mentions this issue: cmd: fix symbols access for AIX

@gopherbot

This comment has been minimized.

Copy link

commented Oct 1, 2018

Change https://golang.org/cl/138736 mentions this issue: runtime: handle 64bits addresses for AIX

gopherbot pushed a commit that referenced this issue Oct 26, 2018
cmd/compile/internal/ssa: fix TestStmtLines for AIX
This commit adapts TestStmtLines for AIX operating system.

Update #25893

Change-Id: I1c76bbd8fc679a66b65ecfbd1ed7745518064eae
Reviewed-on: https://go-review.googlesource.com/c/144958
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@gopherbot

This comment has been minimized.

Copy link

commented Oct 29, 2018

Change https://golang.org/cl/145397 mentions this issue: cmd: add XCOFF objfile and adapt cmd/nm tests

gopherbot pushed a commit that referenced this issue Oct 31, 2018
cmd: add XCOFF objfile and adapt cmd/nm tests
This commit adds a new file format in cmd/internal/objfile for XCOFF.
It also adapts tests inside cmd/nm for AIX.

Updates: #25893

Change-Id: I1e55ea0b7f7d08a871343bee27d11e2d3baad254
Reviewed-on: https://go-review.googlesource.com/c/145397
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

@bradfitz bradfitz added OS-AIX and removed OS-AIX labels Nov 2, 2018

gopherbot pushed a commit that referenced this issue Nov 26, 2018
runtime: handle 64bits addresses for AIX
This commit allows the runtime to handle 64bits addresses returned by
mmap syscall on AIX.

Mmap syscall returns addresses on 59bits on AIX. But the Arena
implementation only allows addresses with less than 48 bits.
This commit increases the arena size up to 1<<60 for aix/ppc64.

Update: #25893

Change-Id: Iea72e8a944d10d4f00be915785e33ae82dd6329e
Reviewed-on: https://go-review.googlesource.com/c/138736
Reviewed-by: Austin Clements <austin@google.com>
@gopherbot

This comment has been minimized.

Copy link

commented Nov 28, 2018

Change https://golang.org/cl/151301 mentions this issue: vendor: update x/net/internal/nettest for aix support

gopherbot pushed a commit that referenced this issue Nov 28, 2018
vendor: update x/net/internal/nettest for aix support
Update golang.org/x/net/internal/nettest to x/net git rev 9b4f9f5ad519
for:

   internal/nettest: add AIX operating system
   https://golang.org/cl/144077

This fixes the build failure of the vendored x/net/internal/nettest on
aix/ppc64.

Additionally this also pulls in:

  all: re-adjust build constraints for JS and NaCl
  https://golang.org/cl/122539

Updates #25893

Change-Id: I9abefc7d4ad158e9e68913362f7f1320321d6f5f
Reviewed-on: https://go-review.googlesource.com/c/151301
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@bcmills

This comment has been minimized.

Copy link
Member

commented Dec 5, 2018

What should we say about the AIX port in the Go 1.12 release notes?
(CC @andybons)

@trex58

This comment has been minimized.

Copy link
Author

commented Dec 6, 2018

Hi Bryan,
Clément, who ports golang, is out today and tomorrow. I need to work with him about what we should say about the AIX port of the Go 1.12 release. So, what is the dead-line?

Here is a first list of subjects we'll have to cover I think:

  • arch: 64bit only (ppc64)
  • processors: Power8 and higher only
  • OS version: AIX 7.2 minimum. Better results with TL2 (need to test with TL3 & ASLR)
  • Go features not yet supported: cgo, race detector (requires LLVM), pprof, net IPv6, crypto/aes for ppc64, memory sanitizer, more ?
  • build-modes not supported: ToBeCompleted
  • tests results: ~100% OK in our dev environment. However still some issues on our Go buildfarm VM
  • gdb on AIX does not handle perfectly Go yet
  • golang.org/x/sys/unix : handles AIX golang (and AIX GCC Go)
  • golang.org/x/... : ToBeCompleted
  • Performance: ?
  • debug/xcoff: ?
  • AIX RPMs required for running golang: git, subversion, mercurial, gcc
  • AIX RPMs required for building golang: gcc-go ?, patch, coreutils, findutils, file, file-libs, perl, pcre-devel.
  • will also be available as AIX RPMs on BullFreeware web-site and then on IBM AIX ToolBox
  • other limitations: ?
  • other: ?
@bcmills

This comment has been minimized.

Copy link
Member

commented Dec 6, 2018

That's probably more information than we need for the release notes proper: most of the detail can probably go on an accompanying wiki page (e.g., https://golang.org/wiki/AIX). The release note itself should mostly just make people aware that it's an option.

For some idea of scale, see https://golang.org/doc/go1.11#ports and https://golang.org/doc/go1.8#ports.

@laboger

This comment has been minimized.

Copy link
Contributor

commented Dec 6, 2018

@Helflym

This comment has been minimized.

Copy link
Contributor

commented Dec 10, 2018

Hi,

I would said something like that:
"Go is now available on AIX 7.2 on POWER8 architecture (aix/ppc64). External linking, cgo, pprof and race detector aren't yet supported. "

I don't know if you wish to be more precise about buildmode but we can add that only -buildemode=exe is available.

It can also be said that golang.org/x/sys/unix now supports aix/ppc64 on both gc and gccgo.

I think this should be enough. Maybe, I'll add a word about gdb (even if there is an issue about that ?) but it should be available before the true release anyway.

About the Wiki page, that's a good idea ! But does it need to be ready before 1.12 first beta ? Or can I made it after ?

@theonewolf

This comment has been minimized.

Copy link

commented Dec 21, 2018

@trex58 do we have documented anywhere the limitations preventing getting this off the ground for AIX 6.X?

@pinacoelho

This comment has been minimized.

Copy link

commented Jan 3, 2019

Wonderful! Will you guys consider AIX 6.1 ?

No. Sorry. AIX 6.1 does not provide all the features required by Go (either GCC Go or golang).

Hi, What's missing in 6.1 ?

@trex58

This comment has been minimized.

Copy link
Author

commented Jan 3, 2019

@theonewolf @pinacoelho We have to build the complete list of requirements golang has on the AIX operating system.
On AIX 7.2, we see that golang works better (no more test failures) on: 7200-02-02-1810 compared to: 7200-00-02-1614 . See: https://build.golang.org/# ("aix" is on the right) for 7200-02-02-1810 .
Please remember that golang on AIX requires Power8 or 9. Do you have AIX 6.1 on P8 or P9 CECs? We have to check if the last AIX 6.1 TL provides what we need or not, probably no.
We could port golang on AIX 7.1, at least 3 months from now. However, we need to know which TLs do provide golang requirements, and there are other priorities.
What do you plan to do with Go on AIX? Why AIX 6.1?

@theonewolf

This comment has been minimized.

Copy link

commented Jan 3, 2019

@trex58 I am trying to produce a binary for some old servers that can not be upgraded to AIX 7.X, they are stuck on 6.X.

@trex58

This comment has been minimized.

Copy link
Author

commented Jan 3, 2019

@trex58 I am trying to produce a binary for some old servers that can not be upgraded to AIX 7.X, they are stuck on 6.X.

golang on AIX, as on Linux/Power, requires Power8 or 9. Are they using such processors with their 6.1 ?
AIX 6.1 is supported on Power8&9 with P7 mode.

@pinacoelho

This comment has been minimized.

Copy link

commented Jan 3, 2019

  1. The AES assembly code does require POWER8 or above, however there is also a pure golang implementation of those functions.
  2. AIX 7.1 introduced the openat family of functions. I'm still trying to figure out if golang needs them or just provides an interface to them. (In the later case, their absence isn't critical - in any case, I'm trying to figure out if it's possible to reimplement them in golang)
  3. There's still a lot of AIX 6.1 LPARs lying around (namely, the VIOS V2), and a lot of them are excluded from immediate upgrade considerations (for whatever factors). Being able to build golang programs to run on all AIX LPARs (AIX 6.1, 7.1, 7.2 and VIOS V2, V3) would be a great step into bringing them into a DevOps world.

I'm currently in a dead-end situation: AIX 6.1 & 7.1 on POWER5+POWER6, I'm trying to bootstrap gcc/gccgo from the GCC V6, and "it ain't easy" isn't even in the picture yet.
Removing the POWER8&AIX 7.2 requirement would allow me to run the toolset on AIX7.1/POWER6 to build golang into a AIX6.1/POWER5.

Alternate option: build a cross-chain gccgo on linux/x86_64 to aix6.1/ppc64 (power5) to produce an initial golang for aix6.1/power5, but it's looking like a double effort for a half return.

@trex58

This comment has been minimized.

Copy link
Author

commented Jan 4, 2019

AFAIK, porting golang on AIX 6.1 seems impossible, due to the lack of some DWARF features even in last TLs.
About AIX 7.1, our focus now is AIX 7.2 only (and there are still plenty things to do!).

@owlmind

This comment has been minimized.

Copy link

commented Jan 17, 2019

I understand it won't work on power7? What is the reason it's power8 or higher only?

@trex58

This comment has been minimized.

Copy link
Author

commented Jan 17, 2019

@owlmind
We are using a work done by IBM Linux/Power team which started to port golang on Linux/Power for Power8 or higher only: the assembler part, which is common for AIX and Linux/Power. Porting golang on AIX is a very very complex task and we still are improving the quality and we are working hard so that golang 1.12 will be officially available on AIX, including cgo & golang.org/x (March). And we develop on AIX 7.2 on Power8 only today for reducing the complexity. Maybe we'll port golang on AIX 7.1 ; not sure. Maybe that the Power8 requirements can be relaxed ; not sure ; however for now, we haven't yet looked at this. Maybe that P8 assembler code is used only for encryption (which is for now only available for LE platform, thus not for AIX) ; so maybe that being compatible with P7 is not very complex ; however, we may look at these possibilities later, in April probably, when golang works perfectly on AIX 7.2 & P8/9. That depends also on how many AIX users want golang on older environments. Probably that customers wanting to use Go are focused on new applications and are using recent environments (AIX 7.2 & P8/9), which means that porting golang on AIX 7.1 and older HW will not be useful for many AIX users though it will take a lot of work.
Why would you like to have golang on Power7?
Do you want to use existing applications written in Go or do you want to write new applications on AIX due to the qualities of the Go language?
Do you plan to use cgo? (Go -> C or C -> Go ?)

@owlmind

This comment has been minimized.

Copy link

commented Jan 18, 2019

@trex58 We have a lot of power7 systems and most likely will have to use them for at least 2-3 years. And end of life for AIX 7.1 is 30 April 2022.
We want to use existing applications, particularly filebeat in this case. I was really surprised to see app supported for 7.2 and not for 7.1. Have never seen that before, software on aix usually have pretty good backward compatibility.

@awilfox

This comment has been minimized.

Copy link

commented Feb 2, 2019

#19074 details the (imo, extremely flimsy) reason that golang does not support POWER7 or lower. There's been some community movement on getting a patchset to support down to POWER5, but it hasn't fully materialised yet. Letting people know just how many of us have non-P8 hardware and want to run Go will help.

@zuraff

This comment has been minimized.

Copy link

commented Mar 26, 2019

@trex58 My use-case is to run certain existing utilities written in Go on our customers' environments, which are AIX 7.1 and AIX 7.2. These utilities depend on existing C libraries, so lack of cgo is a blocker.

@Helflym

This comment has been minimized.

Copy link
Contributor

commented Mar 26, 2019

@zuraff cgo is now available on master or in RPMs delivered by either us http://www.bullfreeware.com/affichage.php?id=4967 or by IBM on AIXToolBox.
However, it's only available on AIX 7.2, as @trex58 said, 7.1 is not our priority for the moment.

@orz--

This comment has been minimized.

Copy link

commented Mar 27, 2019

My customers use POWER8, POWER7 for production's environments.
But many cases use POWER7, POWER6 for testing environments.

@awilfox

This comment has been minimized.

Copy link

commented Mar 27, 2019

I think at this point the only way to get support for stable, older POWER ISAs is GCC Go. The Golang people are not receptive to us, the users of pre-P8 hardware, at all.

@trex58

This comment has been minimized.

Copy link
Author

commented Mar 28, 2019

I have warned the guys who take decisions about the many people asking for golang on older Power machines with AIX 7.2.

Raw porting on Power7 (and lower maybe) with AIX 7.2 does not seem so complicated. But officially managing older Power versions may be a burden since it requires at first a lot of complementary tasks and it generates then more build/test/maintenance work. It has a cost.

Porting golang on AIX 7.1 seems much more difficult. No plan.

Moreover, now there are also work to be done on GCC Go for GCC v9 (v1.10 to v1.12 : lot of changes) and on golang.org/x (in order to make all golang packages aware of AIX).

The priority for now seems to finalize GCC Go v9 and a complete golang on AIX 7.2 / Power8.
Moreover some very rare issues have to be studied. A quick study of performance must be done too, in order to find the top bottle-necks vs Linux/Power.

Looks like there is an interesting discussion at: #19074 . I need to read it completely.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.