Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: runtime: add AlignedN types that can be used to increase alignment #19057

Open
ianlancetaylor opened this issue Feb 13, 2017 · 65 comments

Comments

@ianlancetaylor
Copy link
Contributor

ianlancetaylor commented Feb 13, 2017

The sync/atomic packages have this in the docs in the "Bugs" section: "On both ARM and x86-32, it is the caller's responsibility to arrange for 64-bit alignment of 64-bit words accessed atomically. The first word in a global variable or in an allocated struct or slice can be relied upon to be 64-bit aligned." This makes it difficult to use atomic operations in types that may not necessarily be at the beginning of an allocated struct or slice. For example, sync.WaitGroup does this:

	// 64-bit value: high 32 bits are counter, low 32 bits are waiter count.
	// 64-bit atomic operations require 64-bit alignment, but 32-bit
	// compilers do not ensure it. So we allocate 12 bytes and then use
	// the aligned 8 bytes in them as state.
	state1 [12]byte

and this:

func (wg *WaitGroup) state() *uint64 {
	if uintptr(unsafe.Pointer(&wg.state1))%8 == 0 {
		return (*uint64)(unsafe.Pointer(&wg.state1))
	} else {
		return (*uint64)(unsafe.Pointer(&wg.state1[4]))
	}
}

Further, on x86 there are vector instructions that require alignment to 16 bytes, and there are even some instructions (e.g., vmovaps with VEC.256), that require 32 byte alignment. While those instructions are not currently generated by the gc compiler, one can easily imagine using them in assembler code, which will require the values to be appropriately aligned.

To permit programmers to force the desired alignment, I propose that we add new types to the runtime package: runtime.Aligned2, runtime.Aligned4, runtime.Aligned8, runtime.Aligned16, runtime.Aligned32, runtime.Aligned64, runtime.Aligned128. (We could also use bit values, giving us runtime.Aligned16 through runtime.Aligned1024, if that seems clearer.)

These types will be identical to the type struct{} except that they will have a the alignment implied by the name. This will make it possible to write a struct as

type vector struct {
    vals [16]byte
    _ runtime.Aligned16
}

and ensure that instances of this struct will always be aligned to a 16 byte boundary.

It will be possible to change sync.Waitgroup to be

type WaitGroup struct {
    noCopy noCopy
    _ runtime.Aligned8
    state uint64
    sema uint32
}

simplifying the code.

Although this functionality will not be used widely, it does provide a facility that we need today without requiring awkward workarounds. The drawback is the addition of a new concept to the runtime, though I think it is fairly clear to those people who need to use it.

Another complexity is that we will have to decide whether the size of a value is always a multiple of the alignment of the value. Currently that is the case. It would not be the case for the runtime.AlignedN values. Should it be the case for any struct that contains a field with one of those types? If the size of a value is not always a multiple of the alignment, we will have to modify the memory allocator to support that concept. I don't think that will be particularly difficult, but I haven't really looked.

@ianlancetaylor ianlancetaylor added this to the Proposal milestone Feb 13, 2017
@josharian
Copy link
Contributor

A few minor comments:

  • If we do this, we should probably do go/types: add GCSizes #17586 too. It's not cmd/vet that relies on go/types for information about what cmd/compile will do alignment-wise. Another example is wasted.
  • Your struct vector above will have a byte of padding at the end to avoid GC problems. You probably want the struct{} field as the first field. And we'll want to carefully document that recommendation. I mention this only because if you missed this detail, others definitely will.
  • This could also be done with a //go: annotation on the type. We already have other //go: annotations on types. Don't panic. I'm not suggesting we do this. (Even the mere mention of annotations tends to raise banshee-level howling.) But it does raise the general question about when and why to use magic embedded types vs annotations vs perhaps some other general mechanism that doesn't exist yet.

@ianlancetaylor
Copy link
Contributor Author

It's true that that we do have go:nointerface and go:notinheap annotations on types, but neither is documented and the latter is clearly only for the runtime. I suppose my general reaction to support //go: annotations on types can be summed up as https://www.youtube.com/watch?v=hulm_T_xnwY .

@cespare
Copy link
Contributor

cespare commented Feb 13, 2017

@ianlancetaylor when you wrote

(We could also use runtime.Aligned16, etc., if that seems clearer.)

I don't see how this differs from the previous sentence.

@ianlancetaylor
Copy link
Contributor Author

@cespare I meant to imply using bit values rather than byte values. Updated original comment to clarify. Those names would mean that the AlignedN names would correspond to intN names.

@rsc
Copy link
Contributor

rsc commented Feb 13, 2017

I agree we should fix this problem. I am less certain about how to fix it. Perhaps to start we should just align 64-bit integers to 64-bit addresses on 32-bit platforms. It's called out in sync/atomic because it's basically a bug on our side, one that we've just not fixed.

The solution proposed here essentially assumes the compiler will not reorder fields, at least not if these tags are present. I don't think we've fully closed the door on that (#10014). In that issue (two years ago), I argued that it is important for the programmer to have control over locality, so wholesale reordering of fields is not great (for example, sort by size and then lay out would give optimal packing but I think be too invasive).

At the same time, I am getting tired of looking for uint32-sized or bool-sized holes when adding fields to existing structs, and even more I am getting tired of being forced to choose between "understandable struct definition" and "small-in-memory struct definition". I do wonder if the compiler should be able by default to sift individual small fields up into gaps that would otherwise go unfilled, but not otherwise reorder the definitions. This is getting off-topic for this issue, except that any such scheme would need an override annotation for cgo and so that might give a mechanism for expressing alignment as well; of course any reordering would need to keep alignment in mind. I don't have any good ideas.

Also not every variable that needs alignment is a field in a struct. I'm not sure what to do about that either. Code might declare 'var x [16]byte' and want to pass it to something that requires 16-byte alignment, for example. Maybe that's getting too far ahead of ourselves, but it's worth keeping in mind.

I had hoped that alignment would be a property of a type, not a specific declaration. Are there cases where that's not tenable?

@mdempsky
Copy link
Contributor

mdempsky commented Feb 13, 2017

runtime.AlignedCache (which could just alias an appropriate AlignedN type) would help address #19025.

@ianlancetaylor
Copy link
Contributor Author

I did not mean to imply that this approach meant that the compiler could not reorder fields. The language spec does not state that the alignment of one field in a struct implies anything at all about the alignment of subsequent fields in the struct. I envision any uses of this as being of the form

type Vec struct {
    _ runtime.Aligned16
    b [16]byte
}

Here we know that any instance of Vec is aligned to a 16-byte boundary.

I agree that alignment should be a property of a type, and I believe that that satisfies all alignment needs in Go. The question is how you specify that alignment, and whether it can be done without using a magic comment. This proposal is one approach: in effect, you can only specify the alignment of struct types, and you do so by adding a field of type runtime.AlignedN.

For comparison, in C (with GCC extensions) you specify alignment of a type by writing

typedef T ... __attribute__((aligned(n)));

You can also specify alignment of a specific variable in the same way. Or, you can implement alignment for a specific memory allocation by using memalign or posix_memalign (since C's memory allocator does not understand types, it is generally necessary to use these functions when allocating memory for an aligned type).

Clearly for Go it would be nicer to be able to declare the alignment of any type, rather than this proposal which in effect only permits you to declare the alignment of a struct. If we can figure out a way to do that, we should. But I'd really rather not do it via a magic comment.

@minux
Copy link
Member

minux commented Feb 14, 2017 via email

@ianlancetaylor
Copy link
Contributor Author

For sync/atomic, the biggest problem is usually not the alignment of the
whole structure (because we have at least 8 byte alignment on struct larger
than 8 byte), but ensuring that the next uint64 will be aligned correctly
within the struct.

Yes, understood. With this proposal you write "the next uint64" as a field of type alignedUint64, a struct defined as

type alignedUint64 struct {
    _ runtime.Aligned8
    v uint64
}

Vectors and sync/atomic are not the only uses of aligned types, so I think a more general solution would be a good idea if we can find one.

@cespare
Copy link
Contributor

cespare commented Feb 14, 2017

Here's a slight variation on the original proposal:

  • Add runtime.Align2, ... runtime.Align128. (Note: "align", not "aligned")
  • These are just like struct{}, except that if a runtime.AlignN, is used as a struct field, it specifies the alignment of the following field (in declared order).
  • If a runtime.AlignN is declared as the last struct field, it specifies the alignment of the entire struct.

This gets to @minux's point about aligning particular struct fields. You could even use multiple AlignNs to align multiple fields in a single struct.

(AlignN is more like a magic comment than AlignedN is, but at least it isn't a comment.)

@rsc
Copy link
Contributor

rsc commented Feb 14, 2017

@ianlancetaylor Thanks for the clarification. I was slightly confused by the fact that in your original post 'struct vector' (really should be 'type vector struct') puts the alignment after the field, not before.

@minux
Copy link
Member

minux commented Feb 14, 2017 via email

@josharian
Copy link
Contributor

If a runtime.AlignN is declared as the last struct field, it specifies the alignment of the entire struct.

Zero-sized final fields cause cmd/compile to insert a padding byte at the end. I imagine that is unacceptable in many of the cases for which increasing alignment is important.

@rsc
Copy link
Contributor

rsc commented Feb 14, 2017

As Minux pointed out (hard to tell with the Github-mangling of the email reply), AlignN declared anywhere would end up specifying a min alignment for the entire struct, since a struct can't have alignment less than any of its fields.

@josharian
Copy link
Contributor

Thanks, Russ, I missed that.

Clever, Minux. Seems like it might be worth accepting it is a language change and defining unsafe.AlignedByte and letting folks take it from there.

@ianlancetaylor
Copy link
Contributor Author

alignedbyte is clever but I'm a little uncomfortable with the idea that [13]alignedbyte has an alignment requirement of 13 bytes.

@josharian
Copy link
Contributor

Perhaps it could specify a minimum alignment, and compilers could choose to round up to the nearest power of two.

@cespare
Copy link
Contributor

cespare commented Feb 14, 2017

Regarding Minux's idea:

type Align16 [0]aligned16 // zero size, can be embedded into other structs
to force alignment of the next field (and also the whole structure)

To clarify, isn't there "magic" required to guarantee this relationship with the next field (same as in my AlignN version), if the compiler can reorder fields?

@minux
Copy link
Member

minux commented Feb 14, 2017 via email

@rsc
Copy link
Contributor

rsc commented Feb 14, 2017

@josharian "minimum alignment" is not a well-defined concept unless the space of possible requests are all multiples or divisors of each others. A multiple of 16 is not a multiple of 13.

I tend to agree with Ian that alignedbyte is a little too much rope.

@minux
Copy link
Member

minux commented Feb 14, 2017 via email

@josharian
Copy link
Contributor

josharian commented Feb 14, 2017

Ack.

The thing I am struggling with is that once you start introducing magic, it is unclear which the right magic is. A magic field type in package runtime? A comment-based type annotation? An interpreted field tag (or type tag)? A magic interface in package runtime (check whether a type has an Aligned2 method)?

This does feel a bit like a language change, though, and unsafe does seem like the right home for manually messing with alignments. Maybe there's an alternative unsafe formation that provides less rope? Here's a terrible idea to start: unsafe.AlignedShift: [n]unsafe.AlignedShift has alignment 1<<n. :)

@rsc
Copy link
Contributor

rsc commented Feb 14, 2017

I think the compiler definitely cannot reorder arbitrary structures.

What you think is only a little interesting; telling us why is much more interesting.

Perhaps we can embed a unsafe.PackedStruct to signify that the compiler can rearrange a struct, but I still think packing a struct is better done by another program, not automatic by the compiler.

That's fine for structs that people don't look at. What bothers me most about packing structs explicitly (by hand or with a program) is that doing so rewrites the source code to be less readable.

@bcmills
Copy link
Contributor

bcmills commented Feb 14, 2017

Allowing the compiler to reorder structures would be tricky: we may need to distinguish between unoptimized layouts (which the compiler should obviously fix) and hand-optimized layouts.

If the author has intentionally adjusted cache-line locality or packed the struct to match a kernel or C data structure, how do we tell the compiler not to break that?

@rsc
Copy link
Contributor

rsc commented Feb 15, 2017

@bcmills, struct field reordering is #10014. When I mentioned it above I wrote:

This is getting off-topic for this issue, except that any such scheme would need an override annotation for cgo and so that might give a mechanism for expressing alignment as well.

@minux
Copy link
Member

minux commented Feb 15, 2017 via email

@bcmills
Copy link
Contributor

bcmills commented Feb 15, 2017

I would guess that most uses of alignment fall into one of two categories:

  1. Passing a pointer to an unexported field to an atomic function.
  2. Passing a struct to a C function (via cgo or a syscall).

We normally let the compiler figure out details of allocation and layout. Perhaps we could do the same for alignment most of the time.

We could do something akin to escape analysis to see which fields need to be aligned:

  • If a pointer to a field is passed as a function parameter that requires a particular alignment, then both the field and the struct require at least that alignment. Pointers passed to functions in package atomic must be aligned to their element size.
  • If a pointer to a field or struct is passed to a cgo function call, syscall, or converted to an unsafe.Pointer which may be passed to a cgo function call or syscall, then the field and the struct require C-compatible alignment.

For the few remaining cases (are there any?), perhaps we could add a no-op function call (akin to runtime.KeepAlive):

package runtime

// Align marks its argument as requiring the given alignment.
// The ptr argument must be a pointer to a variable of a struct type,
// or a pointer to a field on a variable of a struct type. 
// The alignment argument must be a compile-time constant.
func Align(ptr interface{}, alignment int)

The only situation I'm aware of that would require explicit calls to Align would be if a pointer is allocated and returned from a function in one package but the alignment constraints occur only in the calling package. I cannot think of any examples of such usage at the moment.

@josharian
Copy link
Contributor

@bcmills a remaining case: argument to user-written assembly routine using vector instructions. (Minux mentioned this above too.)

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/308971 mentions this issue: cmd/compile: add internal/align package for runtime

@rogpeppe
Copy link
Contributor

rogpeppe commented Apr 14, 2021

One other thought that's probably horrible, but I'll leave it here anyway: how about making runtime.Aligned work by using the size of the array that it's part of (Aligned would be zero sized) ?

type WaitGroup struct {
    _ [8]runtime.Aligned
    state uint64
    sema uint32
}

That allows any alignment to be specified without introducing a zillion new types.

Edit: I see that's pretty similar to this.

For the [17]runtime.Aligned case, maybe it's reasonable to give the compiler the freedom to arbitrarily increase the alignment if it's not compatible with hardware constraints. So on a machine that didn't allow unaligned word access, it could round up to the nearest word-size multiple.

@mdempsky
Copy link
Contributor

how about making runtime.Aligned work by using the size of the array that it's part of (Aligned would be zero sized) ?

This is how internal/align.elemT works in https://golang.org/cl/308971.

asiekierka pushed a commit to WonderfulToolchain/gcc-ia16 that referenced this issue May 16, 2022
GCC release freeze.

	* go-backend.c: Include "go-c.h".
	* go-gcc.cc (Gcc_backend::write_export_data): New method.

	* go-gcc.cc (Gcc_backend::Gcc_backend): Declare
	__builtin_prefetch.
	* Make-lang.in (GO_OBJS): Add go/wb.o.

commit 884c9f2cafb3fc1decaca70f1817ae269e4c6889
Author: Than McIntosh <thanm@google.com>
Date:   Mon Jan 23 15:07:07 2017 -0500

    compiler: insert additional conversion for type desc ptr expr
    
    Change the method Type::type_descriptor_pointer to apply an additional
    type conversion to its result Bexpression, to avoid type clashes in
    the back end. The backend expression for a given type descriptor var
    is given a type of "_type", however the virtual calls that create the
    variable use types derived from _type, hence the need to force a
    conversion.
    
    Reviewed-on: https://go-review.googlesource.com/35506


commit 5f0647c71e3b29eddcd0eecc44e7ba44ae7fc8dd
Author: Than McIntosh <thanm@google.com>
Date:   Mon Jan 23 15:22:26 2017 -0500

    compiler: insure tree integrity in Call_expression::set_result
    
    Depending on the back end, it can be problematic to reuse Bexpressions
    (passing the same Bexpression to more than one Backend call to create
    additional Bexpressions or Bstatements). The Call_expression::set_result
    method was reusing its Bexpression input in more than one tree
    context; the fix is to pass in an Expression instead and generate
    multiple Bexpression references to it within the method.
    
    Reviewed-on: https://go-review.googlesource.com/35505


commit 7a8e49870885af898c3c790275e513d1764a2828
Author: Ian Lance Taylor <iant@golang.org>
Date:   Tue Jan 24 21:19:06 2017 -0800

    runtime: copy more of the scheduler from the Go 1.8 runtime
    
    Copies mstart, newm, m0, g0, and friends.
    
    Reviewed-on: https://go-review.googlesource.com/35645


commit 3546e2f002d0277d805ec59c5403bc1d4eda4ed9
Author: Ian Lance Taylor <iant@golang.org>
Date:   Thu Jan 26 19:47:37 2017 -0800

    runtime: remove a few C functions that are no longer used
    
    Reviewed-on: https://go-review.googlesource.com/35849


commit a71b835254f6d3164a0e6beaf54f2b175d1a6a92
Author: Ian Lance Taylor <iant@golang.org>
Date:   Thu Jan 26 16:51:16 2017 -0800

    runtime: copy over more of the Go 1.8 scheduler
    
    In particular __go_go (aka newproc) and goexit[01].
    
    Reviewed-on: https://go-review.googlesource.com/35847


commit c3ffff725adbe54d8283c373b6aa7dc95d6fc27f
Author: Ian Lance Taylor <iant@golang.org>
Date:   Fri Jan 27 16:58:20 2017 -0800

    runtime: copy syscall handling from Go 1.8 runtime
    
    Entering a syscall still has to start in C, to save the registers.
    Fix entersyscallblock to save them more reliably.
    
    This copies over the tracing code for syscalls, which we previously
    weren't doing, and lets us turn on runtime/trace/check.
    
    Reviewed-on: https://go-review.googlesource.com/35912


commit d5b921de4a28b04000fc4c8dac7f529a4a624dfc
Author: Ian Lance Taylor <iant@golang.org>
Date:   Fri Jan 27 18:34:11 2017 -0800

    runtime: copy SIGPROF handling from Go 1.8 runtime
    
    Also copy over Breakpoint.
    
    Fix Func.Name and Func.Entry to not crash on a nil Func.
    
    Reviewed-on: https://go-review.googlesource.com/35913


commit cc60235e55aef14b15c3d2114030245beb3adfef
Author: Than McIntosh <thanm@google.com>
Date:   Mon Feb 6 11:12:12 2017 -0500

    compiler: convert go_write_export_data to Backend method.
    
    Convert the helper function 'go_write_export_data' into a Backend
    class method, to allow for an implementation of this function that
    needs to access backend state.
    
    Reviewed-on: https://go-review.googlesource.com/36357


commit e387439bfd24d5e142874b8e68e7039f74c744d7
Author: Than McIntosh <thanm@google.com>
Date:   Wed Feb 8 11:13:46 2017 -0500

    compiler: insert backend conversion in temporary statement init
    
    Insert an additional type conversion in Temporary_statement::do_get_backend
    when assigning a Bexpression initializer to the temporary variable, to
    avoid potential clashes in the back end. This can come up when assigning
    something of concrete pointer-to-function type to a variable of generic
    pointer-to-function type.
    
    Reviewed-on: https://go-review.googlesource.com/36591


commit c5acf0ce09e61ff623847a35a99da465b8571609
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Wed Mar 1 17:57:53 2017 +0100

    libgo: build tags for aix
    
    Build tags for the libgo source files required to build
    libgo on AIX.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/37633


commit 67ed19616898ea18a101ec9325b82d028cd395d9
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Thu Mar 2 15:41:31 2017 +0100

    libgo: handle AIX tag in match.sh and gotest
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/37638


commit 83ea2d694c10b2dd83fc8620c43da13d20db754e
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Wed Mar 1 17:48:16 2017 +0100

    libgo: add AIX support in configure and Makefile
    
    - support for GOOS=aix
    - CFLAGS/GOCFLAGS/LDFLAGS for AIX
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/37632


commit 35d577fe22ffa16a3ccaadf5dae9f6f425c8ec8c
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Mon Mar 6 15:00:15 2017 +0100

    runtime: adapt memory management to AIX mmap
    
    On AIX:
    * mmap does not allow to map an already mapped range,
    * mmap range start at 0x30000000 for 32 bits processes,
    * mmap range start at 0x70000000_00000000 for 64 bits processes
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/37845


commit 4e49e56a5fd4072b4ca7fcefe4158d6885d9ee62
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Mon Mar 6 13:42:26 2017 +0100

    runtime: add getproccount implementation for AIX
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/37844


commit ff626470294237ac664127894826614edc46a3d0
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Mon Mar 6 17:31:21 2017 +0100

    runtime: handle ERESTART errno with AIX's wait4
    
    On AIX, wait4 may return with errno set to ERESTART, which causes unexepected
    behavior (for instance, go build may exit with the message "wait: restart
    system call" after running a command, even if it was successfull).
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/37846


commit 37daabbfc83d533b826ef9ab10e2dee7406e7198
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Mon Mar 6 11:02:58 2017 +0100

    runtime: support for AIX's procfs tree
    
    On AIX, the process executable file is available under /proc/<pid>/object/a.out
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/37842


commit a0275c039d56acf4bf48151978c1a4ec5758cc2c
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Mar 8 07:00:05 2017 -0800

    libgo/Makefile.am: don't use nonportable \n or \t in sed expression
    
    The resulting zstdpktlist.go is less pretty, but it works.
    
    Reviewed-on: https://go-review.googlesource.com/37940


commit 29b190f76105aafa2b50b48249afdafecc97a4be
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Thu Mar 9 16:02:34 2017 +0100

    runtime: netpoll and semaphores for AIX
    
    semaphore implementation based on Solaris implementation in
    libgo/go/runtime/os_solaris.go
    
    netpoll is just a stub to avoid build failure on AIX.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/37966


commit 55ca6d3f3cddf0ff9ccb074b2694da9fc54de7ec
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Thu Mar 9 15:38:30 2017 +0100

    libmain: ensure initfn is called when loading a go library
    
    AIX does not support .init_array.
    The alterative is to export the __go_init function and tell the linker
    it is an init function with the -Wl,-binitfini:__go_init option.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/37965


commit 349a30d17d880ac8bc1a35e1a2ffee6d6e870ae9
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Fri Mar 10 11:15:08 2017 +0100

    libgo: use an import list for missing symbols
    
    libgo depends on symbols provided by Go programs at runtime. On AIX,
    this requires either to build libgo with -Wl,-berok linker option and
    the programs with -Wl,-brtl, or to provide a list of imported symbols
    when building libgo. The second options seems preferable, to avoid
    requiring an additional option for every Go program.
    
    There are also some symbols that are specific to GNU ld and do not
    exist when linking with AIX ld (__data_start, __edata, __etext and
    __bss_start).
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/37969


commit 91db0ea1ff068ca1d97b9c99612100ea5b96ddb2
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Wed Mar 8 15:34:45 2017 +0100

    crypto/x509: add certificate files locations for AIX
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/37952


commit 92e521c854e91709b949548c47e267377850f26a
Author: Ian Lance Taylor <iant@golang.org>
Date:   Fri Mar 10 14:10:11 2017 -0800

    compiler: fix check for pointer in Temporary_reference_expression
    
    The check for an unrepresentable pointer in
    Temporary_reference_expression::do_get_backend was incorrectly
    translated from C to Go in https://golang.org/cl/14346043.  Fix the
    check to use points_to rather than has_pointer and deref.  This should
    not make any difference in practice as either way the condition will
    only be true for a pointer to void, but points_to is correct and more
    efficient.
    
    Reviewed-on: https://go-review.googlesource.com/38009


commit 9a0b676e59e7171a630c48fdc3d4de6712bad0ca
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Thu Mar 16 16:51:53 2017 +0100

    libgo: add missing _arpcom struct to *sysinfo.go
    
    This struct is filtered due to having a field of type _in6_addr,
    but other types exported to *sysinfo.go are depending on it.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38251


commit 61262a757bdd3d9a595ab6a90f68c0c4ebed7bc1
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Thu Mar 16 18:27:46 2017 +0100

    syscall: raw_ptrace stub for AIX
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38253


commit 8029632b50880fd9b5e39299c738b38e3386595f
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Wed Mar 15 16:58:37 2017 +0100

    libgo: adapt runtime.inc to AIX
    
    * Two AIX types are wrongfully exported to runtime.inc as their names
      make them look like a Go type.
    * The sigset go type conflicts with a system sigset type.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38192


commit 25f3a90d14bc268479369ecc0eada72791612f86
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Wed Mar 15 16:58:37 2017 +0100

    libgo: update Makefile.in, accidentally omitted from last change
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38310


commit d52b4895616b66f93b460366527e74336829aaa5
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Thu Mar 16 18:39:26 2017 +0100

    syscall: TIOCSCTTY does not exist on AIX
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38254


commit ff1ec3847a4472008e5d53a98b6694b1e54ca322
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Thu Mar 16 18:07:34 2017 +0100

    syscall: syscall does not exist on AIX
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38252


commit c1ee60dabf0b243a0b0286215481a5d326c34596
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Fri Mar 17 17:18:18 2017 +0100

    net: EAI_OVERFLOW does not exist on AIX
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38266


commit ad4ad29aed9f70b14b39b488bfeb9ee745382ec4
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Fri Mar 17 17:23:56 2017 +0100

    net: sockopt/sockoptip stubs for AIX
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38267


commit 5d7db2d7542fe7082f426d42f8c2ce14aad6df55
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Fri Mar 17 16:35:05 2017 +0100

    os/user: add listgroups stub for AIX
    
    This is required to build os/user.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38263


commit 4e57a7973e9fa4cb5ab977c6d792e62a8f7c5795
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Wed Mar 22 11:11:30 2017 +0100

    os: fix readdirnames for AIX
    
    Largefile implementation should be used on AIX.
    
    readdir64_r function returns 9 and sets result to NULL when
    reaching end of directory, so this return code should not
    always be considered as an error.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38359


commit b34036967d1ec57b25e3debe077439b4210a1d4a
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Fri Mar 17 17:39:31 2017 +0100

    libgo: adapt sigtab.go to AIX
    
    On AIX, _NSIG is not directly defined to its integer value in
    gen-sysinfo.go.
    The real value is _SIGMAX32+1 or _SIGMAX64+1, depending if we are
    building a 32bit ligbo or a 64bit libgo, so we need to read one of
    those constants to set nsig value in mksigtab.sh
    
    This change also ensures that all signal numbers from 0 to nsig-1
    are referenced in sigtable.
    
    Reviewed-on: https://go-review.googlesource.com/38268


commit 20991c32671a183ec859b4f285df37fdd4634247
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Thu Mar 23 17:28:09 2017 +0100

    syscall: missing import in socket_bsd.go
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38369


commit c34754bd9adf5496c4c26257eaa50793553c11e8
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Wed Mar 22 17:57:01 2017 +0100

    sycall: WCOREDUMP macro is not defined on AIX
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38363


commit 4f38813482227b12ea0ac6ac1b981ff9ef9853ef
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Thu Mar 23 17:44:43 2017 +0100

    libgo: additional build tags for AIX
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38510


commit d117ede6ff5a7083e9c40eba28a0f94f3535d773
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Thu Mar 23 17:48:46 2017 +0100

    go/build: add AIX to "go build" command known OS
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38511


commit 7b0ddaa6a6a71f9eb1c374122d29775b13c2cac5
Author: Ian Lance Taylor <iant@golang.org>
Date:   Thu Mar 23 09:57:01 2017 -0700

    compiler: don't crash if imported package imports this one
    
    When building a test it's OK if test code imports a package that
    imports this one. The go tool is supposed to catch cases where this
    creates an impossible initialization order. The compiler already has
    code to permit this in Gogo::add_import_init_fn. This CL avoids a
    compiler crash on a similar case when writing out the export data.
    
    I have no test case for this. Basically it pushes a compiler crash
    into an error reported elsewhere.
    
    Problem was reported by Tony Reix.
    
    Reviewed-on: https://go-review.googlesource.com/38462


commit 925636975d075e3e3353823b09db3f933f23cb03
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Mar 29 14:14:18 2017 -0700

    runtime: copy finalizer support from Go 1.8 runtime
    
    Reviewed-on: https://go-review.googlesource.com/38794


commit 1ccb22b96cb3b1011db0e427877d9ddecb577fa9
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Thu Mar 30 15:21:06 2017 +0200

    runtime: initcontext and setcontext stubs for AIX
    
    Further investigations are required to understand the clobbering
    issue and implement a proper fix. Until then, those stubs are
    required to allow the build to complete.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38930


commit 27db481f369b54256063c72b911d22390c59199c
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Wed Mar 29 18:07:25 2017 +0200

    os: fix Readlink failure on AIX
    
    AIX readlink routine returns an error if the link is longer
    than the buffer, instead of truncating the link.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38700


commit c93babbf48eddd0bc34d4179ffb302dc60087299
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Wed Mar 29 17:26:35 2017 +0200

    compiler: implement support for reading AIX big archives
    
    This is required to read go export from a Go library.
    
    Code courtesy of Damien Bergamini from Atos Infogérance.
    
    Issue golang/go#19200
    Reviewed-on: https://go-review.googlesource.com/38698


commit 930dd53482bdee3a9074850d168d0b9d7819c135
Author: Ian Lance Taylor <iant@golang.org>
Date:   Thu Apr 6 18:50:11 2017 -0700

    compiler: fix whether conversions are static initializers
    
    The compiler was incorrectly treating type conversions from string to
    int or vice-versa as static initializers.  That doesn't work, as those
    conversions are implemented via a function call.
    
    This case may never actually arise but it seems like the right thing to do.
    
    Reviewed-on: https://go-review.googlesource.com/39872


commit f02691e4195728dbf06f4dde0853c6bccc922183
Author: Ian Lance Taylor <iant@golang.org>
Date:   Thu Apr 6 17:24:08 2017 -0700

    compiler, runtime: don't let slices point past end of memory block
    
    When Go code uses a slice expression like [:len(str)] or [:cap(slice)],
    it's natural for the resulting pointer to point just past the end of
    the memory block.  If the next memory block is not live, we now have a
    live pointer to a dead block, which will unnecessarily keep the block
    alive.  That wastes space, and with the new Go 1.8 GC (not yet
    committed) will trigger an error when using GODEBUG=gccheckmark=1.
    
    This changes the implementation of slice expressions to not move the
    pointer if the resulting string length or slice capacity is 0.  When
    the length/capacity is zero, the pointer is never used anyhow.
    
    Reviewed-on: https://go-review.googlesource.com/39870


commit 17527c35b027e1afcc318faf5563909e1e9d44a6
Author: Ian Lance Taylor <iant@golang.org>
Date:   Thu Apr 6 15:30:11 2017 -0700

    compiler: emit write barriers
    
    The Go 1.8 concurrent GC requires optional write barriers for all
    assignments that may change pointer values in the heap or in a global
    variable.  For details see https://blog.golang.org/go15gc.
    
    This changes the gofrontend code to emit write barriers as needed.
    This is in preparation for future changes.  At the moment the write
    barriers will do nothing.  They test runtime.writeBarrier.enabled,
    which will never be non-zero.  They call simple functions which just
    do a move without doing any of the other operations required by the
    write barrier.
    
    Reviewed-on: https://go-review.googlesource.com/39852


commit c0b00f072bf34b2c288e1271ec8118b88c4f6f6f
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Tue Apr 11 17:47:29 2017 +0200

    libgo: allow building gox files from PIC objects
    
    libtool builds non-PIC objects in the same directory as .lo files
    and PIC objects in a .libs subdirectory.
    BUILDGOX rule uses the non-PIC objects to build the gox files,
    but on AIX only the PIC objects are built.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/40355


commit ea0f3da174c5503a209043f14ddda34871cfec52
Author: Ian Lance Taylor <iant@golang.org>
Date:   Thu Apr 6 19:06:14 2017 -0700

    compiler: add code to generate a ptrmask for a type
    
    The Go 1.8 garbage collector uses a ptrmask for all types below a
    certain size.  A ptrmask is simply a bit vector with a single bit for
    each pointer-sized word in the value.  The bit is 1 if the type has a
    pointer in that position, 0 if it does not.
    
    This change adds code to the compiler to generate a ptrmask.  The code
    is not used by anything yet, it is just compiled.  It will be used
    when we switch over to the Go 1.8 garbage collector.
    
    The new Array_type::int_length method, and the new memory_size
    methods, will also be used by other patches coming later.
    
    Reviewed-on: https://go-review.googlesource.com/39873


commit 3029e1df3be3614d196a03c15e50e68ff850aa4c
Author: Ian Lance Taylor <iant@golang.org>
Date:   Fri Apr 7 10:31:39 2017 -0700

    compiler: add code to generate a gcprog for a type
    
    The Go 1.8 garbage collector uses a gcprog for all types above a
    certain size.  A gcprog describes where the pointers are in the type,
    using a simple bytecode machine that supports repeating bits.  The
    effect is to permit using much less space to describe arrays.  The
    format is described in runtime/mbitmap.go in the docs for runGCProg.
    This is not yet added to the gofrontend, but can be seen in the gc sources.
    
    This change adds code to the compiler to generate a gcprog.  The code
    is not used by anything yet, it is just compiled.  It will be used
    when we switch over to the Go 1.8 garbage collector.
    
    Reviewed-on: https://go-review.googlesource.com/39923


commit 8b01ef1e9176d20f4c9e667972fe031069a4d057
Author: Ian Lance Taylor <iant@golang.org>
Date:   Thu Apr 13 07:00:35 2017 -0700

    compiler: add ptrdata computations and expressions
    
    For the upcoming Go 1.8 GC we need to compute the "ptrdata" of a type:
    the number of bytes in the type that can contain pointers.  For types
    that do not contain pointers this number is zero.  For many types it
    is a number larger than zero but smaller than the total size of the
    type.  The Go 1.8 GC uses this number to make loops looking for
    pointers run faster by not scanning the suffix of a value that can not
    contain a pointer.
    
    Unfortunately there are two subtly different definitions of ptrdata,
    and we need both.  The first is the simple one: the prefix that can
    contain pointers.  The second is the number of bytes described by the
    gcprog for the type.  Recall that we describe the exact position of
    pointers in a type using either a ptrmask or a gcprog.  The ptrmask is
    simpler, the gcprog uses less space.  We use the gcprog for large
    types, currently defined as types that are more than 2048 bytes.  When
    the Go 1.8 runtime expands a gcprog, it verifies that the gcprog
    describes exactly the same number of bytes as the ptrdata field in the
    type descriptor.  If the last pointer-containing portion of a type is
    an array, and if the elements of the array have a ptrdata that is less
    than the size of the element type, then the simple definition of the
    ptrdata will not include the final non-pointer-containing bytes of the
    last element of the array.  However, the gcprog will define the array
    using a repeat count, and will therefore include the full size of the
    last element of the array.  So for a type that needs a gcprog, the
    ptrdata field in the type descriptor must be the size of the data
    described by the gcprog, and that is not necessarily the same as the
    simple ptrdata.
    
    It might seem that we can always use the gcprog version of the ptrdata
    calculation, since that is what will appear in a type descriptor, but
    it turns out that for global variables we always use a ptrmask, not a
    gcprog, even if the global variable is large.  This is because gcprogs
    are handled by expanding them into a ptrmask at runtime, and for a
    global variable there is no natural place to put the ptrmask.  Simpler
    to always use the ptrmask.  That means that we need to describe the
    size of the ptrmask, and that means that we need an expression for the
    simple form of the ptrdata.
    
    This CL implements the ptrdata calculation.  This code is not actually
    used yet.  It will be used later when the Go 1.8 GC is committed.
    
    Reviewed-on: https://go-review.googlesource.com/40573


commit 7a37331303b572412179a08141f1dd35339d40c8
Author: Ian Lance Taylor <iant@golang.org>
Date:   Fri Apr 14 06:55:48 2017 -0700

    compiler: zero length arrays never contain pointers
    
    Reviewed-on: https://go-review.googlesource.com/40696


commit c242f0508a64d3d74a28d498cbaeda785ff76258
Author: Ian Lance Taylor <iant@golang.org>
Date:   Fri Apr 14 07:26:54 2017 -0700

    bytes: disable allocations test on gccgo
    
    It turns out that testing.AllocsPerRun has not been producing correct
    results with the current gccgo memory allocator.  When we update to
    the Go 1.8 memory allocator, testing.AllocsPerRun will work again, and
    this test will fail due to lack of escape analysis.
    
    Reviewed-on: https://go-review.googlesource.com/40700


commit 0dc369f1d63376a36bfb0999a1b0377fd444bfab
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Tue Apr 11 16:22:38 2017 +0200

    os: alternative way to find executable path, using Args[0]
    
    AIX does not provide a proper way to find the original
    executable path from procfs, which contains just an
    hardlink.
    Executable path can be found using Args[0], Getcwd and
    $PATH.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/40353


commit f9bad1342569b338e3b2ea9f12ffc6d3d3fa3028
Author: Ian Lance Taylor <iant@golang.org>
Date:   Fri Apr 14 08:01:19 2017 -0700

    compiler: don't write struct with multiple sink fields to C header file
    
    When writing a struct to the C header file used by the C runtime code,
    a single sink field is fine: it will be called "_", which is valid C.
    There are structs with single sink fields that we want to write out,
    such as finblock.  As it happens, though, the Go 1.8 runtime has a
    struct with two sink fields, gcControllerState, which will produce a C
    definition with two fields named "_", which will fail.  Since we don't
    need to know that struct in C, rather than fix the general case, just
    punt if the struct has multiple sink fields.
    
    After the conversion to the Go 1.8 GC, we may be able to get rid of
    the C header file anyhow.  I'm not sure yet.
    
    Reviewed-on: https://go-review.googlesource.com/40701


commit cfc28901a572aeb15b2f10a38f79eec04c64dfb2
Author: Ian Lance Taylor <iant@golang.org>
Date:   Fri Apr 14 10:07:23 2017 -0700

    runtime: disable allocations test on gccgo
    
    It turns out that testing.AllocsPerRun has not been producing correct
    results with the current gccgo memory allocator.  When we update to
    the Go 1.8 memory allocator, testing.AllocsPerRun will work again, and
    these tests will fail due to lack of escape analysis.
    
    Reviewed-on: https://go-review.googlesource.com/40703


commit 36fedd76edaa48b9ec09709a70d9e4abaddf0caf
Author: Ian Lance Taylor <iant@golang.org>
Date:   Fri Apr 14 10:47:06 2017 -0700

    runtime: remove unused size argument from hash/equal fns
    
    The size argument was removed from hash and equal functions in CL
    34983.  Somehow I missed removing them from three of the predefined
    functions.
    
    Reviewed-on: https://go-review.googlesource.com/40770


commit 90f6accb48d2e78cad8955b9292933f6ce3fe4c8
Author: Ian Lance Taylor <iant@golang.org>
Date:   Fri Apr 14 13:23:05 2017 -0700

    runtime: remove unused stack.go
    
    We're never going to use stack.go for gccgo.  Although a build tag
    keeps it from being built, even having it around can be confusing.
    Remove it.
    
    Reviewed-on: https://go-review.googlesource.com/40774


commit befa71603fc66a214e01ac219f2bba36e19f136f
Author: Ian Lance Taylor <iant@golang.org>
Date:   Fri Apr 14 13:18:34 2017 -0700

    runtime: build fastlog
    
    Take out the build tags which were preventing fastlog2 from being
    built.  It's used by the upcoming Go 1.8 GC.
    
    Reviewed-on: https://go-review.googlesource.com/40773


commit b7e19e9be4ab4c3cd8f4c9506d79a8cd56bace40
Author: Ian Lance Taylor <iant@golang.org>
Date:   Fri Apr 14 10:04:23 2017 -0700

    runtime: add tests from Go 1.8
    
    Some runtime package tests never made it into the gofrontend repo for
    some reason.  Add them now.
    Reviewed-on: https://go-review.googlesource.com/40869


commit 1feef185aebd71bc2a09b9a04287461806096610
Author: Ian Lance Taylor <iant@golang.org>
Date:   Mon Apr 17 16:26:11 2017 -0700

    runtime: change mcall to take a Go function value
    
    For future work in bringing in the Go 1.8 GC, change the mcall
    function to take a Go function value, which means that mcall can take
    a closure rather than just a straight C function pointer.
    
    As part of this change move kickoff from C to Go, which we want to do
    anyhow so that we run the write barriers that it generates.
    
    Reviewed-on: https://go-review.googlesource.com/40935


commit c3db34f4efc2d610f74a01dd2ad7775f48889b29
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Tue Apr 11 16:11:26 2017 +0200

    runtime: netpoll implementation for AIX
    
    Code courtesy of Damien Bergamini from Atos Infogérance.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/40352


commit f5634dff40e53ad9ce61afd67fd07334e3af9d1f
Author: Ian Lance Taylor <iant@golang.org>
Date:   Tue Apr 18 22:06:07 2017 -0700

    runtime: move mstart from Go to C
    
    The assignments done in mstart must be done without write barriers, as
    mstart is running without an m or p.  In the gc toolchain the
    equivalent code to intialize g and g->m is written in assembler;
    on GNU/Linux, it's in the clone function.
    
    Reviewed-on: https://go-review.googlesource.com/40989


commit 671d7c74592f4b6fe3665af279482ba0ea47ca2d
Author: Ian Lance Taylor <iant@golang.org>
Date:   Tue Apr 18 17:47:28 2017 -0700

    compiler: varargs slices do not escape in runtime
    
    Also, don't try to allocate an empty slice on the stack, as it will
    confuse the GCC backend.
    
    Also add a few trivial style, code formatting, and debug output fixes.
    
    Updates golang/go#17431
    
    Reviewed-on: https://go-review.googlesource.com/40983


commit 94699d25f31353bf03419eda56b15993a39f3275
Author: Ian Lance Taylor <iant@golang.org>
Date:   Tue Apr 18 17:30:09 2017 -0700

    compiler: add Ptrmask_symbol_expression
    
    Add an expression to evaluate to the ptrmask for a type.  This will be
    used for global variables, which always use a ptrmask no matter how
    large they are.
    
    Reviewed-on: https://go-review.googlesource.com/40981


commit bfff1654eac5b9288fa6c431e66cba8c9da6a660
Author: Ian Lance Taylor <iant@golang.org>
Date:   Mon Apr 17 10:51:16 2017 -0700

    runtime: change g's in systemstack
    
    The systemstack function in the gc toolchain changes to a different g.
    This is often used to get more stack space; the gofrontend uses a
    different stack growth mechanism that does not require changing g's,
    so we've been running with a version of systemstack that keeps the
    same g.  However, the garbage collector has various tests to verify
    that it is running on g0 rather than on a normal g.  For simplicity,
    change the gofrontend version of systemstack to change to a different
    g just as the gc toolchain does.
    
    This permits us to uncomment some sanity checks in notetsleep.
    Doing that requires us to fix up a couple of places where C code calls
    {start,stop}TheWorldWithSema while not on g0.
    
    Note that this does slow down some code in the runtime package unnecessarily.
    It may be useful to find some places where the runtime calls
    systemstack only to get more stack space and change it to use some
    other function.  That other function would act like systemstack in the
    gc toolchain but simply call the argument in the gofrontend.
    
    Reviewed-on: https://go-review.googlesource.com/40973


commit b2ccc7601ce71a7c5732154cf9b2eeea64681469
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Apr 19 10:36:12 2017 -0700

    compiler, runtime: include ptrmask in GC roots
    
    Change the list of registered GC roots to include a ptrmask,
    and change the data structures to be easily used from Go code.
    The new ptrmask will be used by the Go 1.8 GC to only scan pointers.
    Tweak the current GC to use the new structures, but ignore the new
    ptrmask information for now.
    
    The new GC root data includes the size of the variable.  The size is
    not currently used, but will be used later by the cgo checking code.
    
    Reviewed-on: https://go-review.googlesource.com/41075


commit 9e065149970bc180e4ca83bb99c74d9c4f43b47b
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Apr 19 12:23:16 2017 -0700

    compiler, runtime: don't pass size to __go_new
    
    There is no reason to pass the size to __go_new, as the type
    descriptor includes the size anyhow.  This makes the function
    correspond to the Go 1.8 function runtime.newobject, which is what we
    will use when we update to the Go 1.8 memory allocator.
    
    Reviewed-on: https://go-review.googlesource.com/41080


commit c321de7b738c4a3387c1842919c9305acfa04c57
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Apr 19 13:13:56 2017 -0700

    compiler, runtime, reflect: make type descriptors more like Go 1.8
    
    Change the type descriptor structure to be more like the one in the Go
    1.8 runtime.  Specifically we add the ptrdata field, rename the gc
    field to gcdata and change the type to *byte, and rearrange a few of
    the fields.  The structure is still not identical to the Go 1.8
    structure--we don't use any of the tricks to reduce overall executable
    size--but it is more similar.
    
    For now we don't use the new ptrdata field, and the gcdata field is
    still the old format rather than the new Go 1.8 ptrmask/gcprog format.
    
    Reviewed-on: https://go-review.googlesource.com/41081


commit 7b70c52cddeebea9ebeac003f8c6aad59497e5f0
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Apr 19 14:54:29 2017 -0700

    reflect: make sure to clear unusable hash/equal function
    
    Otherwise we wind up copying the one from the prototype, which is wrong.
    
    Also rewrite the hash/equal functions to look like the ones in Go 1.8,
    mainly a matter of changing names and using arrayAt.
    
    Reviewed-on: https://go-review.googlesource.com/41133


commit 84d26f467f7de8bdbb0d230458135fe1b6b2a99d
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Apr 19 14:59:13 2017 -0700

    runtime: remove duplicate declarations of SetFinalizer/KeepAlive
    
    These should have been removed in CL 38794.  It's a bug that the
    compiler even permits these duplicate declarations.
    
    Reviewed-on: https://go-review.googlesource.com/41134


commit f85ff7e64c24031f6d0bd7c9c426b6176cb95160
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Apr 19 15:56:32 2017 -0700

    runtime: don't crash if panicstring called with no m
    
    It's possible for runtime_panicstring to be called with no m if a
    signal handler, or scheduler innards, do something wrong.  If that
    happens carry on with the panic rather than crashing.
    
    Reviewed-on: https://go-review.googlesource.com/41137


commit 5b362b04f642afb8b20715930416fc3b7d91bb12
Author: Than McIntosh <thanm@google.com>
Date:   Fri Mar 31 14:35:48 2017 -0400

    compiler: fix for expr sharing introduced by Order_eval::statement.
    
    When processing an expression statement with a top-level call
    that returns multiple results, Order_eval::statement can wind up
    creating a tree that has multiple references to the same call,
    which results in a confusing AST dump. Change the implementation
    to avoid introducing this unwanted sharing.
    
    Reviewed-on: https://go-review.googlesource.com/39210


commit b05b4260a68695bf9c9cc29e14ae86ca2699458a
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Apr 19 16:00:28 2017 -0700

    runtime: restore correct m in gtraceback
    
    If gtraceback is used to get a stack trace of a g running in the same m,
    as can happen if we collect a stack trace from a g0, then restore the
    old m value, don't clear it.
    
    Reviewed-on: https://go-review.googlesource.com/41138


commit ca8bbf4dfac19b3f4f7ce21a688b96a418c75031
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Apr 19 16:03:24 2017 -0700

    runtime: set startpc field when starting a new goroutine
    
    This puts the right value in a trace--previously it was always zero.
    
    Reviewed-on: https://go-review.googlesource.com/41139


commit ca8bbf4dfac19b3f4f7ce21a688b96a418c75031
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Apr 19 16:03:24 2017 -0700

    runtime: set startpc field when starting a new goroutine
    
    This puts the right value in a trace--previously it was always zero.
    
    Reviewed-on: https://go-review.googlesource.com/41139


commit 887690dce42d7bf8f711f8ea082e4928fb70f2a5
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Apr 19 17:06:11 2017 -0700

    runtime: add prefetch functions
    
    The Go 1.8 GC uses prefetch functions.  Add versions for gccgo that
    call __builtin_prefetch.  Uncomment the test for them in testAtomic64.
    Don't force the check function to return early, as now taking the
    address of a local variable in the runtime package does not force it
    onto the heap.
    
    Reviewed-on: https://go-review.googlesource.com/41144


commit 4269db69f9184e5a45c54aaee7352425a1f88bff
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Apr 19 17:55:21 2017 -0700

    runtime: split up ticks to get correct alignment
    
    On 32-bit x86 a uint64 variable by itself is aligned to an 8-byte boundary.
    A uint64 field in a struct is aligned to a 4-byte boundary.
    The runtime.ticks variable has a uint64 field that must be aligned
    to an 8-byte boundary.  Rather than rely on luck, split up the struct
    into separate vars so that the required alignment happens reliably.
    
    It would be much nicer if issue golang/go#19057 were fixed somehow,
    but that is for another day.
    
    Reviewed-on: https://go-review.googlesource.com/41143


commit 66926cabdbdbf3431b4f172f7756e195c1c6c513
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Thu Apr 20 17:15:38 2017 +0200

    libgo: fix bad value for O_CLOEXEC on AIX 7.1
    
    On AIX 7.1, O_CLOEXEC is defined as 0x0000001000000000, which
    creates an integer constant overflow error when building libgo.
    
    This affects only 7.1, O_CLOEXEC is not defined on 6.1 (and
    defaults to O in sysinfo.go) and is defined as 0x00800000 on
    AIX 7.2.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/41214


commit af288ff10aeafc47651f5def327ed56425d5be19
Author: Ian Lance Taylor <iant@golang.org>
Date:   Thu Apr 20 17:15:02 2017 -0700

    runtime: preserve stack context in tracebackothers
    
    The tracebackothers function works by saving the current stack context
    in the goroutine's context field and then calling gogo to switch to a
    new goroutine.  The new goroutine will collect its own stack trace and
    then call gogo to switch back to the original goroutine.  This works
    fine, but if the original goroutine was called by mcall then the
    contents of its context field are needed to return from the mcall.
    Fix this by saving the stack context across the calls to the other
    goroutines.
    
    Reviewed-on: https://go-review.googlesource.com/41293


commit 43101e5956e793f1b4de05c15d7738c785e927df
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Fri Apr 21 10:58:52 2017 +0200

    os/user: use _posix_* libc functions
    
    libc getpwnam_r function has a different signature, we must use
    _posix_getpwnam_r instead (by default, the pwd.h system include
     file defines getpwnam_r as a static function calling
    _posix_getpwnam_r, so a C program calling getpwnam_r will indeed
    reference the _posix_getpwnam_r symbol).
    
    Idem for getpwuid_r, getgrnam_r and getgrgid_r.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/41334


commit 71e1fec4d2a536591ea6657a06916a17b5127071
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Apr 19 21:24:48 2017 -0700

    runtime: don't use pointers in g_ucontext_t or stackcontext
    
    The g_ucontext_t type holds registers saved for a goroutine.  We have
    to scan it for pointers, but since registers don't necessarily hold
    pointers we have to scan it conservatively.  That means that it should
    not have a pointer type, since the GC will always scan pointers.
    Instead it needs special treatment to be scanned conservatively.
    The current GC doesn't care when a pointer type holds a non-pointer,
    but the Go 1.8 GC does.
    
    For the current GC this means we have to explicitly scan the
    g_ucontext_t values in a G.
    
    While we're at it change stackcontext to be uintptr too.  The entries
    in stackcontext never hold pointers that the Go GC cares about.
    
    Reviewed-on: https://go-review.googlesource.com/41270


commit eab2960aee91d3e3a6baa5b1bce01262d24c714f
Author: Ian Lance Taylor <iant@golang.org>
Date:   Thu Apr 20 17:08:19 2017 -0700

    runtime/internal/sys: define Goexperiment
    
    The gc toolchain defines Goexperiment based on the environment
    variable GOEXPERIMENT when the toolchain is built.  We just always set
    Goexperiment to the empty string.
    
    Reviewed-on: https://go-review.googlesource.com/41292


commit be4a751943265c0637da859d15a4faf162f5c478
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Thu Apr 20 14:04:35 2017 +0200

    net: sockopt implementation for AIX
    
    This is a copy of the Linux implementation, it allows to
    run some simple client/server applications on AIX, while
    the current sockopt stubs don't.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/41213


commit 46a669c4ca5b80fd6f6a0a42095804d9f704611d
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Wed Mar 29 17:55:06 2017 +0200

    math: fix sign for atan/expm1/log1p(-0)
    
    AIX libc returns +0 for atan(-0), expm1(-0) and log1p(-0),
    while matching Go functions must return -0.
    
    Code courtesy of Tony Reix.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/38699


commit 53b0e809130038a46f0a3d2870e3905f44ab888d
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Wed Apr 26 17:29:22 2017 +0200

    runtime: fix context clobbering on AIX
    
    On AIX 64-bits, r13 is a pointer to thread data.
    setcontext() overwrites r13 with the value saved by getcontext().
    So, when a goroutine is scheduled on a new thread, r13 will point
    to the old thread data after calling setcontext().
    
    Code courtesy of Damien Bergamini.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/41854


commit f8d5ebd71c71e6e777200530d8204b92619157f8
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Wed Apr 26 18:01:19 2017 +0200

    runtime: fix wrong time calculation in semasleep
    
    tv_nsec is added twice when calculating the sleep end time.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/41855


commit ef56097f4ea848d48fbf61eba1c757fe7fce99d3
Author: Matthieu Sarter <matthieu.sarter.external@atos.net>
Date:   Fri Apr 28 10:27:32 2017 +0200

    libgo: pass $(NM) value when running benchmarks
    
    On AIX, we need to use "nm -B" instead of "nm", to have the
    epxected output format, so the configured $(NM) value from
    the Makefile should be exported before running gotest, which
    defaults to "nm" if $NM is not set.
    
    Issue golang/go#19200
    
    Reviewed-on: https://go-review.googlesource.com/42051


commit 0fb550083ae474fb964435927b899ec8e4b62771
Author: Ian Lance Taylor <iant@golang.org>
Date:   Wed Nov 16 21:12:53 2016 -0800

    runtime: copy garbage collector from Go 1.8 runtime
    
    This giant patch replaces the old Go 1.4 memory allocator and garbage
    collector with the new Go 1.8 code.  The memory allocator is fairly
    similar, though now written in Go rather than C.  The garbage
    collector is completely different.  It now uses ptrmask and gcprog
    information, which requires changes in the compiler and the reflect
    package as well as the runtime.  And, of course, the garbage collector
    now runs concurrently with program execution.
    
    In the gc toolchain the garbage collector is strict and precise at all
    levels.  In the gofrontend we do not have stack maps, so stacks, and
    register values, are collected conservatively.  That means that an
    old, no longer used, pointer on a stack or in a register can cause a
    memory object to live longer than it should.  That in turns means that
    we must disable some checks for invalid pointers in the garbage
    collection code.  Not only can we get an invalid pointer on the stack;
    the concurrent nature of the collector means that we can in effect
    resurrect a block that was already unmarked but that the collector had
    not yet gotten around to freeing, and that block can in turn point to
    other blocks that the collector had managed to already free.  So we
    must disable pointer checks in general.  In effect we are relying on
    the fact that the strict pointer checks in the gc toolchain ensure
    that the garbage collector is correct, and we just assume that it is
    correct for the gofrontend since we are using the same code.
    
    Reviewed-on: https://go-review.googlesource.com/41307


commit a95078d501175240d095500a8c5fbfb21bec65cb
Author: Ian Lance Taylor <iant@golang.org>
Date:   Mon Apr 24 16:33:47 2017 -0700

    libgo/Makefile: clean more files
    
    Fix up the mostlyclean, clean, and distclean targets to better follow
    https://www.gnu.org/prep/standards/html_node/Standard-Targets.html.
    
    Reviewed-on: https://go-review.googlesource.com/41625


commit 5956bf1055451cf4239cdfeca259c23b1ded54d8
Author: Ian Lance Taylor <iant@golang.org>
Date:   Mon May 8 13:35:11 2017 -0700

    libgo: delete goc2c
    
    The last .goc file has been removed, so remove goc2c.
    
    The goc2c program was my first contribution to the gc repository that
    was more than 100 lines:
    golang/go@2b57a11
    The program was used in gc for a few years under various guises but
    was finally removed in https://golang.org/cl/132680043.  Now we can
    remove it from gofrontend as well.
    
    Reviewed-on: https://go-review.googlesource.com/42911


commit a222e35d041de0cd42506b61c93b8209e07702b9
Author: Than McIntosh <thanm@google.com>
Date:   Tue May 9 10:33:10 2017 -0400

    compiler: set "need_init_fn" when adding gc root
    
    Variables that back slice initializers in certain cases have to be
    added to the gc roots list, since they can be modified at runtime. The
    code that was doing this addition did not update the flag that tracks
    whether the package being compiled needs an initializer function,
    which resulted in the call in question being left out of the final
    generated code in certain cases. Fix is to change Gogo::add_gc_root()
    to update the "needs init" flag.
    
    Reviewed-on: https://go-review.googlesource.com/43030


commit 822ab419bf7d1c705cdce1c12133e7a11f56be2e
Author: Than McIntosh <thanm@google.com>
Date:   Tue May 9 11:36:51 2017 -0400

    compiler: fix variable context nit in write barrier generation
    
    Update the write barrier generation code to insure that the "lvalue
    context" tag on the space var expression is set only in the case where
    the expr feeds directly into an assignment. This is somewhat
    counter-intuitive, but needed in the case where the backend looks at
    context tags.
    
    Reviewed-on: https://go-review.googlesource.com/43031

From-SVN: r247848
@yuseferi
Copy link

yuseferi commented Jul 4, 2023

sounds like an interesting topic and someone gave a talk at GopherConf Eu 2023,
any update on this proposal?

valyala added a commit to VictoriaMetrics/VictoriaMetrics that referenced this issue Feb 23, 2024
…on' panic on 32-bit architectures

The issue has been introduced in bace9a2
The improper fix was in the d4c0615 ,
since it fixed the issue just by an accident, because Go comiler aligned the rawRowsShards field
by 4-byte boundary inside partition struct.

The proper fix is to use atomic.Int64 field - this guarantees that the access to this field
won't result in unaligned 64-bit atomic operation. See golang/go#50860
and golang/go#19057
valyala added a commit to VictoriaMetrics/VictoriaMetrics that referenced this issue Feb 23, 2024
…on' panic on 32-bit architectures

The issue has been introduced in bace9a2
The improper fix was in the d4c0615 ,
since it fixed the issue just by an accident, because Go comiler aligned the rawRowsShards field
by 4-byte boundary inside partition struct.

The proper fix is to use atomic.Int64 field - this guarantees that the access to this field
won't result in unaligned 64-bit atomic operation. See golang/go#50860
and golang/go#19057
@rip-create-your-account
Copy link

I have a use case that I don't see mentioned here. I'm working on a concurrent hash map and it requires a guaranteed alignment of 64 for the buckets that it allocates internally. I'm storing these buckets in a huge array where a bucket pointer is stored together with a [0, 64) 6-bit integer. I need guaranteed alignment so that I can pack the 6-bit integer into the pointer value like so:

// unsafe.Pointer value is legal as long as it points _somewhere_ into the object. Because
// our buckets always have sizeof >= 64B (and alignment of 64) it means that we can pack the
// 6-bit value into the lowest 6 bits so that the resulting pointer will continue to point into
// the original allocated object.
packed := unsafe.Add(unsafe.Pointer(bucket), smallInt)
atomic.StorePointer(&arrayElement.ptr, packed)

// and unpacking
packed := atomic.LoadPointer(&arrayElement.ptr)
bucket := (*Bucket)(unsafe.Pointer(uintptr(packed) &^ 0b11_1111))
smallInt := uint64(uintptr(packed) & 0b11_1111)

For my benchmark cases the allocator just happens to give the buckets alignment of 64 but I really need the guarantee as this packing produces a very measurable performance boost on multiple metrics compared to storing both values in their individual fields. I would like to see this proposal advance.

@G-M-twostay
Copy link

I have a use case where I wish to use the last bit of the address to generic struct for pointer tagging. The structs are stored in an array and I will use uintptr to array elements to perform pointer tagging.

type SometimesAligned[T constraints.Unsigned] struct{
    a,b,c T //constraints.Unsigned just limit to all unsigned int types.
}

Now SometimesAligned[byte] is the only thing that's aligned to 1 byte and thus isn't safe to use pointer tagging. Using

type Aligned0[T constraints.Unsigned] struct{
    a,b,c T 
    _ uint16
}

gives me 2 bytes alignment but will introduce unnecessary wasted space. It'll waste even more spaces on larger types.

type Aligned1 struct{
    values [2]uint16
}

only wastes 1 byte of space and essentially has the same capability as SometimesAlgined[byte]. However, it's hard to make this generic.

I wish to avoid the wasted spaces as much as possible while keeping the generic capability because these structs can be created in large numbers in arrays.

@randall77
Copy link
Contributor

Does a _ [0]uint16 field work for you?

@MatthiasKunnen
Copy link

I'm interested in this for the purpose of mapping shared memory (named memory in Windows, memory mapped file in Linux). Being able to map structs to memory requires alignment.

This is a snippet of what I am using at moment:

type Header struct {
	Status [4]byte // UTF-8 string

	Version uint32

	Revision uint32

	// The unix time when the last update to the data occurred.
	// Get int using GetLastUpdate.
	LastUpdate [8]byte

	// Offset of the Sensor section from beginning of Header.
	Offset uint32
}
func (info Header) GetLastUpdate() int64 {
	return int64(binary.LittleEndian.Uint64(info.LastUpdate[:]))
}

uint32 is aligned at the moment but this might change, uint64 is not, it has padding, so I'm using [8]byte.

@G-M-twostay
Copy link

Does a _ [0]uint16 field work for you?

Thank you. It looks like

type Aligned[T constraints.Unsigned] struct{
    _ [0]uint16
    a,b,c T
}

works ideally, but I kind of feel like it's a bit unintuitive.

@randall77
Copy link
Contributor

@G-M-twostay If this proposal is accepted, you will be able to write _ runtime.Aligned2 instead.

@randall77
Copy link
Contributor

@MatthiasKunnen Are you saying your LastUpdate field contains a 64-bit integer, but is not 64-bit aligned? Are you trying to match a native Windows API that behaves like that? Or is there another reason you want non-64-bit aligned 64-bit integers?
This proposal won't help that situation (if I'm interpreting your situation correctly). It will only increase alignment, not decrease it.

@MatthiasKunnen
Copy link

@randall77 LastUpdate contains a 64-bit integer but using uint64 as the type takes more space than 64 bits. It was a while ago but I believe this was due to padding or something the compiler does for optimization.

This is a program that shares data by sharing a part of its memory that is packed according to a documented structure.

In essence, I'm trying to mirror C# code like this:

[StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Ansi)]
public struct SharedMemory {
    public UInt32 version;
    public UInt8 rev; // This must be byte in Go because, I believe, uint8 still takes 32 bits
    public long poll_time; // This must be [8]byte in Go because uint64 takes more than 64 bits
};

@rip-create-your-account
Copy link

Not to turn this issue to my personal blog but I have to report my ever increasing thirst for this feature. My concurrent hash map has a len atomic.Int64 field and as one would expect it's getting hammered by all CPU cores on the insert path causing false-sharing and whatever else on the other fields. Manually adding padding so that the field is on its own 128-byte chunk like the following has quite the impact on performance.

type Map[K comparable, V any] struct {
	//...
	pad [128 - 40]byte
	len atomic.Int64
}

old: BenchmarkInsert/size=100000-8 40.77 ns/insert
new: BenchmarkInsert/size=100000-8 27.99 ns/insert

I have zero (0, nil) interest in manually maintaining the correct amount of padding. So I need runtime.Aligned64 to make the map work correctly and runtime.AlignedCacheLine to keep it simple and fast.

@randall77
Copy link
Contributor

@MatthiasKunnen: So it is the Pack=1 part of that C# declaration that is not aligning poll_time to 64-bit boundaries?
In that case, I think the way you are doing it, with [8]byte, is really your only option. This proposal (or any other that I know of) will not help in that situation.

@MatthiasKunnen
Copy link

MatthiasKunnen commented Aug 6, 2024

@randall77, apologies, I was using the wrong terminology. C# does not do any alignment with pack=1. Rather, it packs all fields as tightly as possible with no extra padding. Go does not do this in case of types such as uint64 and uint8. That being said, you are correct that this proposal will not help in this situation (I'm looking for padding decrease, not alignment increase) so I'll refrain from further comments to not derail the discussion. My previous comments can be minimized if desired.

@kostix
Copy link

kostix commented Aug 7, 2024

I'm interested in this for the purpose of mapping shared memory (named memory in Windows, memory mapped file in Linux). Being able to map structs to memory requires alignment.

This is a snippet of what I am using at moment:

type Header struct {
	Status [4]byte // UTF-8 string

	Version uint32

	Revision uint32

	// The unix time when the last update to the data occurred.
	// Get int using GetLastUpdate.
	LastUpdate [8]byte

	// Offset of the Sensor section from beginning of Header.
	Offset uint32
}

<…>

uint32 is aligned at the moment but this might change, uint64 is not, it has padding, so I'm using [8]byte.
<…>
@randall77 LastUpdate contains a 64-bit integer but using uint64 as the type takes more space than 64 bits.
It was a while ago but I believe this was due to padding or something the compiler does for optimization.

Here's the reason: Go makes sure each field in a struct type is aligned using the field's type natural alignment.
If you look at your types, you'll see that Status [4]byte is 32-bit, which makes the next 32-bit-sized field, Version, be naturally aligned; since the next field, Revision is also 32-bit, it's still naturally aligned.
All these three fields take up 4×3=12 bytes, and if the next field, LastUpdate, were to be a 64-bit (8 byte) integer, it would not be naturally aligned "as is", and the compiler would stuff 4 bytes before to make up for 16 bytes preceding space in total, which is wholly divisible by 8 making the field be naturally aligned.

Since a single byte is always naturally aligned, using [8]byte instead of 64-bit integer does not make the compiler insert any padding before that field. This is why you see differences in type size in both cases.

This is a program that shares data by sharing a part of its memory that is packed according to a documented structure.

In essence, I'm trying to mirror C# code like this:

[StructLayout(LayoutKind.Sequential, Pack = 1, CharSet = CharSet.Ansi)]
public struct SharedMemory {
    public UInt32 version;
    public UInt8 rev; // This must be byte in Go because, I believe, uint8 still takes 32 bits
    public long poll_time; // This must be [8]byte in Go because uint64 takes more than 64 bits
};

While your desire is understandable, please note an important thing: some hardware architectures require memory loads and stores to be performed on addresses naturally aligned for the types ("sizes") of data they perform; basically, you cannot perform an (imaginary) machine instruction LOAD $ADDR -> %INT64_REG unless $ADDR is aligned on a 8-byte boundary; performing such an operation would make the CPU (!) generate an error at runtime.
I beleive this is the reason Go makes all fields in structs naturally aligned for their respective types.

x86, which is most probably the architecture you're using, does not have the above restriction: the instructions operating on unaligned memory pefrorm slower but do not fail (while I cannot present any proofs ATM, I also beleive x86 does even still preserve atomicity for such operations).

What I'm leading you to, is that if Go were to allow what you're after — that is, to have LastUpdate be a 64-bit integer not aligned naturally, — an attempt to read or modify it (by machive-level "integer sized" instructions the compiler would likely have generated for that) would fail on some architectures Go supports.

In other words, if Go would have something like #pragma pack(1) of some C compilers or that Pack = 1 of .NET's interop, your code would have needed to be guarded by build constraints making it only compilable on amd64 and 386. (And this also hints at that such a feature, if implemented, would need to somehow require the usage of unsafe as it can break the main guarantee provided by Go — that what its compiler generates, cannot be incorrect from the PoV of the target hardware.)

As you can see, at the moment reading that 64-bit field using something like encoding/binary.LittleEndian.Uint64 definitely looks unwieldy but is guaranteed to work on any H/W arch.

If you're 100% sure this code will only ever work on x86, you could write a helper function which would still read that memory as an integer using type-punning made possible by unsafe (basically take the address of the 1st byte of that [8]byte array, reinterpret it as a pointer to a 64-bit integer and dereference the result).

@MatthiasKunnen
Copy link

@kostix, thank you for this dive into the reasoning behind it. Very interesting and informative!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests