https://github.com/ardanlabs/gotraining/blob/master/topics/courses/go/README.md
All the information below are my personal notes from attending William Kennedy's adapted Ultimate Go (1 day length) workshop at Gophercon EU 2018 in Iceland. Some of the information might be imprecise and even wrong, so make sure to fact check what's written here and push any changes as you see fit.
These are rough notes and do not reflect the quality and content of the workshop that was delivered.
I highly encourage anyone that stumbles upon this repo to take the full Ultimate Go.
- Ultimate Go
- Concurrency
-
Latency:
- I/O.
- Network.
- Timeouts.
-
Garbage collection
- Generate the right type of Garbage.
-
Accessing Data
-
Algorithm efficiency
- Machines are already very powerful, we can be less optimized today.
Data Segment Stacks Heap
See: https://play.golang.org/p/JJMHWiZ9h9
Glossary:
* AF: Active Frame.
* Semantics: Behaviour.
* Mechanics: How it works.
-
Have their own resizeable stack, starting at 2K.
- Segmentation for safety.
-
Calling (executing) a function moves the AF to nother part of the stack.
Any memory below the active frame has no integrity and is stale.
// TODO: Diagram stack.
Pointers are a special variable type 4 to 8 bytes depending on the architecture.
func increment(inc *int) is still a pass by value, the value is the pointer address.
* Immutability.
* No side effects.
* Allocations happen on the stack (Free, sorta).
* Efficiency (Multiple copies of the data on multiple frames).
* Efficiency (Easy to maintain consistency).
* Side effects (Worsened in concurrent programs).
* Allocations on the Heap.
https://play.golang.org/p/sRcPykfoi1d
Allocations are only counted for
Call func => Initialize Stack => Finish func => Switch AF => Invalidates stacks below the AF.
Go compiler performs Static code analysis: * Escape Analysis (Any memory initialized down the stack is shared up).
Any shared variables upwards on the call stack will be allocated in the heap. Avoid pointer semantics on construction unless you return it directly or construct it on the pass val.
func createUserV2() *user {
u := user{
name: "Bill",
email: "bill@ardanlabs.com",
}
println("V2", &u)
return &u
}func createUserV2() *user {
u := &user{
name: "Bill",
email: "bill@ardanlabs.com",
}
println("V2", u)
return u
}
// OR
func createUserV2() user {
u := &user{
name: "Bill",
email: "bill@ardanlabs.com",
}
println("V2", u)
return *u
}Rule of thumb: Use semantics on return to hint what kind of value you're returning
THIS IS SO AWESOME
Optimizing for performance, use -gcflags "-m -m": will output reasoning behind escape analysis.
Escape analysis is about moving stuff from the Stack to the Heap. Value semantics should be the default option because allocations hurt (Even tho more performant).
The size of Stacks are set in stone and are known at compile time. If the compiler does not know the size of something it has to move it to the heap.
https://play.golang.org/p/tpDOwBCvqW
Goroutines are awesome cause they have a 2K stack.
They are resizeable (growable).
- Run out of space in the stack.
- Create a new (bigger) memory space.
In the example below we can the memory positions of a string variable hello that exists in the main.main goroutine stack and how the memory addresses change as the stakc is resized.
0 0x1044dfa0 HELLO
1 0x1044dfa0 HELLO
2 0x10455fa0 HELLO
3 0x10455fa0 HELLO
4 0x10455fa0 HELLO
5 0x10455fa0 HELLO
6 0x10465fa0 HELLO
7 0x10465fa0 HELLO
8 0x10465fa0 HELLO
9 0x10465fa0 HELLO
Thus, it creates new memory addresses to contain the contents of the pointers, thus new pointer addresses.
IN Go, the stack is isolated to that goroutine, and variables and pointers in the stack are self contained.
When memory is shared, the variables (Addr) move to the stack (See upwards behaviour).
When deciding to share memory by pointers, make sure that your usecase actually needs it. i.e. Data integrity.
Concurrent, Mark & Sweep, Garbage collector.
- Value semantics.
- Pointer semantics.
Go tries to keep resource utilisation at the minimum. Go asks:
What's the minimum Heap Size I can have so I can run efficiently (at a reasonable pace) below 100 microseconds.
Collector is allowed to use up to 25% of total CPU capacity on resource contention.
TO stop the world, you've got to stop all of the goroutines and bring them to a safe point the only current mechanism to do so is stopping them at function calls (up to 1.10).
Theres 2 stop the world points:
- Turning the Write barrier on.
- Scan stacks to the Active Frames.
- Everything starts white.
- Things move to grey and get marked either White or Black ocne the usage is decided.
- Once there's no more Grey items, Mark phase is done
- SWEEP => Reclaim WHITE objects not in use.
- OFF => GC Disabled.
White, Gray, Black.
once we're done, everything is White or Black.
Black means the object is used. White means the object is safe to be swept (deleted).
Less is more.
### Example heap distribution
============== 4 MB
Transient ^
-------------- 2 MB
Permanent
==============
### Arrays
https://github.com/ardanlabs/gotraining/blob/master/topics/go/language/arrays/README.md
https://youtu.be/WDIkqP4JbkE?t=1129 Small is fast.
Main memory is broken into cache lines. Cache lines => 64 bytes
Create code that has predictable access patterns to memory.
Prefetches. Prefetches read your program's access to memory and try to predict the access pattern to memory.
Arrays and slices give us contiguous strides of access to the memory.
Arrays are the most important data structure there is today.
Slice is the most important data structure in Go.
Most of the time the slice is the right data structure.
TLB Cache.
When a program comes up is given a full map of virtual memory.
Memory is managed in Memory Pages.
Elements in the link list 4194304
Elements in the matrix 4194304
goos: darwin
goarch: amd64
pkg: github.com/ardanlabs/gotraining/topics/go/testing/benchmarks/caching
BenchmarkLinkListTraverse-8 1000 6767661 ns/op 0 B/op 0 allocs/op
BenchmarkLinkListTraverse-8 1000 7207894 ns/op 0 B/op 0 allocs/op
BenchmarkLinkListTraverse-8 500 6474441 ns/op 0 B/op 0 allocs/op
BenchmarkColumnTraverse-8 500 11459971 ns/op 0 B/op 0 allocs/op
BenchmarkColumnTraverse-8 500 11157890 ns/op 0 B/op 0 allocs/op
BenchmarkColumnTraverse-8 500 10665148 ns/op 0 B/op 0 allocs/op
BenchmarkRowTraverse-8 1000 3959297 ns/op 0 B/op 0 allocs/op
BenchmarkRowTraverse-8 1000 4000582 ns/op 0 B/op 0 allocs/op
BenchmarkRowTraverse-8 1000 3924701 ns/op 0 B/op 0 allocs/op
PASS
ok github.com/ardanlabs/gotraining/topics/go/testing/benchmarks/caching 56.891s
https://github.com/ardanlabs/gotraining/tree/master/topics/go#data-oriented-design
TODO: Watch:
Read Data Oriented Design
Designing for value or pointer semantics
If the alteration changes the core value of what you're changing you want to return a new value without altering the state.
If the alteration modifies a property but doesn't modify the underlying state or meaning of the data then you want to youse pointer semantics.
Avoid mixing semantics in different types
Never go from pointer to value semantics, it causes SHIT TO GO DOWN.
Make the choice between Performance and decoupling. Interfaces provide double indirection which causes the escape analysis to move the types that use the interfaces to the heap even if they're not escaping.
Thread states
- Running (Or executing).
- Runnable (requesting CPU time).
- Waiting (Lot of substatuses).
- CPU Bound work: Thread never goes into waiting state.
- I/O Bound work: Threads go back and forth from Running to Runnable / Waiting.
- Context switches: expensive.
- Less is more, less threads, each thread gets more time.
- Non deterministic switches: Cannot make assumptions in the code.
Go is a cooperative scheduler.
runtime does the cooperation.
// Pprofs to stdout, so if you have any info going to standard out already
// Make sure to point it to some other io.Writer.
pprof.StartCPUProfile(os.Stdout)
defer pprof.StopCPUProfile()$ ./trace > p.out
$ go tool pprof p.out
(pprof) list find
(pprof) web list find
(pprof) top 40 -cum
(pprof) list <function name>### Trace
// Traces out to stdout, so if you have any info going to standard out already
// Make sure to point it to some other io.Writer.
trace.Start(os.Stdout)
defer trace.Stop()# Opens up
$ go tool trace t.out### Benchmarks
// Add memory profile
-memprofile p.out