V0.9.0 (#22)

# v0.9.0 #

The changes made in this PR is aimed at better supporting v0.9.0 of Gorgonia itself.  Along the way there are some new features and optimizations, as well as some bug fixes. 

The majority of the work in supporting v0.9.0 of Gorgonia is to shore up the underlying architecture to support CUDA related engines. This means moving more things to rely on `Engine` while keeping the engine interface overheads low.  Additionally this also means better support for column major data layouts. 


* Heavier reliance on `Engine` for most functions. This allows for extensibility on the data structure.
* Long standing bugbear - concepts of `RowVec` and `ColVec` has been removed (thanks to @matdodgson)
    - Touch points: `ap.go`, `iterator.go`,  `iterator_mult.go`.`shape.go`, and the tests that were correct prior to this change have semantic meaning changes too. 
    - **POTENTIAL TECH DEBT**: `iterator_mult.go` - the solution of filling with ones is a little too dodgy for my liking. The alternative would be to change `BroadcastStrides` which will change even more things (`Concat`, `Stack` etc)
* **Optimization**:
    -  `AP` has been depointerized in `*Dense` (thanks to @docmerlin). This reduces *some* amount of GC pointer chasing, but not all
    - allocation is slightly improved. (`(array).fromSliceOrArrayer`, `(array).fix()` and `(array).forcefix()` are part of the improvement around the logic of allocating data.
* **Bug fixes**:
    - Fixes subtle errors in linear algebra functions. The result is a slightly longer function but easier to reason with.
    - Fixes some subtle bugs in `Concat` - see also gorgonia/gorgonia#218
    - Fixed some small bugs with regards to `SampleIndex` that only show up when the slices have extreme lengths. This API should have been deprecated 2 years ago, but eh... it touched a lot of external projects.
* **API changes**:
    - `Diag` is made available. Relies heavily on an `Engine`'s implementation
    - `NewFlatIterator` is unexported.
    - `NewAP` is unexported.
    - `MakeAP` is used instead.
    - `(Tensor).DataOrder()` is added to the definiiton of what a `Tensor` is. 
    - `(Shape).IsScalarEquiv()`  is a new method. This corresponds to the change of semantics of what a `Shape` should be.
    - `(Shape).CalcStrides()` is exported now. This enables users to correctly calculate strides that are consistent to what the package expects.
    - `(Shape).CalcStridesColMajor()` is exported as the method to calculate the strides of a Col-Major `*Dense`. 
* **New Interfaces**:
    - `NonStdEngine` is an `Engine that does not allocate using the default allocator. This allows for both embedding a `DefaultEngine` while overriding the allocation behaviour.
    - `Diager` - any engine that can return a tensor that only contains the diagonal values of the input
    - `NaNChecker` and `InfChecker` - engines that can check a tensor for NaN and Inf
* **New Features**:
   * Added full support for colmajor tensors. (fixes #10)
        - TODO: colmajor iterator's prev() method (see #34)
   - Added serialization to Protobuf and Flatbuffers
        * TODO: Add example for serialization (see #35 and #36)
   - Added more support for sparse CS tensors.
* **New Subpackages**:
    * `native` is a subpackage that essentially gives users a native, Go-based iterator. Basically the ability to go from a `*Dense` to a `[][]T` or `[][][]T` **without extra allocations** (for the data). This was pulled into `master` earlier, but as of v0.9.0, the generic version is available too.
* **Semantic Changes**:
    - `Shape` has semantic changes regarding whether or not a shape is scalar. A scalar shape is defined to be `Shape{}` or `Shape{1}` only. Formerly, `Shape{1,1}` was also considered to be scalar. Now they're considered to be `ScalarEquivalent` (along with `Shape{1, 1, .... , 1}`)
    - A `Dtype` that is is orderable is also now comparable for equality. If `RegisterOrd` is called with a new `Dtype`, it is also automatically registered as `Eq`.  
* **Cosmetic Changes**:
    - README has been updated to point to correct doc pages

.travis.yml

@@ -7,6 +7,7 @@ branches:
 go:
   - 1.8.x
   - 1.9.x
+  - 1.10.x
   - tip
 
 env:

.travis/test.sh

@@ -6,9 +6,10 @@ go test -v -a -covermode=atomic -coverprofile=test.cover .
 go test -tags='avx' -a -covermode=atomic -coverprofile=avx.cover .
 go test -tags='sse' -a -covermode=atomic -coverprofile=sse.cover .
 go test -tags='inplacetranspose' -a -covermode=atomic -coverprofile=inplacetranspose.cover .
+go test -a -covermode=atomic -coverprofile=native.cover ./native/.
 
 # because coveralls only accepts one coverage file at one time... we combine them into one gigantic one
-covers=(./test.cover ./avx.cover ./sse.cover ./inplacetranspose.cover)
+covers=(./test.cover ./avx.cover ./sse.cover ./inplacetranspose.cover ./native.cover)
 echo "mode: set" > ./final.cover
 tail -q -n +2 "${covers[@]}" >> ./final.cover
 goveralls -coverprofile=./final.cover -service=travis-ci

CONTRIBUTORS.md

@@ -4,6 +4,7 @@
 * Naseer Dari (@ndari) - errors and error handling
 * Joe Kabaka (@kabaka0) - masked array functionality
 * Stuart Carnie (@stuartcarnie) - performance optimization for iterators
+* Jorge Landivar (@docmerlin) - performance optimization for `*Dense`
 
 # Contributors
 
@@ -13,8 +14,8 @@
 * David Soller | @3ygun
 * Davor Kapsa | @dvrkps
 * James Michael DuPont | @h4ck3rm1k3
-* Jorge Landivar | @docmerlin
 * Yuanlin Lian | @alienchow
+* Andrew SnodGrass | @pointlander
 
 
 

README.md

@@ -1,7 +1,8 @@
-# Package `tensor` [![GoDoc](https://godoc.org/github.com/gorgonia/tensor?status.svg)](https://godoc.org/github.com/gorgonia/tensor) [![Build Status](https://travis-ci.org/gorgonia/tensor.svg?branch=master)](https://travis-ci.org/gorgonia/tensor) [![Coverage Status](https://coveralls.io/repos/github/gorgonia/tensor/badge.svg?branch=master)](https://coveralls.io/github/gorgonia/tensor?branch=master) #
+# Package `tensor` [![GoDoc](https://godoc.org/gorgonia.org/tensor?status.svg)](https://godoc.org/gorgonia.org/tensor) [![GitHub version](https://badge.fury.io/gh/gorgonia%2Ftensor.svg)](https://badge.fury.io/gh/gorgonia%2Ftensor)  [![Build Status](https://travis-ci.org/gorgonia/tensor.svg?branch=master)](https://travis-ci.org/gorgonia/tensor) [![Coverage Status](https://coveralls.io/repos/github/gorgonia/tensor/badge.svg?branch=master)](https://coveralls.io/github/gorgonia/tensor?branch=master) [![Go Report Card](https://goreportcard.com/badge/gorgonia.org/tensor)](https://goreportcard.com/report/gorgonia.org/tensor) [![unstable](http://badges.github.io/stability-badges/dist/unstable.svg)](http://github.com/badges/stability-badges)#
+
 Package `tensor` is a package that provides efficient, generic (by some definitions of generic) n-dimensional arrays in Go. Also in this package are functions and methods that are used commonly in arithmetic, comparison and linear algebra operations.
 
-The main purpose of this package is to support the operations required by [Gorgonia](https://github.com/chewxy/gorgonia).
+The main purpose of this package is to support the operations required by [Gorgonia](https://gorgonia.org/gorgonia).
 
 ## Introduction ##
 In the data analysis world, [Numpy](http://http://www.numpy.org/) and [Matlab](https://www.mathworks.com/products/matlab.html) currently reign supreme. Both tools rely heavily on having performant n-dimensional arrays, or tensors. **There is an obvious need for multidimensional arrays in Go**. 
@@ -50,15 +51,15 @@ The `*Dense` tensor is the primary tensor and is represented by a singular flat
 
 ### Compressed Sparse Column Matrix ###
 
-Coming soon
+Documentation Coming soon
 
 ### Compressed Sparse Row Matrix ###
 
-Coming soon
+Documentation Coming soon
 
 ## Usage ##
 
-To install: `go get -u "github.com/chewxy/gorgonia/tensor"`
+To install: `go get -u "gorgonia.org/tensor"`
 
 To create a matrix with package `tensor` is easy:
 
@@ -129,7 +130,7 @@ b.SetAt(1000, 0, 1, 2)
 fmt.Printf("b:\n%v", b)
 ```
 
-There is a whole laundry list of methods and functions available at the [godoc](https://godoc.org/github.com/chewxy/gorgonia/tensor) page
+There is a whole laundry list of methods and functions available at the [godoc](https://godoc.org/gorgonia.org/tensor) page
 
 
 
@@ -198,7 +199,7 @@ The above call will use `myEngine` to allocate memory instead. This is useful in
 
 ### Other failed designs ###
 
-The alternative designs can be seen in the [ALTERNATIVE DESIGNS document](https://github.com/chewxy/gorgonia/blob/master/tensor/ALTERNATIVEDESIGNS.md)
+The alternative designs can be seen in the [ALTERNATIVE DESIGNS document](https://github.com/tensor/blob/master/ALTERNATIVEDESIGNS.md)
 
 ## Generic Features ##
 

ap.go

@@ -26,13 +26,30 @@ type AP struct {
 	Δ Triangle
 }
 
-// NewAP creates a new AP, given the shape and strides
-func NewAP(shape Shape, strides []int) *AP {
-	ap := borrowAP()
+func makeAP(size int) AP {
+	return AP{
+		shape:   Shape(BorrowInts(size)),
+		strides: BorrowInts(size),
+	}
+}
+
+// MakeAP creates an AP, given the shape and strides.
+func MakeAP(shape Shape, strides []int, o DataOrder, Δ Triangle) AP {
+	return AP{
+		shape:   shape,
+		strides: strides,
+		o:       o,
+		Δ:       Δ,
+		fin:     true,
+	}
+}
+
+// Init initalizes an already created AP with a shape and stries.
+// It will panic if AP is nil.
+func (ap *AP) Init(shape Shape, strides []int) {
 	ap.shape = shape
 	ap.strides = strides
 	ap.fin = true
-	return ap
 }
 
 // SetShape is for very specific times when modifying the AP is necessary, such as reshaping and doing I/O related stuff
@@ -46,6 +63,9 @@ func (ap *AP) SetShape(s ...int) {
 	if !ap.fin {
 		// scalars are a special case, we don't want to remove it completely
 		if len(s) == 0 {
+			if ap.shape == nil || ap.strides == nil {
+				ap.shape = Shape{}
+			}
 			ap.shape = ap.shape[:0]
 			ap.strides = ap.strides[:0]
 			return
@@ -102,9 +122,54 @@ func (ap *AP) IsScalar() bool { return ap.shape.IsScalar() }
 // IsMatrix returns true if it's a matrix. This is mostly a convenience method. RowVec and ColVecs are also considered matrices
 func (ap *AP) IsMatrix() bool { return len(ap.shape) == 2 }
 
-// Clone clones the *AP. Clearly.
-func (ap *AP) Clone() (retVal *AP) {
-	retVal = BorrowAP(len(ap.shape))
+// IsZero tell us if the ap has zero size
+func (ap *AP) IsZero() bool {
+	return len(ap.shape) == 0 && len(ap.strides) == 0 && !ap.fin && ap.o == 0 && ap.Δ == 0
+}
+
+// Zero zeros out an AP.
+func (ap *AP) zero() {
+	// log.Printf("ZEROING. Called by %v", string(debug.Stack()))
+
+	// Jorge's original implementation for zeroing a AP is as below
+	// but to cater for the (*Dense).fix() method of the *Dense
+	// a nil shape is used to signal unsetness
+	// so we cannot just truncate the shape even though it would be a lot more efficient
+
+	// ap.shape = ap.shape[:0]
+	// ap.strides = ap.strides[:0]
+	ReturnInts([]int(ap.shape))
+	ReturnInts(ap.strides)
+	ap.zeroOnly()
+}
+
+// side effect free zeroing
+func (ap *AP) zeroOnly() {
+	ap.shape = nil
+	ap.strides = nil
+
+	ap.fin = false
+	ap.o = 0
+	ap.Δ = 0
+}
+
+func (ap *AP) zeroWithDims(dims int) {
+	//ap.shape = BorrowInts(dims)
+	//ap.strides = BorrowInts(dims)
+	if cap(ap.shape) >= dims {
+		ap.shape = ap.shape[:dims]
+	}
+	ap.shape = BorrowInts(dims)
+	if cap(ap.strides) >= dims {
+		ap.strides = ap.strides[:dims]
+	}
+	ap.strides = BorrowInts(dims)
+}
+
+// Clone clones the *AP. Clearly. It returns AP
+func (ap *AP) Clone() (retVal AP) {
+	retVal = makeAP(cap(ap.shape))
+
 	copy(retVal.shape, ap.shape)
 	copy(retVal.strides, ap.strides)
 
@@ -118,21 +183,25 @@ func (ap *AP) Clone() (retVal *AP) {
 	return
 }
 
+func (ap *AP) CloneTo(dest *AP) {
+	dest.shape = append(dest.shape[:0], ap.shape...)
+	dest.strides = append(dest.strides[:0], ap.strides...)
+	dest.fin = ap.fin
+	dest.o = ap.o
+	dest.Δ = ap.Δ
+}
+
 // DataOrder returns the data order of the AP.
 func (ap *AP) DataOrder() DataOrder { return ap.o }
 
 // C returns true if the access pattern is C-contiguous array
-func (ap *AP) C() bool {
-	return ap.o.isRowMajor() && ap.o.isContiguous()
-}
+func (ap *AP) C() bool { return ap.o.IsRowMajor() && ap.o.IsContiguous() }
 
 // F returns true if the access pattern is Fortran contiguous array
-func (ap *AP) F() bool {
-	return ap.o.isColMajor() && ap.o.isContiguous()
-}
+func (ap *AP) F() bool { return ap.o.IsColMajor() && ap.o.IsContiguous() }
 
 // S returns the metadata of the sliced tensor.
-func (ap *AP) S(size int, slices ...Slice) (newAP *AP, ndStart, ndEnd int, err error) {
+func (ap *AP) S(size int, slices ...Slice) (newAP AP, ndStart, ndEnd int, err error) {
 	if len(slices) > len(ap.shape) {
 		// error
 		err = errors.Errorf(dimMismatch, len(ap.shape), len(slices))
@@ -146,7 +215,7 @@ func (ap *AP) S(size int, slices ...Slice) (newAP *AP, ndStart, ndEnd int, err e
 
 	var outerDim int
 	order := ap.o
-	if ap.o.isRowMajor() || ap.IsVector() {
+	if ap.o.IsRowMajor() || ap.IsVector() {
 		outerDim = 0
 	} else {
 		outerDim = len(ap.shape) - 1
@@ -160,12 +229,13 @@ func (ap *AP) S(size int, slices ...Slice) (newAP *AP, ndStart, ndEnd int, err e
 
 		size := ap.shape[i]
 		var stride int
-		if ap.IsVector() {
-			// handles non-vanilla vectors
-			stride = ap.strides[0]
-		} else {
-			stride = ap.strides[i]
-		}
+		stride = ap.strides[i]
+		// if ap.IsVector() {
+		// 	// handles non-vanilla vectors
+		// 	stride = ap.strides[0]
+		// } else {
+		// 	stride = ap.strides[i]
+		// }
 
 		var start, end, step int
 		if start, end, step, err = SliceDetails(sl, size); err != nil {
@@ -196,37 +266,29 @@ func (ap *AP) S(size int, slices ...Slice) (newAP *AP, ndStart, ndEnd int, err e
 
 	if ndEnd-ndStart == 1 {
 		// scalars are a special case
-		newAP = borrowAP()
+		newAP = AP{}
 		newAP.SetShape() // make it a Scalar
 		newAP.lock()
 	} else {
 
 		// drop any dimension with size 1, except the last dimension
+		offset := 0
 		for d := 0; d < dims; d++ {
-			if newShape[d] == 1 /*&& d != t.dims-1  && dims > 2*/ {
+			if newShape[d] == 1 && offset+d <= len(slices)-1 && slices[offset+d] != nil /*&& d != t.dims-1  && dims > 2*/ {
 				newShape = append(newShape[:d], newShape[d+1:]...)
 				newStrides = append(newStrides[:d], newStrides[d+1:]...)
 				d--
 				dims--
+				offset++
 			}
 		}
-
-		//fix up strides
-		if newShape.IsColVec() {
-			stride0 := newStrides[0]
-			ReturnInts(newStrides)
-			newStrides = BorrowInts(1)
-			newStrides[0] = stride0
-		}
-
-		newAP = NewAP(newShape, newStrides)
-		newAP.o = order
+		newAP = MakeAP(newShape, newStrides, order, ap.Δ)
 	}
 	return
 }
 
 // T returns the transposed metadata based on the given input
-func (ap *AP) T(axes ...int) (retVal *AP, a []int, err error) {
+func (ap *AP) T(axes ...int) (retVal AP, a []int, err error) {
 	// prep axes
 	if len(axes) > 0 && len(axes) != ap.Dims() {
 		err = errors.Errorf(dimMismatch, ap.Dims(), len(axes))
@@ -244,7 +306,7 @@ func (ap *AP) T(axes ...int) (retVal *AP, a []int, err error) {
 
 	// if axes is 0, 1, 2, 3... then no op
 	if monotonic, incr1 := IsMonotonicInts(axes); monotonic && incr1 && axes[0] == 0 {
-		return ap, a, noopError{}
+		return ap.Clone(), a, noopError{}
 	}
 
 	currentShape := ap.shape
@@ -270,12 +332,8 @@ func (ap *AP) T(axes ...int) (retVal *AP, a []int, err error) {
 		}
 	}
 
-	retVal = borrowAP()
-	retVal.shape = shape
-	retVal.strides = strides
-	if ap.IsVector() {
-		retVal.strides = retVal.strides[:1]
-	}
+	o := MakeDataOrder(ap.o, Transposed)
+	retVal = MakeAP(shape, strides, o, ap.Δ)
 	retVal.fin = true
 	return
 }
@@ -286,14 +344,21 @@ func (ap *AP) unlock() { ap.fin = false }
 
 func (ap *AP) calcStrides() []int {
 	switch {
-	case ap.o.isRowMajor():
-		return ap.shape.calcStrides()
-	case ap.o.isColMajor():
-		return ap.shape.calcStridesColMajor()
+	case ap.o.IsRowMajor():
+		return ap.shape.CalcStrides()
+	case ap.o.IsColMajor():
+		return ap.shape.CalcStridesColMajor()
 	}
 	panic("unreachable")
 }
 
+// setDataOrder is a method such that any tensor that embeds *AP will have the same method
+func (ap *AP) setDataOrder(o DataOrder) {
+	if !o.HasSameOrder(ap.o) {
+		ap.o = ap.o.toggleColMajor()
+	}
+}
+
 // TransposeIndex returns the new index given the old index
 func TransposeIndex(i int, oldShape, pattern, oldStrides, newStrides []int) int {
 	oldCoord, err := Itol(i, oldShape, oldStrides)