Try out the new Go generics proposal with go-linq

[ahmetb]

Go now has a prototype of generics implementation. Here are some resources:

• https://blog.golang.org/generics-next-step
• https://go.googlesource.com/proposal/+/refs/heads/master/design/go2draft-type-parameters.md
• https://rakyll.org/generics-proposal/

This potentially could give a severe performance boost to go-linq, as well as actually make this library useful.

I haven't taken a closer look at how we would do this yet. For example, we might still end up having to do some type assertions. However, it might help us redesign the package and release a v4 if the generics proposal is adopted.

[cleitonmarx]

Here is my first try and keeping the go-linq original design: https://go2goplay.golang.org/p/gH50GCy32S7
Thoughts?

package main

import (
    "fmt"
)

type Query(type T) struct {
    Iterate func() func() (T, bool)
}

func From(type T)(source []T) Query(T) {
    len := len(source)

    return Query(T){
        Iterate: func() func() (T, bool) {
            index := 0
            return func() (item T, ok bool) {
                ok = index < len
                if ok {
                    item = source[index]
                    index++
                }
                return
            }
        },
    }
}

func (q Query(T)) Where(predicate func(T) bool) Query(T) {
    return Query(T){
        Iterate: func() func() (T, bool) {
            next := q.Iterate()
            return func() (item T, ok bool) {
                for item, ok = next(); ok; item, ok = next() {
                    if predicate(item) {
                        return
                    }
                }
                return
            }
        },
    }
}

func (q Query(T)) ToSlice() (r []T) {
    next := q.Iterate()

    for item, ok := next(); ok; item, ok = next() {
        r = append(r, item)
    }
    return
}

func main() {
    sliceInt := []int{1, 2, 3, 4, 5}
    sliceInt = From(sliceInt).
        Where(func(i int) bool {
            return i < 4
        }).ToSlice()

    fmt.Println("Int:", sliceInt)

    sliceStr := []string{"a", "b", "c", "d", "e"}
    sliceStr = From(sliceStr).
        Where(func(i string) bool {
            return i < "d"
        }).ToSlice()

    fmt.Println("Str:", sliceStr)
}

[ahmetb]

That’s a good start. I didn’t think a whole lot about this. But in this example basically seems the whole chain works only on the initial type provided. What if we say From(ints) but the result of the chain returns []string?

If there was a way to instruct certain methods such as ForEach<int,string>(ints) that could return []string that would be nice. I suspect that doesn’t exist. So that makes me wonder if we need to integrate with generics at all.

[cleitonmarx]

What if we say From(ints) but the result of the chain returns []string?

The proposed way will probably be a breaking change in the library. All projection methods that convert a type to another should be declared as a constructor function passing the source query as a parameter. In .NET, the extension methods make the syntax easier.

https://go2goplay.golang.org/p/CFQH-AKIfCU

package main

import (
	"fmt"
	"strconv"
)

type Query(type T) struct {
	Iterate func() func() (T, bool)
}

func From(type TSource)(source []TSource) Query(TSource) {
	len := len(source)
	return Query(TSource){
		Iterate: func() func() (TSource, bool) {
			index := 0
			return func() (item TSource, ok bool) {
				ok = index < len
				if ok {
					item = source[index]
					index++
				}
				return
			}
		},
	}
}

func Select(type TSource, TResult)(q Query(TSource), selector func(TSource) TResult) Query(TResult) {
	return Query(TResult){
		Iterate: func() func() (TResult, bool) {
			next := q.Iterate()
			return func() (item TResult, ok bool) {
				var it TSource
				it, ok = next()
				if ok {
					item = selector(it)
				}
				return
			}
		},
	}
}

func (q Query(T)) Where(predicate func(T) bool) Query(T) {
	return Query(T){
		Iterate: func() func() (T, bool) {
			next := q.Iterate()
			return func() (item T, ok bool) {
				for item, ok = next(); ok; item, ok = next() {
					if predicate(item) {
						return
					}
				}
				return
			}
		},
	}
}

func (q Query(T)) ToSlice() (r []T) {
	next := q.Iterate()

	for item, ok := next(); ok; item, ok = next() {
		r = append(r, item)
	}
	return
}

func main() {
	sliceStr := []int{1, 2, 3, 4, 5}
	sliceInt := Select(
		From(sliceStr).Where(func(i int) bool { return i < 4 }),
		strconv.Itoa,
	).ToSlice()

	fmt.Println("Select:", sliceInt)
}

[cleitonmarx]

Here is another try, more complex and playing around with the From function for slices, maps, channels and strings. We need to have a constructor for each type: https://gotipplay.golang.org/p/wUTln-sObFy

[ahmetb]

That's excellent. So we can do that after all I guess. Which part of the code you're referring to by "constructor"?

[cleitonmarx]

What I meant was, this kind of implementation would not be possible:

func From(source interface{}) Query {
	src := reflect.ValueOf(source)

	switch src.Kind() {
	case reflect.Slice, reflect.Array:
		return FromSlice(...)
	case reflect.Map:
		return FromMap(...)
	case reflect.String:
		return FromString(...)
	case reflect.Chan:
		return FromChannel(...)
	default:
		return FromIterable(...)
	}
}

We would need to use the specific "Query constructor" function in each particular type.

[ahmetb]

If getting rid of type-agnostic From() and leaving the FromXxx, that’s worth pursuing.

[ahmetb]

Generics proposal is now accepted.
This will be interesting. 😇

[ahmetb]

So I tried to imitate what From..Where..Select.. sort of thing would work.
Each Where/Select/... would have different type parameters:

type Array[K any] []K

func (a Array[K]) Print[V any](v V) {
	for _, i := range a {
		fmt.Printf("%v-%v\n",i,v)
	}
}

However this is not supported by design. https://go.googlesource.com/proposal/+/refs/heads/master/design/go2draft-type-parameters.md#methods-may-not-take-additional-type-arguments

[metalrex100]

@ahmetb are there any plans to still try to implement this lib with generics since go 1.18 beta has been released?

[ahmetb]

Based on my last experimentation, Go generics unfortunately isn't advanced enough to support this by design.

But I also encourage others to try Select().Order() where each method takes a generic type and returns another generic type processed further down the chain.

[szmcdull]

Hi, i made myself an early and naive one here https://github.com/szmcdull/glinq. Would you have a try?

[StevenACoffman]

@ahmetb Have you considered using Facilitators to mitigate the limitation of having no generic parameterized methods? See https://rakyll.org/generics-facilititators/

[ahmetb]

@StevenACoffman sadly the main problem with Go generics is the chaining won't work the way we currently have in this library. If you look at @szmcdull's glinq library, the chaining goes Any(Where(FromSlice(... (versus what we have today is FromSlice().Where().Any(). As method chaining becomes sophisticated, nested method approach is hard to read and reason about.

I am yet to see a feasible way to achieve the chaining approach we currently have with Go generics.

In the meanwhile, other libraries like lo that aren't fully linq (but offers small helpers) has emerged in the ecosystem. But the reason I insist not going the Go generics route is that I want to preserve the existing way of chaining LINQ methods as this library strives to be close to .NET LINQ in terms of look and feel.