database/sql: add generic Null[T]

[methane]

database/sql doesn't have NullUInt64. So drivers and ORMs may implement their own NullUInt64.
Application developers would be confused which NullUInt64 is returned by dynamic Scan methods.

When NullUInt64 was proposed in past, @a8m, @kardianos, @rsc said "wait for generic".
#47953 (comment)

Now we have generics. So can we have it for now? Should it Null[T] or Nullable[T]?

Here is quick implementation: https://go.dev/play/p/Jtx8hP0H1_Z

[seankhliao]

see also #48702

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[rsc]

/cc @kardianos @bradfitz

[rsc]

Looking at the existing NullInt64, etc they name the value field after the type (Int64 for NullInt64). We can't do that for the generic version - it needs a single field name for all the instances. Value is the obvious choice but that has to be a method. Val is the next obvious choice but then the fields are Val and Valid. Maybe X and Valid?

[rsc]

Does someone want to write a CL and see how it looks?

[a8m]

Maybe X and Valid?

I suggest going with V and Valid, and don’t mind changing my CL to support this.

but that has to be a method

Could you please elaborate on why it needs to be a method?

[ianlancetaylor]

Value has to be a method in order to implement database/sql/driver.Valuer.

[methane]

I sent pull request with X and Valid.
It has only test copied from NullString. I will add other tests if needed.

[gopherbot]

Change https://go.dev/cl/501700 mentions this issue: database/sql: add Null[T]

[mateusz834]

I feel like Nullable[T] is a better name for this.

[leaxoy]

I think Null[T] is more useful not only in database/sql, it's a more general abstraction used to describe has a value or nothing there, and should defined outside database/sql.

This abstraction in rust/scala is Option and in haskell is Maybe, all of them use sum type to construct it.

There are many proposals to add sum type to go, latest is #57644. Maybe we can wait it make progress.

[methane]

We have waited for generics. Do you saying that we should wait for sum type again?

[leaxoy]

@methane Not exactly, the Null[T] you proposed in other language is the concept of fp. And not only database/sql depend on it, there are many uses in other places too. For example, when indexing a slice/array or get value from map, Null[T] can indicate result is either some of T or nothing.
Should we move to a separate package like maybe or optional.

[bradfitz]

The database/sql Scan value type is not going to be a general option type that non-SQL users use. Make your own type elsewhere if that's what you want. But even then it won't be widely used by all the libraries so it'll always feel awkward and not idiomatic in Go.

[rsc]

It should be Null[T] to match NullString etc. If those were named NullableString then we'd use Nullable[T]. This is only about sql.

[rsc]

Have all concerns about this proposal been addressed?

[rsc]

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

[rsc]

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

[jba]

http://go.dev/cl/c/go/+/501700 missed go 1.21, but it should be able to get into 1.22. I made some comments on the CL.

We still haven't picked a name for the value field. I know that go/ast uses X a lot, but at least there it might mean "eXpression." Here it just feels odd. How about Val?

[jba]

I missed Russ's point about Val and Valid. Maybe V then?

[jba]

http://go.dev/cl/c/go/+/501700 has been merged. This will be in 1.22.
@methane, as the author of the CL, would you like to add something to the release notes?

[gopherbot]

Change https://go.dev/cl/519555 mentions this issue: doc/go1.22: mention new sql.Null[T]

[LukaGiorgadze]

Before it arrives:

here's the implementation of go null with generics: https://github.com/lomsa-dev/gonull
supports custom types.

package main

import (
	"encoding/json"
	"fmt"

	"github.com/lomsa-dev/gonull"
)

type MyCustomInt int
type MyCustomFloat32 float32

type Person struct {
	Name    string
	Age     gonull.Nullable[MyCustomInt]
	Address gonull.Nullable[string]
	Height  gonull.Nullable[MyCustomFloat32]
}

func main() {
	jsonData := []byte(`{"Name":"Alice","Age":15,"Address":null,"Height":null}`)

	var person Person
	err := json.Unmarshal(jsonData, &person)
	if err != nil {
		panic(err)
	}
	fmt.Printf("Unmarshalled Person: %+v\n", person)

	marshalledData, err := json.Marshal(person)
	if err != nil {
		panic(err)
	}
	fmt.Printf("Marshalled JSON: %s\n", string(marshalledData))
}

[allochi]

I'm sorry that my comment comes late to this proposal, but I do agree with @leaxoy that Null[T] use goes beyond just database/sql package, and it would be nice if it's somewhere else like optional package maybe.

[dolmen]

@leaxoy @allochi We are discussing here about a type that has to implement interfaces database/sql.Scanner and database/sql/driver.Valuer. This is very specific to database/sql.

[arvenil]

One thing is not entirely clear for me, is there a point to use e.g. *NullString once you can use Null[T]? Wouldn't it be more clear for new comers if the old Null types were marked as obsolete?

[jba]

I don't think there is any use for NullString over Null[string].
It's a separate question whether NullString and related types should be deprecated.

[DeedleFake]

Would it break anything for NullString to be declared as type NullString = Null[string]? It would still be a struct { string; bool } with the same methods. Embedding, maybe?

[jba]

It would break reflect.TypeOf(NullString{}) != reflect.TypeOf(Null[string]{}).