[stdlib] Make trait CollectionElement inherit from ExplicitlyCopyable

[gabrieldemarmiesse]

Waiting for the new nightly to be released so I can rebase this PR and shrink the diff

List of issues I have currently:

1 - Passing functions by parameter gives cryptic error message

Here is a minimal reproducer:

fn foo(inout buff: UnsafePointer[Int], len: Int):

    @parameter
    fn _less_than_equal(lhs: Int, rhs: Int) -> Bool:
        return lhs <= rhs

    bar[type=Int, cmp_fn=_less_than_equal](buff, len)

@always_inline
fn bar[
    type: CollectionElement, cmp_fn: fn (type, type) capturing -> Bool
](array: UnsafePointer[type], size: Int):
    ...

The error message is

invalid call to 'bar': callee parameter 'cmp_fn' has 'fn(Int, Int) capturing -> Bool' type, but value has type 'fn(lhs: Int, rhs: Int) capturing -> Bool'

The error message is not clear at all. What can we do to fix it? This issue prevents me from migrating over sort.mojo.

2 - The None constructor of Optional cannot be used anymore. We must use Optional[...]() or Optional[...](_NoneType)

It's still present in the code, but if one tries to use it, then the compiler says "cannot bind MLIR type None to trait CollectionElement". And the error message is often at the wrong place in the code (compiler bug?)

3 - Using Dict[Self, Self] in PythonObject gives recursive reference to declaration

So I had to comment out the method. I don't see how to reintroduce it.

[gabrieldemarmiesse]

Here is an example of the None constructor of Optional not working anymore.

[gabrieldemarmiesse]

Uncommenting this gives "recursive reference to declaration"

[gabrieldemarmiesse]

@ConnorGray would that help if I rebase this PR or do you have an internal branch for this that you're taking care or? (in which case I can just close this PR then)

[gabrieldemarmiesse]

in the meantime we can recommend is None which works well.

[ConnorGray]

I think this will be fixed when we change the compiler to emit NoneType() when it sees None. (Currently it emits the #kgen.none raw MLIR type, which means there's always one extra "hop" of conversion where NoneType is currently used.)

[gabrieldemarmiesse]

@ConnorGray you can take a look at what this PR to get an idea of what it takes to make this work. But I'll likely split it further.

[ConnorGray]

Suggestion I think opt == None will work if you change this signature to:

    # TODO(MSTDL-715):
    #   This method should not be necessary, we should need
    #   only the comparison from a `Optional(None)`.
    fn __ne__(self, rhs: NoneType._mlir_type):

(See the similar comment and pattern on Optional.__init__(inout self, value: NoneType._mlir_type).)

[gabrieldemarmiesse]

I just tried and got the following error:

/projects/open_source/mojo/stdlib/test/collections/test_optional.mojo:148:5: error: cannot bind MLIR type 'None' to trait 'CollectionElement'
def test_optional_equality():
    ^
/projects/open_source/mojo/stdlib/src/builtin/value.mojo:148:8: note: MLIR type cannot satisfy required trait function here
    fn __init__(inout self, *, other: Self):
       ^
/projects/open_source/mojo/stdlib/test/collections/test_optional.mojo:148:5: error: cannot bind MLIR type 'None' to trait 'CollectionElement'
def test_optional_equality():
    ^
/projects/open_source/mojo/stdlib/src/builtin/value.mojo:148:8: note: MLIR type cannot satisfy required trait function here
    fn __init__(inout self, *, other: Self):
       ^

Since I get the same error when I didn't do any modifications to Optional I believe that it's a bug in the overload resolution.

[ConnorGray]

Question Does this work if you use rebind[UnsafePointer[List[Int]]](ptr) wherever the pointer is passed? That might be a simpler workaround (if it does work).

[gabrieldemarmiesse]

I'm not sure I understand where the rebind should go, but I tried this:

@value
struct ValueDestructorRecorder(CollectionElement):
    var value: Int
    var destructor_counter: UnsafePointer[List[Int]]

    fn __init__(inout self, value: Int, destructor_counter: UnsafePointer[List[Int]]):
        self.value = value
        self.destructor_counter = rebind[UnsafePointer[List[Int]]](destructor_counter)

    fn __init__(inout self, *, other: Self):
        self.value = other.value
        self.destructor_counter = other.destructor_counter

    fn __del__(owned self):
        self.destructor_counter[].append(self.value)

And I get the following error:

/projects/open_source/mojo/stdlib/test/memory/test_maybe_uninitialized.mojo:26:32: error: no matching function in initialization
        ValueDestructorRecorder(
        ~~~~~~~~~~~~~~~~~~~~~~~^
/projects/open_source/mojo/stdlib/test/test_utils/types.mojo:120:8: note: candidate not viable: argument #2 cannot be converted from 'UnsafePointer[List[Int], 0, 0]' to 'UnsafePointer[List[Int], 0, 0]'
    fn __init__(inout self, value: Int, destructor_counter: UnsafePointer[List[Int]]):
       ^
/projects/open_source/mojo/stdlib/test/test_utils/types.mojo:124:8: note: candidate not viable: missing 1 required keyword-only argument: 'other'
    fn __init__(inout self, *, other: Self):
       ^
mojo: error: failed to parse the provided Mojo source module

and if I rebind an Int, I just get

/projects/open_source/mojo/stdlib/src/builtin/rebind.mojo:42:52: error: invalid rebind between two unequal types: index to !kgen.pointer<none>

[ConnorGray]

!sync

[gryznar]

Should this code be commented without todo / fixme etc?

[gabrieldemarmiesse]

This won't go into nightly, it's there to show what is needed to make the CI happy

[gryznar]

Same question, is commenting that intentional?

[ConnorGray]

!sync

[gabrieldemarmiesse]

@ConnorGray I found a workaround for the Dict[PythonOject, PythonObject]. Let me know what you think :)

I believe the PR is good enough to be merged.

[JoeLoser]

!sync

[gabrieldemarmiesse]

Feel free to ping me when you have time for a review and want me to rebase the PR