Update to 24.2 WIP #89

anthony-chaudhary · 2024-04-16T00:36:21Z

Related to #88

Work in progress, still getting these errors. I'm new to Mojo so if anyone can assist:

/root/llama2.mojo/llama2.mojo:506:20: error: 'String' value has no attribute 'bitcast'
    buf.data = data.bitcast[DType.uint8]()
               ~~~~^~~~~~~~
/root/llama2.mojo/llama2.mojo:517:29: error: no matching method in call to 'load'
        a.load[width=_nelts](j, a.load[width=_nelts](j) + b.load[width=_nelts](j))
        ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: method argument #1 cannot be converted from 'SIMD[f32, _nelts]' to 'Int'
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 2
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:543:39: error: no matching method in call to 'load'
        o.offset(j).load[width=_nelts](0, val)
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:580:29: error: no matching method in call to 'load'
        x.load[width=_nelts](start + ii, val)
        ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: method argument #1 cannot be converted from 'SIMD[f32, _nelts]' to 'Int'
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 2
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:589:29: error: no matching method in call to 'load'
        x.load[width=_nelts](start + ii, x.load[width=_nelts](start + ii) / ssum)
        ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: method argument #1 cannot be converted from 'SIMD[f32, _nelts]' to 'Int'
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 2
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:821:48: error: no matching method in call to 'load'
                    state.xb.load[width=_nelts](xb_offset + i, xbi)
                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: method argument #1 cannot be converted from 'SIMD[f32, _nelts]' to 'Int'
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 2
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:850:40: error: no matching method in call to 'load'
            state.hb.load[width=_nelts](i, hbi * state.hb2.load[width=_nelts](i))
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: method argument #1 cannot be converted from 'SIMD[f32, _nelts]' to 'Int'
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 2
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:881:32: error: invalid call to 'rand': missing 1 required positional argument: 'size'
    var r = rand[DType.float32](1)
            ~~~~~~~~~~~~~~~~~~~^~~
/root/llama2.mojo/llama2.mojo:1:1: note: function declared here
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:50:37: warning: 'SIMD[T, _width]' value is unused
        self.data.load[width=_width](newVal)
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~
        _ = 
/root/llama2.mojo/llama2.mojo:111:38: error: no matching method in call to 'load'
        return self._data.load[nelts](idx)
               ~~~~~~~~~~~~~~~~~~~~~~^~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 2
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: callee parameter #2 has 'Intable' type, but value has type 'Int'
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:119:38: error: no matching method in call to 'load'
        return self.load[width=nelts](indices[0] * self._shape[1] + indices[1])
               ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/llama2.mojo/llama2.mojo:110:8: note: candidate not viable: unknown keyword parameter: 'width'
    fn load[nelts: Int](self, idx: Int) -> SIMD[DType.float32, nelts]:
       ^
/root/llama2.mojo/llama2.mojo:113:8: note: candidate not viable: unknown keyword parameter: 'width'
    fn load[nelts: Int](self, *indices: Int) -> SIMD[DType.float32, nelts]:
       ^
/root/llama2.mojo/llama2.mojo:121:8: note: candidate not viable: unknown keyword parameter: 'width'
    fn load[
       ^
/root/llama2.mojo/llama2.mojo:129:8: note: candidate not viable: missing 1 required positional argument: 'val'
    fn load[nelts: Int](self, idx: Int, val: SIMD[DType.float32, nelts]):
       ^
/root/llama2.mojo/llama2.mojo:130:38: error: no matching method in call to 'load'
        return self._data.load[nelts](idx, val)
               ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 1 positional argument, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:1:1: note: candidate not viable: expected at most 2 positional arguments, got 3
from algorithm import sum
^
/root/llama2.mojo/llama2.mojo:133:34: error: no matching method in call to 'load'
        return self.load[width=1](idx, val)
               ~~~~~~~~~~~~~~~~~~^~~~~~~~~~
/root/llama2.mojo/llama2.mojo:110:8: note: candidate not viable: expected at most 2 positional arguments, got 3
    fn load[nelts: Int](self, idx: Int) -> SIMD[DType.float32, nelts]:
       ^
/root/llama2.mojo/llama2.mojo:113:8: note: candidate not viable: unknown keyword parameter: 'width'
    fn load[nelts: Int](self, *indices: Int) -> SIMD[DType.float32, nelts]:
       ^
/root/llama2.mojo/llama2.mojo:121:8: note: candidate not viable: expected at most 2 positional arguments, got 3
    fn load[
       ^
/root/llama2.mojo/llama2.mojo:129:8: note: candidate not viable: unknown keyword parameter: 'width'
    fn load[nelts: Int](self, idx: Int, val: SIMD[DType.float32, nelts]):
       ^
/root/llama2.mojo/llama2.mojo:382:40: error: 'List[SIMD[si8, 1]]' value has no attribute 'bitcast'
        var int32_ptr = config_data_raw.bitcast[DType.int32]()
                        ~~~~~~~~~~~~~~~^~~~~~~~
/root/llama2.mojo/llama2.mojo:469:27: error: 'List[SIMD[si8, 1]]' value has no attribute 'bitcast'
            var data = tmp.bitcast[DType.float32]()
                       ~~~^~~~~~~~
/root/.modular/pkg/packages.modular.com_max/bin/mojo: error: failed to parse the provided Mojo

The type parameter for the memcpy function is now automatically inferred. This means that calls to memcpy of the form memcpy[Dtype.xyz](...) will no longer work and the user would have to change the code to memcpy(...).

As per The simd_load(), simd_store(), aligned_simd_load(), and aligned_simd_store() methods on DTypePointer, Buffer, and NDBuffer have been merged into a more expressive set of load() and store() methods with keyword-only width and alignment parameters:

DynamicVector has been renamed to List, and has moved from the collections.vector module to the collections.list module. I

StaticTuple parameter order has changed to StaticTuple[type, size] for consistency with SIMD and similar collection types.

List.push_back() has been removed. Please use the append() function instead.

_steal_ptr

mikowals

Thanks for undertaking this. It is a fairly chunky set of changes for this release.

And I will definitely pull a copy and be able to help figure out any errors later today. Thanks again!

llama2.mojo

mikowals · 2024-04-16T01:05:16Z

llama2.mojo

@@ -379,7 +379,7 @@ struct Config:
        var config_data_raw = f.read_bytes(bytes_of_config_params)
        f.close()
        # correct Tensor type and shape for easy reading, without copying data
-        var int32_ptr = config_data_raw._steal_ptr().bitcast[DType.int32]()
+        var int32_ptr = config_data_raw.bitcast[DType.int32]()


Are you sure we don't still need to steal the pointer also? config_data_raw is going to be deleted after last use which will free this pointer before we are done with it if it is still the owner.

I'm not sure

With it remaining the error is

/home/anthony/llama2.mojo/llama2.mojo:382:40: error: 'List[SIMD[si8, 1]]' value has no attribute '_steal_ptr' var int32_ptr = config_data_raw._steal_ptr().bitcast[DType.int32]()

with it removed the error is

/home/anthony/llama2.mojo/llama2.mojo:382:40: error: 'List[SIMD[si8, 1]]' value has no attribute 'bitcast' var int32_ptr = config_data_raw.bitcast[DType.int32]()

So the theme seems to be 'List[SIMD[si8, 1]]' value has no attribute

I am sure. 😃 The correct thing is to steal the pointer so we take ownership. The method signature has just changed so it is a bit more elaborate now.

The full correct line here is:
var int32_ptr = DTypePointer[DType.int8](config_data_raw.steal_ptr().value).bitcast[DType.int32]().

It can be split into multiple lines if you think it is clearer but the steps are steal the pointer, convert to DTypePointer, and then bitcast to the int32 representation we want.

mikowals · 2024-04-16T01:06:08Z

llama2.mojo

@@ -466,7 +466,7 @@ struct TransformerWeights:
            # So we can't reshape to target shape because dims don't match
            var tmp = f.read_bytes(shape.num_elements() * sizeof[DType.float32]())
            bytes_read += shape.num_elements() * sizeof[DType.float32]()
-            var data = tmp._steal_ptr().bitcast[DType.float32]()
+            var data = tmp.bitcast[DType.float32]()


As above, we don't want tmp to keep ownership.

I am guessing that the bitcast errors may all be similar. I referenced the errors above with it either way

yes, same as above with changes to types is brackets and the variable name.

mikowals · 2024-04-16T01:06:34Z

llama2.mojo

@@ -503,7 +503,7 @@ fn read_file(file_name: String, inout buf: FileBuf) raises:
    var data = fd.read()
    fd.close()
    buf.size = data._buffer.size
-    buf.data = data._steal_ptr().bitcast[DType.uint8]()
+    buf.data = data.bitcast[DType.uint8]()


As above, we don't want data to keep ownership.

llama2.mojo

mikowals · 2024-04-16T02:57:45Z

I pulled this and tried against Mojo 24.2 but I think maybe you are working against the nightly branch? So these changes are likely to be needed in 24.4 assuming we get a release later this month that makes these new Mojo APIs public.

update: This looks like my own mistake not shifting the VS Code extension to 24.2. Your changes here are indeed for 24.2 so ignore this comment.

anthony-chaudhary · 2024-04-16T18:15:29Z

Hi Michael, thanks for all the updates 😎, I'll take a closer look and check version tonight! 😊

…

On Mon, Apr 15, 2024, 7:58 PM Michael Kowalski ***@***.***> wrote: I pulled this and tried against Mojo 24.2 but I think maybe you are working against the nightly branch? So these changes are likely to be needed in 24.4 assuming we get a release later this month that makes these new Mojo APIs public. — Reply to this email directly, view it on GitHub <#89 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEJ6DJAHEC5CLJLKT2XO3YDY5SHT7AVCNFSM6AAAAABGIKGSHOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJYGE2DEMJTHE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

: error: invalid call to 'rand': missing 1 required positional argument: 'size' var r = rand[DType.float32](1)

expected ']' for parameter list fn load[width=width: Int](self, *indices: Int) -> SIMD[DType.float32, width]:

requires a size and pointer declared create new pointer following https://docs.modular.com/mojo/stdlib/random/random#rand alloc(1) for 1 value?

'None' does not implement the '__bool__' method

anthony-chaudhary · 2024-04-17T06:05:42Z

Hello!

Okay so I think I fixed most of the items mentioned. Sorry about simd_store mix up I read the change log a little too fast and missed that.

I fixed a new error
anthony-chaudhary@017081e

The only two sets of remaining errors seem to be

Attempting to add a None check to fix subscript error:

/home/anthony/llama2.mojo/llama2.mojo:886:13: error: 'None' is not subscriptable, it does not implement the `__getitem__`/`__setitem__` or `__refitem__` methods
        if r[0] < cdf:

anthony-chaudhary@394d31f
But it still yields:

/home/anthony/llama2.mojo/llama2.mojo:886:12: error: 'None' does not implement the '__bool__' method
        if r:

Not surer if I'm missing something obvious but it seems like this isn't implemented in mojo the same as python?

The bitcast set of errors (which seems to actually be attribute errors, since it seems to be there either way with or without steal pointer) Update to 24.2 WIP #89 (comment)

anthony@Anthony:~/llama2.mojo$ mojo llama2.mojo stories15M.bin -s 100 -n 256 -t 0.5 -i "Mojo is a language"
/home/anthony/llama2.mojo/llama2.mojo:506:20: error: 'String' value has no attribute 'bitcast'
    buf.data = data.bitcast[DType.uint8]()
               ~~~~^~~~~~~~
/home/anthony/llama2.mojo/llama2.mojo:886:12: error: 'None' does not implement the '__bool__' method
        if r:
           ^
/home/anthony/llama2.mojo/llama2.mojo:382:40: error: 'List[SIMD[si8, 1]]' value has no attribute '_steal_ptr'
        var int32_ptr = config_data_raw._steal_ptr().bitcast[DType.int32]()
                        ~~~~~~~~~~~~~~~^~~~~~~~~~~
/home/anthony/llama2.mojo/llama2.mojo:469:27: error: 'List[SIMD[si8, 1]]' value has no attribute 'bitcast'
            var data = tmp.bitcast[DType.float32]()
                       ~~~^~~~~~~~
mojo: error: failed to parse the provided Mojo

Edit: Sounds like version thing isn't a concern but just in case this is the output of mojo -v
mojo 24.2.1 (58157dc0)

mikowals · 2024-04-17T06:26:18Z

llama2.mojo

@@ -937,12 +939,12 @@ fn print_str(s: PointerString):
    if (s[1].to_int() == ord("0")) and (s[2].to_int() == ord("x")):
        var d1: Int = s[3].to_int()
        var d2: Int = s[4].to_int()
-        print_no_newline(chr(str2num(d1) * 16 + str2num(d2)))
+        print(chr(str2num(d1) * 16 + str2num(d2)))


Since print_no_newline was removed the correct way to mimic this now is with a second argument end="" otherwise the default behaviour adds the newline character.

mikowals · 2024-04-17T06:26:53Z

llama2.mojo

        return
    # print all chars till null character
    var p: Int = 0
    while s[p].to_int() != 0:
-        print_no_newline(chr(s[p].to_int()))
+        print(chr(s[p].to_int()))


as above, end="" needs to be a second argument provided to print

mikowals · 2024-04-17T06:30:21Z

This error:

/home/anthony/llama2.mojo/llama2.mojo:886:13: error: 'None' is not subscriptable, it does not implement the `__getitem__`/`__setitem__` or `__refitem__` methods
        if r[0] < cdf:

Is because rand that creates a random Tensor moved from the random module to the tensor module. So rand needs to be added to the list of imports from tensor at the top of the file. And the from random line can be removed.

tairov · 2024-04-17T08:39:55Z

Hi @Anthony-Sarkis , I added a few fixes. Now it llama2 is compiling fine

anthony-chaudhary · 2024-04-17T16:30:46Z

Awesome, thanks everyone! 😊

…

On Wed, Apr 17, 2024, 1:44 AM Aydyn Tairov ***@***.***> wrote: Merged #89 <#89> into master. — Reply to this email directly, view it on GitHub <#89 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEJ6DJBD56NAAFTDUW25ANTY5YZAHAVCNFSM6AAAAABGIKGSHOVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJSGUYDIMBVGA4DGNY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

anthony-chaudhary added 7 commits April 15, 2024 16:53

Update memcpy

447860b

The type parameter for the memcpy function is now automatically inferred. This means that calls to memcpy of the form memcpy[Dtype.xyz](...) will no longer work and the user would have to change the code to memcpy(...).

Update simd_load to load

08f7ea0

As per The simd_load(), simd_store(), aligned_simd_load(), and aligned_simd_store() methods on DTypePointer, Buffer, and NDBuffer have been merged into a more expressive set of load() and store() methods with keyword-only width and alignment parameters:

Update llama2.mojo

375116a

DynamicVector -> List

ede881b

DynamicVector has been renamed to List, and has moved from the collections.vector module to the collections.list module. I

Update StaticTuple

34ee37d

StaticTuple parameter order has changed to StaticTuple[type, size] for consistency with SIMD and similar collection types.

update width and push_back

93f7cc5

List.push_back() has been removed. Please use the append() function instead.

remove workaround

aecabe2

_steal_ptr

mikowals suggested changes Apr 16, 2024

View reviewed changes

mikowals reviewed Apr 16, 2024

View reviewed changes