Update Makefile for macOS Clang + OpenMP + C++11 by dratman · Pull Request #2 · freelw/cpp-transformer

dratman · 2025-05-27T08:02:07Z

With some help from ChatGPT 4o I was able to build your cpp-transformer project on a MacBook M3.
This pull request modifies the makefile so that the project can successfully compile and run on macOS using Apple Clang.
My Linux system is down so I have not been able to check whether the modified makefile still works there.

Changes:
• Enable OpenMP support via Homebrew’s libomp
• Detect system SDK path using xcrun for compatibility with Clang
• Add -std=c++11 to support C++ features like override and auto
• Clean up compiler vs linker flags to avoid Clang warnings

Tested and confirmed working on:
• MacBook Pro (M3 Max), macOS 14.4.1
• Apple Clang + libomp

The training and inference output on the Mac matches the output shown in your Readme.
Thank you for making this project available.

Ralph Dratman
(https://github.com/dratman/cpp-transformer-rd)

dratman · 2025-05-27T08:06:27Z

I unintentionally closed the pull request!

dratman · 2025-05-27T08:07:26Z

I closed this by mistake then reopened it.

freelw · 2025-05-27T09:41:44Z

I closed this by mistake then reopened it.

Thank you for pr and I have merged it. I am working on a new design at here. After I finish my work I will publish it here. I hope you will like the new version.

freelw · 2025-05-27T10:02:59Z

I found new makefile does not work on x64 linux. I revert it and add a new makefile_osx. #3 Can you test whether makefile_osx works on your MacBook? By this cmd : make -f makefile_osx

…

2025年5月27日 16:02，Ralph Dratman ***@***.***> 写道： Dratman

dratman · 2025-06-05T18:06:46Z

The makefile_osx works for MacOs so that is fine. I was wondering if you have a decoder.cpp which I could use in a language model? Thank you. Ralph

…

On Tue, May 27, 2025 at 6:03 AM freelw ***@***.***> wrote: *freelw* left a comment (freelw/cpp-transformer#2) <#2 (comment)> I found new makefile does not work on x64 linux. I revert it and add a new makefile_osx. #3 Can you test whether makefile_osx works on your MacBook? By this cmd : make -f makefile_osx > 2025年5月27日 16:02，Ralph Dratman ***@***.***> 写道： > > Dratman — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAWZPUD4YRBZ4F63OLKJTT3AQZ6TAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSMJRHEZTMMRRGE> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

dratman · 2025-06-05T21:13:00Z

I am interested in a C++ small language model with no GPU code for experimentation and study. I would pay something for you to build that for me. On Thu, Jun 5, 2025 at 2:06 PM Ralph Dratman ***@***.***> wrote:

…

The makefile_osx works for MacOs so that is fine. I was wondering if you have a decoder.cpp which I could use in a language model? Thank you. Ralph On Tue, May 27, 2025 at 6:03 AM freelw ***@***.***> wrote: > *freelw* left a comment (freelw/cpp-transformer#2) > <#2 (comment)> > I found new makefile does not work on x64 linux. I revert it and add a > new makefile_osx. > #3 > Can you test whether makefile_osx works on your MacBook? By this cmd : > make -f makefile_osx > > > 2025年5月27日 16:02，Ralph Dratman ***@***.***> 写道： > > > > Dratman > > — > Reply to this email directly, view it on GitHub > <#2 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAAWZPUD4YRBZ4F63OLKJTT3AQZ6TAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSMJRHEZTMMRRGE> > . > You are receiving this because you modified the open/close state.Message > ID: ***@***.***> >

freelw · 2025-06-07T03:16:51Z

The makefile_osx works for MacOs so that is fine. I was wondering if you have a decoder.cpp which I could use in a language model? Thank you. Ralph
…
On Tue, May 27, 2025 at 6:03 AM freelw @.> wrote: freelw left a comment (freelw/cpp-transformer#2) <#2 (comment)> I found new makefile does not work on x64 linux. I revert it and add a new makefile_osx. #3 Can you test whether makefile_osx works on your MacBook? By this cmd : make -f makefile_osx > 2025年5月27日 16:02，Ralph Dratman @.> 写道： > > Dratman — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAWZPUD4YRBZ4F63OLKJTT3AQZ6TAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSMJRHEZTMMRRGE . You are receiving this because you modified the open/close state.Message ID: @.***>

I am currently learning GPT-2 and will implement a version of it. By then, there might be the decoder.cpp you want.

dratman · 2025-06-07T07:11:50Z

GPT-2 would be great. I can make a financial contribution if that would be helpful. Meanwhile I transferred your encoder-only project in the XCode IDE on MacOS, building in that environment and soon I think it will be fully running in XCode. (Previously I had just used the MacOS terminal to get the encoder-only project working, creating the same output as yours. XCode is just an alternate way to produce the same results.) Thanks! Ralph

…

On Fri, Jun 6, 2025 at 11:17 PM freelw ***@***.***> wrote: *freelw* left a comment (freelw/cpp-transformer#2) <#2 (comment)> The makefile_osx works for MacOs so that is fine. I was wondering if you have a decoder.cpp which I could use in a language model? Thank you. Ralph … <#m_1281260957338099353_> On Tue, May 27, 2025 at 6:03 AM freelw *@*.*> wrote: freelw left a comment (freelw/cpp-transformer#2 <#2>) <#2 (comment) <#2 (comment)>> I found new makefile does not work on x64 linux. I revert it and add a new makefile_osx. #3 <#3> Can you test whether makefile_osx works on your MacBook? By this cmd : make -f makefile_osx > 2025年5月27日 16:02，Ralph Dratman @.*> 写道： > > Dratman — Reply to this email directly, view it on GitHub <#2 (comment) <#2 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAWZPUD4YRBZ4F63OLKJTT3AQZ6TAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSMJRHEZTMMRRGE . You are receiving this because you modified the open/close state.Message ID: *@*.***> I am currently learning GPT-2 and will implement a version of it. By then, there might be the decoder.cpp you want. — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAWZPXAFU55QH57Z4PLBQL3CJKTRAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNJRGU3TAOJZGA> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

freelw · 2025-06-08T12:16:19Z

GPT-2 would be great. I can make a financial contribution if that would be helpful. Meanwhile I transferred your encoder-only project in the XCode IDE on MacOS, building in that environment and soon I think it will be fully running in XCode. (Previously I had just used the MacOS terminal to get the encoder-only project working, creating the same output as yours. XCode is just an alternate way to produce the same results.) Thanks! Ralph
…
On Fri, Jun 6, 2025 at 11:17 PM freelw @.> wrote: freelw left a comment (freelw/cpp-transformer#2) <#2 (comment)> The makefile_osx works for MacOs so that is fine. I was wondering if you have a decoder.cpp which I could use in a language model? Thank you. Ralph … <#m_1281260957338099353_> On Tue, May 27, 2025 at 6:03 AM freelw @.> wrote: freelw left a comment (freelw/cpp-transformer#2 <#2>) <#2 (comment) <#2 (comment)>> I found new makefile does not work on x64 linux. I revert it and add a new makefile_osx. #3 <#3> Can you test whether makefile_osx works on your MacBook? By this cmd : make -f makefile_osx > 2025年5月27日 16:02，Ralph Dratman @.> 写道： > > Dratman — Reply to this email directly, view it on GitHub <#2 (comment) <#2 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAWZPUD4YRBZ4F63OLKJTT3AQZ6TAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSMJRHEZTMMRRGE . You are receiving this because you modified the open/close state.Message ID: @.> I am currently learning GPT-2 and will implement a version of it. By then, there might be the decoder.cpp you want. — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAWZPXAFU55QH57Z4PLBQL3CJKTRAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNJRGU3TAOJZGA . You are receiving this because you modified the open/close state.Message ID: @.*>

#23 This PR has merged the language model functionality.
The latest version has supported a simple language model. Please refer to the 'Language Model' section in readme.md for details. I hope this is helpful to you.

freelw · 2025-06-08T12:19:07Z

GPT-2 would be great. I can make a financial contribution if that would be helpful. Meanwhile I transferred your encoder-only project in the XCode IDE on MacOS, building in that environment and soon I think it will be fully running in XCode. (Previously I had just used the MacOS terminal to get the encoder-only project working, creating the same output as yours. XCode is just an alternate way to produce the same results.) Thanks! Ralph
…
On Fri, Jun 6, 2025 at 11:17 PM freelw @.> wrote: freelw left a comment (freelw/cpp-transformer#2) <#2 (comment)> The makefile_osx works for MacOs so that is fine. I was wondering if you have a decoder.cpp which I could use in a language model? Thank you. Ralph … <#m_1281260957338099353_> On Tue, May 27, 2025 at 6:03 AM freelw @.> wrote: freelw left a comment (freelw/cpp-transformer#2 <#2>) <#2 (comment) <#2 (comment)>> I found new makefile does not work on x64 linux. I revert it and add a new makefile_osx. #3 <#3> Can you test whether makefile_osx works on your MacBook? By this cmd : make -f makefile_osx > 2025年5月27日 16:02，Ralph Dratman @.> 写道： > > Dratman — Reply to this email directly, view it on GitHub <#2 (comment) <#2 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAWZPUD4YRBZ4F63OLKJTT3AQZ6TAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSMJRHEZTMMRRGE . You are receiving this because you modified the open/close state.Message ID: @.> I am currently learning GPT-2 and will implement a version of it. By then, there might be the decoder.cpp you want. — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAWZPXAFU55QH57Z4PLBQL3CJKTRAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNJRGU3TAOJZGA . You are receiving this because you modified the open/close state.Message ID: @.*>

you can use this script ./build_mac_cpu.sh to build on your macbook.

dratman · 2025-06-09T04:24:11Z

Thank you very much for creating lm.cpp and for the makefile changes! Indeed, that is helpful. 谢谢你

freelw · 2025-06-09T04:49:29Z

Thank you very much for creating lm.cpp and for the makefile changes! Indeed, that is helpful. 谢谢你

You're welcome. Thank you too for replying in Chinese.

freelw · 2025-06-09T05:05:24Z

Thank you very much for creating lm.cpp and for the makefile changes! Indeed, that is helpful. 谢谢你

checkpoint_20250609_123600_7.bin
You can use this pre-trained model for inference. The model was trained for 7 epochs using the full text of The Time Machine novel. The num_steps is 32. Like, run model inference with the command below.
`$ ./lm -c ./checkpoint_20250609_123600_7.bin -e 0
corpus : ./resources/time_machine/timemachine_preprocessed.txt
epochs : 0
batch_size : 16
gpu : 1
learning rate : 0.001
checkpoint : ./checkpoint_20250609_123600_7.bin
max_words_cnt : 256
token_ids_size : 256
Allocating memory
for tensors : 36355416 bytes,
for c_tensors: 17206900 bytes
for grad_tensors: 14209596 bytes
loading from checkpoint : ./checkpoint_20250609_123600_7.bin
loaded from checkpoint
serving mode
test file : ./test_lm.txt
sentence : the time machine
by h g wells i the time traveller for so it will be convenient to speak of him was expounding a recondite matter to us his grey eyes shone and twinkled and his usually pale face was flushed and animated the fire burned brightly and animated the fire burned brightly

sentence : the fire burned brightly
and animated the time traveller for so it will be convenient to speak of him was expounding a recondite matter to us his usually pale face was flushed and animated the fire burned brightly and animated the fire burned brightly and animated the fire burned brightly and animated the fire`

dratman · 2025-06-09T06:12:42Z

Thanks very much again. I will try that checkpoint after some sleep! I assume you must occasionally sleep too.

dratman · 2025-06-10T04:23:56Z

There is some problem with the link for https://cpp-transformer-1252366230.cos.ap-beijing.myqcloud.com/lm/checkpoint_20250609_123600_7.bin i tried several different ways but cannot download the file. Could you try attaching that to an email reply? Thank you. Ralph

…

On Sun, Jun 8, 2025 at 8:19 AM freelw ***@***.***> wrote: *freelw* left a comment (freelw/cpp-transformer#2) <#2 (comment)> GPT-2 would be great. I can make a financial contribution if that would be helpful. Meanwhile I transferred your encoder-only project in the XCode IDE on MacOS, building in that environment and soon I think it will be fully running in XCode. (Previously I had just used the MacOS terminal to get the encoder-only project working, creating the same output as yours. XCode is just an alternate way to produce the same results.) Thanks! Ralph … <#m_-5458829483201424775_> On Fri, Jun 6, 2025 at 11:17 PM freelw *@*.*> wrote: freelw left a comment (freelw/cpp-transformer#2 <#2>) <#2 (comment) <#2 (comment)>> The makefile_osx works for MacOs so that is fine. I was wondering if you have a decoder.cpp which I could use in a language model? Thank you. Ralph … <#m_1281260957338099353_> On Tue, May 27, 2025 at 6:03 AM freelw @.> wrote: freelw left a comment (freelw/cpp-transformer#2 <#2> <#2 <#2>>) <#2 <#2> (comment) <#2 (comment) <#2 (comment)>>> I found new makefile does not work on x64 linux. I revert it and add a new makefile_osx. #3 <#3> <#3 <#3>> Can you test whether makefile_osx works on your MacBook? By this cmd : make -f makefile_osx > 2025年5月27日 16:02，Ralph Dratman @.> 写道： > > Dratman — Reply to this email directly, view it on GitHub <#2 <#2> (comment) <#2 (comment) <#2 (comment)>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAWZPUD4YRBZ4F63OLKJTT3AQZ6TAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSMJRHEZTMMRRGE <https://github.com/notifications/unsubscribe-auth/AAAWZPUD4YRBZ4F63OLKJTT3AQZ6TAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSMJRHEZTMMRRGE> . You are receiving this because you modified the open/close state.Message ID: @.**> I am currently learning GPT-2 and will implement a version of it. By then, there might be the decoder.cpp you want. — Reply to this email directly, view it on GitHub <#2 (comment) <#2 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAWZPXAFU55QH57Z4PLBQL3CJKTRAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNJRGU3TAOJZGA <https://github.com/notifications/unsubscribe-auth/AAAWZPXAFU55QH57Z4PLBQL3CJKTRAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNJRGU3TAOJZGA> . You are receiving this because you modified the open/close state.Message ID: @.**> you can use this script ./build_mac_cpu.sh to build on your macbook. — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAWZPQLENQJZDZ7UBPXYBL3CQS5BAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNJUGAZDANZRGU> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

freelw · 2025-06-10T14:58:35Z

There is some problem with the link for https://cpp-transformer-1252366230.cos.ap-beijing.myqcloud.com/lm/checkpoint_20250609_123600_7.bin i tried several different ways but cannot download the file. Could you try attaching that to an email reply? Thank you. Ralph
…
On Sun, Jun 8, 2025 at 8:19 AM freelw @.> wrote: freelw left a comment (freelw/cpp-transformer#2) <#2 (comment)> GPT-2 would be great. I can make a financial contribution if that would be helpful. Meanwhile I transferred your encoder-only project in the XCode IDE on MacOS, building in that environment and soon I think it will be fully running in XCode. (Previously I had just used the MacOS terminal to get the encoder-only project working, creating the same output as yours. XCode is just an alternate way to produce the same results.) Thanks! Ralph … <#m_-5458829483201424775_> On Fri, Jun 6, 2025 at 11:17 PM freelw @.> wrote: freelw left a comment (freelw/cpp-transformer#2 <#2>) <#2 (comment) <#2 (comment)>> The makefile_osx works for MacOs so that is fine. I was wondering if you have a decoder.cpp which I could use in a language model? Thank you. Ralph … <#m_1281260957338099353_> On Tue, May 27, 2025 at 6:03 AM freelw @.> wrote: freelw left a comment (freelw/cpp-transformer#2 <#2> <#2 <#2>>) <#2 <#2> (comment) <#2 (comment) <#2 (comment)>>> I found new makefile does not work on x64 linux. I revert it and add a new makefile_osx. #3 <#3> <#3 <#3>> Can you test whether makefile_osx works on your MacBook? By this cmd : make -f makefile_osx > 2025年5月27日 16:02，Ralph Dratman @.> 写道： > > Dratman — Reply to this email directly, view it on GitHub <#2 <#2> (comment) <#2 (comment) <#2 (comment)>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAWZPUD4YRBZ4F63OLKJTT3AQZ6TAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSMJRHEZTMMRRGE https://github.com/notifications/unsubscribe-auth/AAAWZPUD4YRBZ4F63OLKJTT3AQZ6TAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSMJRHEZTMMRRGE . You are receiving this because you modified the open/close state.Message ID: @.> I am currently learning GPT-2 and will implement a version of it. By then, there might be the decoder.cpp you want. — Reply to this email directly, view it on GitHub <#2 (comment) <#2 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAWZPXAFU55QH57Z4PLBQL3CJKTRAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNJRGU3TAOJZGA https://github.com/notifications/unsubscribe-auth/AAAWZPXAFU55QH57Z4PLBQL3CJKTRAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNJRGU3TAOJZGA . You are receiving this because you modified the open/close state.Message ID: @.> you can use this script ./build_mac_cpu.sh to build on your macbook. — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAWZPQLENQJZDZ7UBPXYBL3CQS5BAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNJUGAZDANZRGU . You are receiving this because you modified the open/close state.Message ID: @.*>

us link
Can you download this checkpoint? I have reuploaded it to the COS in North America. Try again？

dratman · 2025-06-10T20:02:03Z

Yes -- that download worked perfectly, and generated text.
I have been trying to find a C++ language model for months!
Thank you so much.
非常感谢
With my very best regards,
Ralph

dratman · 2025-06-10T20:08:46Z

I will try training the lm on some other text.
I have been training it on the same time machine text and it went like this:
/lm
corpus : ./resources/time_machine/timemachine_preprocessed.txt
epochs : 10
batch_size : 16
gpu : 1
learning rate : 0.001
checkpoint :
max_words_cnt : 256
Warning: GPU backend is not available in CPU build. Now use cpu instead!!!
Allocating memory
for tensors : 36609236 bytes,
for c_tensors: 3194706328 bytes
for grad_tensors: 1241779004 bytes
epoch 0 : [224/256]loss : 5.62508
epoch 1 : [224/256]loss : 1.47182
epoch 2 : [224/256]loss : 0.2021
epoch 3 : [224/256]loss : 0.0490456
epoch 4 : [224/256]loss : 0.0254087
Terminated: 15

(I sent it a Quit signal from the Mac Task Manager, but it did not output a checkpoint on exit)

freelw · 2025-06-11T00:37:44Z

I will try training the lm on some other text. I have been training it on the same time machine text and it went like this: /lm corpus : ./resources/time_machine/timemachine_preprocessed.txt epochs : 10 batch_size : 16 gpu : 1 learning rate : 0.001 checkpoint : max_words_cnt : 256 Warning: GPU backend is not available in CPU build. Now use cpu instead!!! Allocating memory for tensors : 36609236 bytes, for c_tensors: 3194706328 bytes for grad_tensors: 1241779004 bytes epoch 0 : [224/256]loss : 5.62508 epoch 1 : [224/256]loss : 1.47182 epoch 2 : [224/256]loss : 0.2021 epoch 3 : [224/256]loss : 0.0490456 epoch 4 : [224/256]loss : 0.0254087 Terminated: 15

(I sent it a Quit signal from the Mac Task Manager, but it did not output a checkpoint on exit)

let me test on my macbook

freelw · 2025-06-11T00:44:48Z

I will try training the lm on some other text. I have been training it on the same time machine text and it went like this: /lm corpus : ./resources/time_machine/timemachine_preprocessed.txt epochs : 10 batch_size : 16 gpu : 1 learning rate : 0.001 checkpoint : max_words_cnt : 256 Warning: GPU backend is not available in CPU build. Now use cpu instead!!! Allocating memory for tensors : 36609236 bytes, for c_tensors: 3194706328 bytes for grad_tensors: 1241779004 bytes epoch 0 : [224/256]loss : 5.62508 epoch 1 : [224/256]loss : 1.47182 epoch 2 : [224/256]loss : 0.2021 epoch 3 : [224/256]loss : 0.0490456 epoch 4 : [224/256]loss : 0.0254087 Terminated: 15

(I sent it a Quit signal from the Mac Task Manager, but it did not output a checkpoint on exit)

I have found the cause of the problem. My program can only handle the Ctrl+C signal, not the quit signal. Moreover, after pressing Ctrl+C, it is necessary to wait until the batch training is completed before exiting normally. Do not force an early exit, as this is the only way to generate the checkpoint.

dratman · 2025-06-11T06:52:18Z

Ah, that is helpful. I suggest you change the logic of the program so that it writes out a checkpoint after it completes each epoch. Then it would not be necessary to stop the program to get the checkpoint.

freelw · 2025-06-11T07:38:10Z

Ah, that is helpful. I suggest you change the logic of the program so that it writes out a checkpoint after it completes each epoch. Then it would not be necessary to stop the program to get the checkpoint.

In previous historical versions, it was indeed the case as you said, where a checkpoint was saved after each epoch. However, since each file was approximately 140MB, it filled up my hard drive, so I later changed it to the current setup. If you really need this feature, you can make minor modifications to lines 353 and 354 of lm.cpp, moving them up into the loop above. I don't think it should be too complicated.

dratman · 2025-06-12T08:41:55Z

Recent output illustrating a few minor changes:
--->./lm
corpus : ./resources/time_machine/timemachine_preprocessed.txt
epochs : 5
batch_size : 16
gpu : 1
learning rate : 0.001
checkpoint to load from :
max_words_cnt : 256
token_ids_size : 256
Warning: GPU backend is not available in CPU build. Now use cpu instead!!!
Allocating memory
for tensors : 36609236 bytes,
for c_tensors: 3194706328 bytes
for grad_tensors: 1241779004 bytes
epoch 0 : [224/256]loss : 5.59927
checkpoint saved : ./checkpoints/checkpoint_20250611_210257_-1.bin
epoch 1 : [224/256]loss : 1.48668
checkpoint saved : ./checkpoints/checkpoint_20250612_041930_0.bin
. . . (still running)

freelw · 2025-06-12T13:07:25Z

Should the checkpoint file be correct at this point? Although there are bugs, it should be usable. Can you submit a PR? I'd like to review the code you modified.

Additionally, I'm currently supporting Metal and it's estimated that the capability to invoke the GPU on a Mac will be available next week. Then training on a Mac should be faster. You can give it a try then. You can take a look at this branch. I'm currently implementing it.

Recent output illustrating a few minor changes: --->./lm corpus : ./resources/time_machine/timemachine_preprocessed.txt epochs : 5 batch_size : 16 gpu : 1 learning rate : 0.001 checkpoint to load from : max_words_cnt : 256 token_ids_size : 256 Warning: GPU backend is not available in CPU build. Now use cpu instead!!! Allocating memory for tensors : 36609236 bytes, for c_tensors: 3194706328 bytes for grad_tensors: 1241779004 bytes epoch 0 : [224/256]loss : 5.59927 checkpoint saved : ./checkpoints/checkpoint_20250611_210257_-1.bin epoch 1 : [224/256]loss : 1.48668 checkpoint saved : ./checkpoints/checkpoint_20250612_041930_0.bin . . . (still running)

dratman · 2025-06-12T16:22:51Z

Metal would be fantastic. I thought that would be very difficult and now I am glad to know you do not think so! I will send a PR, but not today because I am not sure of my changes yet.

…

On Thu, Jun 12, 2025 at 9:07 AM freelw ***@***.***> wrote: *freelw* left a comment (freelw/cpp-transformer#2) <#2 (comment)> Should the checkpoint file be correct at this point? Although there are bugs, it should be usable. Can you submit a PR? I'd like to review the code you modified. Additionally, I'm currently supporting Metal and it's estimated that the capability to invoke the GPU on a Mac will be available next week. Then training on a Mac should be faster. You can give it a try then. You can take a look at this branch. <https://github.com/freelw/cpp-transformer/tree/wangli_dev_20250612> I'm currently implementing it. Recent output illustrating a few minor changes: --->./lm corpus : ./resources/time_machine/timemachine_preprocessed.txt epochs : 5 batch_size : 16 gpu : 1 learning rate : 0.001 checkpoint to load from : max_words_cnt : 256 token_ids_size : 256 Warning: GPU backend is not available in CPU build. Now use cpu instead!!! Allocating memory for tensors : 36609236 bytes, for c_tensors: 3194706328 bytes for grad_tensors: 1241779004 bytes epoch 0 : [224/256]loss : 5.59927 checkpoint saved : ./checkpoints/checkpoint_20250611_210257_-1.bin epoch 1 : [224/256]loss : 1.48668 checkpoint saved : ./checkpoints/checkpoint_20250612_041930_0.bin . . . (still running) — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAWZPVLRB2KL6T355I2A6L3DF3SFAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNRWGY2DKOBSGE> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

dratman · 2025-06-13T17:59:59Z

The language model is supposed to complete the "prompt" input, which here is called the "sentence" but I observe no change in the output in response to a new prompt.

freelw · 2025-06-14T00:30:39Z

The result on my Ubuntu computer is as follows: there are two different sentences in ./test_lm.txt, and the output inference results are different.
`
$ cat test_lm.txt
the time machine
the fire burned brightly
$ ./lm -c ./checkpoint_20250609_123600_7.bin -e 0
corpus : ./resources/time_machine/timemachine_preprocessed.txt
epochs : 0
batch_size : 16
gpu : 1
learning rate : 0.001
checkpoint : ./checkpoint_20250609_123600_7.bin
max_words_cnt : 256
token_ids_size : 256
Allocating memory
for tensors : 36355416 bytes,
for c_tensors: 17206900 bytes
for grad_tensors: 14209596 bytes
loading from checkpoint : ./checkpoint_20250609_123600_7.bin
loaded from checkpoint
serving mode
test file : ./test_lm.txt
sentence : the time machine
by h g wells i the time traveller for so it will be convenient to speak of him was expounding a recondite matter to us his grey eyes shone and twinkled and his usually pale face was flushed and animated the fire burned brightly and animated the fire burned brightly

sentence : the fire burned brightly
and animated the time traveller for so it will be convenient to speak of him was expounding a recondite matter to us his usually pale face was flushed and animated the fire burned brightly and animated the fire burned brightly and animated the fire burned brightly and animated the fire
-----------------`
Could you provide me with more information to reproduce your phenomenon? Or you can also send me your checkpoint file.

The language model is supposed to complete the "prompt" input, which here is called the "sentence" but I observe no change in the output in response to a new prompt.

freelw · 2025-06-14T01:11:23Z

The language model is supposed to complete the "prompt" input, which here is called the "sentence" but I observe no change in the output in response to a new prompt.

I forgot to remind you of one thing. If you want to use other texts for training, you need to do some preprocessing and update the vocabulary, which means updating two files: 1 timemachine_preprocessed.txt, 2 vocab/time_machine/vocab.txt.

The methods for preprocessing and constructing the vocabulary can be referred to here and here.

freelw · 2025-06-14T12:45:56Z

Metal would be fantastic. I thought that would be very difficult and now I am glad to know you do not think so! I will send a PR, but not today because I am not sure of my changes yet.
…
On Thu, Jun 12, 2025 at 9:07 AM freelw @.> wrote: freelw left a comment (freelw/cpp-transformer#2) <#2 (comment)> Should the checkpoint file be correct at this point? Although there are bugs, it should be usable. Can you submit a PR? I'd like to review the code you modified. Additionally, I'm currently supporting Metal and it's estimated that the capability to invoke the GPU on a Mac will be available next week. Then training on a Mac should be faster. You can give it a try then. You can take a look at this branch. https://github.com/freelw/cpp-transformer/tree/wangli_dev_20250612 I'm currently implementing it. Recent output illustrating a few minor changes: --->./lm corpus : ./resources/time_machine/timemachine_preprocessed.txt epochs : 5 batch_size : 16 gpu : 1 learning rate : 0.001 checkpoint to load from : max_words_cnt : 256 token_ids_size : 256 Warning: GPU backend is not available in CPU build. Now use cpu instead!!! Allocating memory for tensors : 36609236 bytes, for c_tensors: 3194706328 bytes for grad_tensors: 1241779004 bytes epoch 0 : [224/256]loss : 5.59927 checkpoint saved : ./checkpoints/checkpoint_20250611_210257_-1.bin epoch 1 : [224/256]loss : 1.48668 checkpoint saved : ./checkpoints/checkpoint_20250612_041930_0.bin . . . (still running) — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAWZPVLRB2KL6T355I2A6L3DF3SFAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNRWGY2DKOBSGE . You are receiving this because you modified the open/close state.Message ID: @.>

#28 Cheer up! Metal is now supported!

freelw · 2025-06-15T07:47:52Z

time ./lm -m 10000000 corpus : ./resources/time_machine/timemachine_preprocessed.txt epochs : 10 batch_size : 16 gpu : 1 learning rate : 0.001 checkpoint : max_words_cnt : 10000000 token_ids_size : 32775 Allocating memory for tensors : 36609236 bytes, for c_tensors: 3194706336 bytes for grad_tensors: 1241779004 bytes epoch 0 : [32752/32775]loss : 8.3573 epoch 1 : [144/32775]loss : 8.35684 checkpoint saved : ./checkpoints/checkpoint_20250615_154537_2.bin ./lm -m 10000000 1157.25s user 2434.96s system 24% cpu 4:02:08.01 total

It takes about 4 hours to train one epoch on my MacBook Air M1.Using the complete data of the novel Time Machine, there are a total of 32,775 tokens.

dratman · 2025-06-16T02:53:27Z

The speed is great

…

On Sun, Jun 15, 2025 at 3:48 AM freelw ***@***.***> wrote: *freelw* left a comment (freelw/cpp-transformer#2) <#2 (comment)> time ./lm -m 10000000 corpus : ./resources/time_machine/timemachine_preprocessed.txt epochs : 10 batch_size : 16 gpu : 1 learning rate : 0.001 checkpoint : max_words_cnt : 10000000 token_ids_size : 32775 Allocating memory for tensors : 36609236 bytes, for c_tensors: 3194706336 bytes for grad_tensors: 1241779004 bytes epoch 0 : [32752/32775]loss : 8.3573 epoch 1 : [144/32775]loss : 8.35684 checkpoint saved : ./checkpoints/checkpoint_20250615_154537_2.bin ./lm -m 10000000 1157.25s user 2434.96s system 24% cpu 4:02:08.01 total It takes about 4 hours to train one epoch on my MacBook Air M1. — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAWZPX6N3TBM32CSX67PLD3DUQL5AVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNZTGU3DONRUGA> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

dratman · 2025-06-16T04:14:06Z

But I must be doing something wrong because the loss is not going down much:

time ./lm
corpus : ./resources/time_machine/timemachine_preprocessed.txt
epochs : 10
batch_size : 16
gpu : 1
learning rate : 0.001
checkpoint :
max_words_cnt : 256
token_ids_size : 256
Allocating memory
for tensors : 36609236 bytes,
for c_tensors: 3194706336 bytes
for grad_tensors: 1241779004 bytes
epoch 0 : [224/256]loss : 8.46906
epoch 1 : [224/256]loss : 8.46934
epoch 2 : [224/256]loss : 8.47008
epoch 3 : [224/256]loss : 8.46854
epoch 4 : [224/256]loss : 8.46883
epoch 5 : [224/256]loss : 8.46955
epoch 6 : [224/256]loss : 8.46921
epoch 7 : [224/256]loss : 8.46919
epoch 8 : [224/256]loss : 8.46892
epoch 9 : [224/256]loss : 8.46953
checkpoint saved : ./checkpoints/checkpoint_20250615_235223_9.bin

dratman · 2025-06-16T04:17:55Z

real 2m12.590s
user 0m45.625s
sys 0m13.453s

freelw · 2025-06-16T05:53:12Z

But I must be doing something wrong because the loss is not going down much:

time ./lm
corpus : ./resources/time_machine/timemachine_preprocessed.txt
epochs : 10
batch_size : 16
gpu : 1
learning rate : 0.001
checkpoint :
max_words_cnt : 256
token_ids_size : 256
Allocating memory
for tensors : 36609236 bytes,
for c_tensors: 3194706336 bytes
for grad_tensors: 1241779004 bytes
epoch 0 : [224/256]loss : 8.46906
epoch 1 : [224/256]loss : 8.46934
epoch 2 : [224/256]loss : 8.47008
epoch 3 : [224/256]loss : 8.46854
epoch 4 : [224/256]loss : 8.46883
epoch 5 : [224/256]loss : 8.46955
epoch 6 : [224/256]loss : 8.46921
epoch 7 : [224/256]loss : 8.46919
epoch 8 : [224/256]loss : 8.46892
epoch 9 : [224/256]loss : 8.46953
checkpoint saved : ./checkpoints/checkpoint_20250615_235223_9.bin

Have you made any changes to the code? If so, could you send me a pull request (PR) so that I can review it?

freelw · 2025-06-16T05:55:41Z

Are you using the latest code on the main branch?
git rev-parse HEAD
43f4e87

freelw · 2025-06-16T06:00:45Z

But I must be doing something wrong because the loss is not going down much:

time ./lm
corpus : ./resources/time_machine/timemachine_preprocessed.txt
epochs : 10
batch_size : 16
gpu : 1
learning rate : 0.001
checkpoint :
max_words_cnt : 256
token_ids_size : 256
Allocating memory
for tensors : 36609236 bytes,
for c_tensors: 3194706336 bytes
for grad_tensors: 1241779004 bytes
epoch 0 : [224/256]loss : 8.46906
epoch 1 : [224/256]loss : 8.46934
epoch 2 : [224/256]loss : 8.47008
epoch 3 : [224/256]loss : 8.46854
epoch 4 : [224/256]loss : 8.46883
epoch 5 : [224/256]loss : 8.46955
epoch 6 : [224/256]loss : 8.46921
epoch 7 : [224/256]loss : 8.46919
epoch 8 : [224/256]loss : 8.46892
epoch 9 : [224/256]loss : 8.46953
checkpoint saved : ./checkpoints/checkpoint_20250615_235223_9.bin

you are right. there must be a bug. let me check

freelw · 2025-06-16T14:57:44Z

But I must be doing something wrong because the loss is not going down much:

time ./lm
corpus : ./resources/time_machine/timemachine_preprocessed.txt
epochs : 10
batch_size : 16
gpu : 1
learning rate : 0.001
checkpoint :
max_words_cnt : 256
token_ids_size : 256
Allocating memory
for tensors : 36609236 bytes,
for c_tensors: 3194706336 bytes
for grad_tensors: 1241779004 bytes
epoch 0 : [224/256]loss : 8.46906
epoch 1 : [224/256]loss : 8.46934
epoch 2 : [224/256]loss : 8.47008
epoch 3 : [224/256]loss : 8.46854
epoch 4 : [224/256]loss : 8.46883
epoch 5 : [224/256]loss : 8.46955
epoch 6 : [224/256]loss : 8.46921
epoch 7 : [224/256]loss : 8.46919
epoch 8 : [224/256]loss : 8.46892
epoch 9 : [224/256]loss : 8.46953
checkpoint saved : ./checkpoints/checkpoint_20250615_235223_9.bin

I have now identified a phenomenon: when the allocated memory is too large, triggering swapout, the calculation results become incorrect. However, this issue does not occur in the CUDA environment because CUDA does not use unified memory and thus avoids swap-in/swap-out operations. The temporary solution I adopted was adjusting the default batchsize in lm.cpp, which has prevented swapout on my MacBook Air for now. This problem might stem from my insufficient understanding of the Metal API, but so far I haven’t found any standardized handling procedures for unified memory scenarios in the documentation. I will create an issue to track this problem going forward.

freelw · 2025-06-16T15:01:22Z

But I must be doing something wrong because the loss is not going down much:

time ./lm
corpus : ./resources/time_machine/timemachine_preprocessed.txt
epochs : 10
batch_size : 16
gpu : 1
learning rate : 0.001
checkpoint :
max_words_cnt : 256
token_ids_size : 256
Allocating memory
for tensors : 36609236 bytes,
for c_tensors: 3194706336 bytes
for grad_tensors: 1241779004 bytes
epoch 0 : [224/256]loss : 8.46906
epoch 1 : [224/256]loss : 8.46934
epoch 2 : [224/256]loss : 8.47008
epoch 3 : [224/256]loss : 8.46854
epoch 4 : [224/256]loss : 8.46883
epoch 5 : [224/256]loss : 8.46955
epoch 6 : [224/256]loss : 8.46921
epoch 7 : [224/256]loss : 8.46919
epoch 8 : [224/256]loss : 8.46892
epoch 9 : [224/256]loss : 8.46953
checkpoint saved : ./checkpoints/checkpoint_20250615_235223_9.bin

You can try the latest mainline version again. The loss of the language model (LM) should converge normally now.
I will conduct an experiment tomorrow to carefully compare the calculation results under different environments in detail. By then, there should be more clues to uncover.

dratman · 2025-06-16T23:19:40Z

Yes! The loss goes down very fast! I continue testing in detail. Thanks for this! Ralph

…

On Mon, Jun 16, 2025 at 11:01 AM freelw ***@***.***> wrote: *freelw* left a comment (freelw/cpp-transformer#2) <#2 (comment)> But I must be doing something wrong because the loss is not going down much: time ./lm corpus : ./resources/time_machine/timemachine_preprocessed.txt epochs : 10 batch_size : 16 gpu : 1 learning rate : 0.001 checkpoint : max_words_cnt : 256 token_ids_size : 256 Allocating memory for tensors : 36609236 bytes, for c_tensors: 3194706336 bytes for grad_tensors: 1241779004 bytes epoch 0 : [224/256]loss : 8.46906 epoch 1 : [224/256]loss : 8.46934 epoch 2 : [224/256]loss : 8.47008 epoch 3 : [224/256]loss : 8.46854 epoch 4 : [224/256]loss : 8.46883 epoch 5 : [224/256]loss : 8.46955 epoch 6 : [224/256]loss : 8.46921 epoch 7 : [224/256]loss : 8.46919 epoch 8 : [224/256]loss : 8.46892 epoch 9 : [224/256]loss : 8.46953 checkpoint saved : ./checkpoints/checkpoint_20250615_235223_9.bin You can try the latest mainline version again. The loss of the language model (LM) should converge normally now. I will conduct an experiment tomorrow to carefully compare the calculation results under different environments in detail. By then, there should be more clues to uncover. — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAWZPUQKT6OCIRD34R62MT3D3L5PAVCNFSM6AAAAAB57IZHM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSNZXGAYDKMJXGU> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

Update Makefile for macOS Clang + OpenMP + C++11

8c57a82

dratman closed this May 27, 2025

dratman reopened this May 27, 2025

freelw merged commit d1bb7c7 into freelw:main May 27, 2025

freelw mentioned this pull request Jun 16, 2025

lm heap-buffer-overflow #33

Closed

Conversation

dratman commented May 27, 2025

Uh oh!

dratman commented May 27, 2025

Uh oh!

dratman commented May 27, 2025

Uh oh!

freelw commented May 27, 2025

Uh oh!

freelw commented May 27, 2025 via email

Uh oh!

dratman commented Jun 5, 2025 via email

Uh oh!

dratman commented Jun 5, 2025 via email

Uh oh!

freelw commented Jun 7, 2025

Uh oh!

dratman commented Jun 7, 2025 via email

Uh oh!

freelw commented Jun 8, 2025

Uh oh!

freelw commented Jun 8, 2025

Uh oh!

dratman commented Jun 9, 2025

Uh oh!

freelw commented Jun 9, 2025

Uh oh!

freelw commented Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dratman commented Jun 9, 2025

Uh oh!

dratman commented Jun 10, 2025 via email

Uh oh!

freelw commented Jun 10, 2025

Uh oh!

dratman commented Jun 10, 2025

Uh oh!

dratman commented Jun 10, 2025

Uh oh!

freelw commented Jun 11, 2025

Uh oh!

freelw commented Jun 11, 2025

Uh oh!

dratman commented Jun 11, 2025

Uh oh!

freelw commented Jun 11, 2025

Uh oh!

dratman commented Jun 12, 2025

Uh oh!

freelw commented Jun 12, 2025

Uh oh!

dratman commented Jun 12, 2025 via email

Uh oh!

dratman commented Jun 13, 2025

Uh oh!

freelw commented Jun 14, 2025

Uh oh!

freelw commented Jun 14, 2025

Uh oh!

freelw commented Jun 14, 2025

Uh oh!

freelw commented Jun 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dratman commented Jun 16, 2025 via email

Uh oh!

dratman commented Jun 16, 2025

Uh oh!

dratman commented Jun 16, 2025

Uh oh!

freelw commented Jun 16, 2025

Uh oh!

freelw commented Jun 16, 2025

Uh oh!

freelw commented Jun 16, 2025

Uh oh!

freelw commented Jun 16, 2025

Uh oh!

freelw commented Jun 16, 2025

freelw commented Jun 9, 2025 •

edited

Loading

freelw commented Jun 15, 2025 •

edited

Loading