Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix "'NoneType' object has no attribute 'cast'" error when model is freed #117

Merged
merged 1 commit into from
Jul 19, 2023

Conversation

mczk77
Copy link
Contributor

@mczk77 mczk77 commented Jul 17, 2023

This patch fixes the error by storing the null pointer in the constructor, which is used to set ctx.ptr when the model is freed.

@saharNooby
Copy link
Collaborator

I don't really understand how this works. Is it possible to provide a minimal snippet that reproduces the exception and is fixed by this PR?

@mczk77
Copy link
Contributor Author

mczk77 commented Jul 18, 2023

This error happens on all of the examples in my environment. The simplest way I'm able to reproduce it is by modifying the generate_completions.py script to have it read one token and generate one token. The error it produces is:

Loading 20B tokenizer
System info: AVX=1 AVX2=1 AVX512=0 FMA=1 NEON=0 ARM_FMA=0 F16C=1 FP16_VA=0 WASM_SIMD=0 BLAS=0 SSE3=1 VSX=0
Loading RWKV model
1 tokens in prompt

--- Generation 0 ---

Hello[,]

Took 0.151 sec, 151 ms per token
Exception ignored in: <function RWKVModel.__del__ at 0x7fa783d80ee0>
Traceback (most recent call last):
  File "./rwkv.cpp/rwkv/rwkv_cpp_model.py", line 198, in __del__
  File "./rwkv.cpp/rwkv/rwkv_cpp_model.py", line 193, in free
  File "./rwkv.cpp/rwkv/rwkv_cpp_shared_library.py", line 235, in rwkv_free
AttributeError: 'NoneType' object has no attribute 'cast'

I'm running it on Python 3.10.6.

@saharNooby saharNooby merged commit f685aa4 into RWKV:master Jul 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants