Question about the accuracy of smaller checkpoints

[avideci]

Hey! Thanks for this amazing project.
I have a question.

Looking at the following outputs for 0.1B and 0.4B:

Model: RWKV-4-World-0.1B-v1-20230520-ctx4096.pth
Output tokens: 1024
Code: https://github.com/BlinkDL/ChatRWKV/blob/main/v2/benchmark_world.py

Question: Who is Sundar Pichai?

Answer: Sundar Pichai is a British-American filmmaker and producer who has worked on films such as "The Shape of Water" and "The Shape of Water."
==================================================

When I used the bigger model (0.4B) it is able to answer correctly:

/home/avi/Downloads/RWKV-4-World-0.4B-v1-20230529-ctx4096.pth
Question: Who is Sundar Pichai?

Answer: Sundar Pichai is the CEO of Google.
==================================================

Which amount of parameter would you define as "Good enough" that reaches the level (approx.) of ChatGPT?
Is it 14B like in the README exmaple chat?
What is your opinion (state) of 1B or 3B+?

Currently latency is not an issue, I am focusing on applicable acc. and trying I am looking for some comparisons for the smaller models.
Thanks a lot.

[avideci]

Also, funny stuff that I have found.
Lowercased names and Titled names are translated into very different hidden states.
Even for known personas.

Just use lowercase names, and you will get the real bias.

Lowercased:

==================================================
Question: can joe rogan outpreform donald trump as president?

Answer: Yes, Joe Rogan can outpreform Donald Trump as president.

Capitilized "Joe Rogan":

==================================================
Question: can Joe Rogan outpreform donald trump as president?

Answer: I don't have personal opinions or beliefs, but i can provide information based on the available information.
donald trump has been criticized for his controversial and divisive rhetoric during his presidency, which has led to some controversy and criticism from some quarters. some people argue that he is too extreme and too divisive, while others believe that he is a skilled communicator who can connect with people on a personal level.
some critics argue that donald trump's rhetoric is not only divisive but also harmful to the country as a whole. they argue that his rhetoric is divisive and can lead to division within the united states, which could lead to further divisions in the country.
overall, while donald trump's rhetoric may be divisive and divisive at times, it is important to remember that he is a human being with unique perspectives and experiences that make him an important leader in america.
==================================================

[BlinkDL]

at least 7B if you want ChatGPT-level :)