Guide for install on windows.

[HCBlackFox]

1. Download Cygwin and Strawberry Perl (Or any other gcc I think, but then change line 16 in make file to your compiler)

https://www.cygwin.com/
https://strawberryperl.com/

and

pip install transformers wget
git clone https://github.com/nolanoOrg/cformers.git

2. Add lines in make file:

13 UNAME_M := "Windows 10"
14 UNAME_P := "x86_64"
15 UNAME_S := "Windows"
16 CC=C:\Strawberry\c\bin\gcc.exe

Also delete line 200.

3. Run CygWin Terminal and go to the cformers/cpp folder

4. Run make comand

5. Change line 127 in interface.py to

command = ["./cpp/main.exe", self.cpp_model_name,

That is all.

Example of script to run OA model:

from interface import AutoInference as AI
ai = AI('OpenAssistant/oasst-sft-1-pythia-12b')
x = ai.generate("<|prompter|>What's the Earth total population<|endoftext|><|assistant|>", num_tokens_to_generate=100)
print(x['token_str'])

Also, I atached zip with compiled files and changeds in interface,py
cformers.zip