Skip to content

Commit

Permalink
Fix missing documentation in LLM runtime, and a broken link + some ty…
Browse files Browse the repository at this point in the history
…pos (#1128)

* Fix broken link

* Fix script typos
  • Loading branch information
aahouzi committed Jan 11, 2024
1 parent fdefb9d commit f780a28
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 4 deletions.
3 changes: 2 additions & 1 deletion intel_extension_for_transformers/llm/runtime/graph/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -408,10 +408,11 @@ while True:
outputs = model.generate(inputs, streamer=streamer, interactive=True, ignore_prompt=True, do_sample=True)
```

## How to use: Python script
## How to use: Straightforward Python script
Install from binary
```shell
pip install intel-extension-for-transformers
pip install -r requirements.txt # under graph folder
```

Build from source
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ def main(args_in: Optional[List[str]] = None) -> None:
convert_cmd.extend(["--outfile", Path(work_path, "ne_{}_f32.bin".format(model_type))])
convert_cmd.extend(["--outtype", "f32"])
convert_cmd.append(args.model)
print("convert model ...")
print("Convert model ...")
subprocess.run(convert_cmd)

# 2. quantize
Expand All @@ -185,7 +185,7 @@ def main(args_in: Optional[List[str]] = None) -> None:
if args.use_ggml:
quant_cmd.extend(["--use_ggml"])
quant_cmd.extend(["--build_dir", args.build_dir])
print("quantize model ...")
print("Quantize model ...")
subprocess.run(quant_cmd)

# 3. inference
Expand All @@ -204,7 +204,7 @@ def main(args_in: Optional[List[str]] = None) -> None:
infer_cmd.extend(["--build_dir", args.build_dir])
if args.shift_roped_k:
infer_cmd.extend(["--shift-roped-k"])
print("inferce model ...")
print("Inference model ...")
subprocess.run(infer_cmd)


Expand Down

0 comments on commit f780a28

Please sign in to comment.