Transform a string into input llama2-specific and llama3-specific input ?

[JamieVC]

I try to transform a string into input llama2-specific and llama3-specific input in the function completion_to_prompt()
Is there a way to pass parameter model_option as a input? or else, I should use a global variable to let the function know which model is using?

def completion_to_prompt(completion):
    #return f"<s>[INST] <<SYS>>\n    \n<</SYS>>\n\n{completion} [/INST]"
    return f"<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\n{completion}<|eot_id|><|start_header_id|>    <|end_header_id|>"

@st.cache_resource
def create_model(model_name):
       llm_model = IpexLLM.from_model_id(
            model_name=model_name,
            tokenizer_name=tokenizer_name,
            context_window=512,
            max_new_tokens=64,
            completion_to_prompt=completion_to_prompt,
            generate_kwargs={"do_sample": True, 'temperature': 0.1},
            #messages_to_prompt=messages_to_prompt,
            device_map="cpu",
        )


with st.sidebar:
    st.markdown("## Configuration")
    model_option = st.selectbox(
    "model name",
    ("meta-llama/Llama-2-7b-chat-hf", "meta-llama/Meta-Llama-3-8B-Instruct"),
    index=None,
    placeholder="Select LLM model...",
    disabled=st.session_state.disabled,
    )
    

     st.session_state.disabled = True
     st.write("You selected:", model_option)

Refer to:
https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3/
https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-2/

[ivy-lv11]

As the completion_to_prompt function only accepts the completion and an optional system_prompt as parameters like completion_to_prompt in llama_index.llms.llama_cpp.llama_utils, you can specify the model type by using a global variable.