Just a curious observation: Seems Llama 3 Instruct 8b (and maybe LLMs in general) aren't fully aware of contents outside the messages

[TiagoTiago]

After I figured out I needed a different tokenizer to let Llama 3 8b gguf not glitch in the presence of a system prompt; I've been spending a few hours trying to get it to quote the exact beginning of the conversation; I managed to get it to volunteer the contents of the system prompt easily by having the system prompt instruct it to, but it has a hard time showing those instructions came before my initial message, always putting it after in the citation attempts; it did not understand that it was the system prompt and not just some floating "special text" disconnected from anything, and barely acknowledge the existence of special tokens; I even tried a "fill in the blanks" approach, including a large chunk of the beginning of the context, which included the system prompt, and just replacing a small section with underscores, and it still didn't get anywhere near actually quoting the log correctly.....