Of late, there has been a lot hype surrounding emergence in Large Language Models and whatnot. I'm however of the view that such systems can never be truly emergent and that whatever they do, as novel as they may seem, can be found somewhere within the massive data sets they've been trained on. Can you convince me otherwise?
While it is true that Large Language Models (LLMs) struggle on tasks that are not similar to any seen in the dataset they were trained on, it's not accurate to consider all data that LLMs generate as existing somewhere inside their datasets. In fact, almost all of the sequences they produce are some amalgamation of loads of data that infer some sort of understanding of the concepts beneath. The idea that this equates to emergent intelligence, however, needs much more study, and the difficulty of answering that question also lies in our inability to define human intelligence or consciousness.
Offer me some experiments I could do to further research this subject
Sure! Here are some suggestions for experiments related to language models:

1. Fine-tuning on a specific task: Take a pre-trained language model and fine-tune it on a specific task such as sentiment analysis or named entity recognition. Evaluate its performance on a held-out test set and compare it to other models.

2. Ablation study: Remove various components of a language model and observe the effect on its performance. For example, you can remove the attention mechanism and observe how it affects the output.

3. Generating text in different styles: Use a pre-trained language model to generate text in different styles such as formal, informal, or persuasive. Evaluate the quality of the generated text and observe if the model is able to capture the desired style.

4. Exploring robustness: Evaluate the robustness of a language model by subjecting it to various types of adversarial inputs and observe its behavior. For example, you can add noise to the input or provide a misleading context to see how it affects the output.

5. Evaluating diversity: Evaluate the diversity of generated text from a language model. For example, you can generate multiple outputs for the same input prompt and observe if the outputs are diverse or if they repeat the same pattern.

I hope this gives you some ideas! These experiments can help you understand how large language models (LLMs) work and what factors affect their performance.