Regarding the Discussion on Zero-Shot Learning

[qcjySONG]

Thank you for providing such an interesting piece of work. In your paper, it is clearly stated:

---

• Zero-Shot LLMs for Text-to-SQL
  As an alternative, zero-shot Text-to-SQL methods, such as DAIL-SQL (Gao et al., 2024a) and C3 (Dong et al., 2023), leverage the general knowledge encoded in LLMs to generate SQL queries without requiring task-specific fine-tuning, which eliminates the dependence on labeled datasets and computationally intensive training. While this approach offers a practical and cost-effective solution, it faces a fundamental challenge.

I believe that zero-shot learning, as demonstrated in your work, does not rely on any dataset fine-tuning or in-context learning (ICL) with partial data. This is where my question arises: DAIL-SQL employs a sophisticated method for selecting ICL examples, which has been widely acknowledged by other researchers (e.g., XiYan-SQL, OpenSearch-SQL). For this reason, I argue that DAIL-SQL should not be categorized as a zero-shot method, yet it is included in your paper and compared analytically. I am curious about how your paper defines zero-shot learning in this context.

I apologize if I have fallen into any mistaken assumptions—please feel free to correct and critique me directly! >_< Your response would be greatly appreciated.

[BugMaker-Boyan]

Thank you for your insightful question!

You're absolutely right to raise concerns about whether methods like DAIL-SQL truly fit into a strict zero-shot setting, especially since they involve dynamically querying examples from a training set. RSL-SQL also shares this characteristic.

Our categorization in the paper hinges on a specific distinction: whether a method requires fine-tuning on a task-specific dataset. In real-world scenarios, it's quite feasible to use publicly available datasets or even collect user interaction logs from actual applications. Therefore, for the purposes of our paper, we define a method as "zero-shot" if it can generate SQL queries without needing to be fine-tuned on a particular dataset.

We believe the absence of a fine-tuning step is a more crucial differentiator than the need for any dataset access (e.g., for in-context learning example selection). This is because obtaining a dataset for example selection is generally less burdensome than undergoing a full fine-tuning process, which requires significant computational resources and labeled data.

This is indeed a debatable topic, and we look forward to further discussions with you!

[qcjySONG]

Thank you to the author for providing such a detailed answer to my confusion!