Repository navigation
L. Language's Concept of Data
In Noam Chomsky’s framework, data (e.g., utterances, grammaticality judgments) are seen primarily as evidence for an underlying mental system: the language faculty (I-language.
“Linguistic data are not the object of inquiry; they are evidence for the nature of the underlying generative system.”
— Noam Chomsky
So:
- Data ≠ the thing to be modeled
- Data = clues to a hidden structure (Universal Grammar)
This sets Chomsky apart from empiricist models that treat data as central and sufficient for understanding language.
Chomsky famously distinguishes between:
-
The idealized, internal knowledge of language
-
The messy, real-world use of language (which produces data)
Data from performance (e.g., speech corpora) is considered noisy and incomplete, and not always reliable.
| Linguistic data, by themselves, are not enough. |
|---|
Chomsky argues we need theory guided interpretation: you cannot "learn" language just by looking at statistics of usage.
| View | Chomskyan Linguistics | Statistical Language Models |
|---|---|---|
| What is data? | Evidence of internal, rule-based grammar | Raw input for modeling statistical patterns |
| Trust in data | Limited: messy, needs interpretation | High: more = better |
| Goal | Explain mental language competence | Predict/imitate language behavior |
| Theory | Necessary, innate (Universal Grammar) | Emergent from data |
| Famous critique | “Of course you can make a model that mimics behavior. But does it explain anything?” | (Counterpoint) Explanation isn't always needed for practical success |
Legend:
- 🔰 Start here
- 🏵️ Important Insight
- ⚙️ Technical and applications
- 💻 Some Code
- ⚜️ Advanced Topics
- 00.1 What Is Data?
-
00.2 Three Models of Reality (planned)
- 00.2.1 The Linguistic
- 00.2.2 The Symbolic
- 00.3.3 The Mathematical
- 00.3 Maps and Reality
- 00.4 Aside: Hossack, data as structure
- 00.5 Aside: Chomsky, data as evidence
- 01.1 🔰 Sets and Logic (planned)
- 01.2 Sample Space
- 01.3 Randomness and Pattern
- 01.4 Outcomes and Events
- 01.5 Finite and Infinite Sample Spaces
- 02.1 🔰 Functions (planned)
- 02.2 Types Are Alphabets
- 02.3 Random Variables: X : Ω → 𝒳 (planned)
- 02.4 Measurement Scales
- 02.5 Numerics
- 03.1 🔰 Fractions and Proportions (planned)
- 03.2 Randomness vs Uncertainty
- 03.3 Probability: Events, Conditioning, Bayes
- 03.4 Bayes' Theorem, Step by Step
- 03.5 Expected Value and Variance (planned)
- 03.6 Bernoulli (ES)
- 03.7 Discrete Uniform (ES)
- 03.8 🧗 Kolmogorov: Ω, 𝒜, ℙ
- 04.1 🔰 Powers and Logarithms (planned)
- 04.2 Rarity
- 04.3 Information Theory (Shannon)
- 04.4 Entropy: Three Views
- 05.1 🔰 Sums and Averages (planned)
- 05.2 Data, Measured
- 05.3 Sampling (ES)
- 05.4 Population and Sample (ES)
- 05.5 Types of Sampling (ES)
- 05.6 Choosing Summaries & Visuals
- 05.7 Visualization
- 05.8 Plots
- 06.1 What Is Statistics?
- 06.2 🔰 Curves and Area (planned)
- 06.3 Statistical Thinking
- 06.4 Variation
- 06.5 Uncertainty
- 06.6 Parameters vs Estimators (ES)
- 06.7 Distributions
- 06.8 Statistical Modeling
- 06.9 Statistical Methods
- 07.1 Data Storage: Databases (stub)
- 07.2 ER Diagrams
- 07.3 Chen Diagrams
- 07.4 SQL (planned)
- 07.5 Formal Languages and Regular Expressions (planned)
- 07.6 Reproducibility (planned)
08 · μ(x) — Fuzzy - When data is flexible
- 08.1 Fuzzy Logic (planned)
- 08.2 Fuzzy Numbers (planned)
- 08.3 Possibility vs Probability (planned)
09 · x(t) — Signals and systems - Where patterns live
- 09.1 Signals (stub)
- 09.2 Sampling and Aliasing (planned)
- 09.3 Convolution (planned)
- 09.4 Frequency (planned)
- 10.1 🔰 Vectors and Matrices (planned)
- 10.2 Mathematical Modelling (planned)
- 11.1 🔰 Derivatives (planned)
- 11.2 Convex Optimisation (planned)
- 12.1 Supervised Learning (planned)
- 12.2 Data in Language Models
- 13.1 Cause and Effect (planned)
- 13.2 Capstone Guide (planned)