I asked Gemini the following:
Can you generate another top ten table in a linguistics context that also illustrates Zipf’s Law at work.
Here was its response:
Standard Corpus of Present-Day American English (~1,014,000 words).
Conformance and Dynamics in Linguistics
Unlike national urban populations—which are buffeted by geographic constraints, political borders, and regional planning—natural language corpora fit the general power-law dynamic of Zipf's Law far more reliably.
The Pure Rank-1 to Rank-2
Halving The transition between Rank 1 (the) and Rank 2 (of) demonstrates almost exact adherence to Zipf's formulation:$$\frac{69,971}{2} = 34,986$$The actual count of 36,412 is within 4.1% of theoretical parity.
The Functional Word Plateau (Ranks 3–6)
Between ranks 3 and 6 (and, to, a, in), words appear noticeably more often than simple 1/n scaling predicts. This divergence occurs because English relies heavily on a tiny, closed class of essential structural function words (conjunctions, prepositions, and articles). These words cluster together at high baseline frequencies before the drop into open-class vocabulary begins.
The Pronoun / Auxiliary Cliff (Rank 7)
A sharp drop occurs between Rank 6 (in at 21,341) and Rank 7 (that at 10,594), cutting the count by more than half in a single step. This steep adjustment pulls the actual frequencies back into close alignment with the theoretical curve, bringing Rank 7 (that) within 6.0% of its expected Zipf value (9,996).
Zipf-Mandelbrot Refinement
While the pure model \(F_n = F_1 / n\) models the broad curve, linguists often use the generalized Zipf–Mandelbrot Law to fit natural text more precisely:$$F(n) = \frac{C}{(n + \beta)^{\alpha}}$$Introducing the shift parameter \(\beta\) (which flattens the curve for the highest-frequency grammatical function words) and allowing the scaling exponent \(\alpha\) to deviate slightly from 1.0 accounts directly for the slight elevation seen between ranks 3 and 6.
I asked for a log-log for the dataset:
Plotting Zipf's Law on a log-log scale transforms the expected exponential decay into a straight, diagonal line with a slope of $-1$. This provides a clear visual baseline for identifying where real-world data aligns with or deviates from the mathematical power-law.
- The Power-Law Signature: The red dashed line represents the perfect theoretical Zipfian distribution \(F_n = F_1 / n\)
- On a log-log graph, any dataset that strictly obeys a power-law will plot as a straight line parallel to this ideal.
- The Function Word Bulge: The visual gap between ranks 3 ("and") and 6 ("in") illustrates how heavily English relies on a tight cluster of primary function words. The actual frequency line bulges upward, pulling away from the perfect mathematical prediction.
- The Correction Drop: The steep vertical descent between "in" (rank 6) and "that" (rank 7) is highly visible here. It acts as a structural correction, dropping the actual frequency line back down to intersect tightly with the theoretical Zipf curve for the remaining ranks.
