 |
Figure 1 |
 |
Figure 2: generated by Gemini |
Most of these metal digits would be purchased for house numbers that would certainly conform to Benford's Law. In my case, I grew up in a house numbered 21 and I've lived in a house numbered 12 for the past twenty years.
Here is more detailed information about the law generated by Gemini:
Benford's Law, often called the "First-Digit Law," is a statistical phenomenon where, in many real-life sets of numerical data, the leading digit is likely to be small. For example, the number 1 appears as the leading digit about 30% of the time, while 9 appears less than 5% of the time.
Here is an exploration of its history, utility, and some fascinating details surrounding it.
1. The History of Discovery
Interestingly, Benford's Law is an example of Stigler's Law of Eponymy, which states that no scientific discovery is named after its original discoverer.
The Original Discovery (1881): The phenomenon was actually first noticed by the astronomer Simon Newcomb. While browsing log tables (books used for calculation before calculators), he noticed that the earlier pages, which contained numbers starting with 1, were much more worn and dirty than the later pages. He published a paper on this counter-intuitive observation, stating, "The law of probability of the occurrence of numbers is such that all natural numbers are not equally probable."
The Rediscovery (1938): Over 50 years later, physicist Frank Benford noticed the same pattern in log tables. Unaware of Newcomb's work, Benford tested the theory against 20 different domains, including the surface areas of 335 rivers, the specific heats of thousands of chemical compounds, and even numbers from Reader's Digest issues. He found the pattern held true across these disparate datasets. Because Benford formalized it and provided extensive data to back it up, the law bears his name.
2. Usefulness and Applications
Benford's Law has transitioned from a mathematical curiosity to a powerful tool for forensic analysis.
Fraud Detection (Forensic Accounting): This is the most famous application. Financial data—such as income tax returns, expense reports, and stock prices—should generally conform to Benford's Law. If a company manufactures fake invoices or an individual invents expense numbers, they often distribute the leading digits uniformly (using 1 through 9 roughly equally) or use psychological biases (avoiding "obvious" numbers). When the data is plotted and deviates significantly from the Benford curve (e.g., too many 7s or 8s), it flags the dataset for audit.
Election Integrity: Statisticians have used Benford’s Law to analyze vote counts in elections. While not definitive proof of fraud (since voting districts can have specific demographics that skew numbers), meaningful deviations can indicate where further investigation is needed.
Scientific Data Verification: It is used to detect data manipulation in scientific studies. If a researcher fabricated data rather than collecting it experimentally, the leading digits often fail the Benford test.
- Macroeconomic Reporting: It has been used to assess the veracity of macroeconomic data reported by governments. For example, economists have used it to analyze the economic statistics reported by countries to the EU or IMF to detect potential "massaging" of the figures.
3. Mathematical Explanation
Why does this happen? The most intuitive explanation involves growth and scale invariance.
Scale Invariance: If a rule about numbers is universal, it shouldn't matter what unit you use. If you measure the lengths of rivers in miles, kilometers, or light-years, the distribution of the first digits should remain the same. Benford's distribution is the only distribution that remains stable when you change the scale.
Logarithmic Growth: Many natural phenomena grow geometrically or exponentially (like compound interest or populations). To get from a leading digit of 1 to 2, a quantity must increase by 100%. To get from 8 to 9, it only needs to increase by 12.5%. Therefore, growing entities spend much more time "being" a 1 than they do "being" an 8 or 9.
4. Interesting Facts & Limitations
The Exact Probabilities:
1: 30.1%
2: 17.6%
3: 12.5%
4: 9.7%
5: 7.9%
6: 6.7%
7: 5.8%
8: 5.1%
9: 4.6%
It Doesn't Work on Everything: Benford's Law requires data that spans several orders of magnitude. It does not apply to:
Restricted ranges: The height of human adults (mostly starts with 1 or 2 meters, or 4, 5, 6 feet).
Assigned numbers: Phone numbers, zip codes, or social security numbers (which are generated sequentially or geographically).
Lotteries: These are designed specifically to be uniformly random; if a lottery followed Benford's Law, it would be rigged.
Legal Precedent: Evidence based on Benford's Law has been admitted in federal, state, and local courts in the United States as evidence of fraud.
The absence of zero in Benford's Law often trips people up, but it comes down to how we define a "leading digit" and the mathematical nature of the law itself.
Here is the breakdown of why zero is excluded from the first-digit rule, but prominently included elsewhere.
1. The Definition of "Leading Digit"
Benford's Law specifically analyzes the first significant digit of a number. In standard mathematical notation, a non-zero number can never start with zero.
Whole Numbers: We do not write "54" as "054." If we did, the zero would be considered formatting, not a significant value. The first digit is 5.
Decimals: For a number like 0.0034, the leading zeros are merely placeholders to show magnitude. The first significant digit—the one that tells you the precision of the number—is 3.
The Number Zero: The number $0$ itself is excluded from the dataset entirely. Benford's Law applies to data that spans several orders of magnitude (growth). Zero represents a complete lack of magnitude, so it cannot be analyzed in terms of "growth" or relative size.
2. The Logarithmic Scale
As mentioned in the previous explanation, Benford's Law is based on a logarithmic scale.
To find the probability of a digit, you are essentially looking at the space between logarithms (e.g., $ \log_{10}(1)$ vs $ \log_{10}(2)$).
The logarithm of zero ($ \log_{10}(0)$) is undefined (it approaches negative infinity). Therefore, zero cannot exist on the logarithmic scale that underpins the law.
3. Zero is the "Star" of the Second Digit
While zero is banned from the first position, it actually becomes the most common digit in the second position.
Benford's Law can be generalized to look at the 2nd, 3rd, and 4th digits. As you move to the right, the bias flattens out, but for the second digit, the "Benford skew" is still visible, and zero takes the lead.
Probability of Second Digits:
0: ~12.0% (Highest frequency)
1: ~11.4%
2: ~10.9%
...
9: ~8.5%
Because the first digit has already "absorbed" the massive logarithmic bias (where 1 appears 30% of the time), the second digit has more room to be a 0. This is because after a number grows past 19 (leading digit 1), it hits 20 (second digit 0), then 21, etc.
Summary
Zero isn't in the "First Digit Law" because standard math notation doesn't allow a number to start with 0, and the logarithmic math behind the law breaks if you input 0. However, if you look at the second digit of that same data, you will find 0 is the most frequent number of all.
*****************************************
Below is a table showing percentages for the 2nd, 3rd, 4th and 5th digits:
Gemini comments:
Here are the precise probabilities for the 2nd, 3rd, 4th, and 5th digits according to Benford's Law.
The General Trend
As you move to later positions (3rd, 4th, 5th), the "Benford Bias" rapidly fades. By the 5th digit, the distribution becomes almost perfectly uniform, meaning every number from 0 to 9 has an equal $10\%$ chance of appearing.
Detailed Probability Table
This table compares the likelihood of each digit appearing in the 2nd, 3rd, 4th, and 5th positions.
| Digit | 2nd Position | 3rd Position | 4th Position | 5th Position |
| 0 | 11.97% | 10.18% | 10.02% | 10.00% |
| 1 | 11.39% | 10.14% | 10.01% | 10.00% |
| 2 | 10.88% | 10.10% | 10.01% | 10.00% |
| 3 | 10.43% | 10.06% | 10.01% | 10.00% |
| 4 | 10.03% | 10.02% | 10.00% | 10.00% |
| 5 | 9.67% | 9.98% | 10.00% | 10.00% |
| 6 | 9.34% | 9.94% | 9.99% | 10.00% |
| 7 | 9.04% | 9.90% | 9.99% | 10.00% |
| 8 | 8.76% | 9.86% | 9.99% | 10.00% |
| 9 | 8.50% | 9.83% | 9.98% | 10.00% |
Observations
The 2nd Digit: There is still a clear bias. Zero is the most common (11.97%), and 9 is the least common (8.50%). This is significant enough to be useful in forensic accounting.
The 3rd Digit: The bias has mostly evaporated. The difference between the most likely digit (0) and the least likely (9) is only about 0.35%.
The 4th & 5th Digits: The differences are statistically negligible for most practical purposes. By the 5th digit, the deviation from a perfect 10% is less than 0.01%.
The Mathematics Behind the Fade
The reason the bias disappears is that Benford's Law is based on the relative distance between numbers on a logarithmic scale.
At the "front" of a number (1 vs 2), the relative difference is massive (100% growth).
By the time you get to the 5th digit (e.g., the difference between 10,001 and 10,002), the relative growth required to change that digit is tiny. The logarithmic constraints loosen, and randomness takes over.
For those interested in the exact calculation, the probability $P$ of a specific digit $d$ appearing in position $n$ is given by summing the probabilities of all possible prefixes:
For the 2nd digit, for instance, this sums the probabilities across the ranges 10–19, 20–29, etc.
Would you like me to explain how forensic accountants use this specifically—do they just look at the 1st digit, or do they combine these probabilities for a "First-Two Digits" test?
Since you are interested in the deeper application, here is how forensic accountants actually use these probabilities, specifically focusing on the "First-Two Digits" (F2D) Test.
While the First-Digit test is a good "smoke detector," the First-Two Digits test is the "high-resolution camera." It is much harder for a fraudster to trick.
1. The First-Two Digits (F2D) Test
Instead of sorting data into 9 bins (1 through 9), accountants sort data into 90 bins (from 10 to 99).
This test combines the probabilities of the first and second digits to create a specific curve.
The "10" bin: This is the most common starting pair. It should appear about 4.1% of the time.
The "99" bin: This is the least common pair. It should appear only about 0.4% of the time.
Why this catches fraudsters
If an employee tries to invent expenses, they might know Benford's Law enough to start many numbers with 1 or 2. However, they rarely get the internal distribution right.
Natural Data: A number starting with 10 (like 105, 1,020) is much more common than a number starting with 19 (like 195, 1,900).
Fraudulent Data: A human inventing numbers often treats "10" and "19" as roughly equal because they both start with 1. On an F2D graph, this creates a "jagged" line rather than a smooth curve, immediately signaling manipulation.
2. Common "Red Flag" Signals
When accountants run the F2D test, they aren't just looking for general chaos; they are looking for specific spikes that reveal the method of the fraud.
A. The "Just Under the Limit" Spike
Many companies have approval thresholds. For example, a manager might be allowed to approve invoices up to $500 without a supervisor's signature.
What you see: A massive, unnatural spike in numbers starting with 49 (e.g., $495, $499).
Benford Prediction: Numbers starting with 49 should be rare (~0.9%).
Reality: If they appear 5% or 10% of the time, the auditor knows the employee is structuring payments to bypass authority.
B. The "Psychological Pricing" Spike
In retail or marketing data, you often see spikes at 95 and 99 ($9.99, $19.95).
While this isn't necessarily fraud, it breaks Benford’s Law.
Forensic analysts must "scrub" the data of standard retail prices before running the test to see if the underlying accounting is valid.
C. The "Lazy Repeater"
If a fraudster is lazy, they might repeatedly reuse a specific number they feel is "safe."
Example: A fraudster might use the number $1,245 repeatedly because it looks "random" to them.
Result: You get a statistical spike at 12. While 12 is naturally common, a specific, repeated spike often exceeds even the high Benford probability.
3. The "Last-Two Digits" Counter-Check
While Benford's Law governs the start of numbers, the Uniform Distribution governs the end of numbers.
In a genuine dataset of financial transactions (like grocery bills or wire transfers), the cents (the last two digits) should be purely random.
Expected Pattern: The number .00 should appear just as often as .37 or .99.
The Fraud Signal: If the cents column shows a high frequency of .00 or .50, it suggests the numbers are being estimated or rounded by a human, rather than generated by actual transactions (which usually have taxes and random pricing variations).
4. Summary of Forensic Workflow
Run First-Digit Test: Is the general shape correct? (If no, major issue).
Run F2D Test: Are the 10s more common than the 19s? Are there spikes at 49 or 99?
Run Last-Two Digits: Are the cents random? (If they are mostly .00, the data is likely estimated).
Would you like me to walk you through a specific famous case study, such as the Enron scandal or the Greek debt crisis, to see how these numbers actually looked in a real-world disaster?
I declined to venture further at this point as this post was getting long enough.
No comments:
Post a Comment