Law of Large Numbers

Law of Large Numbers (LLN) is a mathematical theorem that states the average of the results obtained from many independent random samples.

In this article, we have discussed the Law of Large Numbers definition, its limitations, examples and others in detail.

Law of Large Numbers is a concept in probability and statistics that states that the average is closer to the expected or theoretical value as the number of trials or observations increases.

Example: If you flip a fair coin many times, the proportion of heads will get closer to 50% as you increase the number of flips.

Various limitations of the Law of Large Numbers are:

• Sample Size: When the sample size is genuinely huge, the Law of huge Numbers operates most effectively. A limited sample size could mean that the results do not represent the underlying population and that the law does not hold.

• Independence: Law of Large Numbers presupposes that the events or observations are unrelated to one another. The law might not apply correctly if there is any dependence or correlation between the observations.

• Rate of Convergence: Law of Large Numbers states that as the sample size grows, the sample mean will converge to the population mean, but it doesn’t say how quickly this convergence will happen. Sometimes, the convergence could be sluggish, and a huge sample size might be needed to get the appropriate degree of precision.

• Outliers and Extreme Values: The existence of outliers or extreme values in the data can have a significant impact on the Law of Large Numbers. Even with a high sample size, a few extreme observations can considerably affect the sample mean.

• Observations Not Identically Distributed: Law of Large Numbers presupposes that the data come from a single probability distribution. The law might not hold if the underlying distribution varies over time or if the data are drawn from disparate distributions.

• Biased Sampling: Law of Large Numbers may not apply and the sample mean may not converge to the true population mean if the sampling procedure is biased or non-random.

• Finite Population: In general, an infinite population is used to state the Law of Large Numbers. The law might need to be changed or altered when working with a finite population to take into consideration the population’s finite size.

Various of the Law of Large Numbers are:

Weak Law of Large Numbers (WLLN)

Imagine you have a fair coin and flip it several times to determine Weak Law of Large Numbers (WLLN). The Weak Law of Large Numbers indicates that the proportion of heads (or tails) seen will get closer and closer to 0.5 (since the coin is fair, and the probability of obtaining a head or a tail is 0.5). As you increase the number of coin flips.

Generally speaking, the WLLN states that as the number of observations rises the sample average will converge to the theoretical mean if your series of independent and identically distributed random variables (such as coin flips).

Strong Law of Large Numbers (SLLN)

An improved form of the WLLN is the Strong Law of Large Numbers. It says the sample average not only will converge to the expected value but also offers a gauge of the speed at which this convergence occurs.

Returning to the coin flipping example, the SLLN indicates that as the number of coin flips rises the likelihood of the sample average deviating from the expected value (0.5) by a notable amount falls. Stated differently, when more observations are gathered, the likelihood of the sample average deviating significantly from the expected value becomes even more rare.

Both versions essentially mean that with enough trials, the results will stabilize around the expected average.

An evolved variation of the Law of Large Numbers is the Law of the Iterated Logarithm. It offers for the sample average an exact rate of convergence. It says the sample average will fluctuate about the predicted value, but as the number of observations rises the oscillations will get ever less.

In the coin flipping example, the LIL indicates that, as the number of coin flips rises, the sample average will not only converge to 0.5 but also offers a specific range within where it is most likely to fall.

Law of large number is important because of following traits:

• Predictability: It helps in predicting the average outcome of random events over time.

• Reliability: It shows that the average of results from a large number of trials is reliable.

• Practical Use: It is widely used in statistics, economics, finance, and other fields to make informed decisions based on data.

Law of Large Numbers (LLN) and the Central Limit Theorem (CLT) are two fundamental concepts in probability and statistics that describe the behavior of large samples and their definition is:

Law of Large Numbers (LLN)

Law of Large Numbers states that as the number of trials or observations increases, the average of the results obtained will converge to the expected value.

Central Limit Theorem (CLT)

Central Limit Theorem states that the distribution of the sample mean of a sufficiently large number of independent, identically distributed (i.i.d.) random variables approaches a normal distribution, regardless of the original distribution of the variables.

• LLN focuses on the convergence of the sample mean to the population mean as the sample size grows.
• CLT focuses on the distribution of the sample mean, stating that it becomes approximately normal as the sample size grows.

Law of Large Numbers and the Central Limit Theorem are foundational principles in probability and statistics. LLN ensures that averages of large samples are reliable estimates of the population mean, while CLT justifies the use of the normal distribution for making inferences about sample means.

An example explaining law of large numbersis added below:

Imagine your bag contains blue and red balls. Assume the bag holds 50% blue balls and 50% red balls. Drawing just one ball from the bag might result in a red or a blue ball, but it would be difficult to forecast the precise hue.

Now imagine, you take 10 balls from the bag one at a time, noting the colors. You might get six red balls and four blue balls or perhaps seven red balls and three blue balls. Although your little sample’s red to blue ball ratio would not be exactly 50:50, it would most certainly be near.

Law of Large Numbers, however, informs us that the ratio of red balls to blue balls in your total sample would get closer and closer to the theoretical ratio of 50:50 if you kept drawing balls from the bag and tracking the colors hundreds or perhaps thousands of times.

Law of Large Numbers is a fundamental concept in probability theory and statistics that has significant applications in finance. In simple terms, it states that as the sample size (or number of observations) increases, the average of the results observed will become closer and closer to the expected or theoretical average.

Let’s break this down with an example:

Imagine you are flipping a fair coin. The theoretical probability of getting heads is 0.5 (or 50%). If you flip the coin only a few times, say 10 times, the observed proportion of heads may deviate significantly from 0.5 due to random chance. However, if you flip the coin thousands or millions of times, the observed proportion of heads will be very close to 0.5.

In finance, the Law of Large Numbers is particularly relevant in the context of portfolio management and risk analysis. Here are a few examples:

• Investing in a single stock might cause quite erratic and extremely fluctuating returns. On the other hand, if you have a diversified portfolio comprising several stocks, the total portfolio returns usually show better consistency and closer alignment with the average return of the market. This is so because as the portfolio grows the positive and negative deviations of individual equities tend to cancel each other out.

• Insurance firms decide their policy premiums using the Law of Large Numbers. Analyzing a sizable pool of policyholders helps one to more precisely estimate the average frequency and severity of claims. This lets them create equitable and sustainable rates for the whole pool.

• In finance, Monte Carlo simulations help to predict and examine the possible results of several investments or financial plans. The Law of Large Numbers guarantees that the simulated results converge towards the genuine anticipated value or distribution of outcomes as the number of runs rises.

• Financial institutions project the possible losses or benefits connected with different risk factors using the Law of Large Numbers. Analyzing several historical data points or scenarios helps one better estimate and control hazards.

Articles Related to Law of Large Numbers:

• Bayes’ Theorem
• Binomial Distribution
• Normal Distribution
• Bernoulli Distribution
• Probability Theory
• Events in Probability
• Types of Events in Probability

Example 1: A fair six-sided die is rolled repeatedly. What is the expected average value of the outcomes as the number of rolls increases

Solution:

Expected value of rolling a fair six-sided die is the average of the numbers 1 through 6, which is (1 + 2 + 3 + 4 + 5 + 6)/6 = 3.5

According to the Law of Large Numbers, as the number of rolls increases, the average outcome will approach 3.5

Example 2: In a game, a player flips a fair coin. If it lands heads, the player wins $1; if it lands tails, the player loses $1. What is the expected average profit/loss for the player as the number of flips increases?

Solution:

Expected value of flipping a fair coin is 0.5 for heads and 0.5 for tails

Expected profit/loss for each flip is (0.5 × $1) + (0.5 × -$1) = $0

Therefore, the expected average profit/loss for the player as the number of flips increases approaches $0 by the Law of Large Numbers

Example 3: A bag contains 20 red balls and 30 blue balls. A ball is drawn from the bag, and the color is noted. The ball is then returned to the bag, and the process is repeated. What is the expected proportion of red balls drawn as the number of draws increases?

Solution:

Probability of drawing a red ball on any single draw is 20/(20+30) = 20/50 = 2/5

By the Law of Large Numbers, as the number of draws increases, the proportion of red balls drawn will approach 2/5

Example 4: A factory produces light bulbs, and historical data show that 5% of the bulbs are defective. If a random sample of bulbs is taken from the production line, what is the expected proportion of defective bulbs as the sample size increases?

Solution:

Probability of selecting a defective bulb from the production line is 0.05

As the sample size increases, by the Law of Large Numbers, the proportion of defective bulbs in the sample will approach 0.05

Example 5: In a game, a player rolls two fair six-sided dice and wins $10 if the sum of the dice is 7, and loses $5 otherwise. What is the expected average profit/loss for the player as the number of rolls increases?

Solution:

There are 6 possible outcomes when rolling two fair six-sided dice, and only one of these outcomes results in a sum of 7

So, the probability of winning $10 is 1/6, and the probability of losing $5 is 5/6. The expected profit/loss per roll is (1/6 × $10) + (5/6 × -$5) = -$5/6

Therefore, the expected average profit/loss for the player as the number of rolls increases approaches -$5/6 by the Law of Large Numbers

Example 6: A student takes multiple-choice quizzes with 5 questions, each with 4 answer choices. If the student randomly guesses the answers to all questions, what is the expected average score as the number of quizzes increases?

Solution:

Each question has 4 answer choices, so the probability of guessing the correct answer to any question is 1/4

Since there are 5 questions, the expected score for each quiz is (1/4 × 5) = 5/4

Therefore, the expected average score for the student as the number of quizzes increases approaches 5/4 by the Law of Large Numbers

What is the law of large numbers?

Law of large numbers, states that, “as a sample size grows, its mean gets closer to the average of the whole population.”

What is the law of large numbers example?

Example of law of large number is, “if we roll the dice a large number of times, the average result will be closer to the expected value of 3.5.”

What is the weak law of large numbers statement?

Weak Law of Large Numbers (WLLN) states that for a sequence of independent and identically distributed random variables with a finite expected value, the sample average converges in probability to the expected value as the sample size increases.

What is limit theorem law of large numbers?

Law of Large Numbers is often discussed in the context of limit theorems in probability theory. It describes how the average of a large number of trials behaves as the number of trials approaches infinity