Homework 4.1: Normal approximations (40 pts)

a) Imagine I have a univariate continuous distribution with PDF \(f(y)\) that has a maximum at \(y^*\). Assume that the first and second derivatives of \(f(y)\) are defined and continuous near \(y^*\). Show by expanding the log PDF of this distribution in a Taylor series about \(y^*\) that the distribution is locally Normal near the maximum.

In performing the Taylor series, how is the scale parameter \(\sigma\) of the Normal approximation of the distribution related to the log PDF of the distribution it is approximating?

b) Another way you can approximate a distribution as Normal is to use its mean and variance as the parameters as the approximate Normal. We will call this technique “equating moments.” Can you do this if the distribution you are approximating has heavy tails, say like a Cauchy distribution? Why or why not?

c) Make plots of the PDF and CDF of the following distributions with their Normal approximations as derived from the Taylor series and by equating moments. Do you have any comments about the approximations?

  • Beta with α = β = 10

  • Gamma with α = 5 and β = 2

d) Discrete distributions are also often approximated as Normal. In fact, early studies of the Normal distributions arose from it being used to approximate a Binomial distribution. Use the method of equating moments to make a plot of the PMF of the Binomial distribution and the PDF of the Normal approximation of the Binomial distribution for:

  • Binomial with N = 100 and θ = 0.1.

  • Binomial with N = 10 and θ = 0.1.

Comment on what you see.