Chapter 7 Estimation

7.1 Statistical Inference

Nothing on this section.

7.2 Prior and Posterior Distributions

  1. Suppose that the proportion \(\theta\) of defective items in a lot is either \(0.1\) or \(0.2\) and the prior probability function is \(\xi(0.1)=0.6\) and \(\xi(0.2)=0.4\). Suppose that nine items were selected at random, and exactly two of them were defective. Determine the posterior distribution of \(\theta\).

  2. Suppose that the number of mice per hectare follows a \(X\sim \textrm{Poisson}(\lambda)\) distribution, and our prior distribution for \(\lambda\) is Gamma(\(\alpha=2, \beta=1\)) distribution. Suppose that we collect one observation \(x\).
    1. Show that \[\xi(\lambda | x) \propto g(x)h(x,\lambda) = \frac{1}{(x-1)!}\,e^{-2\lambda}\lambda^{x+1}\]
    2. Recognize \(h(x,\lambda)\) as the kernel of a Gamma distribution. What are the parameters of the posterior distribution?

7.3 Conjugate Prior Distributions

  1. Suppose that the proportion \(\theta\) of defective items in a large shipment is unknown and that the prior distribution of \(\theta\) is \(\textrm{Beta}(\alpha=2, \beta=200)\). If 100 itiems are selected at random, and three are found to be defective, what is the posterior distribution of \(\theta\)?

  2. Suppose that the number of defects in a roll of aluminum has a Poisson distribution with rate parameter \(\theta\). Suppose that we use a prior distribution \(\theta \sim \textrm{Gamma}(3, 1)\). Determine the posterior distribution of \(\theta\).

  3. Suppose that \(X \sim \textrm{NegBinomial}(r,p)\) and \(r\) is known, but we are interested in inference on \(p\). Show that \(p \sim \textrm{Beta}(\alpha,\beta)\) is conjugate prior by deriving the posterior distribution.

  4. Suppose that the continuous random variable \(X_i \stackrel{iid}{\sim} \textrm{Uniform}(0, \theta)\). The conjugate prior distribution for \(\theta\) is the Pareto distribution, which has pdf \[f(x|x_0, \alpha) = \frac{\alpha x_0^\alpha}{x^{\alpha+1}}\;I(x > x_0)\]
    1. Graph the Pareto distribution with \(x_0=1\), and \(\alpha=2\) and again with \(\alpha=3\).
    2. Show that the joint distribution of \(X_1,\dots,X_n\) can be written in terms of the sample maximum, which you may denote \(X_{(n)}\). Make sure to use an indicator function to denote the support of the distribution.
    3. Consider the prior distribtion on \(\theta \sim \textrm{Pareto}(\theta_0, \alpha)\). Write down the prior pdf, again making sure to include your indicator function.
    4. If \(x_{(n)} \ge \theta_0\), what is the posterior distribution of \(\theta\)?

7.4 Bayes Estimators

  1. Supposet that a random sample of size \(n\) is taken from a Bernoulli distribution with unknown probability of success \(\theta\). As usual, we will assign the conjugate prior \(\textrm{Beta}(\alpha, \beta)\). Denote the mean of the prior distribution \(\mu_0\).
    1. Show that the Bayes estimator under squared error loss will be a weighted average of the sample mean and \(\mu_0\). That is, show that \[\hat{\theta}_n = \gamma_n \bar{X}_n + (1-\gamma_n)\mu_0\]
    2. Show that \(\hat{\theta}_n\) is a consistent estimator of \(\theta\).
  2. Suppose that a random sample of size \(n\) is taken from a Poisson distribution for which the value of the mean \(\theta\) is unknown, and the prior distribution of \(\theta\) is a gamma distribution for which the mean is \(\mu_0\).
    1. Show that \(\hat{\theta}\), the mean of the posterior distribution of \(\theta\), will be a weighted average of sample mean \(\bar{X}\) and the prior mean \(\mu_0\).
    2. Show that \(\hat{\theta}_n\) is a consistent estimator of \(\theta\).
  3. Suppose that \(X_i \stackrel{iid}{\sim} \textrm{Uniform}(0,\theta)\) and that the prior distribution for \(\theta\) is \(\textrm{Pareto}(\theta_0, \alpha)\) where \(\alpha > 1\). Suppose that we observe \(\max(x_i) > \theta_0\). What is the Bayes estimator of \(\theta\) under the squared error loss function?

7.5 Maximum Likelihood Estimators

  1. Suppose that \(X_1, \dots, X_n\) form a random sample from a Poisson(\(\theta\)) distribution for which the mean \(\theta\) is unknown and \(\theta > 0\).
    1. Determine the MLE of \(\theta\), assuming that at least one of the observed values is different than zero.
    2. Show that the MLE of \(\theta\) does not exist if every observed value is zero.

  2. Suppose that \(X_1, \dots, X_n\) form a random sample from the normal distribution for which the mean \(\mu\) is known, but the variance, \(\sigma^2\), is unknown. Find the MLE of \(\sigma^2\). Hint, it is NOT the sample variance!

  3. Suppose that \(X_1, \dots, X_n\) form a random sample from a distribution with pdf \[f(x|\theta) = \theta x^{\theta-1} \, I( 0 < x < 1 )\] where \(\theta > 0\).
    1. Graph the distribution for various values of \(\theta\).
    2. Find the MLE of \(\theta\).

7.6 Properties of Maximum Likelihood Estimators

  1. Suppose that \(X_i \stackrel{iid}{\sim} \textrm{Exponential}(\theta)\) for a sample of size \(n\).
    1. Find the MLE of \(\theta\).
    2. Find the MLE of the median of the distribution.

  2. Suppose that \(X_i \stackrel{iid}{\sim} N( \mu, \sigma^2 )\). Find the MLE of the 0.95 quantile of the distribution, that is, of the point \(\theta\) such that \(Pr( X \le \theta) = 0.95\).

  3. Suppose that \(X_i \stackrel{iid}{\sim} \textrm{Gamma}(\alpha, \beta)\) where \(\alpha\) is known. Find the MLE of \(\beta\) and then also find the MLE of \(\theta = \alpha/\beta\).

  4. Supppose that \(X_i \stackrel{iid}{\sim} \textrm{Beta}( \alpha, \beta )\). Find the Method of Moment estimators for \(\alpha\) and \(\beta\). Hint: This sort of calculation can be done easily in Mathematica or Wolfram Alpha. It really pays to get comfortable with some software packages.

7.7 Sufficient Statistics

  1. Suppose that \(X_i \stackrel{iid}{\sim} \textrm{NegBinom}(r,p)\) where \(r\) is known. Show that \(T=\sum X_i\) is sufficient for \(p\).

  2. Suppose that \(X_i \stackrel{iid}{\sim} \textrm{Gamma}(\alpha, \beta)\) where \(\alpha\) is known. Show that \(T=\sum X_i\) is sufficient for \(\beta\).

  3. Suppose that \(X_i \stackrel{iid}{\sim} \textrm{Gamma}(\alpha, \beta)\) where \(\beta\) is known.
    1. Show that \(T=\prod X_i\) is sufficient for \(\alpha\).
    2. Show that \(T=\sum \log(X_i)\) is also sufficient for \(\alpha\).