台大應數所(碩士班) 105年度一般考試 機率統計解答

題目：
https://exam.lib.ntu.edu.tw/sites/default/files/exam/graduate/105/105_2016_graduate.pdf

1. Let $X_1, \ldots, X_n$ be a random sample from a continuous (i.e., an absolutely continuous) distribution $F(x)$. Find the distribution of the $i$th order statistics $X_{(1)},\ldots, X_{(n)}$.

(方法) 由(絕對)連續分布抽樣的順序統計量的聯合機率密度函數為$n!f(x_1)\ldots f(x_n)\mathbb{1}(x_1<\ldots<x_n)$。我們對$x_1, \ldots, x_{i-1}$以及$x_{n},\ldots, x_{i+1}$做積分即可。

(解答) First, integral with respect to $x_1$:
$$\int_{-\infty}^{\infty} n!f(x_1)\ldots f(x_n) \mathbb{1}(x_1<\ldots<x_n) dx_1,$$
which can be rewritten as:
$$= \int_{-\infty}^{x_2} n!f(x_1)\ldots f(x_n) \mathbb{1}(x_2<\ldots<x_n) dx_1.$$
Therefore, we have:
$$n! F(x_2) f(x_2) f(x_3)\ldots f(x_n) \mathbb{1}(x_2<\ldots<x_n).$$
Repeating the above procedure, we obtain after computing $\int(\ldots)dx_{i-1}$:

Second, we integral with respect to $x_n$:
$$\int_{x_{n-1}}^{\infty} n! \frac{1}{(i-1)!} F(x_i)^{i-1} f(x_i) f(x_{i+1})\ldots f(x_n) \mathbb{1}(x_{i}<\ldots<x_{n-1}) dx_n.$$

And we obtain:
$$n! \frac{1}{(i-1)!} F(x_i)^{i-1} f(x_i) f(x_{i+1})\ldots f(x_{n-1}) (1-F(x_{n-1})) \mathbb{1}(x_{i}<\ldots<x_{n-1}).$$

By repeating this procedure, we eventually obtain:
$$\frac{n!}{(i-1)!(n-i)!} F(x_i)^{i-1} f(x_i) (1-F(x_{i}))^{n-i}.$$

(補充) 如果你對自己算出來的答案沒有把握，可以代入$i = n$，檢查你的答案是否合理。

2.
(1) Find the limiting distribution of $T \sim t_\nu$, a $t$ distribution with $\nu$ degrees of freedom, when $\nu \rightarrow \infty$.
(2) Find the distribution of $\frac{X_1}{X_1+X_2}$ when $X_1, X_2$ independently follow $\chi^2_{\nu_1}$ and $\chi^2_{\nu_2}$.

(方法) 第一小題使用Slutsky’s theorem，第二小題做變數變換即可。在開始計算前，可以觀察該隨機變數的值域為0到1，因此可以猜出答案應該會是Beta分布。

(解答 (1))

Let $Z_0, Z_1, \ldots, Z_n \stackrel{\rm iid}{\sim} N(0,1)$. Define $T_n \stackrel{\rm def}{=} \frac{Z_0}{\sqrt{\frac{Z_1^2+\ldots+Z_n^2}{n}}}$. Note the following two facts when $n$ goes to infinity:

1. $Z_0 \stackrel{d}{\rightarrow} N(0,1)$ because $Z_0 \sim N(0,1)$
2. $\frac{Z_1^2+\ldots+Z_n}{n} \stackrel{\rm a.s.}{\rightarrow 1}$ by the Strong Law of Large Numbers, thus $\sqrt{\frac{Z_1^2+\ldots+Z_n}{n}}^{-1} \stackrel{\rm a.s.}{\rightarrow 1}$

By the Slutsky’s theorem we have:

(解答 (2))

Let $n = \nu_1, m = \nu_2$. Define $(T,U) \stackrel{\rm def}{=}(\frac{X_1}{X_1+X_2}, X_1+X_2)$. Let $g: \mathbb{R}^2 \mapsto \mathbb{R}$ be an arbitrary bounded continuous function.

Consider $E[g(T,U)]$:

By a variable transformation, we have:
$$=\int_{0<t<1, u>0} g(t,u) \frac{(tu)^{n/2-1}(u(1-t))^{m/2-1}}{\Gamma(n/2)\Gamma(m/2)} \exp(-\frac{u}{2})udu.$$

By a simple rearrangement, we further obtain:
$$=\int_{0<t<1, u>0} g(t,u) \frac{t^{n/2-1}(1-t)^{m/2-1}}{{\rm Be}(n/2, m/2)} \frac{u^{(n+m)/2-1}}{\Gamma((n+m)/2)} \exp(-\frac{u}{2})du,$$
which implies that the joint p.d.f of $(T,U)$ is:
$$\frac{t^{n/2-1}(1-t)^{m/2-1}}{{\rm Be}(n/2, m/2)} \mathbb{1}_{(0,1)}(t) \cdot \frac{u^{(n+m)/2-1}}{\Gamma((n+m)/2)} \exp(-\frac{u}{2}) \mathbb{1}_{(0,\infty)}(u).$$

So $(T,U)$ are independently follow a Beta distribution and a Chi-square distribution. We conclude that $T \sim {\rm Beta}(n/2, m/2)$.

3. Let $U \mid T=t \sim {\rm Unif}(0,t)$ and suppose that the marginal distribution of $T$ is an exponential family with rate $\lambda$. Find $E[U]$ and $V[U]$.

(方法) 期望值的部分用雙重期望值原理，變異數的部分利用全變異數的公式。

(解答)

$E[U] =  EE[U\mid T] = E[T/2] = \frac{1}{2\lambda}$.

$V[U] =  EV[U\mid T] + VE[U\mid T] = E[T^2/12] + V[T/2] = \frac{2}{12\lambda^2} + \frac{1}{4\lambda^2} = \frac{5}{12 \lambda^2}$.

4. Let $X_1, \ldots, X_n \stackrel{\rm iid}{\sim} N(\mu, \sigma^2)$ where $\mu, \sigma^2$ are unknown parameters and let  $\Phi(\cdot)$ stand for the CDF of the standard normal distribution. Find the maximum likelihood estimator for $\Phi(\frac{x-\mu}{\sigma})$ and its asymptotic distribution for a given $x \in \mathbb{R}$.

(方法) 考慮$(\widehat{\mu}_{\rm MLE}, \widehat{\sigma}^2_{\rm MLE})^\top$的漸近分布後，考慮一個函數$g(y_1,y_2)= \Phi(\frac{x-y_1}{\sqrt{y_2}})$以及在$y_1=\mu_1, y_2=\sigma^2$的泰勒展開（也就是multivariate $\delta$-method）。

(解答) First, we find the maximum likelihood estimators for $(\mu, \sigma^2)$. The likelihood function is given by:

So the log-likelihood function is:
$$\ell(\mu,\sigma^2) = -\frac{n}{2}\ln(2\pi\sigma^2) - \frac{1}{2\sigma^2} \sum_{i=1}^n (x_i-\mu)^2.$$

For fixed $\sigma^2>0$, consider the maximization of $\ell(\mu,\sigma^2)$:
$$\frac{\partial \ell}{\partial \mu} = \frac{n(\bar{x}-\mu)}{\sigma^2},$$
from which we can see that $\ell(\dot, \sigma^2)$ attain the maximum value at $\mu = \bar{x}$.

Next, consider the maximization of $\ell(\bar{x}, \sigma^2)$:

$$\frac{\partial \ell(\bar{x}, \sigma^2)}{\partial \sigma^2} = \frac{n}{2\sigma^2}(\frac{1}{n}\sum_{i=1}^n (x_i-\bar{x})^2 - \sigma^2),$$
from which we can see that $\ell(\bar{x}, \sigma^2)$ attains the maximum value at $\sigma^2 = \frac{1}{n} \sum_{i=1}^n (x_i-\bar{x})^2$.

So we conclude that the maximum likelihood estimators for $(\mu,\sigma^2)$ are:
$$(\widehat{\mu}, \widehat{\sigma}^2) = (\bar{x}, \frac{1}{n}\sum_{i=1}^n(x_i-\bar{x})^2).$$

Next, we consider the asymptotic distribution for $(\widehat{\mu}, \widehat{\sigma}^2)$.

Note that $\sqrt{n}(\widehat{\mu}-\mu) \sim N(0,1)$, so the asymptotic distribution is also $N(0,1)$.

To consider the asymptotic distribution of $\widehat{\sigma}^2$, recall that $\sum_{i=1}^n(x_i-\bar{x})^2 \sim \sigma^2 \cdot \chi^2_{n-1}$. When $Y_1, \ldots, Y_{n-1} \stackrel{\rm iid}{\sim} \sigma^2 \cdot \chi_{1}$, the distribution of $\sum_{i=1}^n(x_i-\bar{x})^2$ is identical with that of $\sum_{i=1}^{n-1} Y_i^2$.

So we can consider the asymptotic distribution of $\frac{1}{n} \sum_{i=1}^{n-1} Y_i^2$ to find the asymptotic distribution of $\widehat{\sigma}^2$. By the Central Limit Theorem, we have:

Therefore, we also have:

By the Basu’s Theorem, we also have $\widehat{\mu}, \widehat{\sigma}^2$ are mutually independent. (One is a complete sufficient statistic for $\mu$, and the other is ancillary.) So we obtain:

Finally, consider the following fuction and its Taylor expansion at $\bm{y}=\bm{\theta}$, where $\bm{y}\stackrel{\rm def}{=}(y_1,y_2)^\top$ and $\bm{\theta} \stackrel{\rm def}{=}(\mu, \sigma^2)^\top$:

5. Let $X_1, \ldots, X_n \stackrel{\rm iid}{\sim} f(x\mid \theta)$ where $f(x\mid \theta) = \theta x^{\theta-1} \mathbb{1}_{(0,1)}(x)$. Find the UMVUE for $\theta$ and its asymptotic distribution.

(方法)

(解答)

6. Let $X_1, \ldots, X_n \stackrel{\rm iid}{\sim} N(\mu, \sigma^2)$ be a random sample where $\mu, \sigma^2$ are unknown parameters. Find a size $\alpha$ unbiased test for the hypotheses $H_0: \theta \in [\theta_1, \theta_2]$ vs $H_1: \theta \notin [\theta_1, \theta_2]$.

(方法)

(解答)

7. Let $X_1, \ldots, X_n \stackrel{\rm iid}{\sim} N(\mu, \sigma^2)$ be a random sample where $\mu, \sigma^2$ are unknown parameters. Derive the power function of the size $\alpha$ likelihood ratio test for the hypotheses $H_0: \theta \leq \theta_0$ vs $H_1: \theta > \theta_0$.

(方法)

(解答)