評論: 原題目並未明確表示參數為未知與否，因此在此假設它們為未知。
題目：
https://exam.lib.ntu.edu.tw/sites/default/files/exam/graduate/105/105_2016_graduate.pdf
1. Let X_1, \ldots, X_n be a random sample from a continuous (i.e., an absolutely continuous) distribution F(x). Find the distribution of the ith order statistics X_{(1)},\ldots, X_{(n)}.
(方法) 由(絕對)連續分布抽樣的順序統計量的聯合機率密度函數為n!f(x_1)\ldots f(x_n)\mathbb{1}(x_1<\ldots<x_n)。我們對x_1, \ldots, x_{i-1}以及x_{n},\ldots, x_{i+1}做積分即可。
(解答) First, integral with respect to x_1:
\int_{-\infty}^{\infty} n!f(x_1)\ldots f(x_n) \mathbb{1}(x_1<\ldots<x_n) dx_1,
which can be rewritten as:
= \int_{-\infty}^{x_2} n!f(x_1)\ldots f(x_n) \mathbb{1}(x_2<\ldots<x_n) dx_1.
Therefore, we have:
n! F(x_2) f(x_2) f(x_3)\ldots f(x_n) \mathbb{1}(x_2<\ldots<x_n).
Repeating the above procedure, we obtain after computing \int(\ldots)dx_{i-1}:
Second, we integral with respect to x_n:
\int_{x_{n-1}}^{\infty} n! \frac{1}{(i-1)!} F(x_i)^{i-1} f(x_i) f(x_{i+1})\ldots f(x_n) \mathbb{1}(x_{i}<\ldots<x_{n-1}) dx_n.
And we obtain:
n! \frac{1}{(i-1)!} F(x_i)^{i-1} f(x_i) f(x_{i+1})\ldots f(x_{n-1}) (1-F(x_{n-1})) \mathbb{1}(x_{i}<\ldots<x_{n-1}).
By repeating this procedure, we eventually obtain:
\frac{n!}{(i-1)!(n-i)!} F(x_i)^{i-1} f(x_i) (1-F(x_{i}))^{n-i}.
(補充) 如果你對自己算出來的答案沒有把握，可以代入i = n，檢查你的答案是否合理。
2.
(1) Find the limiting distribution of T \sim t_\nu, a t distribution with \nu degrees of freedom, when \nu \rightarrow \infty.
(2) Find the distribution of \frac{X_1}{X_1+X_2} when X_1, X_2 independently follow \chi^2_{\nu_1} and \chi^2_{\nu_2}.
(方法) 第一小題使用Slutsky’s theorem，第二小題做變數變換即可。在開始計算前，可以觀察該隨機變數的值域為0到1，因此可以猜出答案應該會是Beta分布。
(解答 (1))
Let Z_0, Z_1, \ldots, Z_n \stackrel{\rm iid}{\sim} N(0,1). Define T_n \stackrel{\rm def}{=} \frac{Z_0}{\sqrt{\frac{Z_1^2+\ldots+Z_n^2}{n}}}. Note the following two facts when n goes to infinity:
- Z_0 \stackrel{d}{\rightarrow} N(0,1) because Z_0 \sim N(0,1)
- \frac{Z_1^2+\ldots+Z_n}{n} \stackrel{\rm a.s.}{\rightarrow 1} by the Strong Law of Large Numbers, thus \sqrt{\frac{Z_1^2+\ldots+Z_n}{n}}^{-1} \stackrel{\rm a.s.}{\rightarrow 1}
By the Slutsky’s theorem we have:
\frac{Z_0}{\sqrt{\frac{Z_1^2+\ldots+Z_n^2}{n}}} \stackrel{d}{\rightarrow} N(0,1).(解答 (2))
Let n = \nu_1, m = \nu_2. Define (T,U) \stackrel{\rm def}{=}(\frac{X_1}{X_1+X_2}, X_1+X_2). Let g: \mathbb{R}^2 \mapsto \mathbb{R} be an arbitrary bounded continuous function.
Consider E[g(T,U)]:
E[g(T,U)] = E[g(\frac{X_1}{X_1+X_2}, X_1+X_2)] =\int_{x_1, x_2>0} g(\frac{x_1}{x_1+x_2}, x_1+x_2) \frac{x_1^{n/2-1}x_2^{m/2-1}}{\Gamma(n/2)\Gamma(m/2)} \exp(-\frac{x_1+x_2}{2})dx_1dx_2.By a variable transformation, we have:
=\int_{0<t<1, u>0} g(t,u) \frac{(tu)^{n/2-1}(u(1-t))^{m/2-1}}{\Gamma(n/2)\Gamma(m/2)} \exp(-\frac{u}{2})udu.
By a simple rearrangement, we further obtain:
=\int_{0<t<1, u>0} g(t,u) \frac{t^{n/2-1}(1-t)^{m/2-1}}{{\rm Be}(n/2, m/2)} \frac{u^{(n+m)/2-1}}{\Gamma((n+m)/2)} \exp(-\frac{u}{2})du,
which implies that the joint p.d.f of (T,U) is:
\frac{t^{n/2-1}(1-t)^{m/2-1}}{{\rm Be}(n/2, m/2)} \mathbb{1}_{(0,1)}(t) \cdot \frac{u^{(n+m)/2-1}}{\Gamma((n+m)/2)} \exp(-\frac{u}{2}) \mathbb{1}_{(0,\infty)}(u).
So (T,U) are independently follow a Beta distribution and a Chi-square distribution. We conclude that T \sim {\rm Beta}(n/2, m/2).
3. Let U \mid T=t \sim {\rm Unif}(0,t) and suppose that the marginal distribution of T is an exponential family with rate \lambda. Find E[U] and V[U].
(方法) 期望值的部分用雙重期望值原理，變異數的部分利用全變異數的公式。
(解答)
E[U] = EE[U\mid T] = E[T/2] = \frac{1}{2\lambda} .
V[U] = EV[U\mid T] + VE[U\mid T] = E[T^2/12] + V[T/2] = \frac{2}{12\lambda^2} + \frac{1}{4\lambda^2} = \frac{5}{12 \lambda^2}.
4. Let X_1, \ldots, X_n \stackrel{\rm iid}{\sim} N(\mu, \sigma^2) where \mu, \sigma^2 are unknown parameters and let \Phi(\cdot) stand for the CDF of the standard normal distribution. Find the maximum likelihood estimator for \Phi(\frac{x-\mu}{\sigma}) and its asymptotic distribution for a given x \in \mathbb{R}.
(方法) 考慮(\widehat{\mu}_{\rm MLE}, \widehat{\sigma}^2_{\rm MLE})^\top的漸近分布後，考慮一個函數g(y_1,y_2)= \Phi(\frac{x-y_1}{\sqrt{y_2}})以及在y_1=\mu_1, y_2=\sigma^2的泰勒展開（也就是multivariate \delta-method）。
(解答) First, we find the maximum likelihood estimators for (\mu, \sigma^2). The likelihood function is given by:
L(\mu,\sigma^2)=\prod_{i=1}^n \frac{1}{\sqrt{2\pi\sigma^2}} \exp(-\frac{1}{2\sigma^2}(x_i-\mu)^2).So the log-likelihood function is:
\ell(\mu,\sigma^2) = -\frac{n}{2}\ln(2\pi\sigma^2) - \frac{1}{2\sigma^2} \sum_{i=1}^n (x_i-\mu)^2.
For fixed \sigma^2>0, consider the maximization of \ell(\mu,\sigma^2):
\frac{\partial \ell}{\partial \mu} = \frac{n(\bar{x}-\mu)}{\sigma^2},
from which we can see that \ell(\dot, \sigma^2) attain the maximum value at \mu = \bar{x}.
Next, consider the maximization of \ell(\bar{x}, \sigma^2):
\frac{\partial \ell(\bar{x}, \sigma^2)}{\partial \sigma^2} = \frac{n}{2\sigma^2}(\frac{1}{n}\sum_{i=1}^n (x_i-\bar{x})^2 - \sigma^2),
from which we can see that \ell(\bar{x}, \sigma^2) attains the maximum value at \sigma^2 = \frac{1}{n} \sum_{i=1}^n (x_i-\bar{x})^2.
So we conclude that the maximum likelihood estimators for (\mu,\sigma^2) are:
(\widehat{\mu}, \widehat{\sigma}^2) = (\bar{x}, \frac{1}{n}\sum_{i=1}^n(x_i-\bar{x})^2).
Next, we consider the asymptotic distribution for (\widehat{\mu}, \widehat{\sigma}^2).
Note that \sqrt{n}(\widehat{\mu}-\mu) \sim N(0,1), so the asymptotic distribution is also N(0,1).
To consider the asymptotic distribution of \widehat{\sigma}^2, recall that \sum_{i=1}^n(x_i-\bar{x})^2 \sim \sigma^2 \cdot \chi^2_{n-1}. When Y_1, \ldots, Y_{n-1} \stackrel{\rm iid}{\sim} \sigma^2 \cdot \chi_{1}, the distribution of \sum_{i=1}^n(x_i-\bar{x})^2 is identical with that of \sum_{i=1}^{n-1} Y_i^2.
So we can consider the asymptotic distribution of \frac{1}{n} \sum_{i=1}^{n-1} Y_i^2 to find the asymptotic distribution of \widehat{\sigma}^2. By the Central Limit Theorem, we have:
\sqrt{n}(\frac{1}{n}\sum_{i=1}^{n-1} Y^2_i - \sigma^2) \stackrel{d}{\rightarrow} N(0, 2\sigma^4).Therefore, we also have:
\sqrt{n}(\widehat{\sigma}^2 - \sigma^2) \stackrel{d}{\rightarrow} N(0, 2\sigma^4).By the Basu’s Theorem, we also have \widehat{\mu}, \widehat{\sigma}^2 are mutually independent. (One is a complete sufficient statistic for \mu, and the other is ancillary.) So we obtain:
\sqrt{n}(\begin{pmatrix} \widehat{\mu} \\ \widehat{\sigma}^2\end{pmatrix} -\begin{pmatrix} \mu \\ \sigma^2\end{pmatrix}) \stackrel{d}{\rightarrow} N(\begin{pmatrix} 0 \\ 0\end{pmatrix}, \begin{pmatrix}\sigma^2 & 0 \\ 0 & 2\sigma^4 \end{pmatrix})Finally, consider the following fuction and its Taylor expansion at \bm{y}=\bm{\theta} , where \bm{y}\stackrel{\rm def}{=}(y_1,y_2)^\top and \bm{\theta} \stackrel{\rm def}{=}(\mu, \sigma^2)^\top:
g(\bm{y}) \stackrel{\rm def}{=} \Phi(\frac{x-y_1}{\sqrt{y_2}}).5. Let X_1, \ldots, X_n \stackrel{\rm iid}{\sim} f(x\mid \theta) where f(x\mid \theta) = \theta x^{\theta-1} \mathbb{1}_{(0,1)}(x). Find the UMVUE for \theta and its asymptotic distribution.
(方法)
(解答)
6. Let X_1, \ldots, X_n \stackrel{\rm iid}{\sim} N(\mu, \sigma^2) be a random sample where \mu, \sigma^2 are unknown parameters. Find a size \alpha unbiased test for the hypotheses H_0: \theta \in [\theta_1, \theta_2] vs H_1: \theta \notin [\theta_1, \theta_2].
(方法)
(解答)
7. Let X_1, \ldots, X_n \stackrel{\rm iid}{\sim} N(\mu, \sigma^2) be a random sample where \mu, \sigma^2 are unknown parameters. Derive the power function of the size \alpha likelihood ratio test for the hypotheses H_0: \theta \leq \theta_0 vs H_1: \theta > \theta_0.
(方法)
(解答)