題目：https://exam.lib.ntu.edu.tw/sites/default/files/exam//graduate/104/106_104graduate.pdf
1 . Let X \sim {\rm NB}(r,p) (Negative Binomial). State the conditions such that X converges to a Possion distribution. And prove it.
(方法) 負二項分布是在看到第r次成功之前發生的失敗次數之分布。(r=1時被稱為幾何分布。但幾何分布有兩種定義。其中一種是失敗次數，而另一種定義則是總共試驗次數。)
泊松分配可以想成是發生機率很低的事件次數。由於負二項分布是失敗次數的分布，所以我們可以猜想答案應該會是失敗這個事件的機率很低的情況，也就是p \rightarrow 1。
另外，期望值需要是E[X]=\frac{rq}{p}~(q=1-p) = \lambda (\lambda: 固定)。(但p趨近於1的過程中，那些p使得r為自然數) 最後，依分布收斂等價條件是特徵函數或動差生成函數的收斂。負二項分配有動差生成函數，為了避免討論複數的極限(但相信跟實數的情況一樣)，以下採用動差生成函數。
(解答) Consider a stochastic process \{X_p\}_p with p \rightarrow 1 where X_p \sim {\rm NB}(r, p) and r \in \mathbb{N} is also a function of p satisfying E[X_p] = \frac{rq}{p} = \lambda where \lambda is a given positive number.
Consider the characteristic function of X_p:
M_p(t) \stackrel{\rm def}{=} E[e^{tX_p}]
= E[e^{tY_p}]^r
= \left\{\sum_{y=0}^\infty e^{ty} pq^{y}\right\}^r
= \left\{\sum_{y=0}^\infty p (qe^{t})^y\right\}^r
= \frac{p^r}{(1-qe^{t})^r},
where Y_p \sim {\rm Geo}(p). (This is because when Y_1, \ldots, Y_r \stackrel{\rm iid}{\sim} {\rm Geo}(p), we have \sum_{i=1}^r Y_i \sim {\rm NB}(r,p).)
For conveniene, consider the cumulant generating function:
K_p(t) \stackrel{\rm def}{=} \ln M_p(t) = \lambda p \left\{ \frac{\ln p}{1-p} - \frac{\ln(1-(1-p)e^t)}{1-p}\right\}Using the L’hopital’s rule, we have:
\lim_{p \rightarrow 1} K_p(t)
= \lim_{p \rightarrow 1} \lambda p \left\{ \frac{\ln p}{1-p} - \frac{\ln(1-(1-p)e^t)}{1-p}\right\},
equals to
= \lim_{p \rightarrow 1} \lambda p \left\{ \frac{-1}{p} - \frac{e^t}{1-(1-p)e^t}\right\} = \lambda (-1-e^t),thus, we have
\lim_{p \rightarrow 1} M_p(t) = e^{-\lambda} \exp(-\lambda e^t).(Note that p \rightarrow 1 only goes through p‘s such that r‘s are natural numbers.)
By the Levy’s continuity theorem, such a stochastic process converges to a Poisson distribution.
2 . Let X_1, \ldots, X_n follow a distribution with a probability density function f(x \mid \theta) = \theta x^{\theta-1} \mathbb{1}_{(0,1)}(x). Show that the variance of the UMVUE for \theta can not attain the Cramer-Rao’s lower bound.
test
3 . Let X \mid Y=y\sim {\rm Bin}(y,p), Y \mid \Lambda=\lambda \sim {\rm Po}(\lambda) and \Lambda \sim {\rm exp}(\beta) where \beta is known. Find V[X].
(方法) 雙重期望值原理以及全變異數公式。
(解答) First, note that
V[X] = EV[X|Y] + VE[X|Y] = E[p(1-p)Y] + V[pY] = p(1-p)E[Y]+p^2V[Y].Second, consider E[Y], V[Y]:
E[Y] = EE[Y \mid \Lambda] = E[\Lambda] = \frac{1}{\beta},
and
V[Y] = EV[Y \mid \Lambda] + VE[Y \mid \Lambda] = E[\Lambda] + V[\Lambda] =\frac{\beta+1}{\beta^2} .
4 . Let X, Y be indendent random variables with absolutely continuous distributions. Let the CDFs be F_X, F_Y. Derive the distribution of Y conditioning on X-Y=0
(方法) (我們假設X, Y為相互獨立) 首先考慮 (Y, X-Y)的聯合分布。有了聯合分布，我們就可以算條件分布。
(解答) Recall that a distribution of (X,Y) is characterized by an expected value of g(X,Y) where g(\cdot) is an arbitrary bounded continuous function. We would like to first find the joint distributon of (Y,X-Y).
Let W \stackrel{\rm def}{=} Y, \quad Z \stackrel{\rm def}{=} X-Y . Note that that X = W+Z, \quad Y=W.
Consider the expected value of g(W,Z) where g is an arbitrary bounded continuous function.
E[g(W,Z)] = E[g(Y,X-Y)] = \int_{\mathbb{R}^2} g(y,x-y) f_X(x)f_Y(y)dxdyBy a variable transformation, the above integral is written by
E[g(W,Z)] \stackrel{\rm eq}{=} \int_{\mathbb{R}^2} g(w,z) f_X(w+z)f_Y(w)dwdz,which implies that the joint probability density function of (W,Z) is
f_{WZ}(w,z) \stackrel{\rm eq}{=} f_X(w+z)f_Y(w).Therefore the marginal distribution of Z is
f_{Z}(z) \stackrel{\rm eq}{=} \int_{\mathbb{R}}f_X(w+z)f_Y(w)dw.The conditional probability density funcion W \mid Z is
f_{W\mid Z}(w\mid z) \stackrel{\rm eq}{=} \frac{f_X(w+z)f_Y(w)}{\int_{\mathbb{R}}f_X(w+z)f_Y(w)dw}.Finally, put w \leftarrow y, z \leftarrow 0 and we will obtain the desired distribution.
5 . Let X_1, \ldots, X_n \stackrel{\rm iid}{\sim}N(\mu, \sigma^2). Find a (Borel meaurable) function g(\cdot) such that E[g(S_n^2)] = \sigma^p when n+p>1.
(方法) 利用T_n = \frac{1}{\sigma^2} \sum_{i=1}^n (X_i-\bar{X})^2 \sim \chi^2_{n-1} 以及考慮E[T_n^{p/2}].
(解答)
6 . State and prove the Neyman-Pearson’s lemma.
(解答) Consider simple hypotheses H_0: \theta = \theta_0 vs H_1: \theta = \theta_1. Assuem that \bm{x} \sim f(\bm{x} \mid \theta) where f(\cdot\mid\theta) is a probability density function or a probability mass function. The most powerful test for the above hypotheses is given by
\varphi(\bm{x}) \stackrel{\rm def}{=}
\mathbb{1}\left(\{\frac{f(\bm{x}\mid \theta_1)}{f(\bm{x}\mid\theta_0)} > k\}\right)+ \gamma \mathbb{1}\left(\{\frac{f(\bm{x}\mid \theta_1)}{f(\bm{x}\mid\theta_0)} = k\}\right),
where k and 0 \leq \gamma <1 are chosen so that
E_{\theta_0}[\varphi(\bm{x})] = \alpha.
Let \varphi^*(\bm{x}) be an arbitrary size \alpha
test. Define a probability measure
\mu_\theta(B) \stackrel{\rm def}{=} P(X \in B \mid \theta)
=
\begin{cases}
\int_B f(x\mid\theta)dx & (p.d.f) \\
\sum_{x \in B} f(x \mid \theta) & (p.m.f)
\end{cases}
Consider the following equation:
\int_{\mathbb{R}} (\varphi(\bm{x}) - \varphi^*(\bm{x})) (\mu_{\theta_1} - k\mu_{\theta_0}) (dx),
which can be rewritten as:
=
\begin{cases}
\int (\varphi(\bm{x}) - \varphi^*(\bm{x})) (f(\bm{x}\mid\theta_1) - kf(\bm{x}\mid\theta_0))dx & (p.d.f.) \\
\sum_x (\varphi(\bm{x}) - \varphi^*(\bm{x})) (f(\bm{x}\mid\theta_1) - kf(\bm{x}\mid\theta_0))dx & (p.m.f.)
\end{cases}
7 . Show that a uniformly most powerful test is an unbiased test.
(方法) 跟隨機化檢定的檢定力做比較。
(解答) Assuem that the hypotheses are H_0: \theta \in \Theta_0 vs H_1: \theta \in \Theta_1. Let \varphi(\bm{x}) be a size \alpha UMP test for the given hypotheses.
Given \alpha, define \varphi^*(\bm{x}) = \alpha, a size \alpha randomized test.
Since \varphi(\bm{x}) is a UMP test, the power function \beta(\theta) = E_\theta[\varphi(\bm{x})] of the UMP test is greater than or equal to that of the randomized test \beta^*(\theta) = E_\theta[\varphi^*(\bm{x})] = \alpha for all \theta \in \Theta_1.
\mathrm{i.e.,} ~ \beta(\theta) \geq \alpha, \quad \forall \theta \in \Theta_1.By the definition of size of the test, \alpha \stackrel{\rm def}{=} \sup_{\theta \in \Theta_0} \beta(\theta).
Therefore, for any \theta \in \Theta_1, we have \beta(\theta)\geq \alpha = \sup_{\theta \in \Theta_0} \beta(\theta).
So we have
\inf_{\theta \in \Theta_1} \beta(\theta)\geq \sup_{\theta \in \Theta_0} \beta(\theta),
thus \phi(\bm{x}) is an unbiased test.