Problem 1 (5 points)
Consider the probabilistic graphical model below:
X₁ → X₂ → X₃ → X₄ → X₅
↓ ↓ ↓ ↓ ↓
X₆ X₇ X₈ X₉ X₁₀(i) (1 point)
Write down a full factorization of p(X_{1:10}) implied by the graphical model. Your factorization should be as simple as possible (where simplicity is measured by the number of X_\star terms that show up in your final expression).
The Markov boundary of a variable is the union of its parents, its children, and the parents of children. If a variable is conditioned on its Markov boundary, it becomes independent of all other random variables.
(ii) (1 point)
What is the Markov boundary of X_7?
(iii) (1 point)
What is the Markov boundary of X_3?
(iv) (2 points)
Write down a complete factorization of p(X_{1:2},X_{4:10}|X_3) that is as simple as possible.
Problem 2 (13 points)
Consider the two following MA(4) processes:
\begin{align} X_t &= W_t + \theta_3 W_{t-3} + \theta_4 W_{t-4} + \theta_c \\ Y_t &= W_t + \theta_1 W_{t-1} + \theta_4 W_{t-4} \end{align}
where W_t is drawn from \mathcal{N}(0, \sigma_W^2) and all the \theta_\star are constants.
(i) (2 points)
What is the mean, \mu_X(t), of the \{X_t\} process? Justify your answer.
(ii) (3 points)
What is the covariance, \gamma_X(t,s), of the \{X_t\} process?
(iii) (1 point)
Is \{X_t\} drawn from a weakly stationary process?
(iv) (5 points)
What is the cross-covariance, \gamma_{X,Y}(t,s), between X_t and Y_s?
(v) (2 points)
Is it possible that \gamma_{X}(t,t) = 0? If so, what is one setting of all of the \theta_\star for which this is true? (Limit yourself to the real numbers.)
Problem 3 (10 points)
Consider the following two models:
\begin{align} X_t &= 2.5X_{t-1} - X_{t-2} + W_t - 2 W_{t-1} \\ Y_t &= 0.7Y_{t-1} + 0.3Y_{t-2} + W_t - 0.4 W_{t-1} \end{align}
where W_t is drawn from \mathcal{N}(0, \sigma_W^2).
(i) (3 points)
Identify \{X_t\} as ARMA(p,q). Watch out for parameter redundancy.
(ii) (1 point)
Is \{X_t\} causal? Justify your answer.
(iii) (1 point)
Is \{X_t\} invertible? Justify your answer.
(iv) (3 points)
Identify \{Y_t\} as ARMA(p,q).
(v) (1 point)
Is \{Y_t\} causal? Justify your answer.
(vi) (1 point)
Is \{Y_t\} invertible? Justify your answer.
Problem 4 (7 points)
Consider an AR(2) process with the equations:
\begin{align} P(B) = (1-0.2B)(1+0.2B) \end{align}
Please answer the following questions:
(i) (1 point)
Is the process causal?
(ii) (6 points)
What is the correlation function \rho(t,t+h)=\rho(h)? (Hint: remember that \rho(0) = 1.)