---
title: "IVX-AR: serially correlated errors"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{IVX-AR: serially correlated errors}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
```

```{r setup, message = FALSE}
library(ivx)
```

`ivx_ar()` implements the IVX-AR procedure of Yang, Long, Peng & Cai (2020),
which extends the Kostakis et al. (2015) test to predictive regressions whose
errors are serially correlated — the situation the authors document for
housing price index returns.

## Model

$$
y_t = \mu + \beta' x_{t-1} + u_t, \qquad
u_t = \sum_{j=1}^{q} \phi_j u_{t-j} + \varepsilon_t,
$$

with persistent predictors as in `vignette("ivx")`. Serial correlation in
$u_t$ invalidates the IVX Wald statistic's $\chi^2$ limit (and the
Newey–West correction inside it is designed for the predictor's innovations,
not for $u_t$). Applying the AR filter $\phi(L) = 1 - \sum_j \phi_j L^j$ to
both sides,

$$
\phi(L)\, y_t = \phi(1)\mu + \beta' \phi(L)\, x_{t-1} + \varepsilon_t,
$$

gives a predictive regression with white-noise errors and the same slope
$\beta$, to which IVX applies. Since $\phi$ is unknown and its estimate from
the residuals is biased under persistence, the paper profiles over it.

## Procedure

1. Fit the plain IVX regression and fit an AR($q$) model to its OLS
   residuals, choosing $q$ by an information criterion (`ar = "auto"`,
   `ar_ic`, `ar_max`) or fixing it (`ar = q`).
2. For every point $\phi$ of a grid around the AR estimates (`ar_grid`,
   default $\pm 0.3$ in steps of 0.02 per coefficient), quasi-difference
   $y_t$ and $x_{t-1}$ with $\phi(L)$ and refit IVX.
3. Keep the grid point with the smallest residual variance; report that
   fit's IVX Wald statistics.
4. Additionally test $H_0: \phi_1 = \dots = \phi_q = 0$ with the Wald
   statistic `Wald_AR` (see `ac_test_wald()`); `ar = 0` reduces to `ivx()`.

```{r}
m <- ivx_ar(hpi ~ log(res) + cpi, data = ylpc)
m
summary(m)
```

The estimated AR coefficients of the selected grid point are in
`m$coefficients_ar`, and the automatic order selection can be replaced:

```{r}
m$coefficients_ar
coef(ivx_ar(hpi ~ log(res) + cpi, data = ylpc, ar = 1))
```

The `ylpc` dataset is the authors' quarterly US housing data.

## Caveats

* The grid search is over $q$ coefficients with the default 31 points each,
  so cost grows as $31^q$; keep `ar_max` small or fix `ar`.
* `ar = "forecast"` delegates order selection to `forecast::auto.arima()`
  and needs that package.
* The tuning and standard-error options of `ivx()` (`beta`, `cz`,
  `bandwidth`, `robust`) are passed through to every grid refit.
* `ivx_boot()` does not accept `ivx_ar` objects: the bootstrap schemes are
  defined for the untransformed regression.
* When the residuals show no serial correlation (`ac_test()` on an `ivx` fit
  is a quick check) `ivx()` is the better choice; the profiling adds noise.

## References

* Yang, B., Long, W., Peng, L., & Cai, Z. (2020). Testing the predictability
  of US housing price index returns based on an IVX-AR model. *Journal of the
  American Statistical Association*, 115(532), 1598–1619.
* Kostakis, A., Magdalinos, T., & Stamatogiannis, M. P. (2015). Robust
  econometric inference for stock return predictability. *Review of Financial
  Studies*, 28(5), 1506–1553.