Some local diagnostics for the Fay-Herriot model


Diagnostics and validation

Jean-François Beaumont (Statistics Canada)
Éric Lesage (INSEE) (Speaker)

The increasing need of information leads to the production of survey estimates for domains (e.g., regions) of various sizes. Thus, within the same survey, domain sample sizes can range from a couple of units to more than thousand units. As direct estimators (Horvitz-Thompson or calibration estimators) suffer from a lack of precision in small domains, it may be desirable to use SAE estimators such as the composite estimator proposed by Fay and Herriot (1979).

There exist tools to assess if the composite estimator is globally better than the direct estimator. For instance, standard model diagnostics can be used to verify the validity of the underlying Fay-Herriot model. However, we are not aware of local diagnostics that could be used to determine, domain by domain, which estimator is preferable. The model-based mean square error can be viewed to some extent as a local diagnostic but it relies on the assumption that a model holds for all the domains.

Yet, some users are concerned about a specific domain and are not really interested in a global criterion to assess the quality of the composite estimator for their single domain. Those users would be eager for inferences made conditionally on the domain parameters of interest, such as design-based inferences, as opposed to model-based inferences.

We have adopted this conditional approach to produce local diagnostics, for each area, that give an indication of whether or not the composite estimator is likely to be more precise than the direct estimator. We have found that, depending on the domain sample size and the magnitude of the standardized model residual, it is possible to detect when the composite estimator is expected to have a smaller design mean square error than the direct estimator.

We will present the results of a simulation study, based on the Canadian Labor Force Survey data, that illustrate the effectiveness of our diagnostics.

Fay, R.E. and Herriot, R.A. (1979). Estimation of Income from Small Places: An Application of James–Stein Procedures to Census Data, Journal of the American Statistical Association, 74, 269–277.
Keywords: Fay-Herriot model; Small area estimation; design based approach; local diagnostic

Download presentation.