An official website of the United States government
United States Department of Labor
United States Department of Labor
The .gov means it's official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information,
make sure you're on a federal government site.
The site is secure.
The
https:// ensures that you are connecting to the official website and that any
information you provide is encrypted and transmitted securely.
Office of Survey Methods Research
SHARE ON:
More Information
Published Year: 2010
Survey/Program(s): General Methodology
Catalog: Statistical Survey Paper
Download PDF
Some Thoughts on the Use of Field Tests to Evaluate Survey Questionnaires
Abstract
This paper focuses on field-based evaluation work that incorporates multiple evaluation methods. Field tests are complex, resource-intensive, collaborative operations that draw upon the knowledge/information/data and skills possessed by various sources/agents (e.g., content and design specialists; interviewers and other field staff; respondents; statisticians) to optimize questionnaire design for the purpose of gathering high-quality data about a particular domain-of-interest. Because field tests represent evaluation work that occurs during specific phases of the questionnaire-design-and-evaluation process, the paper begins with a brief overview of this process. This discussion will be limited to a cursory description of a framework that the author has found useful for situating field tests within the broader context of longitudinal (and potentially reiterative) design-and-evaluation work. With the framework as context, the author attempts to categorize some of the methods and techniques that survey practitioners have at their disposal for evaluating questionnaires at various phases of the design-and-evaluation process. At that point, the author moves from the more abstract discussion of frameworks and methods to a more pragmatic discussion of field-test methodology in real-world settings and does so by reconstructing what transpired in the course of an evaluation of a supplement questionnaire that involved a series of three separate field tests. In the process, the author provides examples of method-generated qualitative and quantitative data, reviews how such data were analyzed and integrated, and offers some thoughts as to the utility of the various methods used. In the final section of the paper, some closing thoughts on the collaborative nature field-test methodology are provided.