Monthly Employment Situation Report: Quick Guide to Methods and Measurement Issues
On this page, you can quickly locate BLS documentation about the methods underlying the numbers that appear each month in the Employment Situation news release. This page also provides links to related information pertaining to measurement issues; this includes information on various topics such as the birth/death model used by the payroll survey, the population controls used by the household survey, and comparisons between the two surveys. Also see Frequently Asked Questions about Employment and Unemployment Estimates.
Payroll survey — methods and measurement issues
The Current Employment Statistics (CES) program, also known as the payroll survey or the establishment survey, is a monthly survey of approximately 146,000 businesses and government agencies representing approximately 623,000 worksites throughout the United States. From the sample, CES produces and publishes employment, hours, and earnings estimates for the nation, states, and metropolitan areas at detailed industry levels.
Series covering all employees’ hours and earnings were officially added by CES on February 5, 2010, with estimates beginning in March 2006. Historically, CES hours and earnings series covered only production and nonsupervisory employees.
The CES employment series are estimates of nonfarm wage and salary jobs, not an estimate of employed persons; an individual with two jobs is counted twice by the payroll survey. The CES employment series excludes employees in agriculture, private households, and the self-employed.
For more information, see the Concepts section of Chapter 2 of the BLS Handbook of Methods at www.bls.gov/opub/hom/pdf/homch2.pdf; this section includes definitions of the types of data available from the survey. More information is also available under Available Data in the CES Technical Notes at www.bls.gov/web/empsit/cestn.htm#section4.
A wide array of public and private policy makers use CES data because it is one of the earliest indicators of economic conditions each month. Major users of CES data include many government agencies and entities, financial markets in the United States and around the world, and other business and academic analysts, researchers, and forecasters.
For more information, see the Uses section of Chapter 2 of the BLS Handbook of Methods at www.bls.gov/opub/hom/pdf/homch2.pdf.
The current CES sample design has been in place since 2003 and follows state-of-the-art design principles for an establishment survey; it was developed in consultation with experts in survey design from universities and other leading statistical agencies. The entire sample is redrawn annually, and a supplemental sample of new business births is selected midway through the year. About one-fourth of the sample is rotated out each year and replaced with newly selected businesses.
For more information, see the Sample Design section of Chapter 2 of the BLS Handbook of Methods at www.bls.gov/opub/hom/pdf/homch2.pdf. More information is also available under The Sample in the CES Technical Notes at www.bls.gov/web/empsit/cestn.htm#section1.
All new sample is solicited by computer-assisted telephone interview (CATI), and data are collected for the first 5 months via this mode. After the initiation period, many sample units are transferred to one of several less costly reporting methods that are self-initiated by the respondent. The CES offers responding businesses a choice of reporting modes in an effort to maximize response rates within the program budget. Respondents can report via Fax, Web, Touchtone Data Entry (TDE), Electronic Data Interchange (EDI), or CATI.
For more information, see the Sample Data section of Chapter 2 of the BLS Handbook of Methods at www.bls.gov/opub/hom/pdf/homch2.pdf. More information is also available under Data Collection in the CES Technical Notes at www.bls.gov/web/empsit/cestn.htm#section2. Also, the latest CES revisions and sample collection rates are available in the CES employment revisions and sample collection rates, detailed tables, and technical information at www.bls.gov/ces/cesrevinfo.htm.
CES monthly employment estimates are made using a 2-part estimator. The sample reports are used to estimate month-to-month employment change from continuing businesses and a birth/death model is used to account for new firm births that otherwise would not be sampled in a timely fashion.
Sample-based estimator. CES uses a matched sample concept and weighted link relative estimator to produce employment, hours, and earnings estimates. For more information on these methods and the CES sample-based estimator, see the Estimation Methods section of the CES technical notes at www.bls.gov/web/empsit/cestn.htm#section5.
Birth/death model. The sample alone is not sufficient to estimate a total employment level because each month new firm births generate employment growth, and there is an unavoidable lag between an establishment opening for business, appearance on the sample frame, and availability for inclusion. To account for these components of total employment, CES uses a net business birth/death model.
Technical information on the estimation methods used to account for employment in business births and deaths is available under Birth/Death section in the CES Technical Notes at www.bls.gov/web/empsit/cestn.htm#section5c. For the most recent monthly total nonfarm birth/death adjustments, see CES Net Birth/Death Model. For historical birth/death adjustments, see Historical Net Birth/Death Adjustments available at www.bls.gov/web/empsit/cesbdhst.htm. For additional information, see CES Birth/Death Model Frequently Asked Questions at www.bls.gov/web/empsit/cesbdqa.htm.
Estimate review. CES uses automated edit and screening techniques to identify potentially erroneous
sample data; respondents are re-contacted as needed to validate or correct their reported information. After the microdata edit
process is complete, monthly estimates are calculated. Automated edits of the estimates are supplemented by analysts who look
for errors and outliers and provide final validation of the series before publication.
More information can be found in the Estimating Procedures section of Chapter 2 of the BLS Handbook of Methods at www.bls.gov/opub/opub/hom/pdf/homch2.pdf.
The seasonal adjustment process removes from the series the effects of normal variation from recurring events within a year, such as holidays and weather changes, and helps reveal underlying economic trends. CES uses a concurrent seasonal adjustment methodology, meaning that it incorporates estimates up through and including the current month’s data to achieve the best possible series.
Technical details and related research on the CES implementation of concurrent seasonal adjustment, including input files used during the latest production run can be found on Seasonal Adjustment Files and Documentation available at www.bls.gov/web/empsit/cesseasadj.htm.
CES first preliminary estimates of employment, hours, and earnings are published each month approximately 3 weeks after the reference period. Estimates are then revised twice before being held constant until the annual benchmarking process. Second preliminary estimates for a given month are published the month following the initial release, and final sample-based estimates are published 2 months after the initial release.
For more details on revision, see the Revisions section of the the CES Technical Notes available at www.bls.gov/web/empsit/cestn.htm#section6. See www.bls.gov/web/empsit/cesnaicsrev.htm for a table of revisions to seasonally adjusted total nonfarm over-the-month changes from January 1979 forward.
Sampling Errors (Variances)
The establishment survey, like other sample surveys, is subject to two types of error, sampling and nonsampling error. The magnitude of sampling error, or variance, is directly related to the size of the sample and the percentage of universe coverage achieved by the sample. The establishment survey sample covers over one-third of total universe employment; this yields a relatively small variance on the total nonfarm estimates.
For more information, see the Reliability of Estimates section of Chapter 2 of the BLS Handbook of Methods at www.bls.gov/opub/hom/pdf/homch2.pdf. Also, for the most recent information on error and relative standard error, please see the Reliability section of the CES technical notes www.bls.gov/web/empsit/cestn.htm#section1c.
Unlike most sample surveys that publish sampling error as their only measure of error, CES can derive an annual approximation of total error, on a lagged basis, because of the availability of the independently derived universe data. On an annual basis BLS recalculates nearly 2 years of estimates in a process known as benchmarking. The benchmark process helps correct for sampling and modeling error in the CES estimates. Historically, benchmark revisions have been very small for total nonfarm employment.
The benchmark process re-anchors sample-based estimates to a nearly complete count of employment based primarily on Unemployment Insurance (UI) tax records for March of each year. For example, when BLS published the March 2009 benchmark revision with the Employment Situation news release in February 2010, the employment series was revised from April 2008 through December 2009 on a not seasonally adjusted basis. Five years of seasonally adjusted data are re-calculated and replaced with each benchmark revision.
For more information, see the Benchmark Data section of Chapter 2 of the BLS Handbook of Methods at www.bls.gov/opub/hom/pdf/homch2.pdf. More information is also available under Benchmarks in the CES Technical Notes at www.bls.gov/web/empsit/cestn.htm#section6b. Details on the most recent CES benchmarks and corresponding detailed data tables may be found in the annual CES Benchmark Article available at www.bls.gov/web/empsit/cesbmart.htm.
Household survey — methods and measurement issues
The Current Population Survey (CPS), frequently referred to as the household survey, is a monthly survey of about 60,000 households. The CPS collects information about the civilian noninstitutional population. All persons in the civilian noninstitutional population age 15 and over are classified as employed, unemployed, or not in the labor force, although labor force estimates are published only for those 16 and older.
The CPS estimates are available by various demographic characteristics, including sex, race, Hispanic or Latino ethnicity, age, and educational attainment. The CPS estimates are also available by industry and by occupation. Numerous cross-tabulations of labor force variables by demographic characteristics are available. The CPS also uses supplemental surveys to collect data on various subjects of interest, such as the working poor, volunteering, and worker displacement.
The CPS estimate of employment is for the total number of employed persons. Included are categories of workers that are not covered by the Current Employment Statistics (CES) survey: self-employed persons, private household workers, agriculture workers, unpaid family workers, and workers on leave without pay during the reference period. Multiple jobholders are counted once in the estimate of total employed.
Unemployed persons include those who did not have a job during the reference week, had actively looked for work in the prior 4 weeks, and were available for work. Actively looking for work includes activities such as contacting a possible employer, contacting an employment agency or employment center, having a job interview, sending out resumes, filling out job applications, placing or answering job advertisements, and checking union or professional registers.
The number of persons in the labor force equals the number of employed persons plus the number of unemployed persons. The labor force participation rate represents the proportion of the civilian noninstitutional population who are in the labor force. The unemployment rate is the proportion of the labor force that is unemployed. The employment-population ratio represents the proportion of the civilian noninstitutional population that is employed.
The not in the labor force group includes all persons in the civilian noninstitutional population who are neither employed nor unemployed. The CPS also collects information on those not in the labor force who say they want a job, including those who have stopped looking for work because they believe no jobs are available (discouraged workers).
For further information, see the Concepts section of Chapter 1 of the BLS Handbook of Methods at www.bls.gov/opub/hom/pdf/homch1.pdf; technical notes at www.bls.gov/cps/eetech_methods.pdf; and Chapter 5 of Current Population Survey Design and Methodology, Technical Paper 66, on the Census Bureau website at www.census.gov/prod/2006pubs/tp-66.pdf (PDF size: 3.2 MB).
The CPS data serve as an important and early economic indicator and are the primary source of data on employment status and demographic characteristics of the labor force, emerging trends, and changes. The CPS also provides data on wages that can be used to evaluate the wage rates and earnings trends for demographic groups. CPS data users include businesses, various government entities, academics and students, the media, the general public, international organizations, and policy think tanks.
For further information, see the Presentation and Uses section of Chapter 1 of the BLS Handbook of Methods at www.bls.gov/opub/hom/pdf/homch1.pdf.
The CPS sample is a probability sample of approximately 72,000 assigned housing units from 824 sample areas designed to measure demographic and labor force characteristics of the civilian noninstitutional population. The housing units come from lists of addresses obtained from the 2000 Decennial Census of Population and Housing. The CPS sample consists of independent samples in each state and the District of Columbia. The sample is updated continuously for new housing built after Census 2000.
For further information, see the Sampling section of Chapter 1 of the BLS Handbook of Methods at www.bls.gov/opub/hom/pdf/homch1.pdf; technical notes at www.bls.gov/cps/eetech_methods.pdf; and Chapters 3 and 4 of Current Population Survey Design and Methodology, Technical Paper 66, on the Census Bureau website at www.census.gov/prod/2006pubs/tp-66.pdf (PDF size: 3.2 MB).
The survey is conducted by the U.S. Census Bureau for the Bureau of Labor Statistics. Each month, during the calendar week including the 19th day, Census interviewers contact households by telephone and in person and ask questions regarding the labor market activity of household members during the previous calendar week which included the 12th day of the month—the reference week. Personal visits are preferred in the first month in which the household is in the sample. At the first visit, interviewers prepare a roster of the household members including their demographic characteristics and their relationship to the person maintaining the household, and enter the information via laptop computers, along with responses to all survey questions.
In the months following the first interview, the interview is generally conducted by telephone. The household roster is checked for accuracy and brought up to date in each interview. About 10 percent of households are interviewed via computer-assisted telephone interviewing (CATI) by staff in three centralized calling centers. Other telephone interviews are collected by field representatives. A personal visit is generally attempted for the fifth interview. At the end of each day’s interviewing, the data are transmitted over secure telecommunications lines to the Census Bureau’s central computer in Washington, DC.
For further information, see the Collection Methods section of Chapter 1 of the BLS Handbook of Methods at www.bls.gov/opub/hom/pdf/homch1.pdf; and Chapters 6,7 and 8 of Current Population Survey Design and Methodology, Technical Paper 66, on the Census Bureau website at www.census.gov/prod/2006pubs/tp-66.pdf (PDF size: 3.2 MB).
Under the estimating methods used in the CPS, all of the results for a given month become available simultaneously and are based on returns from all respondents. The estimation procedure involves weighting the data from each sample person by the inverse of the probability of the person being in the sample. This gives a rough measure of the number of actual persons that the sample person represents. In the next step, an adjustment is made to account for nonresponse. Then, ratio estimation is done to bring the sample population distribution as closely into agreement as possible with the known distribution of the entire population.
For further information, see the Estimation Methods section of Chapter 1 of the BLS Handbook of Methods at www.bls.gov/opub/hom/pdf/homch1.pdf; and Chapter 10 of Current Population Survey Design and Methodology, Technical Paper 66, on the Census Bureau website at www.census.gov/prod/2006pubs/tp-66.pdf (PDF size: 3.2 MB).
The process of seasonal adjustment removes from the series the effects of normal seasonal variations, such as holidays, school openings and closings, and weather. This makes it easier for data users to observe fundamental changes in the levels of the series, particularly those associated with general economic expansions and contractions. The seasonal adjustment procedure that the CPS uses is X-12-ARIMA, which is based on moving averages or “filters” that successively average a shifting timespan of data, which then provides estimates of seasonal factors that change smoothly from one year to the next year. The seasonal adjustment is done concurrently in that the program is run monthly as new data come available. At year’s end, the seasonal factors are re-estimated and the most recent 5 years of data are subject to revision.
For further information, see the Seasonal Adjustment section of Chapter 1 of the BLS Handbook of Methods at www.bls.gov/opub/hom/pdf/homch1.pdf; technical notes at www.bls.gov/cps/documentation.htm#sa; and Chapter 10 of Current Population Survey Design and Methodology, Technical Paper 66, on the Census Bureau website at www.census.gov/prod/2006pubs/tp-66.pdf (PDF size: 3.2 MB)
Sampling Errors (Variances)
The coefficient of variation (CV) is 1.9 percent on national monthly estimates of employment level from the CPS, which translates into a change of 0.2 percentage point in the unemployment rate being significant at the 90-percent confidence level. Because the CPS has a much smaller sample than the CES, its margin of error on the measurement of month-to-month employment change is much larger. For a monthly change in CPS employment to be significant, it must be about plus or minus 436,000.
For further information, see the Sampling section of Chapter 1 of the BLS Handbook of Methods at www.bls.gov/opub/hom/pdf/homch1.pdf; technical notes at www.bls.gov/cps/eetech_methods.pdf; and Chapter 14 of Current Population Survey Design and Methodology, Technical Paper 66, on the Census Bureau website at www.census.gov/prod/2006pubs/tp-66.pdf (PDF size: 3.2 MB).
With the release of January data in February, CPS data are adjusted to reflect updated population controls developed by the U.S. Census Bureau. Every 10 years, the population estimates are adjusted to reflect new information from the decennial census. Updates based on the last decennial census normally occur a few years after the census, and after the last two censuses (1990 and 2000), BLS revised CPS time series data back to the census reference year.
Annual adjustments are made due to intercensal population control estimated changes. The intercensal adjustments normally reflect estimated changes in net international migration and updated vital statistics information. These adjustments are usually small, and data are not revised back due to these intercensal adjustments. The CPS program does provide separate smoothed series for labor force and for total employment that take into account level shifts due to population control adjustments. These smoothed series are for research purposes only and do not match the official published estimates.
For further information, see technical notes at www.bls.gov/cps/eetech_methods.pdf and www.bls.gov/cps/documentation.htm#pop; Labor force and employment estimates smoothed for population adjustments, at www.bls.gov/cps/cpspopsm.pdf; and Appendix C of Current Population Survey Design and Methodology, Technical Paper 66, on the Census Bureau website at www.census.gov/prod/2006pubs/tp-66.pdf (PDF size: 3.2 MB).
Comparisons between the payroll survey and household survey
A report comparing the Current Employment Statistics survey (“payroll survey”) and the Current Population Survey (“household survey”) is on the BLS website at www.bls.gov/web/empsit/ces_cps_trends.pdf. The report, titled Employment from the BLS household and payroll surveys: summary of recent trends, is updated every month when the Employment Situation news release is issued. The report includes the following:
- Summary comparison of household and payroll survey concepts, definitions, and methodologies;
- Employment trends as measured by the payroll and household surveys;
- Possible causes of differences in employment trends; and
- Summary of recent changes made to each survey.
Last Modified Date: May 5, 2016