On January 14, 2014, the following corrections were made to this article:
- Fifth paragraph, first sentence: the number of in-scope units was corrected to 24,000
- “New Housing Survey samples” section, second paragraph, third sentence: the first stage was corrected to indicate a 2-year sample augmentation
The Consumer Price Index (CPI) Housing Survey provides the data needed to measure price change for the two housing component indexes: Owners’ equivalent rent of primary residence (OER) and Rent of primary residence (Rent). These CPI components are the largest with 22.6 and 6.5 percent, respectively, of the CPI market basket as of December 2012. The Housing Survey follows the rents of a sample of renter-occupied housing units selected to represent both renter- and owner-occupied housing units in the urban United States. The CPI has 87 geographic pricing areas selected to represent the urban United States, and the Bureau of Labor Statistics (BLS) field staff agents based in those areas regularly collect data for the units of the Housing Survey sample.
Until 2002, the Shelter Index, another component of the CPI, was revised as part of a periodic roughly 10‑year cycle during which all dimensions of the CPI (expenditure patterns, items and stores, rented housing units, urban areas for collection, and computation and collection methods) were revised. The last comprehensive CPI revision occurred in 1998. Beginning in 2002, the BLS replaced these comprehensive periodic revisions and its associated large resource spikes with shorter, more continuous updating. For example, expenditure patterns are now updated every 2 years, items and the sample of outlets priced are updated over a 4-year period, and computer systems and collection methods are regularly maintained. The only dimensions that were not included in the new revision paradigm were the rented housing units used to measure shelter and the urban areas in which data are collected for the monthly CPI.
In 2010, the CPI began implementing a method for continuously updating the sample of rented housing units by replacing one-sixth of the rented housing unit sample every year on the basis of the latest available U.S. Census Bureau data. In addition to reducing the age of the sample, the new process more accurately reflects new construction and changes in where people live, reduces sample attrition, and more efficiently uses CPI field resources.
Since rents are not as volatile as most other consumer prices, collecting a large sample less frequently is more efficient for the Housing Survey. This efficiency is accomplished by assigning each selected neighborhood (called a segment) in a pricing area to one of six panels, each of which represents a subsample of each pricing area and provides sufficient information for the monthly OER and Rent indexes. Each month, a panel is priced, with all six panels being priced twice a year: panel 1 is collected in January and July, panel 2 in February and August, and so on. Every month, BLS field staff collects the rent and other information for one panel. The 6-month price ratio is computed (the current rent divided by the rent 6 months ago) for each unit in the panel. The measures of price change for the two housing components are based on weighted averages of these rent ratios.
When the 1998 Housing Survey sample was introduced in 1999, the sample had about 33,000 in-scope units. This number was far below the target sample of 50,000 units. Because of this shortfall, a sample augmentation was performed in 2000–2001 that increased the in-scope sample to about 36,000 units. This number was still short of the target.
As samples age and become out of date, they become smaller. This result is certainly true of the Housing Survey’s sample. Over time, some renter-occupied units become owner-occupied while others become nonresidential. As the burden on respondents accumulates (for some becomes excessive), more refusals occur. More importantly, because newly built rental units have no opportunity to enter the sample, the sample becomes less representative of the housing universe. For example, rent changes for newer units may be different from those of older ones. By July 2009, when the efforts to improve the sample began, the sample had approximately 16,000 fewer rental housing units than its target size. This result was due primarily to the sample shortfall in 1999, the subsequent augmentation in 2000, and the continuing sample attrition. See appendix A for details on Housing Survey sample selection.
Sample replacement and sample augmentation are ways to counteract these problems. Sample replacement drops the old sample items and brings in new ones, selected from a new, more recent sampling frame. Augmentation supplements an existing sample by adding new sample items drawn from either the original sampling frame or a new frame. An augmentation that uses a new frame requires care to assure consistency of the old and new sample observations, particularly with respect to the sample weights.
The geographic sample of the last CPI revision, which deployed in January 1998, partitioned the urban United States into 38 CPI areas and selected 87 pricing areas from the 1990 Census to represent these areas. These areas were metropolitan areas or urban places and were selected using probability proportional to size (PPS), where the size measured was the 1990 population. For the Housing Survey, the pricing areas were further partitioned into neighborhoods called segments, formed from one (in most cases) or more U.S. Census Bureau block groups and containing at least 50 housing units in large (A-size) self-representing pricing areas and at least 30 in smaller (B/C-size and D-size) non-self-representing pricing areas. With the use of PPS, a sample of segments was selected in each area, in which the size measure was the sum of renters’ actual rents and owners’ estimated implicit rents. The Census Bureau provides the number of renters, the average rents, and the number of owners by block group, whereas the BLS estimates the average owners’ implicit rents. See appendix A for details on Housing Survey sample selection. An average of about five rental housing units was selected within each segment.
New Housing Survey samples
Most of the rental units in the current (early 2012) Housing Survey entered the sample in 1999 when the CPI completely replaced the sample with a new one drawn using the 1990 Decennial Census. Some units built after the 1990 Census were included through a special new-construction survey that covered units built as recently as 2006.
The CPI is undertaking a three-stage effort to improve the Housing Survey. The first and second stages use the 2000 Census. The first stage is a 4-year sample augmentation. Its goal is to add 16,000 units, mainly in neighborhoods with seriously depleted renter samples, and increase the size of the sample to its target. The CPI began using data from this augmentation in the OER and Rent indexes for July 2010.
The second stage is a sample replacement to replace the rental units introduced in 1999. The November 2012 CPI was the first that used a new sample from this stage. The May 2016 CPI will be the first in which the Housing Survey sample will have been drawn entirely from the 2000 Census.
The final stage will be a regular replacement commencing in 2016 and ending in 2022. It will replace the 2000-Census-based sample with one based on the American Community Survey using 2010 Decennial Census geography. This stage will continue into the future and—for the first time—the CPI Housing Survey will have a process that keeps its sample continuously updated.
In the 1998 CPI revision, BLS statisticians created special maps that indicated the selected segments and blocks of the pricing areas. Using these maps, CPI field staff traveled to selected segments and “listed” them, meaning that the field staff agents recorded the addresses of all housing units using Computer-Assisted Data Collection (CADC) software on their tablet computers. Then their tablet computers used a sampling algorithm to select a subset of these addresses. The field staff agents then visited each of these addresses and found a respondent to interview. They determined whether each address was eligible for the Housing Survey and, if so, initiated those addresses into the Housing Survey. In many segments, a majority of the addresses were screened out because they were nonresidential, were owner-occupied, or failed some other requirement.
Because of lessons learned in the 1999 sample’s selection and initiation, the CPI is using two important new survey methods in the augmentation and replacement. The 2010 sample augmentation used purchased address lists and a mail prescreening survey to locate housing units in the segments. These steps greatly increased the chances that an address contacted by a BLS agent would be a renter-occupied housing unit eligible for the Housing Survey. The lists indicate the probability that an address is owner-occupied and the addresses provide a means of determining whether an address is a commercial establishment. This information was used to determine sampling rates for the mail prescreening survey and to determine if selected addresses are commercial or residential and, if they are residential, their tenure (owner- or renter-occupied). Only those addresses the survey identified as renter-occupied and those with no response are sent out for BLS field agents to screen. Preliminary results were successful, so purchased address lists and prescreening will be used in all future sample replacements.
BLS field staff must find an eligible respondent for each address and then use tablet computers with CADC software for all data collection. During screening, the CADC software directs the field agents through a structured series of questions. They verify that the unit is renter-occupied. They further determine that the unit is the primary residence of the occupant, that the occupant is not a relative of the landlord, and that the unit is not institutional or public housing nor an assisted-living facility with activities of daily living provided to an occupant. These questions help the field agents determine if a selected address is eligible for the Housing Survey sample.
The field staff has a multimonth period to screen and initiate the units in a segment. Those units that the field staff is not able to screen (usually because the field agents fail to contact a respondent) go back out “on panel” for another screening attempt. This process should yield an expected number (usually five) of in-scope housing units in each segment that will be initiated into the Housing Survey sample.
Initiation and pricing
Once a selected address has been successfully screened, the CPI field agent immediately proceeds to initiate the housing unit into the Housing Survey sample. Initiation is the initial collection of rent data, which the field agent obtains by asking another structured series of questions. These data include the rent that is paid and specific housing services that are associated with the unit. These data are the basis for all calculations of rent change that will occur during the life of the housing unit in the Housing sample.
Once a unit is initiated, it will be priced on panel every 6 months. In addition, any in-scope units that are not successfully initiated go back on panel for another attempted initiation. A housing unit’s initiation generally does not occur in its on-panel month, so the housing unit must be priced on panel for two cycles to provide a 6-month interval before price changes can be used in the CPIs.
Collected data for each unit in the Housing Survey
During initiation and during each pricing, BLS collects
· contract rent and rental period (monthly, bimonthly, weekly, or for a specified number of days);
· utilities, facilities, and any other such items included in the rent;
· any subsidies (e.g., Section 8) or reductions in the rent in exchange for services the tenant provided;
· any extra charges included in the contract rent for optional items, such as parking;
· the number of rooms, type of housing structure, and other physical characteristics; and
· equipment used for air-conditioning and fuels used for heat and hot water.
In addition, to ensure that the unit remains in scope, BLS staff asks the screening questions every 2 years or when a change of occupant occurs.
Derived data for each unit in the Housing Survey
After housing data come in from the field, Housing Survey software computes or “derives” additional data elements for each unit. Derived data include review flags and the value of utilities and certain facilities used for quality adjustment. See appendix B for details on quality adjustments.
An important derived data element is the normalized rent of each unit. (Normalized means that rents that are not monthly [for example, biweekly rents] are adjusted to a monthly basis.) Normalized rent is the contract rent plus any subsidies the landlord receives and the value of any work that the tenant provides—all normalized, if not so already. In other words, normalized rent is the price of the “housing service” that the unit provided its occupant during the collection month. See appendix C for details on how the normalized rent is derived from the collected data.
However, the service a housing unit provides changes from one collection month to the next collection period 6 months later. At a minimum, the housing unit is 6 months older. Research has shown that as a housing unit ages, the value of housing service that it provides declines. Of course, less subtle changes may exist. For example, the landlord may stop providing heat or electricity or may have changed the unit structurally. Because the service a housing unit provides changes between collection periods, the CPI cannot estimate rent change from the change in the normalized rents as calculated in two different collection periods. The key requirement for the prices that CPI uses to calculate price change is that the prices be for comparable items (goods or services) in two periods.
For these reasons, the Housing Review and Correction Preprocessor (HRCP) calculates two prices each month for the Rent index and, because its underlying concept is different, another two prices for the OER index. “Economic” rent is the CPI price used for calculating Rent index. The HRCP computes the current-period economic rent and the 6-month-previous economic rent for each housing unit each time it is on-panel. Similarly, “pure rent” is the CPI price used for calculating the OER index calculation. The HRCP computes its current-period pure rent and its 6-month-previous pure rent for each housing unit each time it is on-panel.
The Rent index uses a contract rent concept in which the items that the landlord provides at no extra cost (e.g., utilities) are part of the service the housing unit provides. (The rent index must adjust for the quality change if the extras change; but as long as they do not change, they are part of the economic rent.) The concept for OER is defined as what an owner-occupied housing unit would rent for unfurnished and without utilities. Pure rent, therefore, excludes the estimated value of any landlord-provided utilities. See appendix C for details on how the economic and pure rents are derived.
Housing index calculation
Every month, the Housing Price Relative Calculation (PRC) system calculates a 1-month relative of price change for each of the 38 CPI areas for the two Housing components (OER and Rent) for use in calculating basic indexes and higher-level aggregate indexes. Each 1-month relative of price change is the sixth-root of a 6-month relative of price change.
A 6-month housing relative is the ratio of the weighted average current-period prices of the rental units in the CPI area’s Housing Survey sample to the weighted average 6-month-previous prices of the same rental units. For the Rent averages, the current and previous prices of a sampled housing unit are its current and previous economic rents, of course. Its weight is the average number of dollars of rental expenditure the sampled housing unit represents in the renter-occupied housing universe. For the OER averages, the current and previous prices of a sampled housing unit are its current and previous pure rents and its weight is the number of dollars of implicit rental value a sampled housing unit represents in the owner-occupied housing universe. See appendix D for details on Housing Price Relative and Index Calculations.
Appendix A: Housing Survey sample selection
To achieve a geographic distribution of segments within the pricing areas, the 2000 Census-based sample first sorted segments by geography (state, county, and Census tract) and by their average rent level and then selected a sample of segments by applying systematic probability-proportional-to-size sampling, with the cost of housing in a segment as the measure of size. The cost of housing in a segment is the cost of rented housing plus the cost (implicit rent) of owned housing in the segment. This process guarantees that the sample for each pricing area includes segments from different neighborhoods and rent levels.
Segments are one or more block groups (BGs). The 2000 Decennial Census provided the numbers of renters (RBG) and owners (OBG) and the average rent of renter units (RRBG) for each block group. BLS estimated the average implicit rent of the owner units (IRBG) in the block groups using a regression model for each CPI area based on Consumer Expenditure Survey (CE) data:
IRBG = b0 + b1 * ABG
+ b2 * (ABG)2
+ b3 * BBG Tract
+ b4 * CBG ,
where ABG is the average property value of owners in the block group, BBG is the median income of owners in the block group, CBG is the average number of rooms for owners in the block group, and b0, b1, b2, b3, and b4 (the actual regression coefficients) were determined uniquely within each CPI area. The model was chosen for four major reasons:
1. The model is a relatively simple linear regression.
2. The three independent variables (i.e., property value, owner income, and number of rooms) are variables expected to affect OER.
3. The model does not depend on any renter information. Therefore, no assumptions about renter-owner interactions have to be theorized.
4. These three CE variables have comparable housing variables in the Census 2000 Summary File 3 (SF3). This comparability allowed the BLS to use the SF3 files as the data source.
From these four pieces of information, the CPI calculated the total cost of housing (TCBG) in the block groups from the renter costs (RCBG) and the owner costs (OCBG) in the block groups:
TCBG = RCBG + OCBG = (RBG * RRBG) + (OBG * IRBG).
Since segments are one or more BGs,
and
Each segment within each CPI pricing area has a probability of selection (PS) that is the ratio of the cost of housing (TCS) in the segment multiplied by the number of segments to be selected in the pricing area (nPricingArea) divided by the total housing cost in the pricing area:
The segment weight (WS) is the inverse of the probability of selection,
where S is a segment in a pricing area.
Appendix B: Quality adjustments
Most of the quality adjustments made to the economic and pure rents are derived automatically. These quality adjustments are described as follows:
1. Age bias adjustment. The quality of all units deteriorates over time. Units age between observations, so the BLS adjusts the rent for the estimated loss in quality because of aging. A hedonic regression estimates aging effects (among other things). Only age bias adjustments are applied to the previous rents to reduce the previous pure and economic rents and increase the price change to the current period. The annual impact is an increase of about 0.3 percent in the OER and Rent indexes.
2. Structural change adjustment. The hedonic regression is also used to derive factors to adjust for any changes in the numbers of bedrooms, bathrooms, and other rooms and for the conversion to or from central air-conditioning (A/C). If a bedroom, bathroom, and/or other room is added to the unit or if either central A/C is added or noncentral A/C is converted to central A/C, the adjustments are subtracted from the current rent to make it consistent with the previous rent. If a bedroom, bathroom, and/or other room is removed from the unit or if either central A/C is removed or the central A/C is converted to noncentral A/C, the adjustments are added to the current rent to make it consistent with the previous rent.
3. Removal of the cost of landlord-provided utilities (cost of utilities) for the calculation of pure rents.
a. Electricity, fuel oil, and gas. Regression coefficients for energy consumption are derived using data from the Department of Energy’s Residential Energy Consumption Survey (RECS). The characteristics of the housing units and the RECS regression coefficients are used to estimate the consumption of electricity, fuel oil, and gas for each housing unit. Average prices from the CPI Commodities and Services (C&S) survey are used for deriving costs of the three fuels for each housing unit. If the landlord provides the utility, the cost is added to the cost of utilities for the unit.
b. Water and sewer. If the landlord provides water and/or sewer services, dollar adjustments are added to the cost of utilities for the unit. Combined water and sewer amounts for each CPI pricing area were derived from 2011 American Community Survey (ACS) data. C&S data were used to divide the combined water and sewer amounts into separate water and sewer amounts at the CPI pricing-area level. Appropriate CPI data were used to update the water and sewer amounts from 2011 to a current collection period. Water and sewer amounts are also adjusted monthly using appropriate CPI changes. Water and sewer amounts will be updated annually when the ACS data for the following year are available.
4. Utility adjustment for economic rents. If the landlord changes the provision of utilities from the previous period to the current period, the economic rent is affected. The individual utility costs derived earlier in the process are used for deriving the utility adjustments. If the landlord starts providing electricity, fuel oil, gas, and water and/or sewer services, the utility adjustments are subtracted from the current rent to make it consistent with the previous rent. If the landlord stops providing electricity, fuel oil, gas, and water and/or sewer services, the utility adjustments are added to the current rent to make it consistent with the previous rent.
5. Facility adjustments for parking and A/C units.
a. Parking. If the landlord starts or stops providing free parking, parking dollar adjustments are applied at the CPI pricing-area level. If the landlord starts providing free parking, the parking adjustment is subtracted from the current rent to make it consistent with the previous rent. If the landlord stops providing free parking, the parking adjustment is added to the current rent to make it consistent with the previous rent. Parking amounts are adjusted monthly using appropriate index changes.
b. A/C units. If the landlord changes the number of window or through-the-wall A/C units or heat pumps or starts or stops providing them, the dollar amount is adjusted. If the landlord starts providing units or increases the number of units, the A/C unit adjustment is subtracted from the current rent to make it consistent with the previous rent. If the landlord stops providing units or decreases the number of units, the A/C unit adjustment is added to the current rent to make it consistent with the previous rent. The A/C cost adjustment is reevaluated every year. If the A/C equipment changes from window or through-the-wall to central A/C or vice versa, the central A/C structural change adjustment just described is used to adjust for the change.
6. Analyst adjustments. Rarely, one or more of the automated adjustments do not work correctly; in this case, the commodity analysts adjust the current and/or previous economic and/or pure rents to properly account for the change.
Appendix C: Calculation of normalized, economic, and pure rents
Normalized rents
Current normalized rent = contract rent
+ any subsidies the landlord receives
+ the value of any work that the tenant provides)
* calendar adjustment factor,
where the calendar adjustment factor converts the nonmonthly rents to monthly rents.
Economic rents
The economic rents are the normalized rents adjusted for changes in quality. In most cases, quality adjustments are applied in the current period to make the current and previous period rents consistent. If the landlord provides utilities, the costs of those utilities are embedded in the contract rents, the normalized rents, and the economic rents. Economic rents are used in calculating the Rent price relatives.
Current economic rent = normalized rent
± utility adjustment
± facility adjustment
± structural change adjustment
± analyst adjustment.
t‒6 economic rent = current economic rent in t‒6
– age bias adjustment
± analyst adjustment.
Pure rents
The pure rents also start with the normalized rents and then adjust for changes in the quality of the housing unit, but an additional adjustment is made. Since owners pay their own utilities and the C&S expenditure weight for utilities includes the utility expenses both of owners and of those tenants who pay their own utilities, the value of landlord-provided utilities (cost of utilities) must be removed from the normalized rent. Since the cost of utilities is subtracted to derive the pure rents, utility adjustments are not applied. Pure rents are used in calculating the OER price relatives:
Current pure rent = normalized rent
– cost of utilities
± facility adjustment
± structural change adjustment
± analyst adjustment.
t–6 pure rent = current pure rent in t–6
– age bias adjustment
± analyst adjustment.
Appendix D: Housing Price Relative and Index Calculations
Unit weights
The renter and owner costs in the segments are the basis for the renter and owner weights in the segments used in the Price Relative Calculation (PRC).
Renter weight for the PRC
To derive the renter weight in the segment (RWS), the segment weight (WS) must be multiplied by the number of renters in the segment (RS):
RWS = WS * RS .
The renter weight for a unit in a segment (RWS,i) is the renter weight divided by the desired number of renter units in the segment (nS):
Owner weight for the PRC
To derive the owner weight in the segment (OWS), one must multiply the segment weight (WS) by the number of owners in the segment (OS) and adjust the ratio of the estimated average implicit rent of the owner units in the segment (IRS) to the average rent in the segment (RRS):
The owner weight for a unit in a segment (OWS,i) is the owner weight divided by the desired number of renter units in the segment (nS):
The renter and owner weights are ratios of expenditures, not expenditures themselves, so converting them into quantities by dividing them by base rents or base implicit rents is unnecessary.
PRC for Rent and OER
Because of the panel structure of the housing data, 6-month relatives are calculated:
5. 6-month relatives for Rent for a CPI area (A) are calculated with renter weights (RWS,i) and economic rents for the units (i) in the segment in the current month (ERS,i,t) and 6 months previous (ERS,i,t–6).
6. 6-month relatives for OER for a CPI area (A) are calculated with owner weights (OWS,i) and pure rents for the units (i) in the segment in the current month (PRS,i,t) and 6 months previous (PRS,i,t–6).
Housing takes the sixth root of the 6-month relatives to calculate 1-month relatives and passes them to the CPI estimation system:
.
Vacancy imputation for Rent and OER
Vacant units that renters previously occupied are used in calculating RelRentA,t–6,t and RelOERA,t–6,t. The vacancy imputation process incorporates several assumptions about the unobserved rents of vacant units. It is assumed that rents tend to change at a different rate for units that become vacant (and are, therefore, in the process of changing tenants) than for units that are not vacant. After an initial lease period, the vacancy imputation model assumes that expected rents change at a steady rate until the tenant moves out of the unit. When occupancy changes or a unit becomes vacant, the rent is assumed to jump at some rate, referred to as the “jump rate” (J). In markets with generally rising rents, this jump rate is usually greater than the average rate of change for occupied units. BLS estimates the jump rate based on nonvacant sample units in the area that have had a change in tenant between t‒6 and t.
Rent changes for nonvacant units without a tenant change are used to calculate the average continuous rate of change (C). These values are used to impute economic and pure rents for vacant units for period t from their rent in t‒6:
ERS,i,t = ERS,i,t‒6 * J, if the unit was not vacant in t‒6,
ERS,i,t = ERS,i,t‒6 * C, if the unit was vacant in t‒6,
PRS,i,t = PRS,i,t‒6 * J, if the unit was not vacant in t‒6,
and
PRS,i,t = PRS,i,t‒6 * C, if the unit was vacant in t‒6.
Noninterview imputation for Rent and OER
Imputed rents for noninterviewed units are derived by using the average continuous rate of change at 3 rent levels: high (CH), medium (CM), and low (CL). These values are used to impute economic and pure rents for noninterviewed units for period t from their rent in t–6 based on their rent level.
Economic and pure rents in the high rent-level category:
ERS,i,t = ERS,i,t–6 * CH, PRS,i,t = PRS,i,t–6 * CH.
Economic and pure rents in the medium rent-level category:
ERS,i,t = ERS,i,t–6 * CM , PRS,i,t = PRS,i,t–6 * CM.
Economic and pure rents in the low rent-level category:
ERS,i,t = ERS,i,t–6 * CL PRS,i,t = PRS,i,t–6 * CL.
Calculation of Indexes for Rent and OER
The CPI estimation system uses the 1-month relatives to move the indexes from last month (t‒1) into the current month (t):
,
and
United States Department of Labor
