CHAPTER 10.0 HOUSEHOLD SURVEY IMPLEMENTATION
Note: significant portions of this chapter come from the 2008 NCHRP Report 571, with review and contributions by Stacey Bricka and Joanne Pratt. Section 10.3 updated by Elaine Murakami, December 2012.
10.1 Ethics
Ethics describe minimum acceptable standards of conduct or practice. In travel surveys, this relates to how a survey agency conducts itself with respect to those interviewed, the client, any subcontractors, and the public as a whole. It also relates to a survey agency’s actions following the data collection process when data are cleaned, coded, analyzed, and archived.
Ethics reflect what all stakeholders may consider “fair” or “reasonable” conduct by those involved. In practical terms, the application of ethics involves implementation of precautions to protect those affected from adverse effects. Ethics protect the rights of individuals and groups including their privacy and serve to reduce public disapproval and criticism of what is done.
After reviewing documents prepared by various associations on different aspects of ethical conduct in the execution of travel surveys, it is recommended that the following ethical conduct be observed in all future travel surveys:
1. The anonymity of the persons surveyed, and the confidentiality of the information they provide, must be protected at all times;
2. A survey respondent may not be sold anything or asked for money as part of the survey;
3. Persons must be contacted at reasonable times to participate in the survey and must be allowed to reschedule participation in the survey to a different time if that is more convenient for them;
4. Survey personnel must be prepared to divulge their own name, the identity of the research company they represent, the identity of the agency that commissioned the study, and the nature of the survey being conducted, if requested by a respondent;
5. Children (age 15 and under) may not be interviewed without the consent of a parent or responsible adult;
6. A respondent’s decision to refuse participation in a survey, not answer specific questions in the survey, or terminate an interview while in progress must be respected if that is the respondents’ firm decision;
7. Respondents may not be surveyed or observed without their knowledge: methods of data collection such as the use of hidden tape recorders, cameras, one-way mirrors, invisible identifiers on mail questionnaires, global positioning systems, and RFID chips may only be used in a survey if the method has been fully disclosed to the respondent and the respondent agrees to its use.
8. A research agency may not release research findings prior to the public release of the findings by the organization that commissioned the study, unless approval of the client organization is obtained to do so; and
9. A research agency must ensure the reasonable safety of its fieldworkers during the execution of a survey.
10.2 Mailing Materials
Many surveys involve some activity of mailing materials to respondents, whether this is just an initial contact letter telling about the survey to be done, the sending of recruitment materials, or providing the full survey form. There is evidence to suggest that the materials used to mail to households, as well as materials for households to mail back, have an effect on response rates.
It is recommended that the following standardized approaches be adopted with regard to format and appearance of mailing materials for travel surveys:
1. The use of a large white envelope (4" x 9.5" or larger), with the address printed directly onto the envelope, rather than the use of address labels;
2. Print a recognizable return address on the envelope and indicate the contents of the envelope—at least the survey name; and
3. Earlier literature strongly recommends the use of postage stamps, especially commemorative stamps, rather than using a franking machine or pre-printed bulk mail. However, more recent studies suggest that respondents do not pay as much attention to how postage is affixed as in the past. Regardless of the method used, the postage should contribute to the professional look of the materials.
10.3 Incentives (December 2012)
Response rates to recent travel surveys continue to decline. Because of the high proportion of households who are now "cellphone only," there is a recent trend toward using address-based samples. How to encourate participation for a mail-out survey with multiple methods for data collection can be very challenging, especially when we continue to get so much junk mail!
Consideration for using incentives:
Pre-paid incentives or post-paid incentives
Cash, check, VISA/MC, gift cards from specific retailers
Distribution by mail or email
Survey research continues to find that small pre-paid cash incentives can be very effective. The recent National Household Education Survey (NHES) tested $2 and $5 in the "screener" (or initial) survey. $5 performed better than $2 (Han et al, 2012). In the topical survey (only sent to households with at least one child), different incentives were tested (0, $5, $10, and $20), and they found that $5 was still effective, and the higher incentives did not increase results sufficiently to warrant the higher incentive. In another study (Edwards et al) completed in 2002, a large metadata analysis of 292 mail-out surveys found that the "odds of response were more than doubled when a monetary incentive was used, and almost doubled when incentives were not conditional on response.
NCHRP Report 571 "Standardized Procedures for Personal Travel Surveys" (Stopher et al, 2008) recommend that incentives should be offered as pre-completion, and not conditioned on a return being received. In addition, lotteries were discouraged.
While "cash is best" for small monetary incentives, different forms of gift cards and e-gift cards have been tested recently, as an alternative to cash. This can relieve some anxiety of having to secure large amounts of cash. Although activated gift cards also have a street value, they are easier to trace and can be voided if theft occurs. Younger populations who live "on-line" might prefer an e-gift card. A recent study (2011?) for the Utah State Department of Transportation used Amazon e-gift cards that were emailed to respondents.
Only a few studies have attempted the use of Visa or MasterCard cards. Based on the results of this small number of examples, it is not advised to use this approach. In a National Fisheries study, the card had a value of $10, but if the respondent used it in an ATM other than than the issuing bank, a $3 fee was assessed. In a recent travel survey, the cards were purposed to be "used anywhere a VISA card is accepted," but the cards did not work a several gas stations. This problem resulted in many complaints from survey respondents.
Many household travel surveys conducted between 2006 and 2012 have not used pre-paid incentives, choosing instead to use post-paid conditional incentives. Often these incentives have been restricted to "hard-to-reach" households, such as those with lower income, large families, and zero-vehicles. These incentives have ranged from $20 to $50, and differential incentives may be used based on the difficulty in capturing specific types of households. Generally, households asked to complete a GPS-based travel survey are offered at least $25 per person after return of the GPS equipment.
In a multi-stage survey, multi-staged incentives should also be used. This is especially important in GPS studies where the return of GPS equipment is involved.
Including a test of incentives in the Pilot survey is highly recommended. If incentives can improve recruitment and completion rates, then the overall costs of the survey can be lower than without monetary incentives.
10.4 Respondent Questions
In virtually any travel survey, respondents have concerns regarding the legitimacy of the survey and those conducting it. While some of these concerns may be addressed in a cover letter, the typical survey has more nuances than may be explained in a single- (or even double-) page letter. The state of the practice has evolved three methods for respondents to verify the survey, and obtain answers to frequently asked questions. These include the use of
1. Telephone contact numbers;
2. Informational brochures, with frequently asked questions (FAQs); and
3. Internet websites.
As updated standardized procedure for respondent questions, it is recommended that the execution of a travel survey include
· A telephone contact within the sponsoring agency;
· A toll-free telephone contact within the data collection entity (if different from sponsoring agency);
· Email contacts for each entity and
· Detailed instructions in the form of an informational brochure or fact sheet; care should be taken to ensure that the information is presented in an easy to read manner, with appropriate use of graphics where possible.
Where possible, a website with information about the survey, links to sponsoring agencies, answers to FAQs, email and telephone contact for assistance or further information, and the ability to download survey materials be provided.
10.5 Caller ID
“Caller ID,” “Caller Line Identification,” and “Caller Display” are different names for the service provided by many telephone companies that allows the customer to see the telephone number, and sometimes the directory listing, of the person who is calling. With the addition of Call Blocking, telephone customers may automatically block incoming telephone calls that do not permit the display of a telephone number.
In light of the general decline in telephone survey response rates, it is incumbent upon legitimate survey researchers to provide any information that may encourage responses from the full range of households. One of the primary uses of Caller ID is for households to screen out unwanted telephone calls by simply ignoring calls that do not display a known number or identity of the caller.
It is recommended as a standardized procedure that Caller ID be provided by the entity conducting the telephone calls—whether a contracted survey firm, university, or government agency—because existing data indicate that providing any ID at all may assist response rates more than being unrecognized. However, after careful review, it is concluded that there are no standardized procedures that can be recommended regarding Caller ID listings.
10.6 Answering Machines and Repeated Call-Back Requests
There are two related issues encountered by every telephone-based survey: first, when an answering machine is reached, does it assist completion rates if a message is left? Second, when a household requests an interviewer call them back at another time, is there a point beyond which repeated call backs do not increase completion rates?
It is recommended that a standardized procedure be adopted that messages be left on answering machines, as follows:
- When an answering machine is reached on the initial recruitment/screening call, a message should be left at least once in the call rotation before classifying the number as non-responding. The message should identify the client organization and the nature of the survey and provide a toll-free number for the household to contact should they desire to participate. The message should be short (no more than 15 seconds) and preferably provided by a “live” interviewer as opposed to a recorded message.
- When an answering machine is reached on a reminder telephone call, a message should always be left.
- When an answering machine is reached during telephone retrieval of travel information, a message should always be left.
It is also recommended that telephone survey protocols include a process for complying with call back requests, whether they occur in the recruitment or retrieval portion of a telephone survey. After the fifth request for a call back from the same household, the household be categorized as a “soft” refusal and therefore eligible for any “soft refusal” conversion techniques in use.
10.7 Incorrect Reporting of Non-Mobility
In any travel survey, it is to be expected that some portion of respondents will not have traveled from their home during the survey period. However, a claim of non-mobility during the assigned travel period also may be a form of non-response. Some potential respondents may realize that a claim of non-mobility will shorten significantly the length of the interview. The issue is to reduce the incorrect reporting of non-mobility that is made as a form of non-response. Standardized procedures are recommended in three portions of the travel survey process: data collection, data recording, and reporting.
10.7.1 In Data Collection
It is recommended that a question to verify reported non-mobility be asked of all persons who report they did not travel (stayed in one place/did not leave home) during the entire travel period. The question wording in the 2001 National Household Travel Survey (NHTS) (“Does this mean {you/subject} stayed at {the same place/home} all day?”) should suffice. To explore the issue of non-mobility further, it is recommended to include questions that gently challenge persons who report non-mobility by asking for the reason(s) why no travel was made during that day. One response may be that the respondent worked at home that day, which in some surveys may be relevant information to record.
10.7.2 In Data Coding
At a minimum, it is recommended that the data set include an indicator to distinguish between cases where a person indicated that he or she did not travel and those where a person refused to provide travel data.
10.7.3 In Reporting
It is recommended that the survey results report include the percent of non-mobile person days. In single day surveys, this would be determined by the number of persons reporting that they did not travel, divided by the total number of persons reporting. If questions are asked regarding the reasons why no travel was asked, as advanced practice, the report should include analyses of these reasons and the characteristics of persons who reported no travel.
10.8 Recording Time of Day
This item refers to standards for coding time-of-day values for database entry and relates to how data are recorded (i.e., entered by the interviewer) and stored, rather than how respondents provide the information.
It is recommended that time of day for data entry and storage be undertaken using two fields: one for the day number, and one for the time in military time (00:00–23:59).
10.9 Time of Day to Begin and End Reporting
Surveys use various different times at which to start and end the time for a 24-hour (or longer) diary. The aim is usually to choose a time that is expected to interrupt relatively little travel so that respondents will not be put in the awkward situation of trying to respond about travel that had started before the start time of the diary. However, there is wide discrepancy in the selection of this time, which appears to range anywhere from midnight to 5 a.m.
It is recommended that start and end times for 24-hour diaries should be 03:00 a.m. to 02:59a.m. In the case of diaries that cover more than 1 day, end times are extended by 24 hours for each additional day.
10.10 Creation of ID Numbers
Each completed survey requires a unique identification number. In addition, if data are retained on incomplete households, then all contacted households require a unique identification number. The primary issue with respect to identification numbers is that the numbers should permit ready retrieval of specific records and should provide a unique identification for each unit in the survey. In addition, there is the potential to provide some additional information through the identification number, such as the membership in a specific sampling category, thereby permitting easy checking of the sampling progress during the survey and ready identification for purposes of expansion and weighting after the survey is completed.
It is recommended that an ID number should be assigned at the outset to each piece of sample selected for inclusion in the study. This number should remain attached to the sample piece throughout duration of the survey.
REFERENCES
Edwards, P., I Roberts, M. Clarke, C. DiGuiseppi, S. Pratap, R. Wentz, and I. Kwan. "Increasing response rate to postal questionnaires: systematic review" BMJ.com Volume 324, May 18, 2002.
Han, Daifeng; J. M. Brick, and J. Montaquila. "Evaluation of Incentives Experiments in a Two-Phase address-based sample mail survey" Joint Statistical Meetings, San Diego, California, August 1, 2012.
Stopher, P.R.; R. Alsnih; C.G. Wilmot; C. Stecher; J. Pratt et al. Standardized Procedures for Personal Travel Surveys. NCHRP Report 571. Transportation Research Board, Washington, D.C. 2008.
Uploading ....