If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Correspondence to: Jan M. Eberth, PhD, SC Cancer Prevention and Control Program, University of South Carolina, 915 Greene St, Room 234, Columbia, SC 29208. Phone: 803-576-5770.
Local data are often used to plan and evaluate public health interventions and policy. With increasingly fewer public resources to collect sufficient data to support direct estimation of local outcomes, methods for deriving small area estimates are vital. The purpose of this study is to describe the county-level geographic distribution of human papillomavirus (HPV) vaccine coverage among adolescent females in Texas using multilevel small area estimation.
Methods
Multilevel (individual, county, public health region) random-intercept logit models were fit to HPV vaccination data (≥1 dose Gardasil) from the 2008 Behavioral Risk Factor Surveillance System. Using the parameter estimates from the final model, we simulated 10,000 data sets for each regression coefficient from the normal distribution and applied them to the logit model to estimate HPV vaccine coverage in each county.
Results
County-level coverage estimates ranged from 7% to 29%, compared with the state average of 18% (95% confidence interval [CI], 13.59–21.88). Many Southwestern border and metropolitan counties exhibited high coverage estimates. Low coverage estimates were noted in the Panhandle, Southeastern border region, and Northeast. Significant correlations were observed between HPV vaccination and Hispanic ethnicity, county poverty, and public health region poverty.
Conclusion
Harnessing the flexibility of multilevel small area models to estimate HPV vaccine coverage at the county level, we have provided data that may inform the development of health education programs/policies, the provision of health services, and the planning of new research studies. Additionally, we have provided a framework for modeling other health outcomes at the county level using national survey data.
Introduction
The U.S. Food and Drug Administration approved the use of Gardasil (HPV4), a vaccine against four prevalent strains of human papillomavirus (HPV) for females aged 9 to 26 in June 2006 (
). Initiation of the HPV4 series (3 doses given over 6 months) grew over time, increasing to 48.7% of U.S. females aged 13 to 17 in 2010 (32% completed the vaccine series); however, initiation rates varied considerably by geographic region (range across 50 states, 28.8%–73.0%;
found that HPV vaccine coverage differed significantly across states and counties, a finding explained in part by state- and county-level poverty. Although studies indicate the presence of geographic variation in HPV vaccine coverage across states and selected regions, and have found that group-level covariates such as poverty (
) are associated with HPV vaccination, local level estimates are not uniformly available in the literature.
Local data on HPV vaccine coverage are important for several reasons, including the planning of health promotion programs, identifying target populations for services (e.g., free vaccine clinics), and evaluating the impact of policies and programs designed to improve vaccine uptake. National health surveys provide much of the data used to conduct population-based public health research in the United States. Whereas these surveys provide estimates at the national, state, and sometimes county levels, their ability to provide local estimates is limited owing to costs, sample size constraints (i.e., ≥50 respondents per geographic unit recommended for direct estimation;
), and privacy protections. Thus, researchers often rely on small area estimation (SAE), a set of indirect estimation techniques for producing small area estimates using data collected for larger areas and census tabulations for the small area of interest. This study used multilevel, SAE to describe the county-level geographic distribution of HPV vaccine coverage (≥1 dose HPV4) among females aged 11 to 17 years living in Texas. The methodological framework we discuss herein can be used by other researchers to analyze their own health outcome(s) of interest using a variety of national and state-based population surveys.
Analysis
Overview
In multilevel small area models (a type of synthetic estimation), model coefficients are linked with demographic- and geographic-specific population data (hereafter referred to as the auxiliary dataset) to predict the outcome for each small area or population subgroup. We applied multilevel SAE to predict county-level coverage of HPV vaccination. The 2008 BRFSS was used to develop a three-level (individual, county, public health region [PHR]), random-intercept logit model. Several factors informed our choice of covariates and model-building strategy. First, covariates included in the final model (both individual and group levels) were required to be available in both the individual and auxiliary datasets to permit estimation of small area means. Second, all individual- and group-level covariates considered for inclusion had known (
) or hypothesized associations with HPV vaccination in adolescents. Finally, because Texas counties are nested within PHRs and vaccine coverage data in Texas is often presented for PHRs, we included PHR-level random effects in the model to ensure the correct selection of group-level covariates and parameter estimates. A summary of these methods is shown in Figure 1.
Figure 1Summary of data collection, management, and analysis strategy. PCA, principal component analysis; PHR, public health region. aPreliminary model includes age, race/ethnicity, parental education, county and PHR poverty. bTechnical documentation (i.e., equations/Equ.) can be found in Appendix A. cFinal model includes all the variables from the preliminary model except for parental education. Although parental education is an important predictor of HPV vaccination, it could not be included in the final model because the corresponding variable was not present in the Auxiliary Dataset. A post-adjustment method, described by Congdon (25), was used as an alternative to inclusion of parental education in the final model.
SAS Version 9.1 (SAS Institute, Cary, NC) was used for data management and MLwiN Version 2.20 was used to implement multilevel models. Our study was granted exempt status by our Institutional Review Board.
) to obtain the dependent variable and individual-level covariates of interest. County- and PHR-level data were acquired from multiple sources including the Association of Religious Data Archives, U.S. Department of Agriculture Office of Management and Budget, Texas Cancer Registry, Texas Department of Health and Human Services, and U.S. Census Small Area Health Insurance Estimates Program. County-specific population estimates, stratified by gender, age, and race/ethnicity (the auxiliary dataset), were obtained from the Vintage 2008 Bridged-Race Postcensal Population Estimates file available from the Health Center for Health Statistics (
U.S. Centers for Disease Control and Prevention. (2009). U.S. Census populations with bridged race categories. Retrieved January 20, 2011, from http://www.cdc.gov/nchs/nvss/bridged_race.htm.
The BRFSS is a population-based, random digit-dial survey conducted annually by the states, through a cooperative agreement with the U.S. Centers for Disease Control and Prevention. In addition to the core questions asked by all states in 2008, Texas adopted the optional HPV modules. In Texas, the proportion of contacted persons who completed the interview was 75% (
U.S. Centers for Disease Control and Prevention. (2008a). Behavioral Risk Factor Surveillance System: 2008 summary data quality report. Retrieved September 2, 2011, from ftp://ftp.cdc.gov/pub/Data/Brfss/2008_Summary_Data_Quality_Report.pdf.
U.S. Centers for Disease Control and Prevention. (2008a). Behavioral Risk Factor Surveillance System: 2008 summary data quality report. Retrieved September 2, 2011, from ftp://ftp.cdc.gov/pub/Data/Brfss/2008_Summary_Data_Quality_Report.pdf.
When children resided in the home of an adult respondent (referred to as the parent/caregiver), he or she answered questions about one randomly selected child. Our sample consisted of 574 parents/caregivers, all of whom 1) had a county of residence listed in the public use dataset, 2) had a randomly selected 11- to 17-year-old daughter for whom survey data were collected, and 3) reported their level of educational attainment. The daughter's age and race/ethnicity, as well as the parent/caregiver's educational attainment, are known predictors of HPV vaccine uptake (
); thus, any person with missing data on these items was excluded. Additionally, participants who did not know their daughter's HPV vaccination status were excluded. These criteria reduced the sample size from 623 to 574.
In selecting group-level covariates for examination, we made an a priori decision to include county-level poverty because of its known positive association with HPV vaccine initiation (
). Because the majority of covariates of interest had not been previously studied, we implemented an exploratory approach to select additional covariates for examination. We were specifically interested in testing the effects of access, religious adherence, sexually transmitted disease and cervical cancer prevalence, and uptake of other childhood and adolescent vaccines. When there were multiple measures representing a specific concept (e.g., access to care could be measured by density of physicians or Vaccines For Children providers in the county, urban/rural classifications, or designation as a Health Professional Shortage Area), we conducted principal components analysis to extract one representative covariate for each concept. The covariate with the strongest correlation to the first principal component was then examined in our models. Principal components analysis was not done for PHR-level covariates because of limited variability (n = 11). This approach resulted in examination of the group-level covariates listed in Table 1.
Table 1Model-Building Strategy for Determining the Final Model of HPV Vaccine Initiation among Texas Females Aged 11 to 17
Although sampling weights are provided by the BRFSS, they are not efficient for use in small area models because the weights are constructed at the state level. The problem is compounded for multilevel models, which traditionally have not made use of weights associated with complex survey data (
, we rescaled the level 1 sampling weights (i.e., CHILDWT) to produce better point estimates and reduce the risk of biased estimates when the cluster size is small. We also included weights at levels 2 and 3 as proposed by
. Technical documentation for the scaling and creation of these weights is provided in Appendix A.
Statistical Analysis
The outcome was initiation of HPV vaccination (≥1 dose HPV4, yes or no) among females aged 11 to 17. A series of increasingly complex models were fit to the data, starting with the empty model and adding covariates sequentially by level (Table 1). Level 1 covariates were added to the empty model in step 2 and were retained for inclusion regardless of statistical significance in steps 3 through 5. At steps 3 and 4, group-level covariates were added to the model simultaneously. Covariates were removed from the model sequentially, starting with the least significant, until only covariates with p ≤ .1 remained. Using a more conservative 0.25 p-value, as suggested by
, resulted in the same selection of covariates. Because auxiliary data were only stratified by age and race/ethnicity, parental education was removed from the final model (step 5). A post-estimation adjustment for education at the county-level was incorporated as suggested by
(2009; equation in Appendix A). First-order marginal quasi-likelihood, followed by second-order penalized quasi-likelihood (uses marginal quasi-likelihood estimates as starting values; provides robust estimates when sample sizes are small), was used in all models. Interaction terms were not tested owing to the small sample size.
We applied the parameter estimates from the final model to simulate 10,000 probabilities of HPV vaccine uptake for each race- and age-specific subgroup each county. The probabilities were adjusted for the educational mix of each county. We then calculated the mean value, standard deviation, and 95% confidence interval (CI) of these simulated subgroup probabilities. The variance associated with the PHR-level random intercept was very small and thus ignored in the simulation. The county random intercept was only estimated for sampled counties in the 2008 Texas BRFSS. Finally, we multiplied the race- and age-specific adjusted probabilities above by the corresponding county subpopulation to generate summary estimates of HPV vaccine coverage for each county.
To test the accuracy of our multilevel model, we aggregated our county-level estimates up the state level and compared the resulting model-based estimate with the direct estimate of HPV vaccine coverage at the state level. We did the same validation checks with our age- and race-specific county estimates. These analyses allow us to examine whether our small area estimates (aggregated up to the state level) produce the expected results at the state level, where direct estimation of HPV vaccine coverage is possible.
Results
Sample Characteristics
Our sample consisted of 574 parents/caregivers residing in 99 Texas counties (39% of counties). The adolescent females were primarily White (54%) or Hispanic (33%), with educated parents/caregivers (34% had a bachelor's degree or higher and 27% had some college education), and a mean age of 14 years. The large majority of respondents (88%) indicated they were parents (biologic, step, or adoptive) of the adolescent female. Many of the participants came from large metropolitan counties including Bexar (n = 55), El Paso (n = 44), Harris (n = 40), Lubbock (n = 31), Tarrant (n = 31), Travis (n = 36), and Val Verde (n = 30), which is not surprising given their greater probability of being selected as a survey participant.
Model Results
In all models, there was small variability at the county- and PHR-level (Table 2). At the county level, we evaluated the area level variance using the median odds ratio (MOR) (not able to be estimated at the PHR-level). Because the MOR was not far from 1.00 in the final model, we concluded that there were minimal differences between counties in the individual probability of obtaining HPV vaccination.
Table 2Random-Intercept Logit Models of HPV Vaccine Initiation among Texas Females Aged 11 to 17
Hispanic ethnicity was the only significant level 1 covariate. Compared with Whites, Hispanic females aged 11 to 17 were significantly more likely to be vaccinated against HPV (odds ratio [OR], 1.77; 95% CI, 1.28–2.46). The removal of parental education from the final model impacted the strength and significance of the covariate (OR, 1.34; 95% CI, 0.95–1.91), suggesting a possible suppression effect. We suspect the likely reason for this effect is educational distribution across racial/ethnic groups. In our sample, Whites, Blacks, and other races had positively skewed distributions (i.e., toward more education), whereas Hispanics had a negative skew. In contrast with several studies finding a positive effect of age on HPV vaccine uptake (
), we found a null effect that likely reflects how we defined the variable (i.e., continuous vs. categorical).
We found a significant, positive association between county-level poverty and HPV vaccination. The odds of being vaccinated increased by about 9% for every 1% increase in county-level poverty. A more meaningful interpretation, based on the distribution of poverty in Texas (range across counties, 6.2%–36.8%), would be the change in odds for every 5% or 10% change in county-level poverty. For these values, we would expect to see the odds of vaccination increase by 54% and 139%, respectively. Our model also showed a significant, negative association between PHR-level poverty and HPV vaccination. Thus, for a 5% or 10% change in PHR-level poverty (vs. a 1% change, as shown in Table 2), the odds of vaccination decreased by 35% and 58%, respectively.
Small Area Estimates
In 2008, 17.7% (95% CI, 13.59–21.88) of Texas females aged 11 to 17 were vaccinated against HPV (based on direct, weighted estimation of HPV vaccine initiation data from the 2008 Texas BRFSS). That same year, the national average for females aged 11 to 17 was 30.4%. County-level model-based small area estimates ranged from 6.8% to 29.0% (Figure 2; Appendix B). No estimate is provided for Loving county because there were no representative females. The state-level indirect estimate was 15.5% (95% CI, 14.1–16.9), which lies within the confidence interval of its direct estimate of HPV vaccine coverage (95% CI, 13.59–21.88). Our race/ethnicity- and age-specific indirect estimates, scaled up to the state level, were also within the confidence intervals of their respective direct estimates, with the exception of the indirect estimate for HPV vaccine coverage among 13-year-old girls in Texas (Table 3).
Figure 2County-level estimates of HPV vaccine coverage among females aged 11 to 17 in Texas.
Researchers have used model-based SAE to examine a variety of health behaviors and outcomes including asthma, obesity, smoking, mammography utilization, and health insurance coverage (
found that regression models provided better precision than traditional synthetic and spatial smoothing methods in SAE. Multilevel modeling is a particularly useful SAE approach because it simultaneously examines individual and place-based factors, and borrows strength from the global model to make small area inferences. The increasing availability of software products that can be used to perform multilevel SAE using weighted survey data (e.g., MLwiN, STATA's GLAMM, and Mplus), makes it possible for researchers without formal training in statistics to conduct small area research.
Examining health behaviors and outcomes at the local level facilitates policy making, program development and evaluation, and hypothesis generation. At the policy end, legislators need information about their jurisdictions to make informed choices about bills related to health education programs and mandates. For example,
provided asthma prevalence data at the Assembly district level in California to assist in advocacy and policy-making activities. With data on disease rates over space and time, healthcare administrators and researchers can also develop and evaluate health promotion and outreach programs, targeted and tailored to the communities with the greatest needs. Finally, SAE provides a foundation for hypothesis generation. Knowing which areas are most affected by a particular disease, or conversely, which have the best health behaviors, can lead to formulating and testing hypotheses about the causal mechanisms involved.
SAE is ideal for studying HPV vaccination for several reasons, including the following: 1) The Texas ImmTrac Vaccine Registry is a voluntary “opt-in” program, which limits the generalizability of the data, and 2) the data are not actively collected by schools or public health departments since it is not required for entry into Texas public schools. Our findings indicate wide variability in HPV vaccine coverage across counties (7%–29%). Many of the Southwest border counties exhibited high coverage rates, as did many large, metropolitan counties (e.g., Dallas, Brazos). Low coverage rates were common in the Panhandle (i.e., northernmost counties), Southeast border region, and the Northeast. Our multilevel model also showed some counterintuitive findings. Although county-level poverty was associated with increased odds of vaccination, PHR-level poverty was associated with decreased odds. Our findings paralleled those reported by
, in which HPV vaccination was positively associated with county-level poverty and negatively associated with state-level poverty. This suggests that county-level poverty may be more indicative of better financed public health programs and/or is strongly related to the racial/ethnic makeup of the county. However, not all counties were represented in our sample, which may have resulted in a biased measure of association for county-level poverty. The ability to interpret our findings on PHR-level poverty is limited by two issues: 1) PHR-level poverty represents a greater scale of geographic aggregation (i.e., sets of geographically contiguous counties, typically anchored by at least one metropolitan area), resulting in limited data variability and generalizability, and 2) the sample of respondents within each PHR varied widely, with some PHRs having respondents drawn from multiple counties and others having respondents from only one or two counties.
SAE has some inherent limitations. First, not all covariates of interest can be included. At level 1, only those covariates that are present in both the individual-level and auxiliary dataset can be included. It is possible, however, to make post-model adjustments (as we did for education) at the expense of model simplicity. Second, having no or few respondents at higher levels can limit estimation of random effects. In our study, we handled this by assigning the mean of simulated random effects to all non-sampled counties. Alternatively, one can ignore the random effects altogether (as we did for PHRs). Finally, there is often no gold standard for comparison of small area estimates. Without a reference, one cannot easily assess the accuracy of small area estimates. Several methods have been proposed to deal with this issue, including cross-validation using part of the sample, a different dataset, or data from another year (as long as the outcome is stable over time), as well as aggregation of one's estimates to a higher geographic level, where direct estimation and thus comparison and calibration (
Two lines of future research are proposed: Examination of how alternate modeling strategies might affect our estimates, and estimation of HPV vaccine coverage in other population groups. Alternative modeling strategies include 1) use of different sampling weights, 2) specifying other hierarchical structures, 3) removing random effects, 4) adding a spatial component to incorporate neighborhood information, and 5) evaluating the potential of other software programs (e.g., STATA's GLAMM procedure) to perform similar analyses with greater ease. Estimation of vaccine coverage among other population groups, such as males aged 9 to 26 and females aged 18 to 26, is also warranted.
Acknowledgments
This work was supported by National Cancer Institute pre- and post-doctoral fellowships (R25-CA057712, R25-CA57730) and a National Institutes of Health Cancer Center Support Grant (CA016672). The authors wish to acknowledge the excellent editorial contributions of Dr. Sandi Pruitt (UT Southwestern), Dr. Graham Moon (University of Southampton), and the Doctoral Behavioral Science Seminar at the University of Texas, School of Public Health at Houston. We also thank Dr. Christina Dorell, from the Centers for Disease Control & Prevention, whose suggestions were integral to the planning of this study. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the National Cancer Institute, the National Institutes of Health, or the Centers for Disease Control and Prevention. Gardasil is a registered trademark of Merck & Co., Inc.
U.S. Centers for Disease Control and Prevention. (2008a). Behavioral Risk Factor Surveillance System: 2008 summary data quality report. Retrieved September 2, 2011, from ftp://ftp.cdc.gov/pub/Data/Brfss/2008_Summary_Data_Quality_Report.pdf.
U.S. Centers for Disease Control and Prevention. (2009). U.S. Census populations with bridged race categories. Retrieved January 20, 2011, from http://www.cdc.gov/nchs/nvss/bridged_race.htm.
Jan M. Eberth, PhD, is an Assistant Professor of Epidemiology at the University of South Carolina, Arnold School of Public Health. Her research focuses on disparities in access, utilization, and quality of cancer prevention and treatment services.
Md Monir Hossain, PhD, is an Assistant Professor in the Department of Pediatrics at Cincinnati Children's Hospital Medical Center. Dr. Hossain works to develop innovative statistical methods for cluster detection and small area health analyses.
Jasmin A. Tiro, PhD, is an Assistant Professor in the Department of Clinical Sciences at the University of Texas Southwestern Medical Center. Her research centers on evaluating HPV vaccination behavior and cancer prevention delivery in clinical settings.
Xingyou Zhang, PhD, is a geographer and statistician in the Division of Adult and Community Health at the Centers for Disease Control and Prevention. Dr. Zhang's research focuses on the integration of geographic information and technology in epidemiologic studies.
James B. Holt, PhD, is a geographer in the Division of Adult and Community Health at the Centers for Disease Control and Prevention, and Team Leader of the Analytic Methods Branch. He utilizes geospatial data and methods for chronic disease prevention research.
Sally W. Vernon, PhD, is Professor of Epidemiology and Behavioral Sciences at The University of Texas Health Science Center at Houston School of Public Health. She is recognized for her work in the area of behavioral research to promote cancer screening.
Article info
Publication history
Accepted:
December 18,
2012
Received in revised form:
November 13,
2012
Received:
March 27,
2012
Footnotes
At the time this manuscript was accepted for publication, Dr. Eberth was with the MD Anderson Cancer Center in Houston, TX. At present, she is with the South Carolina Cancer Prevention and Control Program.
00112-0&id=gr1.jpg){kind=link}
00112-0&id=gr2.jpg){kind=link}