Advertisement

Enrollment and Retention of Men and Women in Health Services Research and Development Trials

  • Karen M. Goldstein
    Correspondence
    Correspondence to: Karen M. Goldstein, MD, MSPH, Department of Medicine, Duke University Medical Center, 508 Fulton Street, Durham, NC 27705. Phone: (919) 286.0411 ext 7764; fax: (919) 416.5836.
    Affiliations
    Durham Center of Innovation to Accelerate Discovery and Practice Transformation, Durham Veterans Affairs Health Care System, Durham, North Carolina

    Department of Medicine, Duke University Medical Center, Durham, North Carolina
    Search for articles by this author
  • Wei Duan-Porter
    Affiliations
    University of Minnesota Medical School, Minneapolis, Minnesota

    Center for Care Delivery & Outcomes Research, Minneapolis VA Health Care System, Minneapolis, Minnesota
    Search for articles by this author
  • Aviel Alkon
    Affiliations
    Durham Center of Innovation to Accelerate Discovery and Practice Transformation, Durham Veterans Affairs Health Care System, Durham, North Carolina

    Department of Medicine, Duke University Medical Center, Durham, North Carolina
    Search for articles by this author
  • Maren K. Olsen
    Affiliations
    Durham Center of Innovation to Accelerate Discovery and Practice Transformation, Durham Veterans Affairs Health Care System, Durham, North Carolina

    Department of Biostatistics and Bioinformatics, Duke University Medical Center, Durham, North Carolina
    Search for articles by this author
  • Corrine I. Voils
    Affiliations
    Department of Surgery, University of Wisconsin, Madison, Wisconsin

    William S. Middleton Memorial Veterans Hospital, Madison, Wisconsin
    Search for articles by this author
  • Susan N. Hastings
    Affiliations
    Durham Center of Innovation to Accelerate Discovery and Practice Transformation, Durham Veterans Affairs Health Care System, Durham, North Carolina

    Department of Medicine, Duke University Medical Center, Durham, North Carolina

    Geriatric Research, Education, and Clinical Center, Durham VA Medical Center, Durham, North Carolina

    Center for the Study of Human Aging and Development, Duke University, Durham, North Carolina
    Search for articles by this author

      Abstract

      Introduction

      Sex- and gender-specific science is essential to inform patient-centered, evidence-based care. Developing such evidence requires adequate inclusion of both women and men in trials. We sought to describe study participation of women and men in Department of Veterans Affairs Health Services Research and Development trials.

      Methods

      We identified recent health services research trials from one Health Services Research and Development Center of Innovation and compared the participation of women and men from trial recruitment to study completion. We also calculated the participation to prevalence ratio (PPR) by sex for each trial.

      Results

      We included eight trials that started recruitment between 2011 and 2014. Only one study purposefully attempted to boost the recruitment of women. Overall, the PPR for women ranged from 0.2 to 4.5, with seven studies having a PPR of greater than 1, indicating that women participated in these trials at proportions greater than their prevalence in the disease population within the Department of Veterans Affairs. The PPR for men ranged from 0.8 to 1.1. Retention was best with those studies that used administrative data for final outcomes assessment. No studies provided results stratified by sex or conducted analyses to explore treatment effect by sex.

      Conclusions

      At a single site, women participated in Health Services Research and Development trials at similar or greater rates to men without cross-study efforts to enrich the recruitment or retention of women. Adding strategic recruitment approaches could further boost the proportion of women in Department of Veterans Affairs trials and enable adequately powered sex-based analyses.
      Sex- and gender-based differences in the pathophysiology, quality of care, and patient experiences of disease require sex- and gender-specific science to inform the provision of patient-centered, evidence-based care. For example, when compared with men, women with coronary artery disease are less likely to take statins and antiplatelet medications (
      • Johansen M.E.
      • Hefner J.L.
      • Foraker R.E.
      Antiplatelet and statin use in US patients with coronary artery disease categorized by race/ethnicity and gender, 2003 to 2012.
      ), and respond differently to pharmacologic interventions for pain (
      • Bartley E.J.
      • Fillingim R.B.
      Sex differences in pain: Aa brief review of clinical and experimental findings.
      ). Acquiring the evidence needed to optimize patient-centered care for women requires equitable inclusion of women in clinical research. Importantly, we note that “sex” refers to a biologic concept based on genetics and anatomy at birth, and “gender” refers to a construct stemming from an individual's social, cultural, and biological life experience (
      • Clayton J.A.
      • Tannenbaum C.
      Reporting sex, gender, or both in clinical research?.
      ). The relative relevance of sex and gender to a particular study varies with the specific condition of interest. Because we looked across a collection of conditions in this work, we elected to use the single term “sex” and focus on the inclusion of women generally, acknowledging that this approach overlooks these distinctions.
      Historically, women were either categorically excluded or significantly under-represented in clinical trials (
      • Mazure C.M.
      • Jones D.P.
      Twenty years and still counting: Iincluding women as participants and studying sex and gender in biomedical research.
      ,
      • McCarthy C.R.
      Historical background of clinical trials involving women and minorities.
      ), which limited the ability to identify and understand potential treatment differences by sex. To address this problem, the National Institutes of Health and the U.S. Food and Drug Administration established regulations that require appropriate representation of women in studies ranging from basic science to clinical trials, or, if not, to justify why women were not included (
      • Baird K.L.
      The new NIH and FDA medical research policies: targeting Targeting gender, promoting justice.
      ,
      • Mazure C.M.
      • Jones D.P.
      Twenty years and still counting: Iincluding women as participants and studying sex and gender in biomedical research.
      ,
      • Miller L.R.
      • Marks C.
      • Becker J.B.
      • Hurn P.D.
      • Chen W.J.
      • Woodruff T.
      • Clayton J.A.
      Considering sex as a biological variable in preclinical research.
      ). These regulations and other advocacy efforts have led to significant improvements in trial participation by women (
      • Foulkes M.A.
      After inclusion, information and inference: Rreporting on clinical trials results after 15 years of monitoring inclusion of women.
      ), although levels still lag in certain therapeutic areas (e.g., heart failure and cancer research;
      • Melloni C.
      • Berger J.S.
      • Wang T.Y.
      • Gunes F.
      • Stebbins A.
      • Pieper K.S.
      • Newby L.K.
      Representation of women in randomized clinical trials of cardiovascular disease prevention.
      ,
      • Scott P.E.
      • Unger E.F.
      • Jenkins M.R.
      • Southworth M.R.
      • McDowell T.Y.
      • Geller R.J.
      • Woodcock J.
      Participation of women in clinical trials supporting FDA approval of cardiovascular drugs.
      ). In addition, the focus on boosting participation by women has largely centered on clinical and basic science research (
      • Mazure C.M.
      • Jones D.P.
      Twenty years and still counting: Iincluding women as participants and studying sex and gender in biomedical research.
      ) and less so on health services research.
      Achieving adequate participation of women in research is challenging in settings with relatively few women compared with men, such as the Department of Veterans Affairs (VA) health care system. Within the VA, women veterans are the fastest growing subpopulation (
      • Frayne S.M.
      • Phibbs C.
      • Saecho F.
      • Maisel N.C.
      • Friedman S.A.
      • Finlay A.
      • Berg E.
      • Balasubramanian V.
      • Dally S.K.
      • Ananth L.
      • Romodan Y.
      • Lee J.
      • Iqbal S.
      • Hayes P.M.
      • Zephyrin L.
      • Whitehead A.
      • Torgal A.
      • Katon J.G.
      • Haskell S.
      Sourcebook: Women Veterans in the Veterans Health Administration.
      ), but they continue to be a numerical minority in the overall veteran population at 9.4% (
      Fact Sheet: Women Veterans Population
      ). The recruitment of sufficient numbers of women participants is necessary for sex-based analyses. Inadequate participation may help to explain why even studies that include women veterans rarely report sex-based analyses. For example, a recent evidence map of women veterans' health literature identified 350 publications that included both men and women veterans but did not report sex-specific results (
      • Danan E.R.
      • Krebs E.E.
      • Ensrud K.
      • Koeller E.
      • MacDonald R.
      • Velasquez T.
      • Wilt T.J.
      An evidence map of the women veterans' health research literature (2008-2015).
      ). To conduct adequately powered sex-based analyses, most studies require participation among women veterans at rates higher than their population prevalence. The VA has made concerted efforts to enhance the inclusion of women veterans in its research program, including creating a national practice-based research network to boost inclusion of women in VA multisite studies (
      • Pomernacki A.
      • Carney D.V.
      • Kimerling R.
      • Nazarian D.
      • Blakeney J.
      • Martin B.D.
      • Frayne S.M.
      Lessons from initiating the first Veterans Health Administration (VA) Women's Health Practice-based Research Network (WH-PBRN) study.
      ,
      • Yano E.M.
      • Bastian L.A.
      • Bean-Mayberry B.
      • Eisen S.
      • Frayne S.
      • Hayes P.
      • Washington D.L.
      Using research to transform care for women veterans: Advancing the research agenda and enhancing research-clinical partnerships.
      ). How best to increase the recruitment of women veterans into intervention trials is unknown.
      Understanding past experiences with participation of women throughout the trial process (e.g., from recruitment to final outcome assessment) can inform efforts to increase the number of women in VA-based intervention research. To this end, our objectives were as follows: 1) describe participation by sex across nine study stages from identification for recruitment to outcome assessment, 2) identify and describe identified sex-based analyses, and 3) explore the relationship between study characteristics (e.g., telephone-based vs. in-person) and participation rates of women (as defined as the proportion women of total sample at each study stage) in Durham VA Health Services Research & Development Center of Innovation intervention trials over the last 10 years.

      Methods

       Setting

      The VA Health Services Research & Development (HSR&D) funds 18 Centers of Innovation (COIN) across the country. Each COIN has one or more focused areas of research addressing topics of significant importance to the VA and veteran health. The studies described here were conducted at the Durham Center of Innovation to Accelerate Discovery and Practice Transformation (previously known as the Durham Center for Health Services Research in Primary Care), which seeks to conduct high-quality research to enhance the VA's ability to continually improve health care and optimize veteran wellness and independence. Most Durham COIN studies recruit patients from within the Durham VA Health Care System (DVAHCS), which provides care to approximately 70,000 veterans from North Carolina and Southern Virginia; 10.3% of the veterans served by DVAHCS are women.
      The Durham COIN has been conducting trials over the last 35 years and currently supports 41 active investigators. COINs receive programmatic funding that supports resources shared across investigators and studies, including administrative, project management, statistical, and data infrastructure. To date, the Durham COIN has not used strategies across studies to promote the recruitment of women in trials. Since 2008, the Durham COIN created a participant tracking database application to be used for all trials to establish common variables to facilitate recruitment data integrity across the COIN's research program. Each study has its own study database and all studies are accessed via the same software program. This collaboration allows for quick startup time with fewer technical staff and the ability to maintain more studies than when each site uses its own customized data collection system. This method also allows for participant study stage (e.g., consented, randomized, withdrawn) to be coded identically across all studies, making analyses across studies more consistent.

       Study Identification

      Studies were eligible for inclusion if they met the following criteria: 1) had an active institutional review board (IRB) protocol, 2) involved the administration of an intervention to patients or caregivers, 3) had completed data collection, and 4) had data in our participant tracking database application or similar participant data tracking system. Studies were excluded if 1) they were a pilot study or 2) the tracking data were unsuitable for use owing to turnover in personnel and tracking systems during the study period.
      To identify eligible studies, we first compiled a list of studies conducted within our center since 2008 from the Research Development Information System, a budgetary and project data repository for managing all VA funded studies. We focused on funded studies because clinical trials were unlikely to be conducted without funding and pilots would have insufficient participant numbers. We then selected trials conducted since 2008 because that is when the participant tracking database application was created and would support comparison of participation by study stage across studies. We reviewed the initial list of studies for eligibility and worked as a team to establish agreement. Once a final list of studies was identified, we contacted the principal investigators individually to explain the purpose of this project and requested permission to include their study in our analysis. All principal investigators contacted agreed to have their studies included. Because all data for this analysis were either published or provided in aggregate by a member of a study's IRB-approved staff, we were not required to obtain separate IRB approval to conduct these analyses.

       Data Collection

      From the participant tracking database application, we identified the sex of participants at key points in the process of study recruitment and enrollment, including eligible at administrative data pull, recruitment letter sent, attempted telephone contact for screening, screened, refused, consented, completed primary outcome assessment, and completed final outcome assessment (if at a time point different from primary outcome). In addition, we defined characteristics critical to understanding the participation burden and intervention strategies used by individual trials. Specifically, we identified the following data elements: a priori enrollment target, target clinical population, eligibility criteria, recruitment strategy, duration of recruitment, number and timing of assessments and method (e.g., in person or by telephone), duration of intervention, primary outcome construct, intervention burden, and whether specific strategies were used to boost the recruitment of women. In addition, we collected the mean age and race/ethnicity of consented participants.
      Even with a common tracking database application, there were variations in the use of study participation stages driven by individual study structure and study team preferences. Thus, areas of uncertainty were resolved through discussion with the primary study team members. This was done in one of two ways, depending on study team preference. First, study teams could provide study stage of interest by sex in aggregate or, second, study teams could provide the study identification numbers for elements of the CONSORT diagram to the center programmer who is listed on each individual study IRB staff listing. Thus, it is possible that the numbers included for this analysis differ from those in individual study CONSORTs. Differences between the tracking database and CONSORT diagrams were generally small. These differences originated from study data collected as part of outcomes assessments or postrandomization exclusions that had not been incorporated into the tracking database. Neither study identification number nor other identifying information was shared outside those individuals on the IRB staff listing of any study.
      To collect study-specific intervention characteristics, we identified published reports for each study (both protocol and results papers, as applicable). For unpublished studies, we contacted the principal investigator and requested an unpublished draft or IRB-approved protocol, whichever was available. For some aspects of individual study recruitment (e.g., gender of study staff involved in recruitment), study coordinators were contacted directly. Information was abstracted initially by the first author and checked for accuracy by a second team member (W.D.P.). Discrepancies were resolved after discussion and agreement between team members.

       Analytic Plan

      Because the aim of this study was to describe our experience with recruiting for women veterans for trials, we used a descriptive analysis approach. For each included study, we calculated proportions by sex for each recruitment stage. We also calculated the proportion of participants retained by sex from attempted contact to consent and to completion of primary outcome assessment. No formal statistical testing was planned. However, we planned to look for trends across studies to determine if key, a priori identified aspects of studies seem to be related to recruitment and retention by sex.
      We also planned to calculate a participation to prevalence ratio (PPR), which can be used to consider the representation of a group of interest (e.g., women) in the trial relative to their proportion in the overall disease population (
      • Poon R.
      • Khanijow K.
      • Umarjee S.
      • Fadiran E.
      • Yu M.
      • Zhang L.
      • Parekh A.
      Participation of women and sex analyses in late-phase clinical trials of new molecular entity drugs and biologics approved by the FDA in 2007-2009.
      ,
      • Scott P.E.
      • Unger E.F.
      • Jenkins M.R.
      • Southworth M.R.
      • McDowell T.Y.
      • Geller R.J.
      • Woodcock J.
      Participation of women in clinical trials supporting FDA approval of cardiovascular drugs.
      ). We calculated the PPR by dividing the proportion of consented women among total trial participants by the proportion of women in the overall population of individuals with the disease or condition in question. When available, we used the established proportion of women among veterans with a certain disease or condition or, when not available, we estimated this proportion by adjusting the proportion in the civilian population to account for the smaller numbers of women in the veteran population. A ratio of 1 reflects an equivalent prevalence of women in the trial to the prevalence of women among those veterans with the disease or condition in question (
      • Poon R.
      • Khanijow K.
      • Umarjee S.
      • Fadiran E.
      • Yu M.
      • Zhang L.
      • Parekh A.
      Participation of women and sex analyses in late-phase clinical trials of new molecular entity drugs and biologics approved by the FDA in 2007-2009.
      ,
      • Scott P.E.
      • Unger E.F.
      • Jenkins M.R.
      • Southworth M.R.
      • McDowell T.Y.
      • Geller R.J.
      • Woodcock J.
      Participation of women in clinical trials supporting FDA approval of cardiovascular drugs.
      ). A PPR of less than 0.8 has been cited as a threshold to indicate that women were underrepresented in the trial compared with the disease population and a ratio greater than 1.2 would indicate overrepresentation (
      • Poon R.
      • Khanijow K.
      • Umarjee S.
      • Fadiran E.
      • Yu M.
      • Zhang L.
      • Parekh A.
      Participation of women and sex analyses in late-phase clinical trials of new molecular entity drugs and biologics approved by the FDA in 2007-2009.
      ,
      • Scott P.E.
      • Unger E.F.
      • Jenkins M.R.
      • Southworth M.R.
      • McDowell T.Y.
      • Geller R.J.
      • Woodcock J.
      Participation of women in clinical trials supporting FDA approval of cardiovascular drugs.
      ).

      Results

      We identified 46 studies from the Center since 2008, 8 of which met our inclusion criteria (Figure 1). These eight trials started recruitment between 2011 and 2014. Of these trials, one included dyads of patients and caregivers (
      • Van Houtven C.H.
      • Oddone E.Z.
      • Hastings S.N.
      • Hendrix C.
      • Olsen M.
      • Neelon B.
      • Weinberger M.
      Helping Invested Families Improve Veterans' Experiences Study (HI-FIVES): Study design and methodology.
      ), four targeted cardiovascular disease risk factors including obesity and hypertension (
      • Jackson G.L.
      • Weinberger M.
      • Kirshner M.A.
      • Stechuchak K.M.
      • Melnyk S.D.
      • Bosworth H.B.
      • Edelman D.
      Open-label randomized trial of titrated disease management for patients with hypertension: Study design and baseline sample characteristics.
      ,
      • Oddone E.Z.
      • Damschroder L.J.
      • Gierisch J.
      • Olsen M.
      • Fagerlin A.
      • Sanders L.
      • Juntilla K.
      A Coaching by Telephone Intervention for Veterans and Care Team Engagement (ACTIVATE): A study protocol for a hybrid type i effectiveness-implementation randomized controlled trial.
      ,
      • Voils C.I.
      • Gierisch J.M.
      • Olsen M.K.
      • Maciejewski M.L.
      • Grubber J.
      • McVay M.A.
      • Yancy Jr., W.S.
      Study design and protocol for a theory-based behavioral intervention focusing on maintenance of weight loss: Tthe Maintenance After Initiation of Nutrition TrAINing (MAINTAIN) study.
      ,
      • Voils C.I.
      • Grubber J.M.
      • McVay M.A.
      • Olsen M.K.
      • Bolton J.
      • Gierisch J.M.
      • Yancy Jr., W.S.
      Recruitment and retention for a weight loss maintenance trial involving weight loss prior to randomization.
      ,
      • Zullig L.L.
      • Melnyk S.D.
      • Stechuchak K.M.
      • McCant F.
      • Danus S.
      • Oddone E.
      • Bosworth H.B.
      The Cardiovascular Intervention Improvement Telemedicine Study (CITIES): Rrationale for a tailored behavioral and educational pharmacist-administered intervention for achieving cardiovascular disease risk reduction.
      ,
      • Zullig L.L.
      • Stechuchak K.M.
      • Goldstein K.M.
      • Olsen M.K.
      • McCant F.M.
      • Danus S.
      • Bosworth H.B.
      Patient-reported medication adherence barriers among patients with cardiovascular risk factors.
      ), one addressed smoking cessation (
      • Wilson S.M.
      • Hair L.P.
      • Hertzberg J.S.
      • Kirby A.C.
      • Olsen M.K.
      • Lindquist J.H.
      • Calhoun P.S.
      Abstinence Reinforcement Therapy (ART) for rural veterans: Methodology for an mHealth smoking cessation intervention.
      ), one addressed emergency department follow-up (
      • Hastings S.N.
      • Betts E.
      • Schmader K.E.
      • Weinberger M.
      • Van Houtven C.H.
      • Hendrix C.C.
      • Oddone E.Z.
      Discharge information and support for veterans receiving outpatient care in the emergency department: Study design and methods.
      ), and one addressed osteoarthritis of the knee and hip (
      • Allen K.D.
      • Bosworth H.B.
      • Brock D.S.
      • Chapman J.G.
      • Chatterjee R.
      • Coffman C.J.
      • Yancy Jr., W.S.
      Patient and provider interventions for managing osteoarthritis in primary care: protocols Protocols for two randomized controlled trials.
      ; Table 1). The mean participant age was 57.7 years for women and 65.9 years for men. On average, there was greater racial/ethnic diversity among women veterans compared with men; 43%–83% of women participants were African American compared with 33%–60% among men participants (see Table 2).
      Table 1Participation Stage and Study Characteristics of Interest
      StageDefinition
      Recruitment
       Eligible at data pullPatients meeting administrative inclusion criteria
       Letter sentPatients sent a recruitment letter by study team
       Attempted contact for screeningPatients contacted to conduct eligibility screening (in-person or by phone)
       ScreenedCompleted study screening for eligibility (either phone or in-person or both)
       RefusedRefused to participate in study at any time point
       Duration of recruitmentNumber of days between first letter sent and last consent completed
       ConsentedCompleted informed consent process
      Retention
       Primary outcome assessedCompleted assessment for primary outcome
       Final outcome assessedCompleted last outcome assessment (if timing different than primary outcome)
      Study characteristic
       Target populationPatients sought for study participation
       A priori recruitment targetRecruitment goal so stated prior to onset study activities
       Recruitment strategyMeans used by study team to identify patients willing to consent to study
       Number of assessmentsNumber of study outcome assessments
       Duration of interventionDuration of study intervention activities
       Frequency of intervention contactFrequency and type of contact required for study participation
       Telehealth involvementPresence and type of telehealth required for study
      Table 2Study Characteristics
      PRIMO (
      • Allen K.D.
      • Bosworth H.B.
      • Brock D.S.
      • Chapman J.G.
      • Chatterjee R.
      • Coffman C.J.
      • Yancy Jr., W.S.
      Patient and provider interventions for managing osteoarthritis in primary care: protocols Protocols for two randomized controlled trials.
      )
      CITIES (
      • Melnyk S.D.
      • Zullig L.L.
      • McCant F.
      • Danus S.
      • Oddone E.
      • Bastian L.
      • Bosworth H.B.
      Telemedicine cardiovascular risk reduction in veterans.
      ,
      • Zullig L.L.
      • Melnyk S.D.
      • Stechuchak K.M.
      • McCant F.
      • Danus S.
      • Oddone E.
      • Bosworth H.B.
      The Cardiovascular Intervention Improvement Telemedicine Study (CITIES): Rrationale for a tailored behavioral and educational pharmacist-administered intervention for achieving cardiovascular disease risk reduction.
      )
      MAINTAIN (
      • Voils C.I.
      • Gierisch J.M.
      • Olsen M.K.
      • Maciejewski M.L.
      • Grubber J.
      • McVay M.A.
      • Yancy Jr., W.S.
      Study design and protocol for a theory-based behavioral intervention focusing on maintenance of weight loss: Tthe Maintenance After Initiation of Nutrition TrAINing (MAINTAIN) study.
      )
      TDM (
      • Jackson G.L.
      • Weinberger M.
      • Kirshner M.A.
      • Stechuchak K.M.
      • Melnyk S.D.
      • Bosworth H.B.
      • Edelman D.
      Open-label randomized trial of titrated disease management for patients with hypertension: Study design and baseline sample characteristics.
      )
      DISPO-ED (
      • Hastings S.N.
      • Betts E.
      • Schmader K.E.
      • Weinberger M.
      • Van Houtven C.H.
      • Hendrix C.C.
      • Oddone E.Z.
      Discharge information and support for veterans receiving outpatient care in the emergency department: Study design and methods.
      )
      ART (
      • Wilson S.M.
      • Hair L.P.
      • Hertzberg J.S.
      • Kirby A.C.
      • Olsen M.K.
      • Lindquist J.H.
      • Calhoun P.S.
      Abstinence Reinforcement Therapy (ART) for rural veterans: Methodology for an mHealth smoking cessation intervention.
      )
      ACTIVATE (
      • Oddone E.Z.
      • Damschroder L.J.
      • Gierisch J.
      • Olsen M.
      • Fagerlin A.
      • Sanders L.
      • Juntilla K.
      A Coaching by Telephone Intervention for Veterans and Care Team Engagement (ACTIVATE): A study protocol for a hybrid type i effectiveness-implementation randomized controlled trial.
      )
      HI-FIVES (
      • Van Houtven C.H.
      • Oddone E.Z.
      • Hastings S.N.
      • Hendrix C.
      • Olsen M.
      • Neelon B.
      • Weinberger M.
      Helping Invested Families Improve Veterans' Experiences Study (HI-FIVES): Study design and methodology.
      )
      Start of recruitment20112011201220122013201320142014
      Target populationHip and/or knee OAUncontrolled hypertension and/or hyperlipidemiaObesity, interested in losing weightUncontrolled hypertension on medicationsDischarged from ED at high risk of recurrent ED visitsCurrent smokersAt least one modifiable risk factor for CVDCognitively and/or functionally impaired community-dwelling patients
      Mean age (y)
       Women6254606052535268
       Men6762696865625975
      Recruitment strategy
       In-person consentXXXXXX
       Provider referralXX
       Self-referralXX
       women staffXX
      Number of assessmentsBaseline (IP)

      6 mon (Ph)

      12 mon (IP)
      Baseline (IP)

      6 mon (IP)

      12 mon (IP)
      Baseline (IP)

      Week 14 (IP)

      Week 26 (IP)

      Week 42 (IP)

      Week 56 (IP)
      Baseline (IP)

      6 months (IP)

      12 months (IP)

      18 mon (IP)
      Baseline

      30 days

      180 days
      Baseline (Ph)

      3 mon (Ph)

      6 mon (Ph)

      12 mon (Ph)
      Baseline

      1 mon (Ph)

      6 mon (IP)
      Baseline (IP)

      3 mon (Ph)

      6 mon (Ph)

      12 mon (Ph)
      Duration of intervention12 months12 months40 weeks
      Study included a 16-week run-in period followed by a 40-week intervention.
      18 months1 month1.75 months1 month5 months
      Primary outcomeWOMAC at 12 monthsFramingham risk score at 12 monthsMean weight regain at 56 weeksSystolic blood pressureRepeat ED use in 30 days (admin)6 month abstinence (self-reported with bioconfirmation)Enrollment and participation in prevention program at 6 months (self-report)No. of days at home over 12 months (admin)
      Number and type of intervention contacts
       IP visits00000000
       Phone calls181289 to 182 + 15+25
       Group visits00300004
      Abbreviations: ACTIVATE, A Coaching by Telephone Intervention for Veterans and Care Team Engagement; admin, administrative; ART, Abstinence Reinforcement Therapy; CVD, cardiovascular disease; CITIES, Cardiovascular Intervention Improvement Telemedicine Study; DISPO-ED, Discharge Information and Support for Patients receiving Outpatient Care in the Emergency Department; ED, emergency department; HI-FIVES, Helping Invested Families Improve Veterans' Experiences Study; IP, in-person; MAINTAIN, Maintenance After Initiation of Nutrition Training; OA, osteoarthritis; ph, phone; PRIMO, Patient and Provider Intervention for Management of Osteoarthritis; TDM, Titrated Disease Management; WOMAC, Western Ontario and McMaster Universities Osteoarthritis Index.
      Study included a 16-week run-in period followed by a 40-week intervention.
      To see if women were under-represented at any one stage of trial recruitment compared with another, we looked at the proportion of potentially eligible participants who were women across each stage. In doing so, we found that the proportion of women ranged from 5.1% to 13.3% at the initial data pull, 4.4%–11.7% for mailed recruitment letters, 4.4%–19.2% during attempted to contact for screening, 3.3%–22.0% of those consented, and 3.3%–22.2% of those participants who completed the final outcome assessment. Five studies consented more than 10% women (
      • Hastings S.N.
      • Betts E.
      • Schmader K.E.
      • Weinberger M.
      • Van Houtven C.H.
      • Hendrix C.C.
      • Oddone E.Z.
      Discharge information and support for veterans receiving outpatient care in the emergency department: Study design and methods.
      ,
      • Oddone E.Z.
      • Damschroder L.J.
      • Gierisch J.
      • Olsen M.
      • Fagerlin A.
      • Sanders L.
      • Juntilla K.
      A Coaching by Telephone Intervention for Veterans and Care Team Engagement (ACTIVATE): A study protocol for a hybrid type i effectiveness-implementation randomized controlled trial.
      ,
      • Voils C.I.
      • Gierisch J.M.
      • Olsen M.K.
      • Maciejewski M.L.
      • Grubber J.
      • McVay M.A.
      • Yancy Jr., W.S.
      Study design and protocol for a theory-based behavioral intervention focusing on maintenance of weight loss: Tthe Maintenance After Initiation of Nutrition TrAINing (MAINTAIN) study.
      ,
      • Wilson S.M.
      • Hair L.P.
      • Hertzberg J.S.
      • Kirby A.C.
      • Olsen M.K.
      • Lindquist J.H.
      • Calhoun P.S.
      Abstinence Reinforcement Therapy (ART) for rural veterans: Methodology for an mHealth smoking cessation intervention.
      ,
      • Zullig L.L.
      • Melnyk S.D.
      • Stechuchak K.M.
      • McCant F.
      • Danus S.
      • Oddone E.
      • Bosworth H.B.
      The Cardiovascular Intervention Improvement Telemedicine Study (CITIES): Rrationale for a tailored behavioral and educational pharmacist-administered intervention for achieving cardiovascular disease risk reduction.
      ), and all but one of those targeted conditions that are not inherently tied to the patient's age (e.g., smoking, obesity, inactivity, or emergency department use). The one study that did target age-related conditions, specifically hypertension and hyperlipidemia, also aimed to increase recruitment of women (
      • Zullig L.L.
      • Melnyk S.D.
      • Stechuchak K.M.
      • McCant F.
      • Danus S.
      • Oddone E.
      • Bosworth H.B.
      The Cardiovascular Intervention Improvement Telemedicine Study (CITIES): Rrationale for a tailored behavioral and educational pharmacist-administered intervention for achieving cardiovascular disease risk reduction.
      ). Of the five studies with available numbers for participants completing screening and refusals, the proportion of women who refused participation was lower in three studies (
      • Hastings S.N.
      • Betts E.
      • Schmader K.E.
      • Weinberger M.
      • Van Houtven C.H.
      • Hendrix C.C.
      • Oddone E.Z.
      Discharge information and support for veterans receiving outpatient care in the emergency department: Study design and methods.
      ,
      • Oddone E.Z.
      • Damschroder L.J.
      • Gierisch J.
      • Olsen M.
      • Fagerlin A.
      • Sanders L.
      • Juntilla K.
      A Coaching by Telephone Intervention for Veterans and Care Team Engagement (ACTIVATE): A study protocol for a hybrid type i effectiveness-implementation randomized controlled trial.
      ,
      • Wilson S.M.
      • Hair L.P.
      • Hertzberg J.S.
      • Kirby A.C.
      • Olsen M.K.
      • Lindquist J.H.
      • Calhoun P.S.
      Abstinence Reinforcement Therapy (ART) for rural veterans: Methodology for an mHealth smoking cessation intervention.
      ), similar in one (
      • Allen K.D.
      • Bosworth H.B.
      • Brock D.S.
      • Chapman J.G.
      • Chatterjee R.
      • Coffman C.J.
      • Yancy Jr., W.S.
      Patient and provider interventions for managing osteoarthritis in primary care: protocols Protocols for two randomized controlled trials.
      ), and greater in one (
      • Van Houtven C.H.
      • Oddone E.Z.
      • Hastings S.N.
      • Hendrix C.
      • Olsen M.
      • Neelon B.
      • Weinberger M.
      Helping Invested Families Improve Veterans' Experiences Study (HI-FIVES): Study design and methodology.
      ). Of those who consented to study participation, 63.2%–100% of women completed the primary outcome assessment compared with 61.4%–99.6% of men; these proportions were generally similar within specific studies (data not shown). Overall, it seems that women were contacted for screening at similar or higher rates to their prevalence in the potential patient population, and it does not seem that women were disproportionately ineligible during the subsequent screening process (Table 3). Of those who were screened, women completed the consent process at similar or higher rates than men, suggesting that, overall, women are interested in participating if contacted and eligible (Table 3). The PPR ranged from 0.6 to 4.4, with six studies having a PPR of greater than 1 (Table 3). No studies provided results stratified by sex or conducted subgroup analyses by sex (or planned to a priori).
      Table 3Study Participation by Sex
      Study StagePRIMOCITIES
      Missing data owing to the potential for participants to have recruitment attempts and screenings multiple times.
      MAINTAIN
      The number listed is randomized participants because consent was required to complete eligibility assessment.
      TDM
      Missing data owing to the potential for participants to have recruitment attempts and screenings multiple times.
      DISPO-ED
      Missing data because participants were recruitment based on ED visits and not individual patients.
      ARTACTIVATE
      Includes data from participants across three sites.
      HI-FIVES
      TotalWomen, n (%)TotalWomen, n (%)TotalWomen, n (%)TotalWomen, n (%)TotalWomen, n (%)TotalWomen, n (%)TotalWomen, n (%)TotalWomen, n (%)
      Eligible at data pull7359626 (8.5)3427457 (13.3)156141852 (11.8)5812299 (5.1)3029258 (8.5)8041715 (8.9)3746243 (6.5)
      Letters sent1433120 (8.4)108451269 (11.7)2058177 (8.6)3016298 (9.9)181781 (4.4)
      Attempted contact for screening113089 (7.9)1253235 (18.8)2381117 (4.9)1702146 (8.6)181781 (4.4)
      Screened
      Values for screened, refused, consented are all after the run-in period.
      96174 (7.7)30458 (19.1)1378194 (14.1)1425120 (8.4)1800185 (10.3)171975 (4.4)
      Refused
      Values for screened, refused, consented are all after the run-in period.
      50339 (7.8)25237 (14.7)126840 (3.2)75870 (9.2)73157 (7.8)83961 (7.3)124461 (4.9)
      Consented
      Values for screened, refused, consented are all after the run-in period.
      30628 (9.2)43066 (15.3)22234 (15.3)38529 (7.5)535121 (22.6)33338 (11.4)43665 (14.9)2428 (3.3)
      Primary outcome assessed27125 (9.2)35250 (14.2)18927 (14.3)31525 (7.9)513114 (22.2)20524 (11.7)37653 (14.1)2418 (3.3)
      Retention, % (primary outcome/consented)88.589.281.975.885.179.481.886.295.994.261.663.286.281.599.6100
      Abbreviations: ACTIVATE, A Coaching by Telephone Intervention for Veterans and Care Team Engagement; ART, Abstinence Reinforcement Therapy; CITIES, Cardiovascular Intervention Improvement Telemedicine Study; DISPO-ED, Discharge Information and Support for Patients receiving Outpatient Care in the Emergency Department; HI-FIVES, Helping Invested Families Improve Veterans' Experiences Study; MAINTAIN, Maintenance After Initiation of Nutrition Training; PRIMO, Patient and Provider Intervention for Management of Osteoarthritis; TDM, Titrated Disease Management.
      Missing data owing to the potential for participants to have recruitment attempts and screenings multiple times.
      The number listed is randomized participants because consent was required to complete eligibility assessment.
      Missing data because participants were recruitment based on ED visits and not individual patients.
      § Includes data from participants across three sites.
      Values for screened, refused, consented are all after the run-in period.
      Six of the eight studies required an in-person visit to complete the informed consent process. Two studies had all recruitment processes handled by women study team members (
      • Hastings S.N.
      • Betts E.
      • Schmader K.E.
      • Weinberger M.
      • Van Houtven C.H.
      • Hendrix C.C.
      • Oddone E.Z.
      Discharge information and support for veterans receiving outpatient care in the emergency department: Study design and methods.
      ,
      • Oddone E.Z.
      • Damschroder L.J.
      • Gierisch J.
      • Olsen M.
      • Fagerlin A.
      • Sanders L.
      • Juntilla K.
      A Coaching by Telephone Intervention for Veterans and Care Team Engagement (ACTIVATE): A study protocol for a hybrid type i effectiveness-implementation randomized controlled trial.
      ). One study stratified randomization by sex and used a midcourse plan to increase the recruitment of women (
      • Zullig L.L.
      • Melnyk S.D.
      • Stechuchak K.M.
      • McCant F.
      • Danus S.
      • Oddone E.
      • Bosworth H.B.
      The Cardiovascular Intervention Improvement Telemedicine Study (CITIES): Rrationale for a tailored behavioral and educational pharmacist-administered intervention for achieving cardiovascular disease risk reduction.
      ). One study had a two-step recruitment process with cluster randomization of primary care providers and then recruitment of patients of enrolled primary care providers. Randomization was stratified by volume of women patients by primary care provider panel (<15% vs ≥15%;
      • Allen K.D.
      • Bosworth H.B.
      • Brock D.S.
      • Chapman J.G.
      • Chatterjee R.
      • Coffman C.J.
      • Yancy Jr., W.S.
      Patient and provider interventions for managing osteoarthritis in primary care: protocols Protocols for two randomized controlled trials.
      ). Multiple studies used exclusion criteria that disproportionately affect women, such as excluding individuals with the potential to become pregnant or those with female-predominant conditions like fibromyalgia. All studies required at least some phone-based intervention components ranging from 2 to 18 calls over the duration of the study. Two studies required in-person intervention group visits and no studies required in-person individual visits. Four studies featured an intervention duration of 3 months or less and four studies lasted more than 12 months. All but two studies had women-only interventionists (
      • Wilson S.M.
      • Hair L.P.
      • Hertzberg J.S.
      • Kirby A.C.
      • Olsen M.K.
      • Lindquist J.H.
      • Calhoun P.S.
      Abstinence Reinforcement Therapy (ART) for rural veterans: Methodology for an mHealth smoking cessation intervention.
      ,
      • Zullig L.L.
      • Melnyk S.D.
      • Stechuchak K.M.
      • McCant F.
      • Danus S.
      • Oddone E.
      • Bosworth H.B.
      The Cardiovascular Intervention Improvement Telemedicine Study (CITIES): Rrationale for a tailored behavioral and educational pharmacist-administered intervention for achieving cardiovascular disease risk reduction.
      ). Two studies collected primary outcomes via administrative data (
      • Hastings S.N.
      • Betts E.
      • Schmader K.E.
      • Weinberger M.
      • Van Houtven C.H.
      • Hendrix C.C.
      • Oddone E.Z.
      Discharge information and support for veterans receiving outpatient care in the emergency department: Study design and methods.
      ,
      • Van Houtven C.H.
      • Oddone E.Z.
      • Hastings S.N.
      • Hendrix C.
      • Olsen M.
      • Neelon B.
      • Weinberger M.
      Helping Invested Families Improve Veterans' Experiences Study (HI-FIVES): Study design and methodology.
      ), five required in-person visits (
      • Allen K.D.
      • Bosworth H.B.
      • Brock D.S.
      • Chapman J.G.
      • Chatterjee R.
      • Coffman C.J.
      • Yancy Jr., W.S.
      Patient and provider interventions for managing osteoarthritis in primary care: protocols Protocols for two randomized controlled trials.
      ,
      • Jackson G.L.
      • Weinberger M.
      • Kirshner M.A.
      • Stechuchak K.M.
      • Melnyk S.D.
      • Bosworth H.B.
      • Edelman D.
      Open-label randomized trial of titrated disease management for patients with hypertension: Study design and baseline sample characteristics.
      ,
      • Oddone E.Z.
      • Damschroder L.J.
      • Gierisch J.
      • Olsen M.
      • Fagerlin A.
      • Sanders L.
      • Juntilla K.
      A Coaching by Telephone Intervention for Veterans and Care Team Engagement (ACTIVATE): A study protocol for a hybrid type i effectiveness-implementation randomized controlled trial.
      ,
      • Voils C.I.
      • Gierisch J.M.
      • Olsen M.K.
      • Maciejewski M.L.
      • Grubber J.
      • McVay M.A.
      • Yancy Jr., W.S.
      Study design and protocol for a theory-based behavioral intervention focusing on maintenance of weight loss: Tthe Maintenance After Initiation of Nutrition TrAINing (MAINTAIN) study.
      ,
      • Zullig L.L.
      • Melnyk S.D.
      • Stechuchak K.M.
      • McCant F.
      • Danus S.
      • Oddone E.
      • Bosworth H.B.
      The Cardiovascular Intervention Improvement Telemedicine Study (CITIES): Rrationale for a tailored behavioral and educational pharmacist-administered intervention for achieving cardiovascular disease risk reduction.
      ), and one used phone and mail methods (
      • Wilson S.M.
      • Hair L.P.
      • Hertzberg J.S.
      • Kirby A.C.
      • Olsen M.K.
      • Lindquist J.H.
      • Calhoun P.S.
      Abstinence Reinforcement Therapy (ART) for rural veterans: Methodology for an mHealth smoking cessation intervention.
      ). Across all studies, there were three to five study outcome assessments, but two studies used remote or phone assessments only (
      • Hastings S.N.
      • Betts E.
      • Schmader K.E.
      • Weinberger M.
      • Van Houtven C.H.
      • Hendrix C.C.
      • Oddone E.Z.
      Discharge information and support for veterans receiving outpatient care in the emergency department: Study design and methods.
      ,
      • Wilson S.M.
      • Hair L.P.
      • Hertzberg J.S.
      • Kirby A.C.
      • Olsen M.K.
      • Lindquist J.H.
      • Calhoun P.S.
      Abstinence Reinforcement Therapy (ART) for rural veterans: Methodology for an mHealth smoking cessation intervention.
      ).

      Discussion

      Across eight studies recently conducted at our HSR&D COIN, we found that women were a minority of consented participants. However, for five of the eight studies, women were consented at greater proportions than the local (10.3%) and national (9.4%) proportion of women veteran patients (
      Fact Sheet: Women Veterans Population
      ,
      • Williams-Johnson S.G.K.
      VISN 6 Women's Health DASHBOARD.
      ). In addition, most studies enrolled women at a greater percentage than the estimated proportion of women in the veteran population with the condition of interest. This occurred despite only one study using specific strategies to over-recruit women and several studies using design elements that could preferentially exclude women (e.g., high burden of in-person visits, ineligibility criteria related to conditions highly prevalent among women). In addition, the proportion of women generally increased from the initial data pull stage to consent, and women completed final outcomes assessments in similar proportions to men. Of note, studies that used administrative data for primary outcomes measurements had the greatest retention rates.
      Our finding that women veterans consent to study participation and complete study activities at similar or greater rates to men is encouraging. One might expect that women veterans would be less likely to engage in VA research studies given the multiple barriers and misconceptions expressed by women veterans about VA care (). If women and men participate in HSR&D trials at similar rates, then efforts to ensure the adequate representation of women should consider expanding recruitment outreach to potentially eligible women. Such approaches to boost the recruitment of minority populations in non-VA settings have been multipronged, including tailoring study recruitment messages by sex/gender (
      • Bass S.B.
      • Wolak C.
      • Greener J.
      • Tedaldi E.
      • Nanavati A.
      • Ruppert K.
      • Gordon T.F.
      Using perceptual mapping methods to understand gender differences in perceived barriers and benefits of clinical research participation in urban minority HIV+ patients.
      ) and cultural context (
      • Bonevski B.
      • Randell M.
      • Paul C.
      • Chapman K.
      • Twyman L.
      • Bryant J.
      • Hughes C.
      Reaching the hard-to-reach: Aa systematic review of strategies for improving health and medical research with socially disadvantaged groups.
      ), investing in interpersonal relationships with study staff (
      • Barnett J.
      • Aguilar S.
      • Brittner M.
      • Bonuck K.
      Recruiting and retaining low-income, multi-ethnic women into randomized controlled trials: successful Successful strategies and staffing.
      ,
      • BeLue R.
      • Taylor-Richardson K.D.
      • Lin J.
      • Rivera A.T.
      • Grandison D.
      African Americans and participation in clinical trials: Ddifferences in beliefs and attitudes by gender.
      ), and engaging with target populations to direct recruitment efforts (
      • Falcon R.
      • Bridge D.A.
      • Currier J.
      • Squires K.
      • Hagins D.
      • Schaible D.
      • Mrus J.
      Recruitment and retention of diverse populations in antiretroviral clinical trials: Ppractical applications from the gender, race and clinical experience study.
      ). The VA is working on how to apply these principles to the veteran population. First, an ongoing VA Cooperative Studies Program multicenter post-traumatic stress disorder trial is exploring effective options for boosting the recruitment of women in a mixed-sex context (
      • Pomernacki A.
      • Carney D.V.
      • Frayne S.M.
      • Nevedal A.
      • Caudle K.
      • Zehm L.
      • Shih M.
      • Chen T.
      • Chow B.
      • Goldstein K.
      • Bastian L.
      • Chard K.
      • Ruzek J.
      • Huang G.
      • Johnson M.
      • Yano E.M.
      • Schnurr P.
      Qualitative program evaluation design for the Women's Enhanced Recruitment Process (WERP) of the VA Women's Health Practice-Based Research Network (WH-PBRN). Paper presented at the North American Primary Care.
      ). Second, the VA has invested in veteran engagement activities for research, which creates a prime opportunity to obtain input about the relevance and appropriateness of study recruitment strategies from women veterans themselves.
      None of the included studies had planned analyses a priori to identify the relationship between sex and the treatment effect. This finding is not surprising, because most trials are not powered to conduct subgroup analyses. To engage and enroll adequate numbers of women veterans to examine sex-specific effects, we need to oversample women or use other nonprobability sampling approaches, which have been successful in the recruitment of socially disadvantaged populations (
      • Bonevski B.
      • Randell M.
      • Paul C.
      • Chapman K.
      • Twyman L.
      • Bryant J.
      • Hughes C.
      Reaching the hard-to-reach: Aa systematic review of strategies for improving health and medical research with socially disadvantaged groups.
      ,
      • King D.W.
      • Duello T.M.
      • Miranda P.Y.
      • Hodges K.P.
      • Shelton A.J.
      • Chukelu P.
      • Jones L.A.
      Strategies for recruitment of healthy premenopausal women into the African American Nutrition for Life (A NULIFE) Study.
      ). Which studies warrant enhanced enrollment of women depends on thoughtful consideration of treatment and sex effects for the specific condition under study and hypothesis-driven a priori defined analyses. At a minimum, making sex-stratified results available for potential future pooled analyses would be feasible regardless of an individual study's ability to conduct statistically meaningful analyses by sex.
      We identified a number of eligibility criteria used in the included studies that could lead to the preferential exclusion of women over men. Of specific concern were those criteria related to age, potential reproductive status, and health conditions with differential sex-based predominance. First, age is particularly important when disease onset occurs at different ages for women versus men (e.g., cardiovascular disease [
      • Dodd K.S.
      • Saczynski J.S.
      • Zhao Y.
      • Goldberg R.J.
      • Gurwitz J.H.
      Exclusion of older adults and women from recent trials of acute coronary syndromes.
      ,
      • Tsang W.
      • Alter D.A.
      • Wijeysundera H.C.
      • Zhang T.
      • Ko D.T.
      The impact of cardiovascular disease prevalence on women's enrollment in landmark randomized cardiovascular trials: Aa systematic review.
      ] and stroke [
      • Foerch C.
      • Czapowski D.
      • Misselwitz B.
      • Steinmetz H.
      • Neumann-Haefelin T.
      Gender imbalances induced by age limits in stroke trials.
      ]). Moreover, in the VA, age has specific implications in the veteran population because women veterans are on average 10 years younger than men veterans (
      • Frayne S.M.
      • Phibbs C.S.
      • Friendman S.A.
      • Shaw J.G.
      • Romodan Y.
      • Berg E.
      • Lee J.
      • Anath L.
      • Iqbal S.
      • Hayes P.M.
      • Haskell S.
      Sourcebook: Women Veterans in the Veterans Health Administration. Volume 4.
      ). Second, potential reproductive status (or having the potential to become pregnant) should not be an exclusion criterion by default. Whether or not pregnancy is a barrier to participation depends on the intervention in question. When we are uncertain of increased risk in the setting of pregnancy, informed consent procedures can be strengthened to communicate this risk with potential participants and engage them in shared decision making. Finally, the exclusion of health conditions especially common among women (e.g., inflammatory disorders) can inadvertently limit study participation by women. Similar findings regarding the role of eligibility criteria and the preferential exclusion of women and other vulnerable populations have been described among non-VA trials (
      • Clayton J.A.
      • Arnegard M.E.
      Taking cardiology clinical trials to the next level: A call to action.
      ,
      • Humphreys K.
      • Weingardt K.R.
      • Harris A.H.
      Influence of subject eligibility criteria on compliance with National Institutes of Health guidelines for inclusion of women, minorities, and children in treatment research.
      ,
      • Phillips S.P.
      • Hamberg K.
      Doubly blind: A systematic review of gender in randomised controlled trials.
      ).
      One study (
      • Van Houtven C.H.
      • Oddone E.Z.
      • Hastings S.N.
      • Hendrix C.
      • Olsen M.
      • Neelon B.
      • Weinberger M.
      Helping Invested Families Improve Veterans' Experiences Study (HI-FIVES): Study design and methodology.
      ) recruited both veterans and their caregivers. Although the caregiver did not need to be a spouse, it is notable that women veterans are less likely than men to be married or partnered and less likely to have someone accompany them to their doctor visits (
      • Frayne S.M.
      • Parker V.A.
      • Christiansen C.L.
      • Loveland S.
      • Seaver M.R.
      • Kazis L.E.
      • Skinner K.M.
      Health status among 28,000 women veterans. The VA Women's Health Program Evaluation Project.
      ). The implications of eligibility criteria for preferential exclusion of certain subpopulations including women have been examined in detail in particular types of trials (e.g., mental health conditions [
      • Humphreys K.A.
      Review of the impact of exclusion criteria on the generalizability of schizophrenia treatment research.
      ,
      • Humphreys K.
      • Blodgett J.C.
      • Roberts L.W.
      The exclusion of people with psychiatric disorders from medical research.
      ] and substance use disorders [
      • Moberg C.A.
      • Humphreys K.
      Exclusion criteria in treatment research on alcohol, tobacco and illicit drug use disorders: A review and critical analysis.
      ]) and have clear implications for the generalizability of scientific findings. A clear scientific justification should be provided when studies use criteria that potentially excludes one sex over another (
      • Heidari S.
      • Babor T.F.
      • De Castro P.
      • Tort S.
      • Curno M.
      Sex and gender equity in research: Rrationale for the SAGER guidelines and recommended use.
      ).
      The findings presented here should be considered in the context of important limitations. First, only studies conducted in one center in one VA system were included; thus, the findings in other VA contexts and other veteran populations may be different. For example, DVAHCS had one of the first comprehensive women's health clinics in the country and one of the first VA Women's Health Practice Based Research Network sites. It is possible that this historic local focus on women's health influences the way that women participate in research studies. Confirmation of these findings at other sites would strengthen these results. Second, we only considered intervention trials and participation rates may be different for other types of studies with prospective data collection. Third, because data collection about women at different study stages varied somewhat across the included studies, this factor may have introduced inconsistencies. In addition, there were differences in the way study teams defined different study stages, which led to challenges with pooling data across studies. To advance the science in the area of study recruitment and process efficiency, harmonizing measures while allowing for scientific flexibility and innovation is important. Given the current VA and National Institutes of Health requirement for reporting findings by sex, researchers should anticipate this need at the outset of study planning, including when designing database development and tracking systems. Of note, because the stages of interest in this project were sometimes different than those used by a given study's CONSORT flow diagram, there may be some differences in numbers reported here compared with an individual study's publication(s). Finally, the estimates for the percentage of women among veteran disease population used to calculate the PPR were not uniform in their derivation. They were estimated from the best available data source to approximate this value. Specific caveats are listed in the footnotes of Table 4.
      Table 4Participation of Women in HSR&D Trials
      PRIMOCITIES
      Eligibility criteria are not mutually exclusive. However, CITIES estimate is the cumulative estimate of those with uncontrolled hypertension and uncontrolled hyperlipidemia. For ACTIVATE, the lower end of estimate range is for obesity and current smoking alone and upper end of range is for all three prevalence estimates as if mutually exclusive. Range provided because 80% of ACTIVATE participants met criteria for obesity alone and 47.6% had at least two risk factors (Oddone et al., 2017).
      MAINTAINTDMDISPO-EDARTACTIVATE
      Eligibility criteria are not mutually exclusive. However, CITIES estimate is the cumulative estimate of those with uncontrolled hypertension and uncontrolled hyperlipidemia. For ACTIVATE, the lower end of estimate range is for obesity and current smoking alone and upper end of range is for all three prevalence estimates as if mutually exclusive. Range provided because 80% of ACTIVATE participants met criteria for obesity alone and 47.6% had at least two risk factors (Oddone et al., 2017).
      HI-FIVES
      WomenMenWomenMenWomenMenWomenMenWomenMenWomenMenWomenMenWomenMen
      Participation to prevalence ratio1.11.04.50.92.00.92.41.04.50.81.80.951.7–2.10.90.2–0.41.0–1.1
      Percentage of trial participants990.815.384.715.384.77.592.522.677.411.488.614.985.13.396.7
      Percentage among veteran disease population
      (The number of women veteran with a given condition)/(the total number of veterans with a given condition).
      8923.496.67.592.53.296.8595.06.393.77.0–8.891.2–93.07.6–15.584.5–92.4
      Estimated no. of women veterans with condition(s)574,621
      (The number of women veteran with a given condition)/(the total number of veterans with a given condition).
      115,974
      Eligibility criteria are not mutually exclusive. However, CITIES estimate is the cumulative estimate of those with uncontrolled hypertension and uncontrolled hyperlipidemia. For ACTIVATE, the lower end of estimate range is for obesity and current smoking alone and upper end of range is for all three prevalence estimates as if mutually exclusive. Range provided because 80% of ACTIVATE participants met criteria for obesity alone and 47.6% had at least two risk factors (Oddone et al., 2017).
      152,72949,60794,21798,336251,065–1,312,185283,024
      Total number estimated from RAND report (Ramchand et al., 2014).
      Estimated no. men veterans with condition(s)6,631,697
      Rates of osteoarthritis overall, not joint specific.
      3,295,646
      Eligibility criteria are not mutually exclusive. However, CITIES estimate is the cumulative estimate of those with uncontrolled hypertension and uncontrolled hyperlipidemia. For ACTIVATE, the lower end of estimate range is for obesity and current smoking alone and upper end of range is for all three prevalence estimates as if mutually exclusive. Range provided because 80% of ACTIVATE participants met criteria for obesity alone and 47.6% had at least two risk factors (Oddone et al., 2017).
      1,872,5091,510,3391,790,1141,460,5653,333,074–13,526,9391,538,467
      Total number estimated from RAND report (Ramchand et al., 2014).
      DiseaseArthritisUncontrolled hypertension and/or hyperlipidemiaObesityUncontrolled hypertensionDischarged from emergency roomCurrent smokersObesity or current smoker or <150 min moderate/vigorous activity per week
      Applies rates of physical inactivity (<150 min/wk) from Littman et al. 2009 to the total veteran population in 2015.
      Cognitively and/or functionally impaired community-dwelling patients
      Estimate source(s)
      National Center for Veterans Analysis and Statistics
      VETPOP Living veterans by age group, gender, 2015-2045.
      ,
      • Murphy L.B.
      • Helmick C.G.
      • Allen K.D.
      • Theis K.A.
      • Baker N.A.
      • Murray G.R.
      • Barbour K.E.
      Arthritis among veterans - United States, 2011-2013.
      • Frayne S.M.
      • Phibbs C.
      • Saecho F.
      • Maisel N.C.
      • Friedman S.A.
      • Finlay A.
      • Berg E.
      • Balasubramanian V.
      • Dally S.K.
      • Ananth L.
      • Romodan Y.
      • Lee J.
      • Iqbal S.
      • Hayes P.M.
      • Zephyrin L.
      • Whitehead A.
      • Torgal A.
      • Katon J.G.
      • Haskell S.
      Sourcebook: Women Veterans in the Veterans Health Administration.
      ,
      • Fryar C.D.
      • Ostchega Y.
      • Hales C.M.
      • Zhang G.
      • Kruszon-Moran D.
      Hypertension prevalence and control among adults: United States, 2015-2016.
      ,
      Vital signs
      Prevalence, treatment, and control of high levels of low-density lipoprotein cholesterol--United States, 1999-2002 and 2005-200.
      • Breland J.Y.
      • Phibbs C.S.
      • Hoggatt K.J.
      • Washington D.L.
      • Lee J.
      • Haskell S.
      • Frayne S.M.
      The obesity epidemic in the Veterans Health Administration: Prevalence among key populations of women and men veterans.
      • Frayne S.M.
      • Phibbs C.
      • Saecho F.
      • Maisel N.C.
      • Friedman S.A.
      • Finlay A.
      • Berg E.
      • Balasubramanian V.
      • Dally S.K.
      • Ananth L.
      • Romodan Y.
      • Lee J.
      • Iqbal S.
      • Hayes P.M.
      • Zephyrin L.
      • Whitehead A.
      • Torgal A.
      • Katon J.G.
      • Haskell S.
      Sourcebook: Women Veterans in the Veterans Health Administration.
      ,
      • Frayne S.M.
      • Phibbs C.
      • Saecho F.
      • Maisel N.C.
      • Friedman S.A.
      • Finlay A.
      • Berg E.
      • Balasubramanian V.
      • Dally S.K.
      • Ananth L.
      • Romodan Y.
      • Lee J.
      • Iqbal S.
      • Hayes P.M.
      • Zephyrin L.
      • Whitehead A.
      • Torgal A.
      • Katon J.G.
      • Haskell S.
      Sourcebook: Women Veterans in the Veterans Health Administration.
      ,
      • Fryar C.D.
      • Ostchega Y.
      • Hales C.M.
      • Zhang G.
      • Kruszon-Moran D.
      Hypertension prevalence and control among adults: United States, 2015-2016.
      • Kessler C.
      • Chen J.
      • Dill C.
      • Tyndall G.
      • Olszyk M.D.
      State of affairs of emergency medicine in the Veterans Health Administration.
      ,
      • Kessler C.S.
      • Bhandarkar S.
      • Casey P.
      • Tenner A.
      Predicting patient patterns in veterans administration emergency departments.
      Department of Veterans Affairs
      Survey of Veteran Enrollees' Health and Reliance Upon VA: with selected comparisons to the 1999-2010 Surveys.
      • Breland J.Y.
      • Phibbs C.S.
      • Hoggatt K.J.
      • Washington D.L.
      • Lee J.
      • Haskell S.
      • Frayne S.M.
      The obesity epidemic in the Veterans Health Administration: Prevalence among key populations of women and men veterans.
      ,
      • Littman A.J.
      • Forsberg C.W.
      • Koepsell T.D.
      Physical activity in a national sample of veterans.
      • Ramchand R.
      • Taneilian T.
      • Fisher M.P.
      • Vaughan C.A.
      • Trial T.E.
      • Epley C.
      • Voorhies P.
      • Robbins M.W.
      • Robinson E.
      • Ghosh-Dastidar B.
      Hidden heroes: America's military caregivers.
      ,
      • Van Houtven C.H.
      • Smith V.A.
      • Stechuchak K.M.
      • Shepherd-Banigan M.
      • Hastings S.N.
      • Maciejewski M.L.
      • Oddone E.Z.
      Comprehensive support for family caregivers: Impact on veteran health care utilization and costs.
      Abbreviations: ACTIVATE, A Coaching by Telephone Intervention for Veterans and Care Team Engagement; ART, Abstinence Reinforcement Therapy; CITIES, Cardiovascular Intervention Improvement Telemedicine Study; DISPO-ED, Discharge Information and Support for Patients receiving Outpatient Care in the Emergency Department; HI-FIVES, Helping Invested Families Improve Veterans' Experiences Study; MAINTAIN, Maintenance After Initiation of Nutrition Training; PRIMO, Patient and Provider Intervention for Management of Osteoarthritis; TDM, Titrated Disease Management.
      Eligibility criteria are not mutually exclusive. However, CITIES estimate is the cumulative estimate of those with uncontrolled hypertension and uncontrolled hyperlipidemia. For ACTIVATE, the lower end of estimate range is for obesity and current smoking alone and upper end of range is for all three prevalence estimates as if mutually exclusive. Range provided because 80% of ACTIVATE participants met criteria for obesity alone and 47.6% had at least two risk factors (
      • Oddone E.Z.
      • Damschroder L.J.
      • Gierisch J.
      • Olsen M.
      • Fagerlin A.
      • Sanders L.
      • Juntilla K.
      A Coaching by Telephone Intervention for Veterans and Care Team Engagement (ACTIVATE): A study protocol for a hybrid type i effectiveness-implementation randomized controlled trial.
      ).
      (The number of women veteran with a given condition)/(the total number of veterans with a given condition).
      Rates of osteoarthritis overall, not joint specific.
      § Total number estimated from RAND report (
      • Ramchand R.
      • Taneilian T.
      • Fisher M.P.
      • Vaughan C.A.
      • Trial T.E.
      • Epley C.
      • Voorhies P.
      • Robbins M.W.
      • Robinson E.
      • Ghosh-Dastidar B.
      Hidden heroes: America's military caregivers.
      ).
      Applies rates of physical inactivity (<150 min/wk) from
      • Littman A.J.
      • Forsberg C.W.
      • Koepsell T.D.
      Physical activity in a national sample of veterans.
      to the total veteran population in 2015.

      Implications for Practice and/or Policy

      The VA's efforts to promote women's health research have been successful to date, with a significant growth in publications and awarded grants in this area. To achieve equitable benefit of VA research for both women and men, investigators should be strongly encouraged to report results by sex and include sex-based analyses when possible. Fortunately, we found that women veterans participate in HSR&D trials at similar or greater rates as men veterans when invited to participate.
      In alignment with regulatory guidance and scientific rationale, researchers should make an a priori determination about the plausibility of differential treatment effects between men and women, and, if there is a scientific basis for a potential difference, then develop a plan to enhance the recruitment of women to support an adequately powered moderator analysis. When sufficiently enhanced recruitment is not feasible, studies should report results stratified by sex to support future meta-analyses with individual participant data. Studies and entities seeking to increase the research participation of women to support subgroup analyses and/or stratification of findings by sex should consider a multifaceted approach, including potentially oversampling women, minimizing study burden, and engaging women veterans for the tailoring of study materials and activities.
      The ideal PPR to conduct adequately powered subgroup analyses for a given line of inquiry depends on the effect size that a particular study is trying to detect; however, this routine assessment of sex-based participation using this ratio across studies and/or centers could help to identify topic areas and allow research entities to better understand their inclusion of women. Sex-based recruitment calculations might be particularly helpful during the study planning phase and feasibility assessments. Although incorporating additional strategies to boost the inclusion of women participants may be perceived as burdensome, many such strategies will likely benefit all potential participants and could help overall recruitment efficiency. The use of administrative data for key outcome metrics when possible may also be useful to improve retention to outcome assessment for all participants. Finally, embedding recruitment studies within trials could inform areas for further improving the effectiveness and efficiency of the recruitment process and ensure the adequate inclusion of women and other under-represented groups.

      Conclusions

      At a single site, women participated in HSR&D trials at similar or greater rates as men without specific cross-study efforts to enrich recruitment of women. Strategic approaches to sex-based recruitment could further boost the proportion of women veterans included in VA trials and support adequately powered sex-based analyses where appropriate.

      Acknowledgments

      The authors thank all Durham HSR&D COIN staff and investigators.

      Supplementary Data

      References

        • Allen K.D.
        • Bosworth H.B.
        • Brock D.S.
        • Chapman J.G.
        • Chatterjee R.
        • Coffman C.J.
        • Yancy Jr., W.S.
        Patient and provider interventions for managing osteoarthritis in primary care: protocols Protocols for two randomized controlled trials.
        BMC Musculoskeletal Disorders. 2012; 13: 60
        • Baird K.L.
        The new NIH and FDA medical research policies: targeting Targeting gender, promoting justice.
        Journal of Health Politics, Policy and Law. 1999; 24: 531-565
        • Barnett J.
        • Aguilar S.
        • Brittner M.
        • Bonuck K.
        Recruiting and retaining low-income, multi-ethnic women into randomized controlled trials: successful Successful strategies and staffing.
        Contemporary Clinical Trials. 2012; 33: 925-932
        • Bartley E.J.
        • Fillingim R.B.
        Sex differences in pain: Aa brief review of clinical and experimental findings.
        British Journal of Anaesthesia. 2013; 111: 52-58
        • Bass S.B.
        • Wolak C.
        • Greener J.
        • Tedaldi E.
        • Nanavati A.
        • Ruppert K.
        • Gordon T.F.
        Using perceptual mapping methods to understand gender differences in perceived barriers and benefits of clinical research participation in urban minority HIV+ patients.
        AIDS Care. 2016; 28: 528-536
        • BeLue R.
        • Taylor-Richardson K.D.
        • Lin J.
        • Rivera A.T.
        • Grandison D.
        African Americans and participation in clinical trials: Ddifferences in beliefs and attitudes by gender.
        Contemporary Clinical Trials. 2006; 27: 498-505
        • Bonevski B.
        • Randell M.
        • Paul C.
        • Chapman K.
        • Twyman L.
        • Bryant J.
        • Hughes C.
        Reaching the hard-to-reach: Aa systematic review of strategies for improving health and medical research with socially disadvantaged groups.
        BMC Medical Research Methodology. 2014; 14: 42
        • Breland J.Y.
        • Phibbs C.S.
        • Hoggatt K.J.
        • Washington D.L.
        • Lee J.
        • Haskell S.
        • Frayne S.M.
        The obesity epidemic in the Veterans Health Administration: Prevalence among key populations of women and men veterans.
        Journal of General Internal Medicine. 2017; 32: 11-17
        • Clayton J.A.
        • Arnegard M.E.
        Taking cardiology clinical trials to the next level: A call to action.
        Clinical Cardiology. 2018; 41: 179-184
        • Clayton J.A.
        • Tannenbaum C.
        Reporting sex, gender, or both in clinical research?.
        JAMA. 2016; 316: 1863-1864
        • Danan E.R.
        • Krebs E.E.
        • Ensrud K.
        • Koeller E.
        • MacDonald R.
        • Velasquez T.
        • Wilt T.J.
        An evidence map of the women veterans' health research literature (2008-2015).
        Journal of General Internal Medicine. 2017; 32: 1359-1376
        • Department of Veterans Affairs
        Survey of Veteran Enrollees' Health and Reliance Upon VA: with selected comparisons to the 1999-2010 Surveys.
        Office of the Assistant Deputy Under Secretary for Health for Policy and Planning, Washington, DC2011
        • Dodd K.S.
        • Saczynski J.S.
        • Zhao Y.
        • Goldberg R.J.
        • Gurwitz J.H.
        Exclusion of older adults and women from recent trials of acute coronary syndromes.
        Jorunal of the American Geriatrics Society. 2011; 59: 506-511
        • Fact Sheet: Women Veterans Population
        (Available:)
        www.va.gov/womenvet/docs/WomenVeteransPopulationFactSheet.pdf
        Date: 2017
        Date accessed: February 26, 2019
        • Falcon R.
        • Bridge D.A.
        • Currier J.
        • Squires K.
        • Hagins D.
        • Schaible D.
        • Mrus J.
        Recruitment and retention of diverse populations in antiretroviral clinical trials: Ppractical applications from the gender, race and clinical experience study.
        Journal of Womens Health (Larchmont). 2011; 20: 1043-1050
        • Foerch C.
        • Czapowski D.
        • Misselwitz B.
        • Steinmetz H.
        • Neumann-Haefelin T.
        Gender imbalances induced by age limits in stroke trials.
        Neuroepidemiology. 2010; 35: 226-230
        • Foulkes M.A.
        After inclusion, information and inference: Rreporting on clinical trials results after 15 years of monitoring inclusion of women.
        Journal of Womens Health (Larchmont). 2011; 20: 829-836
        • Frayne S.M.
        • Parker V.A.
        • Christiansen C.L.
        • Loveland S.
        • Seaver M.R.
        • Kazis L.E.
        • Skinner K.M.
        Health status among 28,000 women veterans. The VA Women's Health Program Evaluation Project.
        Journal of General Internal MedicineJ Gen Intern Med. 2006; 21: S40-S46
        • Frayne S.M.
        • Phibbs C.S.
        • Friendman S.A.
        • Shaw J.G.
        • Romodan Y.
        • Berg E.
        • Lee J.
        • Anath L.
        • Iqbal S.
        • Hayes P.M.
        • Haskell S.
        Sourcebook: Women Veterans in the Veterans Health Administration. Volume 4.
        in: Longitudinal Trends in Sociodemographics, Utilization, Health Profile, and Geographic Distribution. Women's Health Evaluation Initiative, Women's Health Services, Veterans Health Administration, Department of Veterans Affairs, Washington, DC2018 (Available at:)
        • Frayne S.M.
        • Phibbs C.
        • Saecho F.
        • Maisel N.C.
        • Friedman S.A.
        • Finlay A.
        • Berg E.
        • Balasubramanian V.
        • Dally S.K.
        • Ananth L.
        • Romodan Y.
        • Lee J.
        • Iqbal S.
        • Hayes P.M.
        • Zephyrin L.
        • Whitehead A.
        • Torgal A.
        • Katon J.G.
        • Haskell S.
        Sourcebook: Women Veterans in the Veterans Health Administration.
        in: Sociodemographics, Utilization, Costs of Care, and Health Profile. 3. Women's Health Evaluation Initiative, Washington, DC2014
        • Fryar C.D.
        • Ostchega Y.
        • Hales C.M.
        • Zhang G.
        • Kruszon-Moran D.
        Hypertension prevalence and control among adults: United States, 2015-2016.
        2017 (Available:)
        • Hastings S.N.
        • Betts E.
        • Schmader K.E.
        • Weinberger M.
        • Van Houtven C.H.
        • Hendrix C.C.
        • Oddone E.Z.
        Discharge information and support for veterans receiving outpatient care in the emergency department: Study design and methods.
        Contemporary Clinical Trials. 2014; 39: 342-350
        • Heidari S.
        • Babor T.F.
        • De Castro P.
        • Tort S.
        • Curno M.
        Sex and gender equity in research: Rrationale for the SAGER guidelines and recommended use.
        Research Integrity and Peer Review. 2016; 1: 2
        • Humphreys K.A.
        Review of the impact of exclusion criteria on the generalizability of schizophrenia treatment research.
        Clinical Schizophrenia & Related Psychoses. 2017; 1: 49-57
        • Humphreys K.
        • Blodgett J.C.
        • Roberts L.W.
        The exclusion of people with psychiatric disorders from medical research.
        Journal of Psychiatric Research. 2015; 70: 28-32
        • Humphreys K.
        • Weingardt K.R.
        • Harris A.H.
        Influence of subject eligibility criteria on compliance with National Institutes of Health guidelines for inclusion of women, minorities, and children in treatment research.
        Alcoholism, Clinical and Experimental Research. 2007; 31: 988-995
        • Jackson G.L.
        • Weinberger M.
        • Kirshner M.A.
        • Stechuchak K.M.
        • Melnyk S.D.
        • Bosworth H.B.
        • Edelman D.
        Open-label randomized trial of titrated disease management for patients with hypertension: Study design and baseline sample characteristics.
        Contemporary Clinical Trials. 2016; 50: 5-15
        • Johansen M.E.
        • Hefner J.L.
        • Foraker R.E.
        Antiplatelet and statin use in US patients with coronary artery disease categorized by race/ethnicity and gender, 2003 to 2012.
        American Journal of Cardiology. 2015; 115: 1507-1512
        • Kessler C.
        • Chen J.
        • Dill C.
        • Tyndall G.
        • Olszyk M.D.
        State of affairs of emergency medicine in the Veterans Health Administration.
        American Journal of Emergency Medicine. 2010; 28: 947-951
        • Kessler C.S.
        • Bhandarkar S.
        • Casey P.
        • Tenner A.
        Predicting patient patterns in veterans administration emergency departments.
        Western Journal of Emergency Medicine. 2011; 12: 204-207
        • King D.W.
        • Duello T.M.
        • Miranda P.Y.
        • Hodges K.P.
        • Shelton A.J.
        • Chukelu P.
        • Jones L.A.
        Strategies for recruitment of healthy premenopausal women into the African American Nutrition for Life (A NULIFE) Study.
        Journal of Womens Health (Larchmont). 2010; 19: 855-862
        • Littman A.J.
        • Forsberg C.W.
        • Koepsell T.D.
        Physical activity in a national sample of veterans.
        Medicine and Science in Sports and Exercise. 2009; 41: 1006-1013
        • Mazure C.M.
        • Jones D.P.
        Twenty years and still counting: Iincluding women as participants and studying sex and gender in biomedical research.
        BMC Womens Health. 2015; 15: 94
        • McCarthy C.R.
        Historical background of clinical trials involving women and minorities.
        Academic Medicine. 1994; 69: 695-698
        • Melloni C.
        • Berger J.S.
        • Wang T.Y.
        • Gunes F.
        • Stebbins A.
        • Pieper K.S.
        • Newby L.K.
        Representation of women in randomized clinical trials of cardiovascular disease prevention.
        Circulation. Cardiovascular Quality and Outcomes. 2010; 3: 135-142
        • Melnyk S.D.
        • Zullig L.L.
        • McCant F.
        • Danus S.
        • Oddone E.
        • Bastian L.
        • Bosworth H.B.
        Telemedicine cardiovascular risk reduction in veterans.
        Am Heart J. 2013; 165: 501-508
        • Miller L.R.
        • Marks C.
        • Becker J.B.
        • Hurn P.D.
        • Chen W.J.
        • Woodruff T.
        • Clayton J.A.
        Considering sex as a biological variable in preclinical research.
        FASEB J. 2017; 3: 29-34
        • Moberg C.A.
        • Humphreys K.
        Exclusion criteria in treatment research on alcohol, tobacco and illicit drug use disorders: A review and critical analysis.
        Drug and Alcohol Review. 2017; 36: 378-388
        • Murphy L.B.
        • Helmick C.G.
        • Allen K.D.
        • Theis K.A.
        • Baker N.A.
        • Murray G.R.
        • Barbour K.E.
        Arthritis among veterans - United States, 2011-2013.
        MMWR Morb Mortal Wkly Rep. 2014; 63: 999-1003
        • National Center for Veterans Analysis and Statistics
        VETPOP Living veterans by age group, gender, 2015-2045.
        2016 (Available:)
        • Oddone E.Z.
        • Damschroder L.J.
        • Gierisch J.
        • Olsen M.
        • Fagerlin A.
        • Sanders L.
        • Juntilla K.
        A Coaching by Telephone Intervention for Veterans and Care Team Engagement (ACTIVATE): A study protocol for a hybrid type i effectiveness-implementation randomized controlled trial.
        Contemporary Clinical Trials. 2017; 55: 1-9
        • Phillips S.P.
        • Hamberg K.
        Doubly blind: A systematic review of gender in randomised controlled trials.
        Glob Health Action. 2016; 9: 29597
        • Pomernacki A.
        • Carney D.V.
        • Kimerling R.
        • Nazarian D.
        • Blakeney J.
        • Martin B.D.
        • Frayne S.M.
        Lessons from initiating the first Veterans Health Administration (VA) Women's Health Practice-based Research Network (WH-PBRN) study.
        Journal of the American Board of Family Medicine. 2015; 28: 649-657
        • Pomernacki A.
        • Carney D.V.
        • Frayne S.M.
        • Nevedal A.
        • Caudle K.
        • Zehm L.
        • Shih M.
        • Chen T.
        • Chow B.
        • Goldstein K.
        • Bastian L.
        • Chard K.
        • Ruzek J.
        • Huang G.
        • Johnson M.
        • Yano E.M.
        • Schnurr P.
        Qualitative program evaluation design for the Women's Enhanced Recruitment Process (WERP) of the VA Women's Health Practice-Based Research Network (WH-PBRN). Paper presented at the North American Primary Care.
        Research Group Practice-Based Research Network Conference, Bethesda, MD2018
        • Poon R.
        • Khanijow K.
        • Umarjee S.
        • Fadiran E.
        • Yu M.
        • Zhang L.
        • Parekh A.
        Participation of women and sex analyses in late-phase clinical trials of new molecular entity drugs and biologics approved by the FDA in 2007-2009.
        Journal of Womens Health (Larchmont). 2013; 22: 604-616
        • Ramchand R.
        • Taneilian T.
        • Fisher M.P.
        • Vaughan C.A.
        • Trial T.E.
        • Epley C.
        • Voorhies P.
        • Robbins M.W.
        • Robinson E.
        • Ghosh-Dastidar B.
        Hidden heroes: America's military caregivers.
        2014 (Available:)
        • Scott P.E.
        • Unger E.F.
        • Jenkins M.R.
        • Southworth M.R.
        • McDowell T.Y.
        • Geller R.J.
        • Woodcock J.
        Participation of women in clinical trials supporting FDA approval of cardiovascular drugs.
        Journal of the American College of Cardiology. 2018; 71: 1960-1969
        • Study of Barriers to Care for Women Veterans
        Final Report.
        2015
        • Tsang W.
        • Alter D.A.
        • Wijeysundera H.C.
        • Zhang T.
        • Ko D.T.
        The impact of cardiovascular disease prevalence on women's enrollment in landmark randomized cardiovascular trials: Aa systematic review.
        Journal of General Internal Medicine. 2012; 27: 93-98
        • Van Houtven C.H.
        • Oddone E.Z.
        • Hastings S.N.
        • Hendrix C.
        • Olsen M.
        • Neelon B.
        • Weinberger M.
        Helping Invested Families Improve Veterans' Experiences Study (HI-FIVES): Study design and methodology.
        Contemporary Clinical Trials. 2014; 38: 260-269
        • Van Houtven C.H.
        • Smith V.A.
        • Stechuchak K.M.
        • Shepherd-Banigan M.
        • Hastings S.N.
        • Maciejewski M.L.
        • Oddone E.Z.
        Comprehensive support for family caregivers: Impact on veteran health care utilization and costs.
        Medical Care Research and Review. 2017;
        • Vital signs
        Prevalence, treatment, and control of high levels of low-density lipoprotein cholesterol--United States, 1999-2002 and 2005-200.
        MMWR Morb Mortal Wkly Rep. 2011; 60: 109-114
        • Voils C.I.
        • Gierisch J.M.
        • Olsen M.K.
        • Maciejewski M.L.
        • Grubber J.
        • McVay M.A.
        • Yancy Jr., W.S.
        Study design and protocol for a theory-based behavioral intervention focusing on maintenance of weight loss: Tthe Maintenance After Initiation of Nutrition TrAINing (MAINTAIN) study.
        Contemporary Clinical Trials. 2014; 39: 95-105
        • Voils C.I.
        • Grubber J.M.
        • McVay M.A.
        • Olsen M.K.
        • Bolton J.
        • Gierisch J.M.
        • Yancy Jr., W.S.
        Recruitment and retention for a weight loss maintenance trial involving weight loss prior to randomization.
        Obesity Science & Practice. 2016; 2: 355-365
        • Williams-Johnson S.G.K.
        VISN 6 Women's Health DASHBOARD.
        2018 (Available:)
        biooffice.pa.cdw.va.gov
        Date accessed: February 26, 2019
        • Wilson S.M.
        • Hair L.P.
        • Hertzberg J.S.
        • Kirby A.C.
        • Olsen M.K.
        • Lindquist J.H.
        • Calhoun P.S.
        Abstinence Reinforcement Therapy (ART) for rural veterans: Methodology for an mHealth smoking cessation intervention.
        Contemporary Clinical Trials. 2016; 50: 157-165
        • Yano E.M.
        • Bastian L.A.
        • Bean-Mayberry B.
        • Eisen S.
        • Frayne S.
        • Hayes P.
        • Washington D.L.
        Using research to transform care for women veterans: Advancing the research agenda and enhancing research-clinical partnerships.
        Womens Health Issues. 2011; 21: S73-S83
        • Zullig L.L.
        • Melnyk S.D.
        • Stechuchak K.M.
        • McCant F.
        • Danus S.
        • Oddone E.
        • Bosworth H.B.
        The Cardiovascular Intervention Improvement Telemedicine Study (CITIES): Rrationale for a tailored behavioral and educational pharmacist-administered intervention for achieving cardiovascular disease risk reduction.
        Telemedicine Journal and E Health. 2014; 20: 135-143
        • Zullig L.L.
        • Stechuchak K.M.
        • Goldstein K.M.
        • Olsen M.K.
        • McCant F.M.
        • Danus S.
        • Bosworth H.B.
        Patient-reported medication adherence barriers among patients with cardiovascular risk factors.
        Journal of Managed Care and Specialty Pharmacy. 2015; 21: 479-485

      Biography

      Karen M. Goldstein, MD, MSPH, is affiliated with the Durham VA Center for Innovation to Accelerate Discovery and Practice Transformation and the Division of General Internal Medicine at Duke University. Her interests include peer support, cardiovascular disease, women's health, and evidence synthesis.
      Wei Duan-Porter, MD, PhD, is affiliated with the Center for Care Delivery & Outcomes Research at the Minneapolis VA Health Care System and the University of Minnesota Medical School. Her interests include physical function and disability, psychosocial determinants of health, and evidence synthesis.
      Aviel Alkon, BS, is affiliated with the Durham VA Center for Innovation to Accelerate Discovery and Practice Transformation and Duke University's Department of Medicine. His interest includes computer programming and software design.
      Maren K. Olsen, PhD, is affiliated with the Durham VA Center for Innovation to Accelerate Discovery and Practice Transformation and Duke University's Department of Biostatistics and Bioinformatics. Her interests include longitudinal data methods and missing data methods.
      Corrine I. Voils, PhD, is affiliated with the Department of Surgery at the University of Wisconsin and the William S. Middleton Memorial Veterans Hospital. Her interests include quantitative and qualitative methodologies, treatment adherence, and genomics health services research.
      Susan N. Hastings, MD, MHS, is affiliated with the Durham VA Center for Innovation to Accelerate Discovery and Practice Transformation and the Duke Center for Study of Human Aging and Development. Her research interests include care transitions and emergency care for older adults.