Advertisement

Evaluating Bystander Intervention Training to Address Patient Harassment at the Veterans Health Administration

Published:August 20, 2020DOI:https://doi.org/10.1016/j.whi.2020.06.006

      Abstract

      Purpose

      One in four women veteran patients report experiencing sexual and gender harassment when attending the Veterans Health Administration (VA) for health care. Bystander intervention—training community members how to intervene when witnessing inappropriate behaviors—is a common approach for addressing harassment in school and military settings. We evaluated implementation of a VA harassment awareness and bystander intervention training that teaches health care staff how to identify and intervene in the harassment of women veteran patients.

      Methods

      Participants included 180 VA staff, including both providers and administrative staff from one VA state health care system, who participated in harassment training during the first year of implementation. Pretest and post-test evaluation surveys included questions on acceptability of training length and relevance, staff experiences with harassment, perceptions of the training, and four short-term attitudinal outcomes: awareness of harassment, barriers to intervening, self-efficacy for intervening, and intentions to intervene.

      Results

      At pretest, most staff reported witnessing harassment, yet fewer than one-half had intervened. By post-test, staff reported significantly decreased barriers to intervening and increased awareness, self-efficacy, and intentions to intervene. Belief that harassment is a problem increased from 42.4% to 75.0%. The majority of staff found the training relevant and appropriate in length. Staff felt the most useful aspects of the training were learning how to intervene, group discussion, effective facilitation, and information on harassment.

      Conclusions

      We found that a bystander approach was acceptable to health care staff and efficacious on short-term outcomes. Bystander intervention may be a promising strategy to address harassment among patients in medical facilities.
      Although harassment has received significant recent attention due to the #MeToo movement, the Veterans Health Administration (VA) has spent the past few years addressing a specific type of harassment—harassment of women veteran patients attending VA for health care. Women are the fastest growing segment of veterans (
      National Center for Veterans Analysis and Statistics
      Women veterans report: The past, present and future of women veterans.
      ), with almost one-half of a million women veterans using VA health care in fiscal 2015 (
      National Center for Veterans Analysis and Statistics
      Women veterans report: The past, present and future of women veterans.
      ). A recent survey of women veterans at 12 sites nationwide revealed that one in four experienced harassment on VA grounds (
      • Klap R.
      • Darling J.E.
      • Hamilton A.B.
      • Rose D.E.
      • Dyer K.
      • Canelo I.
      • Yano E.M.
      Prevalence of stranger harassment of women veterans at Veterans Affairs medical centers and impacts on delayed and missed care.
      ). Although harassment from patients toward staff is a common problem in health care settings (
      • Lanctôt N.
      • Guay S.
      The aftermath of workplace violence among healthcare workers: A systematic literature review of the consequences.
      ), this study found high rates of patients experiencing harassment from other patients.
      Although harassment is associated with several negative outcomes for women, the VA recognized that harassment could have particular consequences for women veteran patients. Experiencing harassment in public is associated with increased anxiety, lower perceived safety, body shame, monitored clothing and appearance, increased fear of sexual assault, and avoidance of public spaces (
      • Bastomski S.
      • Smith P.
      Gender, fear, and public places: How negative encounters with strangers harm women.
      ;
      • Davidson M.M.
      • Butchko M.S.
      • Robbins K.
      • Sherd L.W.
      • Gervais S.J.
      The mediating role of perceived safety on street harassment and anxiety.
      ;
      • Davidson M.M.
      • Gervais S.J.
      • Sherd L.W.
      The ripple effects of stranger harassment on objectification of self and others.
      ;
      • Fairchild K.
      • Rudman L.A.
      Everyday stranger harassment and women’s objectification.
      ;
      • Macmillan R.
      • Nierobisz A.
      • Welsh S.
      Experiencing the streets: Harassment and perceptions of safety among women.
      ). Women veterans who reported experiencing harassment at VA indicated feeling less safe and welcome at VA, decreased access to and engagement with VA health care, and decreased patient satisfaction (
      • Klap R.
      • Darling J.E.
      • Hamilton A.B.
      • Rose D.E.
      • Dyer K.
      • Canelo I.
      • Yano E.M.
      Prevalence of stranger harassment of women veterans at Veterans Affairs medical centers and impacts on delayed and missed care.
      ;
      • Klap R.
      • Golden R.E.
      Preliminary findings from wave 1 of the WH-PBRN stranger harassment card study.
      ). Of significant concern, veterans who reported experiencing harassment at VA were more likely to indicate intentionally delaying or missing health care appointments (
      • Klap R.
      • Darling J.E.
      • Hamilton A.B.
      • Rose D.E.
      • Dyer K.
      • Canelo I.
      • Yano E.M.
      Prevalence of stranger harassment of women veterans at Veterans Affairs medical centers and impacts on delayed and missed care.
      ). Creating a VA environment free from harassment is particularly critical, because approximately 38% of women veterans and active-duty military service members indicate experiencing military sexual trauma, which may include repeated harassment or assault (
      • Wilson L.C.
      The prevalence of military sexual trauma: A meta-analysis.
      ). Women with military sexual trauma histories have higher odds of reporting harassment at VA and missing care to avoid interactions with other veterans (
      • Klap R.
      • Darling J.E.
      • Hamilton A.B.
      • Rose D.E.
      • Dyer K.
      • Canelo I.
      • Yano E.M.
      Prevalence of stranger harassment of women veterans at Veterans Affairs medical centers and impacts on delayed and missed care.
      ;
      • Shipherd J.C.
      • Darling J.E.
      • Klap R.S.
      • Rose D.
      • Yano E.M.
      Experiences in the veterans health administration and impact on healthcare utilization: Comparisons between LGBT and non-LGBT women veterans.
      ).
      After findings about harassment of women veterans were presented to VA leadership, the VA developed a national End Harassment Workgroup to better understand harassment at the VA and determine best practices for addressing harassment. The Workgroup focused on two forms of patient harassment: Unwanted sexual attention (“expressions of romantic or sexual interest that are unwelcome, unreciprocated, and offensive to the recipient”;
      • Leskinen E.A.
      • Cortina L.M.
      • Kabat D.B.
      Gender harassment: Broadening our understanding of sex-based harassment at work.
      , p. 26) and gender harassment (behaviors that convey insulting, hostile, or degrading attitudes about women;
      • Fitzgerald L.F.
      • Gelfand M.J.
      • Drasgow F.
      Measuring sexual harassment: Theoretical and psychometric advances.
      ). Notably, women veterans reported a unique form of gender harassment: men questioning women's veteran status by making statements that imply women do not belong at the VA or are not veterans (
      • Klap R.
      • Darling J.E.
      • Hamilton A.B.
      • Rose D.E.
      • Dyer K.
      • Canelo I.
      • Yano E.M.
      Prevalence of stranger harassment of women veterans at Veterans Affairs medical centers and impacts on delayed and missed care.
      ). Additionally, VA Women's Health Services supported efforts to identify evidence-based ways to address patient harassment. These efforts included discussion groups with men and women veterans, stakeholder interviews with VA health care workers, reviewing evidence-based programs outside of VA, and hosting an expert panel (
      • Dyer K.E.
      • Potter S.J.
      • Hamilton A.B.
      • Luger T.M.
      • Bergman A.A.
      • Yano E.M.
      • Klap R.
      Gender differences in veterans’ perceptions of harassment on Veterans Health Administration grounds.
      ;
      • Klap R.
      • Darling J.E.
      • Hamilton A.B.
      • Rose D.E.
      • Dyer K.
      • Canelo I.
      • Yano E.M.
      Prevalence of stranger harassment of women veterans at Veterans Affairs medical centers and impacts on delayed and missed care.
      ;
      • Klap R.
      • Golden R.E.
      Preliminary findings from wave 1 of the WH-PBRN stranger harassment card study.
      ).
      As first steps toward decreasing harassment, the VA in 2017 began rolling out staff trainings across facilities to raise awareness about harassment and teach bystander intervention strategies and launched a VA-wide social marketing campaign (e.g., posters aimed at culture change). Bystander interventions are an evidenced-based approach to preventing interpersonal violence by encouraging observers and community members to intervene (
      • Banyard V.L.
      The promise of a bystander approach to violence prevention.
      ). In schools, bystander trainings have been found to be effective for increasing students’ intervening behaviors and decreasing sexual violence and harassment (
      • Coker A.L.
      • Bush H.M.
      • Cook-Craig P.G.
      • DeGue S.A.
      • Clear E.R.
      • Brancato C.J.
      • Recktenwald E.A.
      RCT testing bystander effectiveness to reduce violence.
      ;
      • Jouriles E.N.
      • Krauss A.
      • Vu N.L.
      • Banyard V.L.
      • McDonald R.
      Bystander programs addressing sexual violence on college campuses: A systematic review and meta-analysis of program outcomes and delivery methods.
      ;
      • Katz J.
      • Moore J.
      Bystander education training for campus sexual assault prevention: An initial meta-analysis.
      ;
      • Kettrey H.H.
      • Marx R.A.
      The effects of bystander programs on the prevention of sexual assault across the college years: A systematic review and meta-analysis.
      ). Bystander programs have also been successfully tailored to military settings (
      • Orchowski L.M.
      • Berry-Caban C.S.
      • Prisock K.
      • Borsari B.
      • Kazemi D.M.
      Evaluations of sexual assault prevention programs in military settings: A synthesis of the research literature.
      ). However, we are not aware of any bystander trainings that have been evaluated for decreasing harassment in the VA or other health care settings.
      This study evaluates a VA harassment awareness and bystander training during the first year of implementation within one state VA health care system. Our first aim was to assess the frequency of staff experiences with witnessing, intervening, and experiencing harassment. Given that VA staff have reported barriers to intervening in observed harassment (
      • Klap R.
      • Potter S.
      • Yano E.
      • Bergman A.
      • Hamilton A.
      • Darling J.
      “Hey baby, smile” VA staff perspectives on addressing harassment of women veterans at VA medical centers.
      ), we hypothesized that staff would report witnessing patient harassment, yet intervening in fewer situations. Second, to determine whether any groups of staff had specific training needs, we examined whether pretest measures differed by staff background characteristics. Given the lack of research on health care staff intervening in harassment among patients, we made no hypotheses how groups would differ. Third, we aimed to assess training impacts on short-term attitudinal outcomes associated with intervening, including barriers to intervening, awareness of harassment, self-efficacy for intervening, and intentions to intervene (
      • Burn S.M.
      A situational model of sexual assault prevention through bystander intervention.
      ;
      • McMahon S.
      • Peterson N.A.
      • Winter S.C.
      • Palmer J.E.
      • Postmus J.L.
      • Koenick R.A.
      Predicting Bystander Behavior to Prevent Sexual Assault on College Campuses: The Role of Self-Efficacy and Intent.
      ). As a meta-analysis found that brief (20–60 minute) college bystander programs had significant effects on beliefs and attitudes, we expected VA training to improve short-term outcomes (
      • Jouriles E.N.
      • Krauss A.
      • Vu N.L.
      • Banyard V.L.
      • McDonald R.
      Bystander programs addressing sexual violence on college campuses: A systematic review and meta-analysis of program outcomes and delivery methods.
      ). Our fourth aim was to assess training acceptability. We hypothesized that most staff would find the training relevant and of adequate length. Our fifth, qualitative, aim was to understand staff perceptions of the most useful or important components of the training.

      Methods

      Study Design and Sample

      We conducted a program evaluation during the first year of training using a pretest–post-test design. This quality improvement project received a determination of nonresearch from the VA Connecticut Healthcare System Institutional Review Board. Staff from three clinical centers within one statewide VA medical system were invited to participate with the goal of training as many staff as possible through routine mandatory staff meetings and voluntary invited seminars. The Women Veterans Program Manager contacted individual department heads and in some cases was invited by department heads, who notified staff about the training. Based on supervisor and trainer estimates, approximately 285 staff attended one of the 14 trainings. The majority of the trainings (9/14) occurred in fiscal 2018 at one large VA medical center with 2,968 permanent employees. Thus, approximately 5.1% of permanent employees were trained during the first year of implementation at the main facility. Participation in the evaluation was anonymous and voluntary. Of the 285 staff who attended a training, 180 (63.2%) participated in the evaluation, with 153 completing both pretests and post-tests. Although we do not have demographic information on those who did not complete evaluation materials, a higher percentage of staff chose to complete evaluation surveys during trainings delivered to mental health staff (90.1%) than during trainings delivered to primary care staff (53.7%) or at voluntary invited seminars (30.4%).

      Program Design

      The VA End Harassment Training Workgroup
      Members include Lynette Adams, Maggie Czarnogorski, Jane Driver, Angie Fodor, Lana Frankenfield, Jessica Keith, Laura Miller, Jenny Sitzer, and Dawne Vogt.
      developed the “It's Our Responsibility to End Harassment Staff Awareness Training” to raise awareness of patient harassment and teach bystander intervention strategies. The single-session 45- to 60-minute training followed
      • Latané B.
      • Darley J.M.
      The unresponsive bystander: Why doesn't he help?.
      five-stage model for bystander intervention (i.e., notice, identify, recognize, know how, take action). The session provided information on VA patient harassment (i.e., statistics, examples, consequences, definitions) and four options for intervening (i.e., the four Ds) taken from the college bystander intervention literature: directly intervening by saying something, providing a distraction, delegating someone else to intervene, or delaying a response until after the incident has occurred. Training also included group discussions of harassment scenarios to allow staff to share harassment experiences and problem solve how to intervene. All trainings were conducted by one of the training developers (last author) and focused roughly equal amounts of time on raising awareness and teaching staff how to intervene. Harassment scenarios were tailored to audience, roles, and work settings. Conceptually, training staff how to identify and address unacceptable behaviors helps to change the VA culture over time by teaching veterans acceptable behaviors and supporting women veterans, thereby decreasing the incidence of harassment and increasing women veterans' safety and engagement at VA (Figure 1).
      Figure thumbnail gr1
      Figure 1Theoretical outcomes and impact of training Veterans Health Administration (VA) staff to intervene.

      Data Collection

      Before trainings, staff were provided an information sheet describing the evaluation and invited to participate. Staff who agreed completed pretest surveys. Staff completed post-tests immediately following the training or returned surveys by interdepartmental mail. Pretests and post-tests were linked using unique identifiers preprinted on the surveys.

      Measures

      The evaluation survey consisted of open- and closed-ended questions (Appendix). Because there were no existing measures of attitudes concerning intervening in patient harassment, we identified relevant bystander measures for interpersonal violence (described elsewhere in this article) and eliminated items not relevant to patient harassment. We next used previously collected qualitative data from veterans and stakeholders and expert feedback from VA's Harassment Workgroup to generate new items and adapt wording. We performed cognitive interviews with four staff and piloted measures with 14 staff to eliminate redundant or socially desirable items (e.g., items for which all participants selected scale end points). Finally, we used evaluation pretest data to perform a psychometric assessment of internal consistency reliability and validity.
      At pretest only, we assessed demographics (age, gender, VA position, previous service in the armed forces), prior training in how to intervene in harassment (if any), and staff experiences with harassment. Harassment of staff was assessed with one item adapted from
      • Klap R.
      • Darling J.E.
      • Hamilton A.B.
      • Rose D.E.
      • Dyer K.
      • Canelo I.
      • Yano E.M.
      Prevalence of stranger harassment of women veterans at Veterans Affairs medical centers and impacts on delayed and missed care.
      that asked how often staff had personally experienced inappropriate or unwanted comments or behavior from veterans at VA in the prior year (never, sometimes, usually, always). Experiences with witnessing patient harassment were assessed with two questions that asked how often staff witnessed a man engage in either unwanted sexual attention or gender harassment toward a woman veteran in the prior year, from 0 (never) to 4 (≥10 times). Frequency of intervening was assessed by asking staff who had witnessed harassment how often they tried to intervene, from 0 (never) to 4 (≥10 times). Experience with harassment disclosures was assessed with one question that asked how many times women veterans had ever disclosed harassment to the staff member, from 0 (never) to 4 (≥10 times). History of intervening was assessed with one question asking whether staff believed they ever had tried to intervene to stop harassment from happening to a woman veteran at VA (never had an opportunity, yes, no).
      We assessed four outcome measures at both pretest and post-test. Awareness of harassment was assessed with one item adapted from the No Awareness Scale (
      • Banyard V.L.
      • Moynihan M.M.
      • Cares A.C.
      • Warner R.
      How do we know if it works? Measuring outcomes in bystander-focused abuse prevention on campuses.
      ) that asked whether staff believed harassment of women veterans was a problem at this VA (disagree, agree, don't know). Barriers to intervening was assessed with a 17-item scale adapted from
      • Burn S.M.
      A situational model of sexual assault prevention through bystander intervention.
      barriers to intervening scales. Staff answered how much they agreed with each item on a Likert scale, from 1 (strongly disagree) to 5 (strongly agree). To assess self-efficacy for intervening, we used a seven-item scale modeled on the bystander Self-Efficacy Scale (
      • Banyard V.L.
      Measurement and correlates of prosocial bystander behavior: the case of interpersonal violence.
      ). We used one item from
      • Nickerson A.B.
      • Aloe A.M.
      • Livingston J.A.
      • Feeley T.H.
      Measurement of the bystander intervention model for bullying and sexual harassment.
      and created six additional items to cover a range of intervention strategies (directly intervening, distracting, delegating, delaying). Participants answered how certain they were that they could do each behavior on a scale from 1 (can't do) to 7 (completely certain). Intentions to intervene in patient harassment was assessed with three items, two adapted from the Intent to Help Scale (
      • Banyard V.L.
      Measurement and correlates of prosocial bystander behavior: the case of interpersonal violence.
      ;
      • Banyard V.L.
      • Moynihan M.M.
      • Cares A.C.
      • Warner R.
      How do we know if it works? Measuring outcomes in bystander-focused abuse prevention on campuses.
      ) and one created for intervening in sexual harassment. Participants answered how likely they would be to intervene in each situation on a scale from 1 (not at all likely) to 5 (extremely likely). All survey items were modified or created to address intervening in VA patient harassment.
      At post-test only, we assessed training acceptability by asking how staff perceived the training length (too short, just right, too long) and relevance (not at all relevant, somewhat relevant, moderately relevant, very relevant) as well as an open-ended question on the most impactful or useful part of the training.

      Data Analysis

      Aim 1

      We calculated frequencies to determine how often staff experienced, witnessed, and intervened in harassment.

      Aim 2

      We first used Cronbach's alpha to assess internal reliability of the adapted measures and correlations to assess for convergent and discriminant validity. We then performed analyses of variance and Welch's t tests to examine whether pretest measures differed by baseline characteristics, including gender, age, staff position, service history, or harassment experience (dichotomized as 0 = none, 1 = any harassment).

      Aim 3

      To assess changes in awareness (dichotomized as 0 = don't know/disagree harassment is a problem, 1 = agree), we used McNemar's χ2 test with Cohen's G (
      • Cohen J.
      Statistical power analysis for the behavioral sciences.
      ) for effect size (0.05 = small, 0.15 = medium, 0.25 = large). To assess changes in continuous measures (barriers, self-efficacy, intentions), we used paired sample t tests with Cohen's D (
      • Cohen J.
      Statistical power analysis for the behavioral sciences.
      ) for effect size (0.20 = small, 0.50 = medium, 0.80 = large). We used
      • Moynihan M.M.
      • Banyard V.L.
      • Arnold J.S.
      • Eckstein R.P.
      • Stapleton J.G.
      Engaging intercollegiate athletes in preventing and intervening in sexual and intimate partner violence.
      criteria of +1 standard deviation to indicate significant improvement and −1 standard deviation to indicate significant decline (a worsening of attitudes which
      • Moynihan M.M.
      • Banyard V.L.
      • Arnold J.S.
      • Eckstein R.P.
      • Stapleton J.G.
      Engaging intercollegiate athletes in preventing and intervening in sexual and intimate partner violence.
      called “backlash”). We further examined whether this impact varied by baseline characteristics using an N – 1 corrected χ2 test, which corrects for small cell sizes while retaining power to detect effects (
      • Campbell I.
      Chi-squared and Fisher–Irwin tests of two-by-two tables with small sample recommendations.
      ).

      Aim 4

      We calculated frequencies to determine staff perceptions of the training length and relevance.

      Aim 5

      Last, we conducted inductive thematic content analysis of the open-ended question (which parts of the program staff found most impactful or useful) with two coders using
      • Hruschka D.J.
      • Schwartz D.
      • St John D.C.
      • Picone-Decaro E.
      • Jenkins R.A.
      • Carey J.W.
      Reliability in coding open-ended data: Lessons learned from HIV behavioral research.
      method to ensure intercoder reliability. Coding continued until reaching agreement on all codes.

      Results

      Baseline Staff Characteristics

      Of the 180 staff who participated, roughly two-thirds were women (Table 1). The majority were between the ages of 30 and 60. Around one-third were medical professionals (nurses or physicians), one-third were mental health staff (psychologists), and the rest were other employees. Approximately 15% had served in the military.
      Table 1Demographic Characteristics of Bystander Training Participants
      VariableN%
      Employee sex
       Women11865.6
       Men3620.0
       Unknown2614.4
      Age (in years)
       20–291810.0
       30–395731.7
       40–493418.9
       50–594525.0
       ≥601810.0
       Unknown84.4
      Served in military2715.0
      Type of position
       Physician147.8
       Nursing4223.3
       Mental health5430.0
       Clinicians (not specified)158.3
       Trainee84.4
       Support staff84.4
       Administrative73.9
       Other95.0
       Unknown2312.8
      Note. N = 180.

      Frequency of Staff Witnessing and Intervening in Harassment

      At pretest, 66.1% of staff reported witnessing harassment of women veterans in the prior 12 months, with more staff observing unwanted sexual attention (i.e., sexual harassment) than gender harassment (58.4% vs. 38.9%; Table 2). Staff who witnessed harassment were most likely to report witnessing harassment only once or twice in the prior year; 10% witnessed unwanted sexual attention 10 or more times. Approximately one-half of staff reported that women veterans told them about experiencing harassment at the VA. The majority of staff also personally experienced unwanted or inappropriate comments or behavior from veterans, with more women reporting harassment than men (83.1% vs. 38.9%; odds ratio, 7.70; 95% confidence interval, 3.38–17.6). Yet, despite often witnessing and hearing about patient harassment, at pretest only 42.4% of staff believed that harassment of women veterans was a problem at their facility. Women and men staff showed no differences in the belief that harassment of women veterans was a problem.
      Table 2Prior Experiences with Patient Harassment among Staff Participants
      VariableNever1–2 Times3–5 Times6–9 Times≥10 Times
      Unwanted sexual attention in prior 12 months
       Witnessed73 (40.6%)47 (26.1%)28 (15.6%)12 (6.7%)18 (10.0%)
       Intervened when witnessed34 (18.9%)41 (22.8%)13 (7.2%)5 (2.8%)2 (1.1%)
      Gender harassment in prior 12 months
       Witnessed108 (60.0%)45 (25.0%)18 (10.0%)5 (2.8%)2 (1.1%)
       Intervened when witnessed25 (13.9%)32 (17.8%)5 (2.8%)1 (0.6%)0 (0.0%)
      Women veterans disclosed being harassed at VA88 (48.9%)47 (26.1%)18 (10.0%)10 (5.6%)14 (7.8%)
      N%
      Ever intervened to stop patient harassment5832.2%
      Personally experienced harassment from patients13273.3%
      Abbreviation: VA, Veterans Health Administration.
      Note: N = 180.
      Only 32.2% of staff reported that they had ever intervened in patient harassment, yet nearly 82% stated that they intervened in at least one situation when they had the opportunity to do so (Figure 2). Notably, some staff who witnessed harassment may not have believed they had an opportunity to intervene. Out of the 119 people who reported witnessing patient harassment in the past 12 months, 39.0% reported that they had never before had an opportunity to intervene to stop harassment.
      Figure thumbnail gr2
      Figure 2Veterans Health Administration (VA) staff experiences with witnessing and intervening in patient harassment.

      Psychometric Analysis of Adapted Measures

      Correlations among pretest measures and history of intervening confirmed our adapted measures had good convergent validity (Table 3). Intentions and self-efficacy for intervening were positively correlated with each other and negatively related to barriers to intervening. Intentions and self-efficacy were positively correlated (and barriers negatively correlated) with both prior year intervening and ever intervening. Awareness of harassment was correlated with witnessing harassment, intervening in the past year, and ever intervening. Discriminant validity was also supported as intentions and self-efficacy were not correlated with awareness nor with witnessing harassment.
      Table 3Correlations Among Pretest Bystander Attitudes and Behaviors
      Variable12345678
      1. Intentions
      2. Self-efficacy0.60
      p < .001.
      3. Barriers−0.49
      p < .001.
      −0.32
      p < .001.
      4. Awareness−0.05−0.07−0.02
      5. Ever intervened0.15
      p < .05.
      0.23
      p < .01.
      −0.25
      p < .01.
      0.47
      p < .001.
      6. Past year saw SH−0.04−0.040.070.38
      p < .001.
      0.50
      p < .001.
      7. Past year saw GH−0.10−0.08−0.030.41
      p < .001.
      0.44
      p < .001.
      0.34
      p < .001.
      8. Past year intervened in SH0.26
      p < .05.
      0.22
      p < .05.
      −0.29
      p < .01.
      0.33
      p < .01.
      0.48
      p < .001.
      NA0.27
      p < .01.
      9. Past year intervened in GH0.44
      p < .001.
      0.28
      p < .05.
      −0.27
      p < .05.
      0.30
      p < .05.
      0.48
      p < .001.
      0.27
      p < .05.
      NA0.35
      p < .05.
      Abbreviations: GH, gender harassment; SH, sexual harassment (i.e., unwanted sexual attention).
      Note. For all correlations other than correlations involving past year intervening, ns = 163–178; for past year intervening, ns = 48–95.
      p < .001.
      p < .05.
      p < .01.

      Examining Differences in Staff Training Needs

      At baseline, neither intentions nor barriers differed by baseline characteristics. Pretest self-efficacy was higher among men than women (5.82 vs. 5.17), Welch's t (84.26) = −3.47, p < .001, and was higher among staff who never personally experienced harassment compared with those who had (5.66 vs. 5.18), Welch's t (111.78) = 2.57, p < .05.

      Training Efficacy on Short-Term Outcomes

      Staff showed significant positive changes on all outcome variables between pretest and post-test (Table 4). Staff showed decreases in perceived barriers to intervening, increases in self-efficacy, and increases in intentions to intervene, although the effect sizes were small. The only large effect was in perceived awareness, with 42.4% of staff perceiving harassment to be a problem at pretest, compared with 75.0% at post-test.
      Table 4Pretest to Post-Test Change in Bystander Training Outcomes
      VariablePretestPost-TestPaired t TestCohen's D
      M (SD)M (SD)p Value
      Barriers (α = 0.91)2.07 (0.70)2.00 (0.71).020
      p < .05.
      .20
      Self-efficacy (α = 0.91)5.31 (1.28)5.70 (1.04)<.001
      p < .001.
      .38
      Intentions (α = 0.84)3.78 (0.90)4.02 (0.79)<.001
      p < .001.
      .29
      PretestPost-TestMcNemar's χ2Cohen's G
      Awareness42.4%75.0%<.001
      p < .001.
      .41
      Note. N = 180.
      p < .05.
      p < .001.
      We also examined which staff showed potential negative responses (i.e., backlash) to the program (a negative impact by 1+ standard deviation). Overall, 7.7% (n = 11) exhibited backlash in barriers, 2.7% (n = 4) in self-efficacy, and 6.1% (n = 9) in intentions. Background characteristics were not associated with backlash in barriers or self-efficacy. A small subset of staff experienced a significant decrease in intentions to intervene, and this decrease was associated with a disbelief that harassment was a problem at VA (Fisher's exact test, p < .01). All nine participants who exhibited this backlash in intentions had stated at pretest that they either did not believe, or were not sure, that harassment was a problem at the VA; none of the staff who believed that harassment was a problem exhibited backlash.
      When examining a meaningful positive impact (improvement by 1+ standard deviation), 9.8% (n = 14) experienced meaningful improvement in barriers, 14.8% (n = 22) in self-efficacy, and 21.8% (n = 32) in intentions. Women were more likely to experience a meaningful impact in barriers, with 15.1% showing impact compared with 0% of men (Fisher's exact test, p < .05). Neither the impact in self-efficacy nor in intentions was associated with background characteristics.

      Training Acceptability

      Staff also seemed to find the training to be an acceptable length and relevant. Almost one-half (47.8%) reported the training very relevant, 16.7% moderately relevant, and 17.2% somewhat relevant (19.9% did not complete post-test or left this question blank). No staff believed the training was not at all relevant. Furthermore, 62.8% stated the training length was just right, with only 2.8% stating the training was too long and 12.2% indicating it was too short (22.2% did not complete the post-test or left this question blank).

      Staff Perceptions of Useful Training Components

      In describing the most impactful or useful part of the training (Table 5), the most common theme was learning how to intervene, including receiving specific strategies, preparing what to say, and identifying barriers to overcome. The second most common theme involved the usefulness of group discussion, including the ability to share and problem solve real life situations and hear others', particularly women's, experiences with harassment. Staff also felt it was impactful for the facilitator to be knowledgeable and engaging, create an empowering and nonthreatening environment, normalize discomfort while intervening, and encourage staff to push themselves outside their comfort zone. The final theme identified was information on harassment, including local statistics, how to identify harassment, and local resources for addressing harassment. Although several staff stated that all of the training was impactful, one respondent indicated the training was not useful or impactful.
      Table 5Themes and Example Quotes Regarding the most Impactful or Useful Aspects of Bystander Training
      ThemesExample Quotes in Support of Overarching Themes
      Learning how to intervene
       Options for intervening“Discussing the options for addressing inappropriate behavior; i.e., direct, distract, delegate, delay.”
      “Giving many options of how to intervene.”
       Preparing what to say“Having a plan or a line or two to say when people are being offensive.”
      “The HOW part—learning specific verbiage & being given options of how to respond was helpful.”
       Identifying barriers“Getting strategies for intervening and discussing with others the barriers to doing so.”
      “It was helpful to think about what gets in the way of intervening.”
      Use of group discussion
       Problem solve situations“I liked the opportunity to try to share experiences and problem solve.”
      “Discussion of scenarios people have been in.”
       Hearing others' experiences“It was helpful to hear others' responses too—I am not the only one who feels paralyzed in these moments.”
      “Hearing women staff stories.”
      Effective facilitation
       Ability“Instructor knowledge.”
      “Presenter is great. Easy to follow. Keeps audience's attention.”
       Atmosphere and style“It was presented in a supportive and nonthreatening environment.”
      “Presenter's calm and confident and empowering style of providing information.”
       Normalizing discomfort and encouragement to intervene“I liked being challenged to push my comfort zone a bit and encouraged to intervene in situations where I might not have otherwise.”
      “Specifying that there is no one right answer or response and we should always try, or try the next time—others are watching.”
      “The facilitator sharing her own experience and discomfort she felt addressing harassing behavior.”
      Information on harassment“How much harassment goes on.”
      “Learning the different types of harassment, how to identify it.”
      “Resource info.”

      Discussion

      We present a pilot evaluation of an awareness-raising and bystander intervention training for health care staff that addresses harassment of women patients. Results indicated that the training was acceptable and showed efficacy on intended outcomes. Participants exhibited large increases in awareness of patient harassment as well as smaller yet significant changes in perceived barriers, self-efficacy, and intentions to intervene.
      We found that VA staff frequently witnessed and experienced patient harassment. Prior research found that women veterans experienced more sexual than gender-based harassment at the VA (
      • Klap R.
      • Darling J.E.
      • Hamilton A.B.
      • Rose D.E.
      • Dyer K.
      • Canelo I.
      • Yano E.M.
      Prevalence of stranger harassment of women veterans at Veterans Affairs medical centers and impacts on delayed and missed care.
      ). Our finding that staff also witnessed more sexual than gender-based harassment supports the feasibility of a bystander approach, because it implies that staff are seeing harassment as women veterans describe it and, therefore, have an opportunity to intervene. We also found that many staff, especially women staff, personally experience inappropriate patient behavior themselves, which replicates prior work, predominantly outside VA, indicating that health care providers commonly experience harassment and violence from patients (
      • Boissonnault J.S.
      • Cambier Z.
      • Hetzel S.J.
      • Plack M.M.
      Prevalence and risk of inappropriate sexual behavior of patients toward physical therapist clinicians and students in the united states.
      ;
      • Cogin J.
      • Fish A.
      Sexual harassment–a touchy subject for nurses.
      ;
      • DeMayo R.A.
      Patient sexual behavior and sexual harassment: A national survey of female psychologists.
      ;
      • Lanctôt N.
      • Guay S.
      The aftermath of workplace violence among healthcare workers: A systematic literature review of the consequences.
      ;
      • Nielsen M.B.D.
      • Kjær S.
      • Aldrich P.T.
      • Madsen I.E.
      • Friborg M.K.
      • Rugulies R.
      • Folker A.P.
      Sexual harassment in care work–Dilemmas and consequences: A qualitative investigation.
      ;
      • Phillips S.P.
      • Schneider M.S.
      Sexual harassment of female doctors by patients.
      ;
      • Spector P.E.
      • Zhou Z.E.
      • Che X.X.
      Nurse exposure to physical and nonphysical violence, bullying, and sexual harassment: A quantitative review.
      ). Staff also discussed their own experiences of harassment during trainings discussion, indicating a desire for ways to address such harassment. Such experiences indicate that staff also need skills in how to respond when witnessing harassment of other staff and training in how to respond when personally harassed. Such situations may present further opportunities for educating patients on appropriate behavior in a medical setting.
      Before this intervention, most staff reported having never intervened in harassment before, in part because many staff did not believe they ever had an opportunity to intervene. Even among staff who had intervened, staff intervened in fewer situations of harassment than they observed. In college settings, studies show that students do not always intervene and report many barriers to intervening, including not noticing a situation, not identifying the situation as high risk, not taking responsibility, having a skills deficit, and inhibition based on social pressures or fear (
      • Bennett S.
      • Banyard V.L.
      • Garnhart L.
      To act or not to act, that is the question? Barriers and facilitators of bystander intervention.
      ;
      • Burn S.M.
      A situational model of sexual assault prevention through bystander intervention.
      ). Future research should examine factors that determine whether and how staff intervene in health care settings.
      Similar to
      • Moynihan M.M.
      • Banyard V.L.
      • Arnold J.S.
      • Eckstein R.P.
      • Stapleton J.G.
      Engaging intercollegiate athletes in preventing and intervening in sexual and intimate partner violence.
      , many staff showed meaningful improvement on bystander outcomes and few staff reported decreases after the training. Research indicates that most attendees of sexual harassment and assault programs have positive reactions to these programs, yet a small subset have negative reactions (
      • Roehling M.V.
      • Huang J.
      Sexual harassment training effectiveness: An interdisciplinary review and call for research.
      ), particularly men with less gender egalitarian views or those more likely to engage in sexually coercive or harassing behavior (
      • Bingham S.G.
      • Scherer L.L.
      The unexpected effects of a sexual harassment educational program.
      ;
      • Malamuth N.
      • Huppin M.
      • Linz D.
      Sexual assault interventions may be doing more harm than good with high-risk males.
      ;
      • Robb L.A.
      • Doverspike D.
      Self-reported proclivity to harass as a moderator of the effectiveness of sexual harassment-prevention training.
      ). Similarly, backlash in intentions to intervene in our study, although rare, was associated with not believing harassment was a problem. Identifying who will experience backlash in programs may be difficult. The large majority of those who initially did not believe harassment was a problem increased awareness and had positive outcomes. Research should examine how to engage staff who do not see harassment as a problem. Women were more likely than men to report a large decrease in barriers to intervening. Women also reported less self-efficacy at pretest, so this finding could reflect regression to the mean. Future studies should assess which barriers are most important for intervening in patient harassment and whether barriers vary by gender.
      We found the training was acceptable to staff. The majority believed the training was relevant and the length appropriate. Notably, staff stated that some of the most useful parts of the training involved group discussions where staff could hear from others, problem-solve strategies for response, and push their comfort zone within supportive and nonthreatening environments. These findings support the theory that in-person interactions are important for harassment training as they provide opportunities to ask questions and may be effective ways to change social norms and attitudes (
      • Best C.L.
      • Smith D.W.
      • Raymond Sr J.R.
      • Greenberg R.S.
      • Crouch R.K.
      Preventing and responding to complaints of sexual harassment in an academic health center: A 10-year review from the Medical University of South Carolina.
      ;
      • Bicchieri C.
      • Mercier H.
      Norms and beliefs: How change occurs.
      ).

      Limitations

      This study had multiple limitations. The pretest–post-test design was beneficial for assessing short-term outcomes, yet lacked a control group or behavioral outcomes. Furthermore, all measures had to be adapted heavily because no prior measures assessed intervening in patient harassment. Although the measures showed reasonable internal validity, sensitivity to change, and evidence of construct validity, more validation research is necessary to ensure that future measures capture the full range of patient harassment witnessing and intervening experiences. The approximately two-thirds of staff who self-selected to participate in the evaluation may not be representative of staff who were trained but did not participate or staff who were not trained. Furthermore, selection bias may have affected who chose to participate in the evaluation. Specifically, staff at trainings delivered to mental health workers were more likely to complete evaluation materials than staff at trainings delivered to primary care workers or voluntary seminars, and thus were over-represented. Also, given the high numbers of staff who reported experiencing harassment, staff who were more interested in the topic or had personal experiences with harassment may have been more likely to participate. We do not have additional demographic data on nonresponders and so are not able to determine how they may vary from responders. Also, the VA's End Harassment Workgroup encourages trainers to adapt content to fit local needs, so the proportion of time spent on raising awareness of harassment and/or teaching intervention strategies varies greatly across sites. Therefore, these results may not generalize to sites that conduct trainings of different length or content. Furthermore, many completed the post-test immediately after the training, while the presenter was still in the room, so results may have been affected by social desirability. Last, training and data collection coincidentally began around 15 days after the start of the #MeToo movement. We were piloting items during this time and able to remove items that suffered from socially desirable ceiling or floor effects. Nonetheless, staff responses to the remaining items may have been affected by social desirability at pretest, limiting the ability to see improvements and leading to underestimates of effect sizes. Alternatively, staff may have been more engaged, which could have made the training appear more effective.
      Whether staff will intervene in harassment is unknown, yet, small effects on attitudinal measures are common for bystander programs that have found changes in behaviors (
      • Jouriles E.N.
      • Krauss A.
      • Vu N.L.
      • Banyard V.L.
      • McDonald R.
      Bystander programs addressing sexual violence on college campuses: A systematic review and meta-analysis of program outcomes and delivery methods.
      ;
      • Kettrey H.H.
      • Marx R.A.
      The effects of bystander programs on the prevention of sexual assault across the college years: A systematic review and meta-analysis.
      ). Also, this training and other prevention efforts at the VA grew out of findings that women veterans who were harassed at the VA were more likely to report feeling unwelcome, feeling unsafe, and delaying/missing care at the VA (
      • Klap R.
      • Darling J.E.
      • Hamilton A.B.
      • Rose D.E.
      • Dyer K.
      • Canelo I.
      • Yano E.M.
      Prevalence of stranger harassment of women veterans at Veterans Affairs medical centers and impacts on delayed and missed care.
      ). We do not know how intervening will affect women veterans' sense of belonging, safety, or engagement at the VA. Randomized trials or stronger quasi-experimental designs are needed to assess whether the training is associated with increased bystander behaviors, decreased harassment, and/or improvements in patients’ experiences of care. Importantly, two constructs that changed here—self-efficacy and intentions—have been shown to partially mediate the effect of a bystander program on intervening behaviors (
      • McMahon S.
      • Peterson N.A.
      • Winter S.C.
      • Palmer J.E.
      • Postmus J.L.
      • Koenick R.A.
      Predicting Bystander Behavior to Prevent Sexual Assault on College Campuses: The Role of Self-Efficacy and Intent.
      ). We intend to conduct future trials to examine the impacts of the program on behavioral outcomes at the VA. Research is also needed to determine the acceptability and impact of bystander interventions for harassment in non-VA hospital settings. Last, research is necessary to determine whether prevention programs and educational campaigns targeted directly towards male patients could be successful at decreasing harassing behavior.

      Implications for Practice and/or Policy

      Patient harassment, and workplace violence from patients more broadly, has been reported in health care settings for decades and is associated with significant costs, negative outcomes for staff, decreased patient care and engagement, and lower staff retention and time spent at work (
      • Lanctôt N.
      • Guay S.
      The aftermath of workplace violence among healthcare workers: A systematic literature review of the consequences.
      ). In response to findings that women veterans who experience harassment at the VA are more likely to feel unsafe and delay or miss care at the VA (
      • Klap R.
      • Darling J.E.
      • Hamilton A.B.
      • Rose D.E.
      • Dyer K.
      • Canelo I.
      • Yano E.M.
      Prevalence of stranger harassment of women veterans at Veterans Affairs medical centers and impacts on delayed and missed care.
      ), the VA is ramping up efforts to find effective ways to reduce harassment, improve safety, and remove this potential barrier to treatment. This evaluation indicates that a bystander approach to patient harassment is acceptable to staff and that 45 to 60 minutes is sufficient to modify attitudinal antecedents to intervening. For health care institutions considering a bystander approach to patient harassment, these results provide some guidance. Increasing staff awareness of harassment seems to be important to both avoid reactance and motivate staff to intervene. Given that both staff and patients experience harassment, trainings should prepare staff to intervene in both situations. Last, staff reported that the group discussions were critical training components. Therefore, institutions considering electronic trainings as a low-cost alternative to in-person trainings should evaluate the efficacy of such trainings before dissemination. Increased experimentation with novel ways to address patient harassment is needed to continue to build an evidence base for reducing sexual and gender harassment in health care settings.

      Acknowledgments

      The authors thank the employees of VA CT Healthcare system and Dawne Vogt, PhD, who provided feedback on the survey instruments. We are also grateful to all the employees who participated in the evaluation.

      References

        • Banyard V.L.
        Measurement and correlates of prosocial bystander behavior: the case of interpersonal violence.
        Violence and Victims. 2008; 23: 83-97
        • Banyard V.L.
        The promise of a bystander approach to violence prevention.
        in: Toward the next generation of bystander prevention of sexual and relationship violence. Springer, New York2015: 7-23
        • Banyard V.L.
        • Moynihan M.M.
        • Cares A.C.
        • Warner R.
        How do we know if it works? Measuring outcomes in bystander-focused abuse prevention on campuses.
        Psychology of Violence. 2014; 4: 101
        • Bastomski S.
        • Smith P.
        Gender, fear, and public places: How negative encounters with strangers harm women.
        Sex Roles. 2017; 76: 73-88
        • Bennett S.
        • Banyard V.L.
        • Garnhart L.
        To act or not to act, that is the question? Barriers and facilitators of bystander intervention.
        Journal of Interpersonal Violence. 2014; 29: 476-496
        • Best C.L.
        • Smith D.W.
        • Raymond Sr J.R.
        • Greenberg R.S.
        • Crouch R.K.
        Preventing and responding to complaints of sexual harassment in an academic health center: A 10-year review from the Medical University of South Carolina.
        Academic Medicine. 2010; 85: 721-727
        • Bicchieri C.
        • Mercier H.
        Norms and beliefs: How change occurs.
        in: The complexity of social norms. Springer, New York2014: 37-54
        • Bingham S.G.
        • Scherer L.L.
        The unexpected effects of a sexual harassment educational program.
        Journal of Applied Behavioral Science. 2001; 37: 125-153
        • Boissonnault J.S.
        • Cambier Z.
        • Hetzel S.J.
        • Plack M.M.
        Prevalence and risk of inappropriate sexual behavior of patients toward physical therapist clinicians and students in the united states.
        Physical therapy. 2017; 97: 1084-1093
        • Burn S.M.
        A situational model of sexual assault prevention through bystander intervention.
        Sex Roles. 2009; 60: 779-792
        • Campbell I.
        Chi-squared and Fisher–Irwin tests of two-by-two tables with small sample recommendations.
        Statistics in Medicine. 2007; 26: 3661-3675
        • Cogin J.
        • Fish A.
        Sexual harassment–a touchy subject for nurses.
        Journal of Health Organization and Management. 2009; 23: 442-462
        • Cohen J.
        Statistical power analysis for the behavioral sciences.
        2nd ed. Erlbaum, Hillsdale, NJ1988
        • Coker A.L.
        • Bush H.M.
        • Cook-Craig P.G.
        • DeGue S.A.
        • Clear E.R.
        • Brancato C.J.
        • Recktenwald E.A.
        RCT testing bystander effectiveness to reduce violence.
        American Journal of Preventive Medicine. 2017; 52: 566-578
        • Davidson M.M.
        • Butchko M.S.
        • Robbins K.
        • Sherd L.W.
        • Gervais S.J.
        The mediating role of perceived safety on street harassment and anxiety.
        Psychology of Violence. 2016; 6: 553
        • Davidson M.M.
        • Gervais S.J.
        • Sherd L.W.
        The ripple effects of stranger harassment on objectification of self and others.
        Psychology of Women Quarterly. 2015; 39: 53-66
        • DeMayo R.A.
        Patient sexual behavior and sexual harassment: A national survey of female psychologists.
        Professional Psychology: Research and Practice. 1997; 28: 58
        • Dyer K.E.
        • Potter S.J.
        • Hamilton A.B.
        • Luger T.M.
        • Bergman A.A.
        • Yano E.M.
        • Klap R.
        Gender differences in veterans’ perceptions of harassment on Veterans Health Administration grounds.
        Women's Health Issues. 2019; 29: S83-S93
        • Fairchild K.
        • Rudman L.A.
        Everyday stranger harassment and women’s objectification.
        Social Justice Research. 2008; 21: 338-357
        • Fitzgerald L.F.
        • Gelfand M.J.
        • Drasgow F.
        Measuring sexual harassment: Theoretical and psychometric advances.
        Basic and Applied Social Psychology. 1995; 17: 425-445
        • Hruschka D.J.
        • Schwartz D.
        • St John D.C.
        • Picone-Decaro E.
        • Jenkins R.A.
        • Carey J.W.
        Reliability in coding open-ended data: Lessons learned from HIV behavioral research.
        Field methods. 2004; 16: 307-331
        • Jouriles E.N.
        • Krauss A.
        • Vu N.L.
        • Banyard V.L.
        • McDonald R.
        Bystander programs addressing sexual violence on college campuses: A systematic review and meta-analysis of program outcomes and delivery methods.
        Journal of American College Health. 2018; 66: 457-466
        • Katz J.
        • Moore J.
        Bystander education training for campus sexual assault prevention: An initial meta-analysis.
        in: Perspectives on College Sexual Assault: Perpetrator, Victim, and Bystander. Springer, New York2013: 183-196
        • Kettrey H.H.
        • Marx R.A.
        The effects of bystander programs on the prevention of sexual assault across the college years: A systematic review and meta-analysis.
        Journal of Youth and Adolescence. 2019; 48: 212-227
        • Klap R.
        • Darling J.E.
        • Hamilton A.B.
        • Rose D.E.
        • Dyer K.
        • Canelo I.
        • Yano E.M.
        Prevalence of stranger harassment of women veterans at Veterans Affairs medical centers and impacts on delayed and missed care.
        Women's Health Issues. 2019; 29: 107-115
        • Klap R.
        • Golden R.E.
        Preliminary findings from wave 1 of the WH-PBRN stranger harassment card study.
        in: VA Women’s Health Services Harassment Workgroup Online Meeting. VA Veterans Health Administration National, 2018 (February 21, 2018)
        • Klap R.
        • Potter S.
        • Yano E.
        • Bergman A.
        • Hamilton A.
        • Darling J.
        “Hey baby, smile” VA staff perspectives on addressing harassment of women veterans at VA medical centers.
        in: Academy Health 2017 Annual Research Meeting. New Orleans, LA. 2017
        • Lanctôt N.
        • Guay S.
        The aftermath of workplace violence among healthcare workers: A systematic literature review of the consequences.
        Aggression and Violent Behavior. 2014; 19: 492-501
        • Latané B.
        • Darley J.M.
        The unresponsive bystander: Why doesn't he help?.
        Appleton-Century-Crofts, Norwalk, CT1970
        • Leskinen E.A.
        • Cortina L.M.
        • Kabat D.B.
        Gender harassment: Broadening our understanding of sex-based harassment at work.
        Law and Human Behavior. 2011; 35: 25-39
        • Macmillan R.
        • Nierobisz A.
        • Welsh S.
        Experiencing the streets: Harassment and perceptions of safety among women.
        Journal of Research in Crime and Delinquency. 2000; 37: 306-322
        • Malamuth N.
        • Huppin M.
        • Linz D.
        Sexual assault interventions may be doing more harm than good with high-risk males.
        Aggression and Violent Behavior. 2018; 41: 20-24
        • McMahon S.
        • Peterson N.A.
        • Winter S.C.
        • Palmer J.E.
        • Postmus J.L.
        • Koenick R.A.
        Predicting Bystander Behavior to Prevent Sexual Assault on College Campuses: The Role of Self-Efficacy and Intent.
        American journal of community psychology. 2015; 56: 46-56
        • Moynihan M.M.
        • Banyard V.L.
        • Arnold J.S.
        • Eckstein R.P.
        • Stapleton J.G.
        Engaging intercollegiate athletes in preventing and intervening in sexual and intimate partner violence.
        Journal of American College Health. 2010; 59: 197-204
        • National Center for Veterans Analysis and Statistics
        Women veterans report: The past, present and future of women veterans.
        Author, Washington, DC2017
        • Nickerson A.B.
        • Aloe A.M.
        • Livingston J.A.
        • Feeley T.H.
        Measurement of the bystander intervention model for bullying and sexual harassment.
        Journal of Adolescence. 2014; 37: 391-400
        • Nielsen M.B.D.
        • Kjær S.
        • Aldrich P.T.
        • Madsen I.E.
        • Friborg M.K.
        • Rugulies R.
        • Folker A.P.
        Sexual harassment in care work–Dilemmas and consequences: A qualitative investigation.
        International Journal of Nursing Studies. 2017; 70: 122-130
        • Orchowski L.M.
        • Berry-Caban C.S.
        • Prisock K.
        • Borsari B.
        • Kazemi D.M.
        Evaluations of sexual assault prevention programs in military settings: A synthesis of the research literature.
        Military Medicine. 2018; 183: 421-428
        • Phillips S.P.
        • Schneider M.S.
        Sexual harassment of female doctors by patients.
        New England Journal of Medicine. 1993; 329: 1936-1939
        • Robb L.A.
        • Doverspike D.
        Self-reported proclivity to harass as a moderator of the effectiveness of sexual harassment-prevention training.
        Psychological Reports. 2001; 88: 85-88
        • Roehling M.V.
        • Huang J.
        Sexual harassment training effectiveness: An interdisciplinary review and call for research.
        Journal of Organizational Behavior. 2018; 39: 134-150
        • Shipherd J.C.
        • Darling J.E.
        • Klap R.S.
        • Rose D.
        • Yano E.M.
        Experiences in the veterans health administration and impact on healthcare utilization: Comparisons between LGBT and non-LGBT women veterans.
        LGBT Health. 2018; 5: 303-311
        • Spector P.E.
        • Zhou Z.E.
        • Che X.X.
        Nurse exposure to physical and nonphysical violence, bullying, and sexual harassment: A quantitative review.
        International Journal of Nursing Studies. 2014; 51: 72-84
        • Wilson L.C.
        The prevalence of military sexual trauma: A meta-analysis.
        Trauma, Violence, & Abuse. 2018; 19: 584-597

      Biography

      Mark R. Relyea, PhD, is a community psychologist and statistician at VA Connecticut Healthcare System, and associate research scientist at Yale School of Medicine. His focus is on understanding how to prevent sexual assault and harassment and improve outcomes for survivors.
      Galina A. Portnoy, PhD, is a psychologist at VA Connecticut Healthcare System and Associate Research Scientist at Yale School of Medicine. Her research interests include trauma, women's health, and the detection, prevention, and treatment of intimate partner violence, particularly perpetration.
      Ruth Klap, PhD, is a Research Health Scientist at the VA HSR&D Center for the Study of Healthcare Innovation, Implementation and Policy and National Consortium Program Manager for the VA Women's Health Research Network.
      Elizabeth M. Yano, PhD, MSPH, is Director, VA HSR&D Center for the Study of Innovation, Implementation and Policy, and Adjunct Professor, Health Policy and Management, UCLA Fielding School of Public Health.
      Angie Fodor, MS, is a Project Manager in Women's Health Services, Veterans Health Administration.
      Jessica Keith, PhD, is a clinical psychologist at Bay Pines VA Healthcare System and Associate Professor of Psychology at University of Central Florida College of Medicine.
      Jane A. Driver, MD, MPH, is a geriatrician who researches improving care for older veterans. She is Associate Director-Clinical of the New England Geriatric Research and Clinical Center at VA Boston and Associate Professor of Medicine at Harvard Medical School.
      Cynthia A. Brandt, MD, MPH, is board certified in preventive medicine and clinical informatics. Her research is interdisciplinary and focuses on issues related to the design, development, and use of informatics tools and health services research.
      Sally G. Haskell, MD, is Deputy Director, Women's Health Services, Veterans Health Administration; Core Investigator, VA HSR&D PRIME Center, VA Connecticut Healthcare System, and Professor of Medicine, Yale University.
      Lynette Adams, PhD, is a Project Manager for Comprehensive Women's Health, Women's Health Services, Veterans Health Administration and an Assistant Clinical Professor in the School of Medicine, Department of Psychiatry at Yale University.