Research

The gold standard method to research the impact of a program is the Randomized Control Trial (RCT). In essence, it allows us to disentangle the impact of our program from everything else going on to recruit and support candidates in a state. This is the same method medical research uses to prove drugs are effective. It works.

 

Contest Every Race runs RCTs in two dimensions, geography and individuals. In geographic control trials, we randomly choose half our target races or counties and leave them out of the program. In individual control trials, within our target races, we send texts to most, but not all, of the target individuals eligible to run for those races, while remaining silent to a significant share of individuals randomly chosen as a control group. We also conduct surveys to assess how much our texts improve perceptions of the local Democratic Party.

 

This kind of research method comes with some downsides – it reduces our program’s impact where we implement it, it’s expensive and time consuming, and it requires staff with statistical and experimental design experience. But the result is that we can say with confidence exactly what our impact was.

In Georgia, Republicans we targeted were 17% more likely to draw a challenger.

Bar graph titled GA 2019 Elections. Uncontested races dropped 17%; the treatment group ratio is roughly 0.5 and the control ratio is about 0.7.

Across all of our states, Democrats we texted were 4% more favorable toward the local Democratic party.

A bar graph titled Party favorability rises 4 points among those texted. The control group has an average Dem favorability score of 49, while the treatement scored 56.

In counties our program targeted in Virginia, Democrats received a 7% higher vote share than in counties randomly selected as controls.

A bar graph titled Virginia House 9019 Elections: Impact of CER on 2017-2019 Dem net vote percent difference. The difference in control counties was about +2%, while the difference in treatment counties was about +10%.

Across all of our work, individuals to whom we sent an initial text were overall 28% more likely to file than those to whom we didn’t.

RCT Research

State / RacesDateTypeDifferenceP Value
TXJul-Aug 2018Filing Rate50%0.128
OK SchoolNov-Dec 2018Filing Rate223%0.001
OK SchoolNov-Dec 2018Uncontested Races-7%0.049
OK MuniJan-Feb 2019Filing Rate60%0.043
OK MuniJan-Feb 2019Uncontested Races-11%0.005
MSFeb 2019Filing Rate-22%0.197
VAMay-Jun 2019Filing Rate52%0.053
GA 1Jul-Aug 2019Uncontested Races-17%0.001
COJul-Aug 2019Filing Rate8%0.421
NCNov-Dec 2019Filing Rate25%0.248
GA 2Jan-Feb 2020Filing Rate19%0.25
OverallAug 2018-Feb 2020Filing Rate28%0.04

RCT Research

State / RacesDateTypeDifferenceP Value
TXJul-Aug 2018Filing Rate50%0.128
OK SchoolNov-Dec 2018Filing Rate223%0.001
OK SchoolNov-Dec 2018Uncontested Races-7%0.049
OK MuniJan-Feb 2019Filing Rate60%0.043
OK MuniJan-Feb 2019Uncontested Races-11%0.005
MSFeb 2019Filing Rate-22%0.197
VAMay-Jun 2019Filing Rate52%0.053
GA 1Jul-Aug 2019Uncontested Races-17%0.001
COJul-Aug 2019Filing Rate8%0.421
NCNov-Dec 2019Filing Rate25%0.248
GA 2Jan-Feb 2020Filing Rate19%0.25
OverallAug 2018-Feb 2020Filing Rate28%0.04

We are committed to sharing the results of our RCTs back, warts and all, and providing information that supports the movement’s understanding of how to best recruit and support progressive candidates representative of their communities.