I’ve been racking my brain for the last couple of hours trying to understand how to apply DeclareDesign to my specific situation. I was hoping somebody could at least point me in the right direction.
I’m running an experiment where I randomly pair participants up with one another to play a game. Participants can be Republican or Democrat. By definition, you can be paired either with a member of your in-group (so Republican-Republican, Democrat-Democrat), or a member of your out-group (Republican-Democrat, Democrat-Republican).
Some of the DVs are before and after measures that tracks, for example, openness to the political outgroup, whereas other DVs are behavioral games (so post-measures only).
My base model looks like the following:
model_base <- lm_robust(outcome_of-interest ~ politics*outgroup_pairing, cluster = teamID, se = "stata", data = data) summary(model_base)
politics represents the politics of a given respondent, and
outgroup_pairing is a dummy variable that represents 1 if the participant was paired with a member of the outgroup. In other specifications, I add a vector of controls to increase precision. I’m clustering standard errors at the “team” level to resolve inter-team correlations (because the team plays a game together, they will have identical scores on the game, and this may influence outcomes).
We’ve already run a pilot consisting of 70 DD pairings, 80 “mixed pairings”, and 80 RR pairings, and find that the coefficient on “outgroup_pairing” is large and significant, and the interaction is as well – so Republicans are more affected by an outgroup_pairing than Democrats. We now want to run a pre-registered version, and want to do power calculations to have a more accurate estimate of the sample size needed to detect an effect at least as large as the one detected in the pilot.
I couldn’t find anything in the documentation that sheds light on what to do here as this isn’t exactly a pre-post design with a clear-cut treatment condition. Another wrinkle here is that we will be running multiple games with the same pairing philosophy (in-group vs out_group), and will be comparing the effects in one game with the effects of another game. So we want to ask, for example, does a Democrat paired with a Republican in Game 1 show a larger change on the DV than a Democrat paired with a Republican in Game 2?
I’m not expecting a comprehensive answer, but I was hoping I could at least get some direction as to how to do power analysis in this context.