Fabricating correlated likert items

fabricatr
#1

I am about to start my first DeclareDesign, which is also my first R-project. Unfortunately, I already have problems in the first lines of code. Help is much appreciated.

I will run a simple survey-experimental design with two arms and 2+3 experimental groups. To increase efficiency, I want to make use of pre-treatment covariates, using the lm_lin-model.

In the first step, therefore, I wanted to fabricate the pre-treatment covariates.

I would have thought that, in a second step, I would simulate the dependent variable, specifying the assumed correlation between DV and pre-treatment covariates so that the DeclareDesign power analysis (which I would specify later) could then take this information into account.

(I was somewhat confused that a recent blog-post on power strategies that discussed pre-treatment covariate adjustment fabricatr was not mentioned at all. Is my approach entirely wrong?)

Pre-treatment variable 1 is the mean value of three 5-point likert-scale indicators, which I assumed to be correlated at rho = .8
The DV is also the mean value of three 5-point likert-scale indicators, which I assumed to be correlated at rho = .8
I assume a correlation between pre-treatment variable 1 and DV of rho = .7

fabricated_data <- fabricate(

N = 100,
pretreatmentvar1_ind1 = draw_likert(x = rnorm(n = N),
                          type = 5),
pretreatmentvar1_ind2 = correlate(given = pretreatmentvar1_ind1 , rho = 0.8, draw_likert)
)

I noticed in that correlated does not support draw_likert, which confused me as multiple indicators with likert scales are very frequent. Do I have to prent that these are count variables in order to make this work? However, in a response by @nfultz in a related thread I saw that he does suggest simulating correlated Likert variables.

Thank you for any help with these first baby steps and thank you for setting all this up in the first place!

#2

draw_likert hasn’t been wired up for correlate() yet - but I think it would be a good feature for the next version.

In the meantime, what you could do:

  • sample from multivariate normal (eg using the MASS package)
  • use draw_likert on those.
1 Like
#3

We’re in the process of fixing draw_likert… but in the meantime, how about something like this:

rm(list = ls())

library(DeclareDesign)
library(tidyverse)

# Treatment effect in SDs
tau <- 0.2

# slight rewrite of the draw_likert function
my_likert <- 
  function(x, breaks = c(-Inf,-1.5,-0.5, 0.5, 1.5, Inf)){
  as.numeric(cut(x, breaks = breaks))
}


pre_score_design <- 
  declare_population(N = 500, 
                     latent = rnorm(N), 
                     Y_pre = my_likert(latent)) +
  # here's that correlate function... you can change rho to increase/decrease correlation.
  declare_potential_outcomes(Y ~ my_likert(correlate(given = latent, rho = 0.5, rnorm) + tau * Z)) + 
  declare_assignment(prob = 0.5, simple = TRUE) +
  declare_estimand(ate = mean(Y_Z_1 - Y_Z_0)) +
  declare_reveal(Y) +
  declare_estimator(Y ~ Z, model = lm_robust, estimand = "ate", label = "unadjusted") +
  declare_estimator(Y ~ Z, covariates = ~ Y_pre, model = lm_lin, estimand = "ate", label = "adjusted") 

draw_estimates(pre_score_design)

# actual simulations
simulations <- simulate_design(pre_score_design, sims = 250)

simulations <- 
  simulations %>%
  mutate(significant = p.value <= 0.05)

simulations %>%
  group_by(estimator_label) %>%
  summarise(mean(significant))

simulations %>%
  ggplot(aes(estimate, fill = significant)) +
  geom_histogram() +
  facet_wrap(~estimator_label)
#4

Thank you very much. This was so helpful. Very much appreciated. Thank you.

1 Like