forked from Renatagppr93/Engagement-Techreport
/
04-analyses.Rmd
395 lines (287 loc) · 18.6 KB
/
04-analyses.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
# Analyses
```{r data, echo=FALSE, warning=FALSE, message=FALSE}
temp <- read.csv("Data/qualtrics_pilot_data.csv", header=FALSE, na.strings="")
x <- temp[2,]
data <- temp[-c(1:3),]
colnames(data) <- x
num <- nrow(data)
## getting conditions into one large file below - 12/9/20
data$Cond1 <- rowSums(is.na(data[18:53]))
data$Cond2 <- rowSums(is.na(data[54:89]))
data$Cond3 <- rowSums(is.na(data[90:125]))
data$Cond4 <- rowSums(is.na(data[126:161]))
data$Condition[data$Cond1 < 36] <- 1
data$Condition[data$Cond2 < 36] <- 2
data$Condition[data$Cond3 < 36] <- 3
data$Condition[data$Cond4 < 36] <- 4
cond1 <- data[ which(data$Condition==1), ]
cond2 <- data[ which(data$Condition==2), ]
cond3 <- data[ which(data$Condition==3), ]
cond4 <- data[ which(data$Condition==4), ]
cond1.red <- cond1[,c(6, 18:53, 162:165, 171)] ## using Cond1 ordering
cond2.red <- cond2[,c(6, 62:65, 70:73, 82:85, 58:61, 74:77, 86:89, 66:69, 78:81, 54:57, 162:165, 171)]
cond3.red <- cond3[,c(6, 94:97, 106:109, 118:121, 98:101, 110:113, 122:125, 102:105, 114:117, 90:93, 162:165, 171)]
cond4.red <- cond4[,c(6, 138:161, 130:137, 126:129, 162:165, 171)] ## 171 versus 172 because testing script has extra "hours" variable
names(cond1.red) = gsub(pattern = "C*.* - ", replacement = "", x = names(cond1.red)) ## Getting rid of condition markers so rbind will work
names(cond2.red) = gsub(pattern = "C*.* - ", replacement = "", x = names(cond2.red))
names(cond3.red) = gsub(pattern = "C*.* - ", replacement = "", x = names(cond3.red))
names(cond4.red) = gsub(pattern = "C*.* - ", replacement = "", x = names(cond4.red))
together <- rbind(cond1.red, cond2.red, cond3.red, cond4.red) ## we'll be using this object for analyses
num_valid <- nrow(together)
i <- c(1:37) ## Changing item responses to numerics
together[ , i] <- apply(together[ , i], 2, # Specify own function within apply
function(x) as.numeric(as.character(x)))
#library(descr)
# freq(together$`Duration (in seconds)`)
```
## Pilot analyses
`r num_valid` people responded. We may want to further screen these `r num_valid` responses for short response durations. According to communications with Qualtrics in early December, this is the number of seconds to complete the entire survey. Currently, the "number of people who clicked on the assessment link (`r num`)" versus valid *n* (`r num_valid`) may take care of our very low duration respondents. The shortest response duration in the `r num` datafile is `r min(data$Duration)` whereas the lowest in the `r num_valid` datafile is `r min(together$Duration)`. All analyses were performed via `R` version `r R.version.string` [@R-base].
### Classical test theory
In addition to the below interactive plot [via `plotly` version `r packageVersion("plotly")`; @R-plotly], a full inter-item correlation matrix is located in Appendix \@ref(corrs).
```{r interactive_plot, echo=FALSE, warning=FALSE, message=FALSE}
# Interactive plot
library(reshape2)
mcor<-together[,2:37]
mcor<-mcor[c(33,34,35,36,
1,2,3,4,
13,14,15,16,
25,26,27,28,
5,6,7,8,
17,18,19,20,
29,30,31,32,
9,10,11,12,
21,22,23,24)]
mcor<-round(cor(mcor, use="na.or.complete" ),2)
get_upper_tri <- function(mcor){
mcor[lower.tri(mcor)]<- NA
return(mcor)}
upper_tri<-get_upper_tri(mcor)
melted_cor<-melt(upper_tri, na.rm = TRUE)
library(ggplot2)
cor.plot<-ggplot(data=melted_cor, aes(Var1, Var2, fill=value))+
geom_tile(color="white")+
scale_fill_gradient2(low="blue",high="red",mid="white",
midpoint = 0, limit= c(-1,1), space = "Lab",
name="Attitudinal Correlation")+
theme_minimal()+
theme(axis.text = element_blank())+
coord_fixed()
library(plotly)
ggplotly(cor.plot)
```
```{r alphas, echo=FALSE, warning=FALSE, message=FALSE}
## recodes 12/9/20
## AFFECTIVE:
together$`Most days, I feel happiest when the workday is soon to be complete.` <- 7 - together$`Most days, I feel happiest when the workday is soon to be complete.`
together$`This job drains my energy.` <- 7 - together$`This job drains my energy.`
## BEHAVIORAL (NONE):
## COGNITIVE:
together$`Thinking about work saps my energy.` <- 7 - together$`Thinking about work saps my energy.`
together$`I often think about finding another job.` <- 7 - together$`I often think about finding another job.`
affect <- psych::alpha(together[2:13])
behave <- psych::alpha(together[14:25])
cogni <- psych::alpha(together[26:37])
absorb <- psych::alpha(together[c(2:5, 14:17, 34:37)])
vigor <- psych::alpha(together[c(26:29, 6:9, 18:21)])
dedic <- psych::alpha(together[c(30:33, 10:13, 22:25)])
aff.abs <- psych::alpha(together[2:5])
aff.vig <- psych::alpha(together[6:9])
aff.ded <- psych::alpha(together[10:13])
beh.abs <- psych::alpha(together[14:17])
beh.vig <- psych::alpha(together[18:21])
beh.ded <- psych::alpha(together[22:25])
cog.abs <- psych::alpha(together[34:37])
cog.vig <- psych::alpha(together[26:29])
cog.ded <- psych::alpha(together[30:33])
one <- together[ which(together$Condition == 1), ]
two <- together[ which(together$Condition == 2), ]
three <- together[ which(together$Condition == 3), ]
four <- together[ which(together$Condition == 4), ]
aff.1 <- psych::alpha(one[2:13])
beh.1 <- psych::alpha(one[14:25])
cog.1 <- psych::alpha(one[26:37])
abs.1 <- psych::alpha(one[c(2:5, 14:17, 34:37)])
vig.1 <- psych::alpha(one[c(26:29, 6:9, 18:21)])
ded.1 <- psych::alpha(one[c(30:33, 10:13, 22:25)])
aff.2 <- psych::alpha(two[2:13])
beh.2 <- psych::alpha(two[14:25])
cog.2 <- psych::alpha(two[26:37])
abs.2 <- psych::alpha(two[c(2:5, 14:17, 34:37)])
vig.2 <- psych::alpha(two[c(26:29, 6:9, 18:21)])
ded.2 <- psych::alpha(two[c(30:33, 10:13, 22:25)])
aff.3 <- psych::alpha(three[2:13])
beh.3 <- psych::alpha(three[14:25])
cog.3 <- psych::alpha(three[26:37])
abs.3 <- psych::alpha(three[c(2:5, 14:17, 34:37)])
vig.3 <- psych::alpha(three[c(26:29, 6:9, 18:21)])
ded.3 <- psych::alpha(three[c(30:33, 10:13, 22:25)])
aff.4 <- psych::alpha(four[2:13])
beh.4 <- psych::alpha(four[14:25])
cog.4 <- psych::alpha(four[26:37])
abs.4 <- psych::alpha(four[c(2:5, 14:17, 34:37)])
vig.4 <- psych::alpha(four[c(26:29, 6:9, 18:21)])
ded.4 <- psych::alpha(four[c(30:33, 10:13, 22:25)])
```
#### Internal consistency estimates
Condition 1 administered items within the substantive dimensions (with successive randomized blocks of Cognitive, Affective, and Behavioral items). Condition 2 administered items within the attitudinal dimensions (with successive randomized blocks of Absorption, Vigor, and Dedication items). Condition 3 stressed the substantive dimensions (with items fully randomized regardless of attitudinal association). Condition 4 stressed the attitudinal dimensions (with items fully randomized within attitudinal dimension regardless of substantive scale association, see Chapter \@ref(conds) and Appendix \@ref(pilot2)). All internal consistency estimates were generated via `psych` version `r packageVersion("psych")` [@R-psych]. Alphas for the candidate 12-item scales were:
| Dimension | Undifferentiated | Condition 1 | Condition 2 | Condition 3 | Condition 4 |
|----------------|-----------------------------------------|-----------------------------------|-----------------------------------|-----------------------------------|-----------------------------------|
| Affective | `r round(affect$total$raw_alpha, 2)` |`r round(aff.1$total$raw_alpha, 2)`|`r round(aff.2$total$raw_alpha, 2)`|`r round(aff.3$total$raw_alpha, 2)`|`r round(aff.4$total$raw_alpha, 2)`|
| Behavioral | `r round(behave$total$raw_alpha, 2)` |`r round(beh.1$total$raw_alpha, 2)`|`r round(beh.2$total$raw_alpha, 2)`|`r round(beh.3$total$raw_alpha, 2)`|`r round(aff.4$total$raw_alpha, 2)`|
| Cognitive | `r round(cogni$total$raw_alpha, 2)` |`r round(cog.1$total$raw_alpha, 2)`|`r round(cog.2$total$raw_alpha, 2)`|`r round(cog.3$total$raw_alpha, 2)`|`r round(aff.4$total$raw_alpha, 2)`|
| | | | | | |
| Absorption | `r round(absorb$total$raw_alpha, 2)` |`r round(abs.1$total$raw_alpha, 2)`|`r round(abs.2$total$raw_alpha, 2)`|`r round(abs.3$total$raw_alpha, 2)`|`r round(aff.4$total$raw_alpha, 2)`|
| Vigor | `r round(vigor$total$raw_alpha, 2)` |`r round(vig.1$total$raw_alpha, 2)`|`r round(vig.2$total$raw_alpha, 2)`|`r round(vig.3$total$raw_alpha, 2)`|`r round(aff.4$total$raw_alpha, 2)`|
| Dedication | `r round(dedic$total$raw_alpha, 2)` |`r round(ded.1$total$raw_alpha, 2)`|`r round(ded.2$total$raw_alpha, 2)`|`r round(ded.3$total$raw_alpha, 2)`|`r round(aff.4$total$raw_alpha, 2)`|
"Cell" level alphas (4 items each scale, responses collapsed across administrative conditions) were:
| Cell | Alpha |
|----------------------------|--------------------------------------|
| Affective - Absorption |`r round(aff.abs$total$raw_alpha, 2)` |
| Affective - Vigor |`r round(aff.vig$total$raw_alpha, 2)` |
| Affective - Dedication |`r round(aff.ded$total$raw_alpha, 2)` |
| | |
| Behavioral - Absorption |`r round(beh.abs$total$raw_alpha, 2)` |
| Behavioral - Vigor |`r round(beh.vig$total$raw_alpha, 2)` |
| Behavioral - Dedication |`r round(beh.ded$total$raw_alpha, 2)` |
| | |
| Cognitive - Absorption |`r round(cog.abs$total$raw_alpha, 2)` |
| Cognitive - Vigor |`r round(cog.vig$total$raw_alpha, 2)` |
| Cognitive - Dedication |`r round(cog.ded$total$raw_alpha, 2)` |
Corrected item-total correlations are presented in Appendix \@ref(rdrops)
### Confirmatory factor analyses
We used `lavaan` version `r packageVersion("lavaan")` [@R-lavaan] and `semPlot` version `r packageVersion("semPlot")` [@R-semPlot]
```{r CFA, echo=FALSE, warning=FALSE, message=FALSE}
options("scipen"=100, "digits"=2) ## suppresses scientific notation
library(lavaan)
library(sem)
library(semPlot)
library(dplyr)
CFAdata<-together[,2:37]
CFAdata<-CFAdata%>%rename(
Item_1=`I’m able to concentrate on my work without distractions.`,
Item_2=`I have a hard time detaching mentally from my work.`,
Item_3=`Time passes quickly while I’m working.`,
Item_4=`I find it difficult to mentally disconnect from work.`,
Item_5=`I enjoy thinking about work even when I’m not at work.`,
Item_6=`Most days, I feel happiest when the workday is soon to be complete.`,
Item_7=`I am happiest when I am immersed in a project.`,
Item_8=`I love starting my workday.`,
Item_9=`I devote more time than is expected of me.`,
Item_10=`I have to be reminded to take breaks while I’m at work.`,
Item_11=`I never miss a work deadline.`,
Item_12=`I never allow distractions to interfere with my work.`,
Item_13=`I devote my full attention to my work tasks throughout the day.`,
Item_14=`Thinking about work saps my energy.`,
Item_15=`I would rather direct my focus toward a work task than a personal task.`,
Item_16=`I’m able to maintain good levels of energy throughout the workday.`,
Item_17=`I enjoy spending time completing my job tasks.`,
Item_18=`Most days I feel enthusiastic about starting my work day.`,
Item_19=`I feel motivated to go beyond what is asked of me.`,
Item_20=`This job drains my energy.`,
Item_21=`When work is slow I find ways to be productive.`,
Item_22=`I express enthusiasm for my job while at work.`,
Item_23=`I try my best to perform well at work.`,
Item_24=`If I notice my energy level is low, I take corrective steps to re-energize.`,
Item_25=`I plan my future with this company.`,
Item_26=`I believe this company cares about my career goals.`,
Item_27=`I often think about finding another job.`,
Item_28=`This organization challenges me to work at my full potential.`,
Item_29=`I am proud to be a member of this organization.`,
Item_30=`I feel supported by my supervisor when I fail at a task.`,
Item_31=`I feel proud of my accomplishments within this organization.`,
Item_32=`My job makes me feel like I’m part of something meaningful.`,
Item_33=`I make valued contributions to the organization.`,
Item_34=`I embrace challenging situations at work.`,
Item_35=`I speak positively about this organization to others.`,
Item_36=`This organization provides the resources necessary for me to successfully perform my job.`
)
Sub_Model<-'
Absorption=~Item_1+Item_2+Item_3+Item_4+Item_5+Item_6+Item_7+Item_8+Item_9+Item_10+Item_11+Item_12
Vigor=~Item_13+Item_14+Item_15+Item_16+Item_17+Item_18+Item_19+Item_20+Item_21+Item_22+Item_23+Item_24
Dedication=~Item_25+Item_26+Item_27+Item_28+Item_29+Item_30+Item_31+Item_32+Item_33+Item_34+Item_35+Item_36
'
Fit1.1<-lavaan::cfa(Sub_Model, data = CFAdata)
semPlot::semPaths(Fit1.1, "col", "std", "lisrel", "circle")
fit1.1 <- as.data.frame(fitMeasures(Fit1.1))
fit1.1$rounded <- round(fit1.1$`fitMeasures(Fit1.1)`, 2)
## summary(Fit1.1, fit.measure=TRUE)
Att_Model<-'
Cognitive=~Item_1+Item_2+Item_3+Item_4+Item_13+Item_14+Item_15+Item_16+Item_25+Item_26+Item_27+Item_28
Affective=~Item_5+Item_6+Item_7+Item_8+Item_17+Item_18+Item_19+Item_20+Item_29+Item_30+Item_31+Item_32
Behavioral=~Item_9+Item_10+Item_11+Item_12+Item_21+Item_22+Item_23+Item_24+Item_33+Item_34+Item_35+Item_36
'
Fit1.2<-lavaan::cfa(Att_Model, data = CFAdata)
semPlot::semPaths(Fit1.2,"col", "std", "lisrel", "circle")
fit1.2 <- as.data.frame(fitMeasures(Fit1.2))
fit1.2$rounded <- round(fit1.2$`fitMeasures(Fit1.2)`, 2)
## CFA comparison
# Substantive Scale
CFA1 <-'
Absorption=~Item_1+Item_2+Item_3+Item_4+Item_5+Item_6+Item_7+Item_8+Item_9+Item_10+Item_11+Item_12
Vigor=~Item_13+Item_14+Item_15+Item_16+Item_17+Item_18+Item_19+Item_20+Item_21+Item_22+Item_23+Item_24
Dedication=~Item_25+Item_26+Item_27+Item_28+Item_29+Item_30+Item_31+Item_32+Item_33+Item_34+Item_35+Item_36
Engagement =~ Absorption + Vigor + Dedication
'
# fit the model
fit1 <- lavaan::cfa(CFA1, data = CFAdata) # dont know is using missing = "ML" instead
######################################### summary(fit1, fit.measures = TRUE, standardized = TRUE, rsq = TRUE)
# into data frames
fit1.H <- as.data.frame(fitMeasures(fit1))
fit1.H$rounded <- round(fit1.H$`fitMeasures(fit1)`, 2)
# Attitudinal scale
CFA2 <- '
Cognitive=~Item_1+Item_2+Item_3+Item_4+Item_13+Item_14+Item_15+Item_16+Item_25+Item_26+Item_27+Item_28
Affective=~Item_5+Item_6+Item_7+Item_8+Item_17+Item_18+Item_19+Item_20+Item_29+Item_30+Item_31+Item_32
Behavioral=~Item_9+Item_10+Item_11+Item_12+Item_21+Item_22+Item_23+Item_24+Item_33+Item_34+Item_35+Item_36
Engagement =~ Cognitive + Affective + Behavioral
'
# Fit the model
fit2 <- lavaan::cfa(CFA2, data = CFAdata)
######################################### summary(fit2, fit.measures = TRUE, standardized = TRUE, rsq = TRUE)
# into data frame
fit2.A <- as.data.frame(fitMeasures(fit2))
fit2.A$rounded <- round(fit2.A$`fitMeasures(fit2)`, 2)
# Visuals
library(semPlot)
# http://sachaepskamp.com/semPlot/examples?page_id=489
semPaths(fit1,"std", edge.label.cex = 0.5, curvePivot = TRUE, rotation=2)
semPaths(fit2,"std", edge.label.cex = 0.5, curvePivot = TRUE, rotation=4)
```
| Model | $\chi^2$ | *df* | RMSEA | SRMR | CFI | TLI | AIC |
|--------------------------|------------------|--------------------|-----------------|-----------------|----------------|-----------------|------------------|
| 3-factor substantive |`r fit1.1[3,2]` |`r fit1.1[4,2]` |`r fit1.1[23,2]` |`r fit1.1[29,2]` |`r fit1.1[9,2]` |`r fit1.1[10,2]` |`r fit1.1[19,2]` |
| Hierarchical substantive |`r fit1.H[3,2]` |`r fit1.H[4,2]` |`r fit1.H[23,2]` |`r fit1.H[29,2]` |`r fit1.H[9,2]` |`r fit1.H[10,2]` |`r fit1.H[19,2]` |
| 3-factor attitudinal |`r fit1.2[3,2]` |`r fit1.2[4,2]` |`r fit1.2[23,2]` |`r fit1.2[29,2]` |`r fit1.2[9,2]` |`r fit1.2[10,2]` |`r fit1.2[19,2]` |
| Hierarchical attitudinal |`r fit2.A[3,2]` |`r fit2.A[4,2]` |`r fit2.A[23,2]` |`r fit2.A[29,2]` |`r fit2.A[9,2]` |`r fit2.A[10,2]` |`r fit2.A[19,2]` |
#### Multigroup analyses
+ using experimental condition as "multiple groups" (measurement invariance)
#### Bifactor analyses
bifactor analysis are most commonly applied in the exploration of common method variance [see, for example, @reise_rediscovery_2012; @rodriguez_evaluating_2016]. @giordano_exploratory_2020 provide an overview regarding past and potential applications of *exploratory* bifactor analysis and cite @reise_rediscovery_2012 as an influential impetus for the resurgence of bifactor models in general.
```{r bifactor, echo=FALSE, message=FALSE, warning=FALSE}
## Bifactor below (5/19/21):
Bifactor_Model<-'
Cognitive=~Item_1+Item_2+Item_3+Item_4+Item_13+Item_14+Item_15+Item_16+Item_25+Item_26+Item_27+Item_28
Affective=~Item_5+Item_6+Item_7+Item_8+Item_17+Item_18+Item_19+Item_20+Item_29+Item_30+Item_31+Item_32
Behavioral=~Item_9+Item_10+Item_11+Item_12+Item_21+Item_22+Item_23+Item_24+Item_33+Item_34+Item_35+Item_36
Absorption=~Item_1+Item_2+Item_3+Item_4+Item_5+Item_6+Item_7+Item_8+Item_9+Item_10+Item_11+Item_12
Vigor=~Item_13+Item_14+Item_15+Item_16+Item_17+Item_18+Item_19+Item_20+Item_21+Item_22+Item_23+Item_24
Dedication=~Item_25+Item_26+Item_27+Item_28+Item_29+Item_30+Item_31+Item_32+Item_33+Item_34+Item_35+Item_36
Cognitive ~~ Affective
Cognitive ~~ Behavioral
Affective ~~ Behavioral
Absorption ~~ Vigor
Absorption ~~ Dedication
Vigor ~~ Dedication'
Fit.Bi <- lavaan::cfa(Bifactor_Model, data = CFAdata, orthogonal=TRUE)
semPlot::semPaths(Fit.Bi, bifactor = c("Cognitive", "Affective", "Behavioral"), "std", layout = "tree3",
rotation = 2, curvePivot=TRUE, style="lisrel", nCharNodes = 0) ## exoCov=FALSE deletes all covariances
title("Initial pilot bifactor analysis (36 candidate items)")
summary(Fit.Bi)
```
Currently pretty messed up prior to item deletion (Vigor especially). Model also technically not converging need Renata to fix.
Deese guys also do bifactor stuff: @mansolf_when_2017
### Summary
Recommendation for final instrument based on consideration of all of the above pieces of evidence
## Construct and Criterion-related Validation
Use Gallup for construct validation [@thackray_gallup_2005; @harter_relationship_2013]. Also the Utrecht Work Engagement Scale [UWES; @schaufeli_measurement_2002; @schaufeli_defining_2010]