Table 3 Overview of the analytic process using a Rasch model approach

1

Rating scale functioning: Does the rating scale function consistently across items? (substantive validity)

• Average measures for each step category and threshold on each item should advance monotonically

• Rating scale met criteria for all items but item 2. Scale steps 1 and 2 reversed. Recoded into 1(2)34scale

• Rating scale met criteria

• z-values < 2.0 in outfit mean square (MnSq) values for step category calibrations^b

2

Internal scale validity: How well do the actual item responses match the expected responses from the Rasch model? (content validity)

Item goodness-of-fit statistics

• MnSq values < 1.3^c

• 3 items failed to meet criterion^d: • Item 2: MnSq=1.64 (1) • Item 3: MnSq=1.39 (2) • Item 1: MnSq=1.38 (3)

• All items met criterion

3

Internal scale validity: Is the scale unidimensional (i.e., does it measure a single construct)? (structural validity)

Principal component analysis • ≥ 50% of total variance explained by first component (general self-efficacy)^e • Any additional component explains < 5% (or eigenvalue<2.0) of the remaining variance after removing first component^e No more than 5% (or 1 out of 20) of the residual correlations >.30

• First component explained 61.3% of total variance • Second component • explained 6.9% of total variance, but eigenvalue <2.0 (1.8) • One out of 45 (2.2%) residual correlations >.30 (#3 - #8: r = -.31)

• First component explained 64.5% of total variance • Second component explained 8.9% of total variance, but eigenvalue <2.0 (1.7) • One out of 21 (4.8%) residual correlations >.30 (#4 - #9: r = -.33)

4

Person-response validity: How well do the individual responses match expected responses from the Rasch model? (substantive validity)

Person goodness-of-fit statistics • Infit MnSq values < 1.5 and z- value ≤ 2.0^f • ≤ 5% of sample fails to demonstrate acceptable goodness-of-fit values^f

• 13/14 respondents (9.2/9.9% of sample) failed to demonstrate acceptable goodness-of-fit values

• 8/9 respondents ( 5.7/6.4% of sample) failed to demonstrate acceptable goodness-of-fit values

• Infit MnSq values < 1.5 and z- value ≤ 2.0^f

• ≤ 5% of sample fails to demonstrate acceptable goodness-of-fit values^f

5

Person-separation reliability: Can the scale distinguish ≥3 distinct groups of self-efficacy in the sample tested? (reliability)

Person-separation index • ≥ 2.0^g

• 2.75

• 2.67

6

Internal consistency: Are item responses consistent with each other? (reliability)

Cronbach’s alpha coefficient • > 0.8^g

• 0.93

• 0.93