Aspect of validity evidence measured | Statistical approach and criteria | Initial LFS short form scale (5 items) | Final LFS short form scale (3 items) |
---|---|---|---|
Step 1: Evidence based on internal structure | Rating scale functioning The average measures for each response category should advance monotonically, with outfit MnSq values < 2.0 for each of the step calibrations | Criteria were met | Criteria were met |
Step 2: Evidence based on internal structure (item misfit) | Item goodness-of-fit statistics A sample-size adjusted criterion for item goodness-of-fit requiring infit MnSq values between 0.7 and 1.3 | #17 close eyes: MnSq = 1.78 #16 conversation: MnSq = 1.42 #5 worn out: MnSq = 0.75 #1 tired: MnSq = 0.67 #4 fatigued: MnSq = 0.66 | #5 worn out: MnSq = 1.02 #1 tired: MnSq = 1.10 #4 fatigued: MnSq = 0.80 |
Step 3: Evidence based on internal structure (unidimensionality) | Principal component analysis of the residuals The criterion was set for at least 50% of the total variance to be explained by the first latent variable | 63.2% | 81.6% |
Step 4: Evidence based on response processes | Person goodness-of-fit statistics A criterion of < 5% of the sample has unacceptable person goodness-of-fit, as indicated by infit MnSq values < 1.4 and standardized z values < 2.0. | 24 (4.6%) | 29 (5.6%) |
Floor and ceiling effects A criterion of up to 10% of the sample could demonstrate minimum or maximum scores. | Maximum score = None Minimum score = 77 (14.0%) | Maximum score = 3 (0.6%) Minimum score = 88 (16.9%) | |
Step 5: Item stability | Differential item functioning (DIF) No Mantel-Haenzel statistics have a p < .01. | Items 4 and 5 demonstrated DIF by diagnosis | No item demonstrated DIF by diagnosis |
Step 6: Person-separation reliability | Person-separation index An index of > 2.0 was required to ensure that the scale could differentiate the sample into at least three different levels of fatigue severity | Separation index = 1.82 | Separation index = 2.49 |
Step 7: Internal consistency | Person reliability (Rasch) Cronbach’s alpha coefficient (Cr-α) (raw scores) Relationship between individual sum scores and Rasch-based measures The criterion was that all values should be > 0.80 | Person reliability (Rasch) = .77 Cr-α (raw score) = .87 Correlation between sum scores and Rasch measures = .91 | Person reliability (Rasch) = .89 Cr-α (raw score) = .91 Correlations between sum scores and Rasch measures = .98 |