Research design
The study used a cross-sectional design. Data were collected from September 2020 to January 2021. The study was approved by the Research Ethics Committee of Universitas Ahmad Dahlan, Yogyakarta, with ethical approval number 012007028 on 22 September 2020 and was divided into two main phases, adaptation and validation.
Instrument
The instrument consisted of sociodemographic characteristics and the HIV-KQ-18 instrument. HIV-KQ-18 is the short version of HIV-KQ-45 [9, 13]. Permission to translate the HIV-KQ-18 instrument was obtained from Prof. Michael P. Carey, PhD (Director, Centers for Behavioral and Preventive Medicine, The Miriam Hospital) on February 11, 2020. The HIV-KQ-18 instrument is more focused on how to prevent infection and transmission of HIV/AIDS. This instrument consists of 18 items, and each item has 3 options, namely “true”, “false” or “don’t know”. Five items (no 1, 4, 11, 14, 17) are true statements, while the other 13 items are false. The correct response is scored 1, while 0 is used for wrong or “don’t know” responses.
We collected sociodemographic data included gender, age, occupation, education level, marital status and monthly expenses. We asked for monthly expenditure data instead of monthly income since participants indicated that they were more comfortable reporting expenses than income. Participants were classified concerning having an educational background in health sciences (medicine, pharmacy, nursing, midwifery, and public health) and whether they had attended a workshop about a HIV/AIDS education. To maintain confidentiality, participants were given the right to only write their name initials and age. Only researchers had the right to access the dataset.
Cross-cultural adaptation and validation
Indonesian researchers conducted a literature review and agreed to continue the study of the HIV-KQ-18 instrument's validation with several caveats, including: (i) Indonesia does not yet have a validated instrument that can be used to assess HIV/AIDS knowledge; and (ii) there are some very interesting questions that need to be explored further, such as condom knowledge and the method of transmitting HIV/AIDS. Based on our findings and experiences, these two themes are still relatively rarely discussed in public. The original HIV-KQ-18 instrument was translated to Bahasa Indonesia using a forward–backward translation. As a result, the Indonesia research team is continuing to adaptation processes.
The research team consists of scientists and professionals in the fields of HIV, health, and psychometrics study. There are nine Indonesian authors in the team and their origins also represent three regions of Indonesian (West, Central and East Indonesia). Five authors are from Western Indonesia (MRR, Z, DAP, RN, and RSP), three authors from the Central Region (BA, MM, and MNM), and one author from the Eastern Region (SR). Because of the various dialects in each region, this representation can reduce the chance of bias or ambiguity. In addition, DAP and BA are Indonesian researchers who have conducted similar studies several times. Furthermore, all Indonesian authors are lecturers, and lecturers in Indonesia are required to 'do community service.' The majority of them have taken part in HIV/AIDS education programs for the community, particularly for middle and high school teachers and students. Several of the authors have conducted at least one similar study (some are in the process of submitting manuscripts and some are in the process of collecting data).
Some psychometric study components, such as content validity, face validity, and construct validity [14,15,16], were investigated in this study. As illustrated in Fig. 1, an expert panel analyzed and consolidated the Indonesian translation of the original version in order to obtain consensus on inconsistencies and produce a version 1 for field testing to 60 participants. At the end of the instrument, we added a question “Out of the 18 items, which statement was the most difficult or took a long time to answer?”. This question was used to identify additional obstacles by participants to understand the HIV-KQ-18 Bahasa Indonesia instrument.
Participants
Participants were Indonesians at least 17 years old, who consented to participate in the study. Participants involved in the study were sampled from six of Indonesia's main regions, namely: Sumatra, Java, Sulawesi, Kalimantan, 'Bali and Nusa Tenggara', and 'Maluku and Papua'. It is important to highlight that the 'research site' in this study refers to the participant's domicile location (district and province) as submitted on the online link that we distributed.
Sample size
The sample size calculation was based on a study suggesting at least 100 participants should be the minimum limit for psychometric study [17]. Two previous studies stated that the minimum number of participants should be 200 if the number of items in the instrument is not more than 40 [18, 19]. Another study recommends that the minimum number of participants involved in a psychometric study should be the number of items in the instrument to be validated multiplied by ten [20]. Accordingly, the minimum number of participants for each region included in the study is 180 (18 items × 10). Ergo, the minimum total of participants in this study was set at 1080 (180 × 6) participants.
Data collection
The initial Indonesian version of HIV-KQ-18, after forward and backward translations, was evaluated by the Indonesian research team. After obtaining ethical clearance, in the adaptation phase we requested feedback for the initial version of the HIV-KQ-18 from HIV activists, an obstetrician, general practitioners and 60 pilot participants (10 participants from each region). Feedback was reviewed by the Indonesian research team to develop the final version of HIV-KQ-18 Bahasa Indonesia.
In the validation phase, we distributed the online link of the final version of HIV-KQ-18 Bahasa Indonesia through several social media from 29 September 2020 to 6 April 2021, namely: WhatsApp, Facebook inbox, email, Instagram, and Twitter. If participants joined through various media, we removed the duplicate participants based on the initials and date of birth. Figure 1 provides an overview of the study procedure.
Data analysis
During the adaptation phase, we discussed all of the participants’ suggestions. Furthermore, the Indonesian core research team compiled and analyzed the pilot data for the best choice of words and word order in one single sentence (item) through consensus. Notably, whenever differences emerged in this phase, these issues were resolved by consensus.
Participants’ characteristics in the adaptation phase were analysed descriptively. Item analysis was conducted by calculating the percentage of the correct answers of each item and corrected item-total correlation. Items with the percentage of correct answers being between 30 and 80% were considered appropriate [21], because it avoids floor and ceiling effects and allows for additional scores to capture knowledge gains after an education program. Items with a corrected item-total correlation lower than 0.3 were considered to indicate that the items were candidates for deletion [22]; however, an item with a corrected item-total correlation higher than 0.25 was still considered acceptable.
Validity and reliability tests were used to analyze the psychometric properties of HIV-KQ-18 Bahasa Indonesia. Cronbach’s alpha was used to analyze the internal consistency, and a Cronbach’s alpha of higher than 0.7 is considered as a reliable instrument [22].
Validity was measured using known-group validity and construct validity. Known-group validity was determined by comparing the HIV knowledge between participants based on four variables, namely education level, educational background, experience in attending a workshop about HIV and monthly expenses. Previous studies conducted in the Asian general population have demonstrated that different subgroups based on education levels [23,24,25,26,27,28,29], HIV education [28] and socioeconomic status [23, 24, 27,28,29] have different HIV knowledge levels. Therefore, the a priori hypothesis was made that participants with higher education level, educational background in health sciences, experience in attending a workshop about HIV and higher monthly expense had significant higher HIV knowledge than participants with lower education level, without educational background in health sciences, never experience in attending a workshop about HIV and lower monthly expense. The difference of HIV knowledge levels was assessed using an independent t-test for a variable with two subgroups, while ANOVA test for comparing HIV knowledge for a variable with more than two subgroups. If the result from ANOVA was significant, a post hoc analysis was carried using a Bonferroni test.
HIV-KQ-18 has dichotomous options; therefore, an exploratory factor analysis (EFA) using a polychoric correlation matrix was conducted to assess the construct validity [30, 31]. Muñoz-Quezada et al. (2021) used polychoric correlation to conduct factor analysis when validating an instrument with dichotomous options to measure the perception and knowledge about exposure to pesticides [32]. To eliminate the subjectivity in interpreting the scree plot, the number of factors that could be retained was determined based on eigenvalues, parallel analysis, optimal coordinates, and acceleration factor. If these parameters had different recommendations regarding the number of factors to be retained, other considerations were applied in which a factor should have at least 3 items [20] and the possibility of underlying factors to be interpreted. Goodness of fit was assessed using the root mean square of residuals (RMSR) and the Root Mean Square Error of Approximation (RMSEA). A factor loading of 0.4 or higher is required to indicate a good relationship between each item and underlying factor [20].
All statistical analyses were performed using the Statistical Package for Social Sciences (SPSS) version 26 (IBM Corp, Armonk, New York, USA), except for the EFA that was analyzed in R version 4.1.0 and RStudio Version 1.4.1717 using packages of polycor, nFactors and psych. The level of statistical significance was set at p < 0.05. Atlas.ti (Scientific Software Development GmbH, Berlin, Germany) was used to analyze the feedback from the participants regarding the most difficult item in responding to the HIV-KQ-18 Bahasa Indonesia.