Arikan, S. & Aybek, E.C. (2022), A Special Case of Brennan's Index for Tests That Aim to Select a Limited Number of Students: A Monte Carlo Simulation Study. Educational Measurement: Issues and Practice, 41: 35-49. https://doi.org/10.1111/emip.12528

Abstract

Many scholars compared various item discrimination indices in real or simulated data. Item discrimination indices, such as item-total correlation, item-rest correlation, and IRT item discrimination parameter, provide information about individual differences among all participants. However, there are tests that aim to select a very limited number of students, examinees, or candidates for allocated schools and job positions. Thus, there is a need to evaluate the performances of CTT and IRT item discrimination indices when the test purpose is to select a limited number of students. The purpose of the current Monte Carlo study is to evaluate item discrimination indices in the case of selecting a limited number of high-achieving students. The results showed that a special case of Brennan's index, B10–90, provided more accurate information for this specific test purpose. Additionally, the effects of various factors, such as test length, ability distributions of examinees, and item difficulty variance on item discrimination indices were investigated. The performance of each item discrimination index is discussed in detail.

Celen, U. & Aybek, E. C. (2022). A novel approach for calculating the item discrimination for Likert type of scales. International Journal of Assessment Tools in Education, 9(3), 772-786. doi: 10.21449/ijate.1173356

Abstract

Item analysis is performed by developers as an integral part of the scale development process. Thus, items are excluded from the scale depending on the item analysis prior to the factor analysis. Existing item discrimination indices are calculated based on correlation, yet items with different response patterns are likely to have a similar item discrimination index. This study proposed a new item discrimination index that can be used in Likert type of scales and examined its effect on factor analysis results. For this purpose, simulative datasets were generated, and items were excluded from the analysis according to the .20, .30 and .35 item discrimination index criteria, and exploratory factor analysis was performed for a single factor. Accordingly, it was found that more variance could be explained by a single factor with fewer items compared to other discrimination indices when the .20 criterion of the slope coefficient was used as suggested in this study. Similar findings were obtained using the .35 criterion with other discrimination indices. In this context, it is recommended to use the slope coefficient as an additional discrimination index calculation method in the scale development process.

Aybek, E.C. & Toraman, C. (2022). How many response categories are sufficient for Likert type scales? An empirical study based on the Item Response Theory. International Journal of Assessment Tools in Education, 9(2), 534-547. doi: 10.21449/ijate.1132931

Abstract

The current study investigates the optimum number of response categories for the Likert type of scales under the item response theory (IRT). The data was collected from university students attend to mainly the faculty of medicine and the faculty of education. A form of the “Social Gender Equity Scale” developed by Gozutok et al. (2017) was prepared, which had 3, 5 and 7-point response categories. The graded response model (GRM) was used for item calibrations. The results of the study have revealed that using a 5-point response option provides advantages over using a 3-point response category in terms of reliability and test information perspective in the scale development process. The-5 point scale also provides easier responding process for the respondents while it does not pose a major disadvantage compared to a 7-point response category in the terms of reliability. Therefore, based on the findings of the study, researchers are recommended to use a 5-point response category in their scale development process.

Kula Kartal, S., Aybek, E.C., & Yasar, M. (2022). Investigating the Wording Effect in Scales Based on the Different Dimension Reduction Techniques Journal of Uludag University Faculty of Education, 35(1), 44-67. https://doi.org/10.19171/uefad.1033284

Abstract

This study aims to examine the dimensionality of a dataset obtained from the application of a multidimensional scale which is not balanced in terms of the numbers of the positively and negatively worded items based on the item response theory, DETECT, and factor analysis. To this aim, a scale developed to measure parents' perceptions of their children's school was utilized. The study group consisted of 1,388 parents. The dimensionality of the dataset obtained from the scale was examined based on the item response theory, DETECT and factor analyses. The results of the three methods commonly revealed that the negatively worded items formed a separate cluster. Based on this finding, it can be stated that relations among the negatively worded items cannot only be explained by the intended trait with the scale, and the wording of the items also affect the relations among the negatively worded items. No matter what dimension reduction technique was used to analyze the data, the study results evidenced the existence of a strong wording effect in the data set. Based on the study results, some recommendations were given to researchers and practitioners regarding how to examine the dimensionality of a dataset obtained from a scale including both positively and negatively worded items.

Aybek, E.C. (2021). catIRT tools: A "Shiny" Application for Item Response Theory Calibration and Computerized Adaptive Testing Simulation. Journal of Applied Testing Technology, 22(1). 23-27.

Abstract

The study aims to introduce catIRT tools which facilitates researchers’ Item Response Theory (IRT) and Computerized Adaptive Testing (CAT) simulations. catIRT tools provides an interface for mirt and catR packages through the shiny package in R. Through this interface, researchers can apply IRT calibration and CAT simulations although they do not have any coding skills. Dichotomous and polytomous IRT models are supported in IRT calibration and Yen’s Q3 statistics is calculated for the estimation of local independence. In CAT simulation, researchers can use their own parameters and responses, and can produce items or responses. In addition to several item selection and ability estimation methods, researchers can also decide on the specific stopping rule to be used.

Aybek, E. C., & Demirtasli, R. N. (2017). Computerized Adaptive Test (CAT) Applications and Item Response Theory Models for Polytomous Items. International Journal of Research in Education and Science, 3(2), 475-487.

Abstract

This article aims to provide a theoretical framework for computerized adaptive tests (CAT) and item response theory models for polytomous items. Besides that, it aims to introduce the simulation and live CAT software to the related researchers. Computerized adaptive test algorithm, assumptions of item response theory models, nominal response model, partial credit and generalized partial credit models and graded response model are described carefully to reach that aim. Likewise, item selection methods, such as maximum Fisher information, maximum expected information, minimum expected posterior variance, maximum expected posterior weighted-information, and ability prediction methods, such as expected a posteriori and maximum a posteriori, are expounded as well as stopping rules for the computerized adaptive tests.