Study on Optimizing Senior High English Assessment via Multi-dimensional Item Set Effect

Zhiyi Wu

doi:10.63944/n1e.NHE

Authors

Zhiyi Wu Faculty of Education, Sichuan Normal University Author

DOI:

https://doi.org/10.63944/n1e.NHE

Keywords:

Partial Credit Model; Multidimensional Ability; Local Dependence; English Reading Assessment; Markov Chain Monte Carlo

Abstract

Currently, English reading, as a core module in the middle school English assessment system, accounts for a high proportion usually exceeding 70%. Addressing the limitations of traditional models in English reading assessment, this study proposes the Multidimensional Partial Credit Testlet Model (MPCTM) based on the Partial Credit Model (PCM), by integrating the multidimensional ability space, multi-level scoring mechanism, and testlet random effects. Results of the simulation study show that the error and bias of MPCTM in parameter estimation are significantly lower than those of traditional models. Findings from the empirical study confirm that MPCTM has better explanatory power and goodness of fit in complex assessment scenarios; in the future, it is necessary to increase the proportion of mixed question types and expand its interdisciplinary applications.

References

[1] Wang, D. X., & Guo, Y. Y. (2022). Development and application of multilevel scoring IRT model for fusion reaction time. Psychological Exploration, 42, 269-278.

[2] Zhan, P. D., Wang, W., & Wang, L. J. (2013). Topic group response theory of new development of item response theory. Advances in Psychological Science, 21, 265-280.

[3] Tu, D. B., Cai, Y., & Paint, S. G. (2009). The new development of item response theory—The realization of problem group model and its parameter estimation. Psychological Science, 32, 143-145. https://doi.org/10.16719/j.cnki.1671-6981.2009.06.026

[4] Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika, 34(S1), 91-97.

[5] Andrich, D. (1982). An extension of the Rasch model for ratings providing both location and dispersion parameters. Psychometrika, 47(1), 105-113.

[6] Wainer, H., & Kiely, G. L. (1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24(3), 185-201.

[7] DeMars, C. E. (2006). Application of the bi‐factor multidimensional item response theory model to testlet‐based tests. Journal of Educational Measurement, 43(2), 145-168.

[8] Zhan, P. D., Wang, W., Wang, L. J., et al. (2014). Multi-dimensional problem group effect Rasch model. Journal of Psychology, 46, 108-122.

[9] Zhan, P. D., Li, X. M., Wang, W., et al. (2015). Cognitive diagnosis model of multi-dimensional problem group effect. Journal of Psychology, 47, 689-701.

[10] Zhan, P., Jiao, H., & Liao, D. (2018). Cognitive diagnosis modelling incorporating item response times. British Journal of Mathematical and Statistical Psychology, 71(2), 262-286.

[11] Cai, L. (2010). A two-tier full-information item factor analysis model with applications. Psychometrika, 75(4), 581-612.

[12] Zhou, W. J., Tong, W. W., & Guo, L. (2023). Multilevel score cognitive diagnostic question group model. Applied Psychology, 29, 470-479. https://doi.org/10.20058/j.cnki.cjap.022163

[13] Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149-174.

[14] Gelman, A., & Shalizi, C. R. (2013). Philosophy and the practice of Bayesian statistics. British Journal of Mathematical and Statistical Psychology, 66(1), 8-38.

[15] Wei, D., Liu, H. Y., & Zhang, D. H. (2017). Multidimensional problem group response model: Application extension of multidimensional random coefficient multiple logistic model. Journal of Psychology, 49, 104-114.

[16] Embretson, S. E., & Reise, S. P. (2013). Item response theory for psychologists. Psychology Press.

[17] Baker, F. B., & Kim, S. H. (2017). The basics of item response theory using R (Vol. 10). Springer.

[18] Fisher, R. A. (1922). On the mathematical foundations of theoretical statistics. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 222(594-604), 309-368.

[19] Akaike, H. (1974). A new look at the statistical model identification problem. IEEE Transactions on Automatic Control, 19(6), 716-723.

[20] Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461-464.

[21] McCutcheon, A. L. (1987). Latent class analysis (Vol. 64). Sage.