Towards Personalized English Learning Diagnosis: Cognitive Diagnostic Modelling for EFL Listening

Authors

  • xiaomei Ma School of Foreign Studies, Xi'an Jiaotong University, Xi'an, China
  • Yaru Meng School of Foreign Studies, Xi'an Jiaotong University, Xi'an, China

Keywords:

Personalized learning, EFL listening comprehension, Cognitive diagnostic assessment, Diagnostic model

Abstract

ABSTRACT: This paper aims to discuss how an EFL listening diagnostic model is constructed based on the theory of Cognitive Diagnostic Assessment (CDA). Compared with the traditional classical assessment, CDA can offer fine-grained feedbacks on the learner’s knowledge structure and cognitive process. The diagnostic test model construction starts with the identification and definition of EFL attributes relevant to listening comprehension, followed by the hypothetic Q-matrix model and diagnostic test design, and finally, a psychometric G-DINA model analysis. The validated diagnostic model can offer reliable diagnostic feedbacks for group as well as individual levels both for listening competence and attribute mastery profiles, and  is therefore qualified for online diagnostic purposes.

Author Biographies

xiaomei Ma, School of Foreign Studies, Xi'an Jiaotong University, Xi'an, China

Professor and PhD supervisor of SLA in Language and Culture Systematics of the Center for Graduate Studies of Foreign Languages in the School of Foreign Studies, Xi’an Jiaotong University.

Her research areas: Second Language Acquisition; Language Teaching &Technology

 

Yaru Meng, School of Foreign Studies, Xi'an Jiaotong University, Xi'an, China

MengYaru, PhD，Associate professor of the Center for College English in the School of Foreign Studies, Xi’an Jiaotong University.

Her research directions:  L2 Teaching and Technology; L2 assessment.

References

Alderson, J. C. (2005) Diagnosing Foreign Language Proficiency---The Interface between Learning and Assessment [M]. Britain: Continuum. 70-78.

Anderson JR. (2000). Cognitive Psychology and its Implications [M]. New York: Worth Publishers.

Buck, G., Tatsuoka, K., & Kostin, I. (1997). The subskills of reading: Rule-space analysis of a multiple-choice test of second language reading comprehension. Language Learning, 47(3), 423-466.

Buck, G., & Tatsuoka, K. (1998). Application of the rule-space procedure to language testing: Examining attributes of a free response listening test. Language Testing, 15(2), 119-157.

Birenbaum, M., Nasser, F., &Tatsuoka, C. (2005). Large-scale diagnostic assessment: Mathematics

performance in two educational systems. Educational Research and Evaluation, 11(5), 487-507

Buck, G (2011). Assessing Listening [M], Beijing: Foreign Language Teaching an Research Press. 56-57

Cai, Y. (2010). Group-level Ability Assessment and Cognitive Diagnosis on English Reading Problem Solving. Unpublished doctorate dissertation, JiangXi Normal University.

Cai, Y. Tu, D.B. & Ding., S. L. (2010). Theory andMethod on Compilation of Cognitive Diagnosis Test. Examination Research(3): 79-92.

Chapelle, C.A. & Douglas (2010), Assessing Language through Computer Technology [M], Beijing：Foreign Language Teaching and Research Press, 114-117.

Chen, J., Torre, J. d. l., & Zhang, Z. (2013). Relative and Absolute Fit Evaluation in Cognitive Diagnosis Modeling. Journal of Educational Measurement 50 (2): 123–140

DeCarlo LT.(2011) On the analysis of fraction subtraction data: The DINA model, classification, latent class sizes, and the Q-matrix [J]. Applied Psychological Measurement. 35(1): 8-26.

De la Torre J. (2008a) An empirically based method of Q-matrix validation for the DINA model: Development and applications [J]. Journal of Educational Measurement. 45(4): 343-362.

De la Torre J.(2008b) The generalized DINA model. The International Meeting of the Psychometric Society. Durham, NH

De La Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76(2), 179-199.

De la Torre J. (2010) Estimation Code for the G-DINA Model [C]. in Rupp AA. Software for calibrating diagnostic classification models: An overview of the current state of the art. Maryland: University of Maryland.18-22.

Graham, S. (2006). Listening comprehension: The learners’ perspective. System, 34(2), 165-182.

Hargreaves, A. & Shirley, D. (2009). The Fourth Way: The Inspiring Future for Educational Change. Corwin: London.

Jang, E. E. (2005). A validity narrative effects of reading skills diagnosis on teaching and learning in the context of NG TOEFL: University of Illinois at Urbana-Champaign.

Jang, E. E. (2009a). Cognitive diagnostic assessment of L2 reading comprehension ability: Validity arguments for fusion model application to languedge assessment. Language Testing, 26(1), 031-073.

Jang Eunice Eunhee. (2009b) Demystifying a Q-Matrix for Making Diagnostic Inferences About L2 Reading Skills. Language Assessment Quarterly, 6: 210-238.

Kasai M. (1997). Application of the rule space model to the reading comprehension section of the test of English as a foreign language [D]. Urbana: University of Illinois at Urbana-Champaign.

Lee, Y.W. & Y. Sawaki.(2009). Cognitive Diagnosis Approaches to Language Assessment: An Overview [J]. Language Assessment Quarterly, , 6(3)：172-189.

Leighton, J. P. & M. J. Gierl. (2007). Cognitive Diagnostic Assessment for Education-- Theory and Application [M]. Cambridge University Press,. 3-18.

Liao, Y. F. (2009). A construct validation study of the GEPT reading and listening sections: Re-examining the models of L2 reading and listening abilities and their relations to lexico-grammatical knowledge. Teachers College, Columbia University.

Luo, Z. S. (2012). Item Response Theory. Beijng: Beijing Normal University Publishing Group.

Ma, X., Meng Y., He, H., & R. Liu,(2012) Personalized EFL. Audio-Visiondiagnostic model construction and system development［J］Foreign Language Education（5）:59-63

Meng, Y. R. (2013). Developing a Model of Cognitive Diagnostic Assessment for College EFL Listeng, Unpublished doctorate dissertation, Shanghai International Studies University.

Richards JC.(1983) Listening comprehension: Approach, design, procedure. TESOL Quarterly., 17(2): 219-240.

Rupp A. A & Templin J. (2008) The effects of Q-matrix misspecification on parameter estimates and classification accuracy in the DINA model [J]. Educational and Psychological Measurement., 68(1): 78-96.

Rupp, A., Templin A., J.& Henson, R.(2010) Diagnostic Measurement :Theory, Method, and Applications [M], New York London: the Guilford Press,. 26-48.

Sinharay, S., & Almond, R. G. (2007). Assessing Fit Of Cognitive Diagnostic Models A Case Study. Educational and psychological measurement, 67(2), 239-257.

Swaffar, J. &Bacon, S. (1993) Reading and Listening Comprehension: Perspectives on Research and Implications for the Classroom. In A.O. Hadley Lincolnwood, ILL: National Textbook Co., 1993. 124-55.

Tatsuoka KK.(1983) Rule space: an approach for dealing with misconceptions based on item response theory [J]. Journal of Educational Measurement., 20(4): 345-354.

Tu, D. , Qi, S. & Dai H..(2008)Cognitive Diagnostic Assessment in Educational Testing[J]. Testing Research. 4(4):4-15

Wang, C. Mark P. & J Gierl (2011). Using the Attribute Hierarchy Method to Make Diagnostic Inferences about Examinees’ Cognitive Skills in Critical Reading. Journal of Educational Measurement. Vol.48-2:165-187.

Vandergrift, L. (1999). Facilitating Second Language Listening Comprehension: Acquiring Successful Strategies. ELT Journal, 53(3), 168-176.

Zhang WM.(2006) Detecting differential item functioning using the DINA model [D]. Greensboro: The University of North Carolina..

Zou S. (2005) Language Assessment[M], Shanghai: Shanghai Foreign Language Education Press.

Downloads

Published

2014-10-15

How to Cite

Ma, xiaomei, & Meng, Y. (2014). Towards Personalized English Learning Diagnosis: Cognitive Diagnostic Modelling for EFL Listening. Asian Journal of Education and E-Learning, 2(5). Retrieved from https://ajouronline.com/index.php/AJEEL/article/view/1669

Issue

Vol. 2 No. 5 (2014): October 2014

Section

Articles

License

  • Papers must be submitted on the understanding that they have not been published elsewhere (except in the form of an abstract or as part of a published lecture, review, or thesis) and are not currently under consideration by another journal published by any other publisher.
  • It is also the authors responsibility to ensure that the articles emanating from a particular source are submitted with the necessary approval.
  • The authors warrant that the paper is original and that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required.
  • The authors ensure that all the references carefully and they are accurate in the text as well as in the list of references (and vice versa).
  • Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Attribution-NonCommercial 4.0 International that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
  • Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
  • Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
  • The journal/publisher is not responsible for subsequent uses of the work. It is the author's responsibility to bring an infringement action if so desired by the author.