Assessing Undergraduate Cognitive System Thinking Instruments in Genetics Lectures: A Rasch Model Analysis

Iffa Ichwani Putri; Adi Rahmat; Riandi Riandi; Lala Septem Riza

doi:10.35445/alishlah.v16i3.5621

Assessing Undergraduate Cognitive System Thinking Instruments in Genetics Lectures: A Rasch Model Analysis

Iffa Ichwani Putri, Adi Rahmat, Riandi Riandi, Lala Septem Riza

Abstract

This study aims to test the reliability, validity and wright person-items of the test instrument. It used to measure students' cognitive system thinking in genetics lectures. This cognitive system dimension (New Marzano Taxonomy) consists of 4 levels: retrieval, comprehension, analysis, and knowledge utilization. The research method used was descriptive quantitative, with respondents as many as 116 undergraduate students who had attended genetics lectures. The instrument used consisted of 20 multiple-choice questions developed from 4 levels of the cognitive dimension of the system. Data analysis using the Rasch Model-Dichotom to generate more accurate estimates based on individual abilities and difficulty levels of question items. The results showed that the Alpha Cronbach and Item Reliability value on the instrument was 0.81 (Very Good) to 0.92 (Best). Furthermore, person reliability is 0.63 (Moderate). It can be concluded the interaction between the question items and the quality of the question items on the test instrument indicates that the instrument effectively assesses constructs and is relevant for measuring the level of cognitive system thinking. Revisions are made to items that are too easy or too difficult, as well as a review of items with similar difficulties. So it can be assessed that this test has the potential to be widely used in measuring cognitive ability, but some improvements are needed to further improve its reliability for other population.

Keywords

Biology; Cognitive; Genetics; Rasch; Thinking

References

Aivelo, T., & Uitto, A. (2018). Teachers ’ approaches to genetics teaching mirror their perceptions of teaching controversial , societal and sensitive issues Teachers ’ approaches to genetics teaching mirror their perceptions of teaching controversial , societal and sensitive issues. (July). https://doi.org/10.1101/350710

Aivelo, T., & Uitto, A. (2021). Factors explaining students’ attitudes towards learning genetics and belief in genetic determinism. International Journal of Science Education, 43(9), 1408–1425. https://doi.org/10.1080/09500693.2021.1917789

Aramendiz, V. R. R., & Córdova, K. E. G. (2019). Decisions in evaluation: Virtual postgraduate environments, ex post-facto study. Revista de Educación a Distancia, 1(59). https://doi.org/10.6018/red/59/06

Betu, F. S. (2023). KOMPONEN TUJUAN DALAM NEW TAXONOMY MARZANO & KENDALL DAN RELEVANSINYA BAGI PENDIDIK. Atma Reksa : Jurnal Pastoral Dan Kateketik, 7(1). https://doi.org/10.53949/ar.v7i1.142

Bond, T. G., Yan, Z., & Heene, M. (2020). Applying the rasch model: Fundamental measurement in the human sciences. In Applying the Rasch Model: Fundamental Measurement in the Human Sciences. https://doi.org/10.4324/9780429030499

Boone, W. J. (2019). Rasch methods for beginners. Tidsskriftet “Pædagogisk Psykologisk Tidsskrift”S, 1.

Boone, W. J., Staver, J. R., & Yale, M. S. (2014a). Item Measures. In Rasch Analysis in the Human Sciences. https://doi.org/10.1007/978-94-007-6857-4_5

Boone, W. J., Staver, J. R., & Yale, M. S. (2014b). Understanding Person Measures. In Rasch Analysis in the Human Sciences. https://doi.org/10.1007/978-94-007-6857-4_4

Chan, S W, Looi, C. K., & Sumintono, B. (2021). Assessing computational thinking abilities among Singapore secondary students: A Rasch model measurement analysis. Journal of Computers in Education. https://doi.org/10.1007/s40692-020-00177-2

Chan, Shiau Wei, Ismail, Z., & Sumintono, B. (2014). A Rasch Model Analysis on Secondary Students’ Statistical Reasoning Ability in Descriptive Statistics. Procedia - Social and Behavioral Sciences, 129. https://doi.org/10.1016/j.sbspro.2014.03.658

DeLuca, C., Chapman-Chin, A. E. A., LaPointe-McEwan, D., & Klinger, D. A. (2018). Student perspectives on assessment for learning. Curriculum Journal, 29(1), 77–94. https://doi.org/10.1080/09585176.2017.1401550

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, Supplement, 14(1), 4–58. https://doi.org/10.1177/1529100612453266

Etobro, A. B., & Banjoko, S. O. (2017). Misconceptions of genetics concepts among pre-service teachers. Global Journal of Educational Research, 16(2), 121. https://doi.org/10.4314/gjedr.v16i2.6

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United States of America, 111(23), 8410–8415. https://doi.org/10.1073/pnas.1319030111

Kristyasari, M. L., Asmaningrum, H. P., & Hanif, R. F. (2022). Wright-Map to Analyze Students’ Abilities on Chemical Bond Test. SHS Web of Conferences, 149. https://doi.org/10.1051/shsconf/202214901050

Laliyo, L. A. R., Tangio, J. S., Sumintono, B., Jahja, M., & Panigoro, C. (2020). Analytic approach of response pattern of diagnostic test items in evaluating students’ conceptual understanding of characteristics of particle of matter. Journal of Baltic Science Education, 19(5). https://doi.org/10.33225/jbse/20.19.824

Machluf, Y., Gelbart, H., Ben-Dor, S., & Yarden, A. (2017). Making authentic science accessible-the benefits and challenges of integrating bioinformatics into a high-school science curriculum. Briefings in Bioinformatics, 18(1), 145–159. https://doi.org/10.1093/bib/bbv113

Pane, J. F., Steiner, E. D., Baird, M. D., Hamilton, L. S., & Pane, J. D. (2017). Informing Progress. (July), Ahmed, M. & Change, O. (2017) Academics’ e-learnin.

Pratiwi, M. K., Kuntjoro, S., Sunarti, T., & Budiyanto, M. (2023). TOSLS Cognitive Instrument to Measure Students’ Scientific Literacy Abilities. IJORER : International Journal of Recent Educational Research, 4(6). https://doi.org/10.46245/ijorer.v4i6.432

Prochazkova, K., Novotny, P., Hancarova, M., Prchalova, D., & Sedlacek, Z. (2019). Teaching a difficult topic using a problem-based concept resembling a computer game: Development and evaluation of an e-learning application for medical molecular genetics. BMC Medical Education, 19(1), 1–8. https://doi.org/10.1186/s12909-019-1817-2

Raof, S. A., Mustaâ, A. H., Zamzuri, F. K., & Salleh, M. H. (2021). Validity and reliability of students perceptions on OBE approach in Malaysian VC using Rasch model. Journal of Innovation in ….

Sari, W. K., & Nada, E. I. (2022). Marzano Taxonomy-Based Assessment Instrument to Measure Analytical and Creative Thinking Skills. Jurnal Pendidikan Kimia Indonesia, 6(1). https://doi.org/10.23887/jpk.v6i1.40117

Schmiemann, P., Nehm, R. H., & Tornabene, R. E. (2017). Assessment of Genetics Understanding: Under What Conditions Do Situational Features Have an Impact on Measures? Science and Education, 26(10), 1161–1191. https://doi.org/10.1007/s11191-017-9925-z

Septi Purnama, D., Farozin, M., & Astuti, B. (2022). The ryff’s psychological well-being scale for indonesian higher education students: A RASCH model analysis how to cite. IRJE |Indonesian Research Journal in Education, 6(2).

Sumintono, B. (2017). Rasch model measurement as tools in assessment for learning. eprints.um.edu.my.

Sumintono, Bambang. (2015). Pemodelan Rasch pada Asesmen Pendidikan: Suatu Pengantar. Konferensi Guru Dan Dosen Nasional (KGDN) 2015, (November 2015).

Sumintono, Bambang. (2016). Aplikasi Pemodelan Rasch pada asesmen pendidikan: Implementasi penilaian formatif (assessment for learning). Makalah Dipresentasikan Dalam Kuliah Umum Pada Jurusan Statistika, Institut Teknologi Sepuluh November, Surabaya, 17 Maret 2016., (March).

Sumintono, Bambang. (2018). Rasch Model Measurements as Tools in Assesment for Learning. https://doi.org/10.2991/icei-17.2018.11

Taylor, H. A., Rapp, D. N., & Brunye, T. A. D. T. (2007). Repetition and Dual Coding in Procedural Multimedia Presentations. Applied Cognitive Psychology, 22(September 2007), 877–895. https://doi.org/10.1002/acp

Tsui, L. (2002). Fostering critical thinking through effective pedagogy: Evidence from four institutional case studies. Journal of Higher Education, 73(6). https://doi.org/10.1080/00221546.2002.11777179

Wicaksono, D. A., Roebianto, A., & Sumintono, B. (2021). Internal Validation of the Warwick-Edinburgh Mental Wellbeing Scale:Rasch Analysis in the Indonesian Context. Journal of Educational, Health and Community Psychology, 10(2). https://doi.org/10.12928/jehcp.v10i2.20260

Zhang, L., Liu, X., & Feng, H. (2023). Development and validation of an instrument for assessing scientific literacy from junior to senior high school. Disciplinary and Interdisciplinary Science Education Research, 5(1). https://doi.org/10.1186/s43031-023-00093-2

DOI: https://doi.org/10.35445/alishlah.v16i3.5621