Design and Validation of a Competency Assessment for Electrical Power Systems in Engineering Education

Adi Sutopo; Arif Rahman; Dadang Mulyana

doi:10.35445/alishlah.v16i4.5861

Design and Validation of a Competency Assessment for Electrical Power Systems in Engineering Education

Adi Sutopo, Arif Rahman, Dadang Mulyana

Abstract

This research aims to design, develop, and validate a competency test for assessing students in the Electric Power System field. Given the need for rigorous, competency-based assessments in Electrical Engineering, this study addresses the gap by creating a test aligned with specific graduate competencies. The competency test was developed using the Borg and Gall development model, following seven systematic steps. Test items were structured around the competencies expected of graduates from the Electrical Engineering program, categorized according to Bloom’s taxonomy levels 4, 5, and 6 to assess higher-order cognitive skills. A two-phase trial was conducted with 80 students to refine and validate the test. Validation by material and evaluation experts rated the test highly feasible, with scores of 71.0 and 45.55, respectively. Empirical validity testing, using biserial correlation, confirmed that all items were valid (correlation > 0.302), and the reliability test via KR-20 yielded a high reliability coefficient (α = 0.854). Item difficulty was balanced, with 10% easy, 83% moderate, and 0.7% difficult questions. Item discrimination levels were 20% moderate, 33% good, and 47% very good. The results suggest that the competency test is a reliable and valid tool for assessing critical competencies in Electric Power Systems, effectively distinguishing between different levels of student ability. This validated competency test provides a robust assessment tool, supporting competency-based education in Electrical Engineering. Its design and validation process can serve as a model for future competency assessments in technical disciplines.

Keywords

electrical power; system competency test; empirical validity

References

Akhan, O., & Öztürk, A. (2021). A study on the testing and evaluation questions in the T.R. history of revolution and Ataturkism textbooks in the 8th grade according to the revised Bloom taxonomy. Kesit Akademi, 28(28). https://doi.org/10.29228/kesit.52245

Andrei, H., Andrei, P. C., Constantinescu, L. M., Beloiu, R., Cazacu, E., & Stanculescu, M. (2017). Electrical power systems. In Power Systems. https://doi.org/10.1007/978-3-319-51118-4_1

Betin, F., Capolino, G. A., Casadei, D., Kawkabani, B., Bojoi, R. I., Harnefors, L., Levi, E., Parsa, L., & Fahimi, B. (2014). Trends in electrical machines control: Samples for classical, sensorless, and fault-tolerant techniques. IEEE Industrial Electronics Magazine, 8(2). https://doi.org/10.1109/MIE.2014.2313752

Chang, J. C., & Shih, H. F. (2020). What is the core competency of EECS students in technological higher education? International Journal of Electrical Engineering Education. https://doi.org/10.1177/0020720920981534

Eckes, T. (2012). Examinee-centered standard setting for large-scale assessments: The prototype group method. Psychological Test and Assessment Modeling, 54(3).

Gall, M. D., Borg, W. R., & Gall, J. P. (1996). Educational research: An introduction, 6th ed. In Educational research: An introduction, 6th ed. Longman Publishing.

Guedes, C. R., & Encarnacao, L. F. (2018). Hoist crane electrical optimization using a field weakening vector control frequency drive. SBSE 2018 - 7th Brazilian Electrical Systems Symposium. https://doi.org/10.1109/SBSE.2018.8395693

Haryudo, S. I., Ekohariadi, Munoto, Nurlaela, L., Asto, B. I. G. P., & Susila, I. W. (2019). Measure critical thinking ability: Validity and reliability multiple choice test. International Journal of Innovation, Creativity and Change, 8(1).

Kadam, A. A., Verma, A., & Khare, A. S. (2021). Are difficulty index and discrimination index useful tools for assessing the quality of an MCQ? A cross-sectional study. European Journal of Molecular and Clinical Medicine, 8, 1672+. https://link.gale.com/apps/doc/A698308270/AONE?u=anon~439a8991&sid=googleScholar&xid=0b9c66f6

Khulda, R. M., & Wandebori, H. (2020). Proposed Strategy for Electricity Industry of PT PLN (Persero) Electricity Maintenance Center (PLN Pusharlis). European Journal of Business and Management Research, 5(4). https://doi.org/10.24018/ejbmr.2020.5.4.419

Lapa, H. E., Kökce, A., Aldemir, D. A., Özdemir, A. F., & Altındal, Ş. (2020). Effect of illumination on electrical parameters of Au/(P3DMTFT)/n-GaAs Schottky barrier diodes. Indian Journal of Physics, 94(12). https://doi.org/10.1007/s12648-019-01644-y

Meda, L., & Swart, A. J. (2018). Analysing learning outcomes in an Electrical Engineering curriculum using illustrative verbs derived from Bloom’s Taxonomy. European Journal of Engineering Education, 43(3). https://doi.org/10.1080/03043797.2017.1378169

Sutopo, A., Sriadhi, S., Rahman, A., & Mulyana, D. (2024, February 9). The Urgency of Electrical Power System Competency Test in Electrical Engineering Bachelors. https://doi.org/10.4108/eai.24-10-2023.2342349

Syaichu, A., & Akademik, R. K. (2018). Panduan kurikulum program studi sarjana teknik elektro dan sejenis (Vol. 11).

Weiss, L. G. (n.d.). Standardized Assessment for Clinical Practitioners: A Primer.

DOI: https://doi.org/10.35445/alishlah.v16i4.5861