Developing the Technology-Integrated Assessment Framework
DOI:
https://doi.org/10.18357/otessaj.2024.4.1.63Keywords:
assessment, technology, digital learning, technology-integrated assessment, Indigenous education, framework, higher education, educational technologyAbstract
The purpose of this paper is to describe the development of a new framework for understanding technology-integrated assessment in higher education based on a review of the literature using the assessment design in a digital world framework (Bearman et al., 2022) as a lens. Our review (Madland et al., 2024) revealed both congruities and incongruities between the literature and the framework, leading to the need for further work to accurately conceptualize technology-integrated assessment. In this article, we contribute to the literature on technology-integrated assessment in higher education by proposing the technology-integrated assessment framework. This paper marks an important step in extending our understanding of the factors influencing instructors who integrate technology into their assessment practice and promoting ethical and equitable approaches to technology-integrated assessment in higher education.
References
Abrahart, R. (2023, June 12). Rejection of legal duty of care marks a bad day for students’ rights. Wonkhe. https://wonkhe.com/blogs/rejection-of-legal-duty-of-care-marks-a-bad-day-for-students-rights/
Alexander v University of Lethbridge. (2022). https://canlii.ca/t/jpxvz
Almaiah, M. A., Alamri, M. M., & Al-Rahmi, W. (2019). Applying the UTAUT model to explain the students’ acceptance of mobile learning system in higher education. IEEE Access, 7, 174673–174686. https://doi.org/10.1109/ACCESS.2019.2957206 DOI: https://doi.org/10.1109/ACCESS.2019.2957206
Aluko, F. R., & Omidire, M. F. (2020). A critical review of student assessment practices in distance education in an emerging economy: Benchmarking practices against policy. Africa Education Review, 17(5), 76–94. https://doi.org/10.1080/18146627.2021.1920842 DOI: https://doi.org/10.1080/18146627.2021.1920842
Baniasadi, A., Salehi, K., Khodaie, E., Bagheri Noaparast, K., & Izanloo, B. (2023). Fairness in Classroom Assessment: A Systematic Review. The Asia-Pacific Education Researcher, 32(1), 91–109. https://doi.org/10.1007/s40299-021-00636-z DOI: https://doi.org/10.1007/s40299-021-00636-z
BC Ministry of Advanced Education and Skills Training. (2022). Digital Learning Advisory Committee Draft Recommendations. https://www2.gov.bc.ca/assets/gov/education/post-secondary-education/institution-resources-administration/digital-learning-strategy/digital_learning_strategy.pdf
BC Teachers’ Council. (2019). Professional Standards for BC Educators. https://www2.gov.bc.ca/assets/gov/education/kindergarten-to-grade-12/teach/teacher-regulation/standards-for-educators/edu_standards.pdf
Bearman, M., Nieminen, J., & Ajjawi, R. (2022). Designing assessment in a digital world: An organising framework. Assessment & Evaluation in Higher Education, 48(3). https://doi.org/10.1080/02602938.2022.2069674 DOI: https://doi.org/10.1080/02602938.2022.2069674
Bennett, S., Dawson, P., Bearman, M., Molloy, E., & Boud, D. (2017). How technology shapes assessment design: Findings from a study of university teachers. British Journal of Educational Technology, 48(2), 672–682. https://doi.org/10.1111/bjet.12439 DOI: https://doi.org/10.1111/bjet.12439
Biasutti, M. (2017). A coding scheme to analyse the online asynchronous discussion forums of university students. Technology Pedagogy And Education, 26(5), 601–615. https://doi.org/10.1080/1475939X.2017.1365753 DOI: https://doi.org/10.1080/1475939X.2017.1365753
Birch, A., & Irvine, V. (2009). Preservice teachers’ acceptance of ICT integration in the classroom: applying the UTAUT model. Educational Media International, 46(4), 295–315. https://doi.org/10.1080/09523980903387506 DOI: https://doi.org/10.1080/09523980903387506
Boud, D., & Soler, R. (2016). Sustainable assessment revisited. Assessment & Evaluation in Higher Education, 41(3), 400–413. https://doi.org/10.1080/02602938.2015.1018133 DOI: https://doi.org/10.1080/02602938.2015.1018133
Celik, I., Gedrimiene, E., Silvola, A., & Muukkonen, H. (2022). Response of learning analytics to the online education challenges during pandemic: Opportunities and key examples in higher education. Policy Futures In Education. https://doi.org/10.1177/14782103221078401 DOI: https://doi.org/10.1177/14782103221078401
Cleveland-Innes, M., & Emes, C. (2005). Social and academic interaction in higher education contexts and the effect on deep learning. NASPA Journal, 42, 241–262. https://doi.org/10.2202/1949-6605.1475 DOI: https://doi.org/10.2202/1949-6605.1475
Combrinck, M., & Vollenhoven, W. van. (2020). Computer-assisted assessment: An old remedy for challenges in open distance learning. Independent Journal of Teaching and Learning, 15(1), 22–34. http://ref.scielo.org/bh77hf
Dawson, P. (2020). Defending Assessment Security in a Digital World: Preventing E-Cheating and Supporting Academic Integrity in Higher Education (1st ed.). Routledge. https://doi.org/10.4324/9780429324178 DOI: https://doi.org/10.4324/9780429324178-1
DeLuca, C., LaPointe-McEwan, D., & Luhanga, U. (2016). Approaches to classroom assessment inventory: A new instrument to support teacher assessment literacy. Educational Assessment, 21, 248–266. https://doi.org/gfgtsg DOI: https://doi.org/10.1080/10627197.2016.1236677
Deneen, C. C., Brown, G. T. L., & Carless, D. (2018). Students’ conceptions of eportfolios as assessment and technology. Innovations in Education and Teaching International, 55(4), 487–496. https://doi.org/10.1080/14703297.2017.1281752 DOI: https://doi.org/10.1080/14703297.2017.1281752
Dewey, J. (1916). Democracy and education. Macmillan.
Dron, J. (2022). Educational technology: what it is and how it works. AI & Society, 37(1), 155–166. https://doi.org/10.1007/s00146-021-01195-z DOI: https://doi.org/10.1007/s00146-021-01195-z
Duncan, A., & Joyner, D. (2022). On the necessity (or lack thereof) of digital proctoring: Drawbacks, perceptions, and alternatives. Journal of Computer Assisted Learning, 38(5), 1482–1496. https://doi.org/10.1111/jcal.12700 DOI: https://doi.org/10.1111/jcal.12700
Earl, L. M. (2013). Assessment as learning: Using classroom assessment to maximize student learning (2nd edition). Corwin Press. https://us.sagepub.com/en-us/nam/book/assessment-learning-1
Fawns, T., & Nieminen, J. H. (2023). The only way is ethics: A dialogue of assessment and social good (L. Czerniewicz & C. Cronin, Eds.; 1st ed., pp. 533–554). Open Book Publishers. https://doi.org/10.11647/obp.0363.23 DOI: https://doi.org/10.11647/obp.0363.23
Gallavan, N., Huffman, S., & Shaw, E. (2017). Ensuring Ethics and Equity With Classroom Assessments and Mobile Technology: Advancing Online Education. In M. Mills & D. Wake (Eds.), Empowering Learners With Mobile Open-Access Learning Initiatives (pp. 220–241). https://doi.org/10.4018/978-1-5225-2122-8.ch011 DOI: https://doi.org/10.4018/978-1-5225-2122-8.ch011
Gamage, K. A. A., Pradeep, R. G. G. R., & Silva, E. K. de. (2022). Rethinking assessment: The future of examinations in higher education. Sustainability, 14(6), 3552. https://doi.org/10.3390/su14063552 DOI: https://doi.org/10.3390/su14063552
Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2, 87–105. https://doi.org/10.1016/S1096-7516(00)00016-6 DOI: https://doi.org/10.1016/S1096-7516(00)00016-6
Gilliard, C., & Selwyn, N. (2023). Automated surveillance in education. Postdigital Science and Education, 5(1), 195–205. https://doi.org/10.1007/s42438-022-00295-3 DOI: https://doi.org/10.1007/s42438-022-00295-3
Harris, L. D., & Wasilewski, J. (2004). Indigeneity, an alternative worldview: four R’s (relationship, responsibility, reciprocity, redistribution) vs. two P’s (power and profit). Sharing the journey towards conscious evolution. Systems Research and Behavioral Science, 21(5), 489–503. https://doi.org/10.1002/sres.631 DOI: https://doi.org/10.1002/sres.631
Hilliger, I., Ruipérez-Valiente, J. A., Alexandron, G., & Gašević, D. (2022). Trustworthy remote assessments: A typology of pedagogical and technological strategies. Journal of Computer Assisted Learning. https://doi.org/10.1111/jcal.12755 DOI: https://doi.org/10.1111/jcal.12755
hooks, bell. (1994). Teaching to transgress: education as the practice of freedom. Routledge. https://www.routledge.com/Teaching-to-Transgress-Education-as-the-Practice-of-Freedom/hooks/p/book/9780415908085
Horton, M., & Freire, P. (1990). We make the road by walking: Conversations on education and social change (B. Bell, J. Gaventa, & J. Peters, Eds.). Temple University Press. https://tupress.temple.edu/books/we-make-the-road-by-walking
Hussein, M. J., Yusuf, J., Deb, A. S., Fong, L., & Naidu, S. (2020). An evaluation of online proctoring tools. Open Praxis, 12(4), 509–525. https://doi.org/gmbv3m DOI: https://doi.org/10.5944/openpraxis.12.4.1113
Kirkness, V. J., & Barnhardt, R. (1991). First nations and higher education: The four r’s– respect, relevance, reciprocity, responsibility. Journal of American Indian Education, 30(3), 1–15. https://eric.ed.gov/?id=EJ438242
Lake, J., & Atkins, H. (2021). Facilitating online learning with the 5R’s. BCcampus. https://pressbooks.bccampus.ca/the5rsonline/
Lambert, S. R. (2018). Changing our (Dis)Course: A Distinctive Social Justice Aligned Definition of Open Education. Journal of Learning for Development - JL4D, 5(3). http://www.jl4d.org/index.php/ejl4d/article/view/290 DOI: https://doi.org/10.56059/jl4d.v5i3.290
Lave, J., & Wenger, E. (2003). Situated learning: Legitimate peripheral participation. Cambridge University Press. https://doi.org/10.1017/CBO9780511815355 DOI: https://doi.org/10.1017/CBO9780511815355
Madland, C., Irvine, V., DeLuca, C., & Bulut, O. (2024). Technology-integrated assessment: A literature review. The Open/Technology in Education, Society, and Scholarship Association Journal, 4(1), 1-42. https://doi.org/10.18357/otessaj.2024.4.1.57 DOI: https://doi.org/10.18357/otessaj.2024.4.1.57
Massey, K. D., DeLuca, C., & LaPointe-McEwan, D. (2020). Assessment Literacy in College Teaching: Empirical Evidence on the Role and Effectiveness of a Faculty Training Course. To Improve the Academy, 39(1). https://doi.org/gj5ngz DOI: https://doi.org/10.3998/tia.17063888.0039.109
Moorhouse, B., & Kohnke, L. (2022). Conducting formative assessment during synchronous online lessons: University teachers’ challenges and pedagogical strategies. Pedagogies: An International Journal, 18(3). https://doi.org/10.1080/1554480X.2022.2065993 DOI: https://doi.org/10.1080/1554480X.2022.2065993
Nadarajah, S. (2021). Student Suicide on-Campus: Tort Liability of Canadian Universities and Determining a Duty of Care. Appeal: Review of Current Law and Law Reform, 26, 97–120. https://heinonline.org/HOL/P?h=hein.journals/appeal26&i=110
Nieminen, J. H., & Carless, D. (2022). Feedback literacy: A critical review of an emerging concept. Higher Education, 85, 1381–1400. https://doi.org/10.1007/s10734-022-00895-9 DOI: https://doi.org/10.1007/s10734-022-00895-9
Nutbrown, S., Higgins, C., & Beesley, S. (2016). Measuring the impact of high quality instant feedback on learning. Practitioner Research in Higher Education, 10(1), 130–139. https://files.eric.ed.gov/fulltext/EJ1129863.pdf
OpenAI. (2023). https://openai.com/
Or, C., & Chapman, E. (2022). Development and validation of an instrument to measure online assessment acceptance in higher education. British Journal of Educational Technology, 53(4), 977–997. https://doi.org/10.1111/bjet.13180 DOI: https://doi.org/10.1111/bjet.13180
Pires Pereira, I. S., Fernandes, E. L., & Flores, M. A. (2021). Teacher education during the COVID-19 lockdown: Insights from a formative intervention approach involving online feedback. Education Sciences, 11(8), 400. https://doi.org/10.3390/educsci11080400 DOI: https://doi.org/10.3390/educsci11080400
Restoule, J.-P. (2008). The five R’s of Indigenous research: Relationship, respect, relevance, responsibility, and reciprocity. [Paper presentation]. Wise Practices II: Canadian Aboriginal AIDS Network Research and Capacity Building Conference, Toronto
Rodriguez-Triana, M. J., Prieto, L. P., Holzer, A., & Gillet, D. (2020). Instruction, student engagement, and learning outcomes: A case study using anonymous social media in a face-to-face classroom. IEEE Transactions on Learning Technologies, 13(4), 718–733. https://doi.org/10.1109/TLT.2020.2995557 DOI: https://doi.org/10.1109/TLT.2020.2995557
Sargent, J., & Lynch, S. (2021). ’None of my other teachers know my face/emotions/thoughts’: Digital technology and democratic assessment practices in higher education physical education. Technology, Pedagogy and Education, 30(5), 693–705. https://doi.org/10.1080/1475939X.2021.1942972 DOI: https://doi.org/10.1080/1475939X.2021.1942972
Stödberg, U. (2012). A research review of e-assessment. Assessment & Evaluation in Higher Education, 37(5), 591–604. https://doi.org/10.1080/02602938.2011.557496 DOI: https://doi.org/10.1080/02602938.2011.557496
Styres, S. D., & Zinga, D. M. (2013). The community-first land-centred theoretical framework: Bringing a" good mind" to indigenous education research? Canadian Journal of Education, 36(2). https://journals.sfu.ca/cje/index.php/cje-rce/article/view/1315
Tai, J., Ajjawi, R., Boud, D., & Jorre de St Jorre, T. (2022). Promoting equity and social justice through assessment for inclusion. In R. Ajjawi, J. Tai, D. Boud, & T. Jorre De St Jorre (Eds.), Assessment for Inclusion in Higher Education: Promoting Equity and Social Justice in Assessment (1st ed.). Routledge. https://doi.org/10.4324/9781003293101. DOI: https://doi.org/10.4324/9781003293101
Tessaro, D., Restoule, J.-P., Gaviria, P., Flessa, J., Lindeman, C., & Scully-Stewart, C. (2018). The five r’s for indigenizing online learning: A case study of the first nations schools’ principals course. Canadian Journal of Native Education, 40(1), 125–143. https://www.researchgate.net/publication/328289320_The_Five_R%27s_for_Indigenizing_Online_Learning_A_Case_Study_of_the_First_Nations_Schools%27_Principals_Course
The Truth and Reconciliation Commission of Canada. (2015). Truth and Reconciliation Commission of Canada: Calls to Action. Truth and Reconciliation Commission of Canada. https://ehprnh2mwo3.exactdn.com/wp-content/uploads/2021/01/Calls_to_Action_English2.pdf
Thompson, C. J. (2009). Educational statistics authentic learning CAPSULES: Community action projects for students utilizing leadership and e-based statistics. Journal of Statistics Education, 17(1). https://doi.org/10.1080/10691898.2009.11889508 DOI: https://doi.org/10.1080/10691898.2009.11889508
Timmis, S., Broadfoot, P., Sutherland, R., & Oldfield, A. (2016). Rethinking assessment in a digital age: opportunities, challenges and risks. British Educational Research Journal, 42(3), 454–476. https://doi.org/gftz95 DOI: https://doi.org/10.1002/berj.3215
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425–478. https://doi.org/gc8zn2 DOI: https://doi.org/10.2307/30036540
Vygotsky, L. S. (1978). Mind in society. Harvard University Press.
Woo, S. E., LeBreton, J. M., Keith, M. G., & Tay, L. (2023). Bias, Fairness, and Validity in Graduate-School Admissions: A Psychometric Perspective. Perspectives on Psychological Science, 18(1), 3–31. https://doi.org/10.1177/17456916211055374 DOI: https://doi.org/10.1177/17456916211055374
Yang, M., Tai, M., & Lim, C. P. (2016). The role of e-portfolios in supporting productive learning. British Journal of Educational Technology, 47(6), 1276–1286. https://doi.org/10.1111/bjet.12316 DOI: https://doi.org/10.1111/bjet.12316
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Colin Madland, Valerie Irvine, Chris DeLuca, Okan Bulut
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors contributing to the OTESSA Journal agree to release their articles under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the OTESSA Journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in the OTESSA Journal.
