Designing an Online Collaborative Exam: Infusing Opportunities for Regulation of Collaboration

Mariel Miller; Safoura Askari

doi:10.18357/otessaj.2024.4.1.64

Authors

Mariel Miller University of Victoria
Safoura Askari University of Victoria

DOI:

https://doi.org/10.18357/otessaj.2024.4.1.64

Keywords:

collaborative exams, online assessment, computer-supported collaborative learning, online collaboration, self-regulated learning, socially shared regulation of learning, collaborative testing

Abstract

The ability to collaborate online is an essential skill that allows learners to bring together diverse perspectives to deepen understanding regardless of physical location. As online learning and hybrid workplaces have become more prevalent since the COVID-19 global pandemic, the critical nature of this skill has become particularly relevant for post-secondary graduates. While exams are traditionally viewed as solitary endeavors, collaborative exams offer invaluable opportunities for learners to develop these skills. However, this form of collaborative assessment can be challenging for both learners and educators, and few studies offer guidance for the effective design of collaborative exams. As such, in this paper, we report on the design and implementation of a synchronous collaborative midterm exam in a large first-year undergraduate course. Specifically, we describe how we drew on a theoretical framework of self- and socially shared regulation of learning to design a three-phase exam fostering learners’ engagement in key processes of planning, strategic enactment, and reflection on collaboration processes and products. Finally, we discuss key considerations that arose during the design and implementation of the exam, including ensuring an emphasis on process and authenticity, ethical use of video, and equity of access.

References

Bakhtiar, A., Webster, E. A., & Hadwin, A. F. (2018). Regulation and socio-emotional interactions in a positive and a negative group climate. Metacognition and Learning, 13, 57–90. https://doi.org/10.1007/s11409-017-9178-x DOI: https://doi.org/10.1007/s11409-017-9178-x

Bannert, M., Reimann, P., & Sonnenberg, C. (2014). Process mining techniques for analysing patterns and strategies in students’ self-regulated learning. Metacognition and Learning, 9, 161–185. https://doi.org/10.1007/s11409-013-9107-6 DOI: https://doi.org/10.1007/s11409-013-9107-6

Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences, 12(3), 307–359. https://doi.org/10.1207/s15327809jls1203_1 DOI: https://doi.org/10.1207/S15327809JLS1203_1

Biggs, J., & Tang, C. (2011). Teaching for quality learning at university: What the student does. Open University Press.

Bloom, D. (2009). Collaborative test taking: Benefits for learning and retention. College Teaching, 57(4), 216–220. https://doi.org/10.1080/87567550903218646 DOI: https://doi.org/10.3200/CTCH.57.4.216-220

Bruno, B., Engels, J., Ito, G., Gillis-Davis, J., Dulai, H., Carter, G., Fletcher, C. R., & Böttjer-Wilson, D. (2017). Two-stage exams: a powerful tool for reducing the achievement gap in undergraduate oceanography and geology classes. Oceanography, 30(2), 198–208. https://doi.org/10.5670/oceanog.2017.241 DOI: https://doi.org/10.5670/oceanog.2017.241

Cao, Y., & Porter, L. (2017, March). Evaluating student learning from collaborative group tests in introductory computing. In Proceedings of the ACM SIGCSE Technical Symposium on Computer Science Education (pp. 99–104). https://doi.org/10.1145/3017680.3017729 DOI: https://doi.org/10.1145/3017680.3017729

Chen, J., Wang, M., Kirschner, P. A., & Tsai, C. C. (2018). The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis. Review of Educational Research, 88(6), 799–843. https://doi.org/10.3102/0034654318791584 DOI: https://doi.org/10.3102/0034654318791584

Cooper, B., & Cowie, B. (2010). Collaborative research for assessment for learning. Teaching and Teacher Education, 26(4), 979–986. https://doi.org/10.1016/j.tate.2009.10.040 DOI: https://doi.org/10.1016/j.tate.2009.10.040

Cooper, S. (2017). A collaborative assessment of students’ placement learning. Assessment & Evaluation in Higher Education, 42(1), 61–76. https://doi.org/10.1080/02602938.2015.1083093 DOI: https://doi.org/10.1080/02602938.2015.1083093

Cortázar, C., Nussbaum, M., Alario-Hoyos, C., Goñi, J., & Álvares, D. (2022). The impacts of scaffolding socially shared regulation on teamwork in an online project-based course. The Internet and Higher Education, 55, 100877. https://doi.org/10.1016/j.iheduc.2022.100877 DOI: https://doi.org/10.1016/j.iheduc.2022.100877

Cortright, R.N., Collins, H.L., Rodenbaugh, D.W. & Dicarlo, S.E. (2003). Student Retention of Course Content Is Improved by Collaborative-Group Testing. Advances in Physiology Education, 27,102-8. https://doi.org/10.1152/advan.00041.2002 DOI: https://doi.org/10.1152/advan.00041.2002

De Backer, L., Van Keer, H., & Valcke, M. (2022). The functions of shared metacognitive regulation and their differential relation with collaborative learners’ understanding of the learning content. Learning and Instruction, 77, 101527. https://doi.org/10.1016/j.learninstruc.2021.101527 DOI: https://doi.org/10.1016/j.learninstruc.2021.101527

Dignath, C., & Büttner, G. (2008). Components of fostering self-regulated learning among students. A meta-analysis on intervention studies at primary and secondary school level. Metacognition and Learning, 3, 231–264. https://doi.org/10.1007/s11409-008-9029-x DOI: https://doi.org/10.1007/s11409-008-9029-x

Dillenbourg, P. (1999). What do you mean by collaborative learning? Collaborative-Learning: Cognitive and Computational Approaches, 1–19. https://telearn.hal.science/hal-00190240/document

Dhawan, S. (2020). Online learning: A panacea in the time of COVID-19 crisis. Journal of Educational Technology Systems, 49(1), 5–22. https://doi.org/10.1177/0047239520934018 DOI: https://doi.org/10.1177/0047239520934018

Dobie, T. E., & MacArthur, K. (2022). Exploring shifts in student attitudes toward group exams in college calculus: The case of Dane. International Journal of Research in Undergraduate Mathematics Education, 8(1), 149–175. https://doi.org/10.1007/s40753-021-00148-7 DOI: https://doi.org/10.1007/s40753-021-00148-7

Dohms, M. C., Collares, C. F., & Tibério, I. C. (2020). Video-based feedback using real consultations for a formative assessment in communication skills. BMC Medical Education, 20, Article 57. https://doi.org/10.1186/s12909-020-1955-6 DOI: https://doi.org/10.1186/s12909-020-1955-6

Eaton, T. T. (2009). Engaging students and evaluating learning progress using collaborative exams in introductory courses. Journal of Geoscience Education, 57(2), 113–120. https://doi.org/10.5408/1.3544241 DOI: https://doi.org/10.5408/1.3544241

Eastridge, J. A., & Benson, W. L. (2019). Comparing two models of collaborative testing for teaching statistics. Teaching of Psychology, 47(1), 68–73. https://doi.org/10.1177/0098628319888113 DOI: https://doi.org/10.1177/0098628319888113

Eastwood, J. L., Kleinberg, K. A., & Rodenbaugh, D. W. (2020). Collaborative testing in medical education: student perceptions and long-term knowledge retention. Medical Science Educator, 30(2), 737–747. https://doi.org/10.1007/s40670-020-00944-x DOI: https://doi.org/10.1007/s40670-020-00944-x

Edwards, M., & Clinton, M. (2018). A study exploring the impact of lecture capture availability and lecture capture usage on student attendance and attainment. Higher Education, 77, 403–421. https://doi.org/10.1007/S10734-018-0275-9. DOI: https://doi.org/10.1007/s10734-018-0275-9

Fjermestad, J. (2004). An analysis of communication mode in group support systems research. Decision Support Systems, 37, 239–263. https://doi.org/10.1016/S0167-9236(03)00021-6 DOI: https://doi.org/10.1016/S0167-9236(03)00021-6

Garaschuk, K. and Cytrynbaum, E. N. (2019). Feasibility and effectiveness of group exams in mathematics courses. Primus, 29(10), 1061–1079. https://doi.org/10.1080/10511970.2018.1472684 DOI: https://doi.org/10.1080/10511970.2018.1472684

Gilley, B. H., & Clarkston, B. (2014). Collaborative testing: Evidence of learning in a controlled in-class study of undergraduate students. Journal of College Science Teaching, 43(3), 83–91. https://doi.org/10.2505/4/jcst14_043_03_83 DOI: https://doi.org/10.2505/4/jcst14_043_03_83

Goodman, J., Melkers, J., & Pallais, A. (2019). Can online delivery increase access to education? Journal of Labor Economics, 37(1), 1–34. https://doi.org/10.1086/698895 DOI: https://doi.org/10.1086/698895

Hadwin, A. F., Bakhtiar, A., & Miller, M. (2018). Challenges in online collaboration: Effects of scripting shared task perceptions. International Journal of Computer-Supported Collaborative Learning, 13, 301–329. https://doi.org/10.1007/s11412-018-9279-9 DOI: https://doi.org/10.1007/s11412-018-9279-9

Hadwin, A. F., Järvelä, S., & Miller, M. (2011). Self-regulated, co-regulated, and socially shared regulation of learning. Handbook of Self-Regulation of Learning and Performance, 30, 65–84. https://doi.org/10.4324/9780203839010.ch5 DOI: https://doi.org/10.4324/9780203839010.ch5

Hadwin, A., Järvelä, S., & Miller, M. (2017). Self-regulation, co-regulation, and shared regulation in collaborative learning environments. In D. H. Schunk & DOI: https://doi.org/10.4324/9781315697048-6

J. A. Greene (Eds.), Handbook of Self-Regulation of Learning and Performance (2nd ed., pp. 83–106). Routledge/Taylor & Francis Group. https://doi-org.ezproxy.library.uvic.ca/10.4324/9781315697048-6

Hall, D., & Buzwell, S (2012). The problem of free-riding in group projects: Looking beyond social loafing as reason for non-contribution. Active Learning in Higher Education 14(1), 37–49. https://doi.org/10.1177/1469787412467123 DOI: https://doi.org/10.1177/1469787412467123

Han, F., & Ellis, R. A. (2021). Patterns of student collaborative learning in blended course designs based on their learning orientations: a student approaches to learning perspective. International Journal of Educational Technology in Higher Education, 18(1). https://doi.org/10.1186/s41239-021-00303-9 DOI: https://doi.org/10.1186/s41239-021-00303-9

Helmericks, S. G. (1993). Collaborative testing in social statistics: Toward gemeinstat. Teaching Sociology, 21(3), 287–297. https://doi.org/10.2307/1319027 DOI: https://doi.org/10.2307/1319027

Homer-Dixon, T., Renn, O., Rockstrom, J., Donges, J. F., & Janzwood, S. (2021). A call for an international research program on the risk of a global polycrisis. Cascadia Institute. Technical Paper #2022-3. https://cascadeinstitute.org/technical-paper/a-call-for-an-international-research-program-on-the-risk-of-a-global-polycrisis/ DOI: https://doi.org/10.2139/ssrn.4058592

Isohätälä, J., Järvenoja, H., & Järvelä, S. (2017). Socially shared regulation of learning and participation in social interaction in collaborative learning. International Journal of Educational Research, 81, 11–24. https://doi.org/10.1016/J.IJER.2016.10.006. DOI: https://doi.org/10.1016/j.ijer.2016.10.006

Janssen, J., Erkens, G., Kirschner, P., & Kanselaar, G. (2009). Influence of group member familiarity on online collaborative learning. Computers in Human Behavior, 25, 161–170. https://doi.org/10.1016/j.chb.2008.08.010 DOI: https://doi.org/10.1016/j.chb.2008.08.010

Järvelä, S., Hadwin, A., Malmberg, J., & Miller, M. (2018). Contemporary perspectives of regulated learning in collaboration. In International Handbook of the Learning Sciences (pp. 127–136). Routledge. https://doi.org/10.4324/9781315617572-13 DOI: https://doi.org/10.4324/9781315617572-13

Järvelä, S., Kirschner, P. A., Hadwin, A., Järvenoja, H., Malmberg, J., Miller, M., & Laru, J. (2016). Socially shared regulation of learning in CSCL: Understanding and prompting individual-and group-level shared regulatory activities. International Journal of Computer-Supported Collaborative Learning, 11, 263–280. https://doi.org/10.1007/s11412-016-9238-2 DOI: https://doi.org/10.1007/s11412-016-9238-2

Johnson, D. W., Johnson, R. T., & Smith, K. (2007). The state of cooperative learning in postsecondary and professional settings. Educational psychology review, 19, 15–29. https://doi.org/10.1007/s10648-006-9038-8 DOI: https://doi.org/10.1007/s10648-006-9038-8

Kapitanoff, S. H. (2009). Collaborative testing: Cognitive and interpersonal processes related to enhanced test performance. Active Learning in Higher Education, 10(1), 56–70. https://doi.org/10.1177/1469787408100195 DOI: https://doi.org/10.1177/1469787408100195

Kinnear, G. (2021). Two-stage collaborative exams have little impact on subsequent exam performance in undergraduate mathematics. International Journal of Research in Undergraduate Mathematics Education, 7(1), 33–60. https://doi.org/10.1007/s40753-020-00121-w DOI: https://doi.org/10.1007/s40753-020-00121-w

Kofoed, M., Gebhart, L., Gilmore, D., & Moschitto, R. (2021). Zooming to class? Experimental evidence on college students’ online learning during COVID-19. IZA Discussion Paper No. 14356. https://doi.org/10.2139/ssrn.3846700 DOI: https://doi.org/10.2139/ssrn.3846700

Koivuniemi, M., Järvenoja, H., & Järvelä, S. (2018). Teacher education students' strategic activities in challenging collaborative learning situations. Learning, Culture and Social interaction, 19, 109–123. https://doi.org/10.1016/j.lcsi.2018.05.002 DOI: https://doi.org/10.1016/j.lcsi.2018.05.002

LaCaille, R. A., LaCaille, L. J., & Maslowski, A. K. (2019). Metacognition, course performance, and perceived competence for learning: An examination of quiz and exam wrappers. Scholarship of Teaching and Learning in Psychology, 5(3), 209–222. https://doi.org/10.1037/stl0000114 DOI: https://doi.org/10.1037/stl0000114

Lavric, A. (2024). Self-regulation of student learning and teamwork: The role of video feedback, self-reflection and lecturer feedback. INTED2024 Proceedings, pp. 133–137. https://doi.org/10.21125/inted.2024.0073 DOI: https://doi.org/10.21125/inted.2024.0073

Le, H. T. T., Janssen, J., & Wubbels, T. (2017). Collaborative learning practices: teacher and student perceived obstacles to effective student collaboration. Cambridge Journal of Education, 48(1), 103–122. https://doi.org/10.1080/0305764x.2016.1259389 DOI: https://doi.org/10.1080/0305764X.2016.1259389

Lee, A., O’Donnell, A. M., & Rogat, T. K. (2015). Exploration of the cognitive regulatory sub-processes employed by groups characterized by socially shared and other-regulation in a CSCL context. Computers in Human Behavior, 52, 617–627. https://doi.org/10.1016/j.chb.2014.11.072 DOI: https://doi.org/10.1016/j.chb.2014.11.072

Levine, R. E., Borges, N. J., Roman, B. J., Carchedi, L. R., Townsend, M. H., Cluver, J. S., Frank, J., Morey, O., Haidet, P., & Thompson, B. M. (2018). High-stakes collaborative testing: Why not? Teaching and Learning in Medicine, 30(2), 133–140. https://doi.org/10.1080/10401334.2017.1365719 DOI: https://doi.org/10.1080/10401334.2017.1365719

Lobczowski, N. G., Lyons, K., Greene, J. A., & McLaughlin, J. E. (2021). Socially shared metacognition in a project-based learning environment: A comparative case study. Learning, Culture and Social Interaction, 30, 100543. https://doi.org/10.1016/j.lcsi.2021.100543 DOI: https://doi.org/10.1016/j.lcsi.2021.100543

LoGiudice, A. B., Heisz, J. J., & Kim, J. A. (2023). Does collaborative testing in the classroom enhance delayed transfer of knowledge? Scholarship of Teaching and Learning in Psychology, 9(3), 312–325. https://doi.org/10.1037/stl0000267 DOI: https://doi.org/10.1037/stl0000267

Mahoney, J. W., & Harris-Reeves, B. (2019). The effects of collaborative testing on higher order thinking: Do the bright get brighter? Active Learning in Higher Education, 20(1), 25–37. https://doi.org/10.1177/1469787417723243 DOI: https://doi.org/10.1177/1469787417723243

Malmberg, J., Järvelä, S., & Järvenoja, H. (2017). Capturing temporal and sequential patterns of self-, co-, and socially shared regulation in the context of collaborative learning. Contemporary Educational Psychology, 49, 160–174. https://doi.org/10.1016/j.cedpsych.2017.01.009 DOI: https://doi.org/10.1016/j.cedpsych.2017.01.009

Michalsky, T., & Cohen, A. (2021). Prompting socially shared regulation of learning and creativity in solving STEM problems. Frontiers in Psychology, 12, 722535. https://doi.org/10.3389/fpsyg.2021.722535 DOI: https://doi.org/10.3389/fpsyg.2021.722535

Miller, M., & Hadwin, A. (2015). Scripting and awareness tools for regulating collaborative learning: Changing the landscape of support in CSCL. Computers in Human Behavior, 52, 573–588. https://doi.org/10.1016/j.chb.2015.01.050 DOI: https://doi.org/10.1016/j.chb.2015.01.050

Miller, M., & Hadwin, A. (in press). Comparing the effectiveness of CSCL scripts for shared task perceptions in socially shared regulation of collaborative learning. International Journal of Computer-Supported Collaborative Learning.

Moore L (2010). Students’ attitudes and perceptions about the use of cooperative exams in an introductory leadership class. Journal of Leadership Education, 9(2), 72–85. https://doi.org/10.12806/V9/I2/RF6 DOI: https://doi.org/10.12806/V9/I2/RF6

Mort, J. R., & Hansen, D. J. (2010). First-year pharmacy students' self-assessment of communication skills and the impact of video review. American Journal of Pharmaceutical Education, 74(5), Article 78. https://doi.org/10.5688/aj740578 DOI: https://doi.org/10.5688/aj740578

Nokes-Malach, T., Richey, J., & Gadgil, S. (2015). When Is It Better to Learn Together? Insights from Research on Collaborative Learning. Educational Psychology Review, 27, 645–656. https://doi.org/10.1007/S10648-015-9312-8. DOI: https://doi.org/10.1007/s10648-015-9312-8

Organization for Economic Co-Operation and Development. (2019). An OECD Learning Framework 2030.In G. Bast, E. G. Carayannis, & D. F. J. Campbell (Eds.), The Future of Education and Labor (pp. 23–35). Springer. https://doi.org/10.1007/978-3-030-26068-2_3 DOI: https://doi.org/10.1007/978-3-030-26068-2_3

Pandey, C., & Kapitanoff, S. (2011). The influence of anxiety and quality of interaction on collaborative test performance. Active Learning in Higher Education, 12(3), 163–174. https://doi.org/10.1177/1469787411415077 DOI: https://doi.org/10.1177/1469787411415077

Paterson, T., & Prideaux, M. (2020). Exploring collaboration in online group based assessment contexts: Undergraduate Business Program. Journal of University Teaching & Learning Practice, 17(3), Article 3. https://doi.org/10.53761/1.17.3.3 DOI: https://doi.org/10.53761/1.17.3.3

Rao, S. P., Collins, H. L. & Dicarlo, S. E. (2002). Collaborative testing enhances student learning. Advances in Physiology Education, 26, 37–41. https://doi.org/10.1152/advan.00032.2001 DOI: https://doi.org/10.1152/advan.00032.2001

Rennstich, J. K. (2019). Creative online collaboration: A special challenge for co-creation. Education and Information Technologies, 24, 1835–1836. https://doi.org/10.1007/s10639-019-09875-6 DOI: https://doi.org/10.1007/s10639-019-09875-6

Roediger III, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255. https://doi.org/10.1111/j.1467-9280.2006.01693.x DOI: https://doi.org/10.1111/j.1467-9280.2006.01693.x

Roschelle, J., & Teasley, S. D. (1995). The construction of shared knowledge in collaborative problem solving. In C. O’Malley (Ed.), Computer supported collaborative learning (pp. 69–97). Springer. https://doi.org/10.1007/978-3-642-85098-1_5 DOI: https://doi.org/10.1007/978-3-642-85098-1_5

Rogat, T. K., & Linnenbrink-Garcia, L. (2011). Socially shared regulation in collaborative groups: An analysis of the interplay between quality of social regulation and group processes. Cognition and Instruction, 29(4), 375–415. https://doi.org/10.1080/07370008.2011.607930 DOI: https://doi.org/10.1080/07370008.2011.607930

Sandahl, S. S. (2010). Collaborative testing as a learning strategy in nursing education. Nursing Education Perspectives, 31(3), 142–147.

Straus, S. G. (1997). Technology, group process, and group outcomes: Testing the connections in computer-mediated and face-to-face groups. Human–Computer Interaction, 12(3), 227–266. https://doi.org/10.1207/s15327051hci1203_1 DOI: https://doi.org/10.1207/s15327051hci1203_1

Strijbos, J. W., Kirschner, P. A., & Martens, R. L. (Eds.). (2004). What we know about CSCL: And implementing it in higher education. Kluwer Academic. DOI: https://doi.org/10.1007/1-4020-7921-4

Vuorenmaa, E., Järvelä, S., Dindar, M., & Järvenoja, H. (2022). Sequential patterns in social interaction states for regulation in collaborative learning. Small Group Research, 54(4), 512–550. https://doi.org/10.1177/10464964221137524 DOI: https://doi.org/10.1177/10464964221137524

Webb, N. M., & Palincsar, A. S. (1996). Group processes in the classroom. Prentice Hall International.

Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D.J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277–304). Lawrence Erlbaum Associates Publishers.

Xu, D., & Xu, Y. (2019). The promises and limits of online higher education: Understanding how distance education affects access, cost, and quality. American Enterprise Institute. https://www.aei.org/research-products/report/the-promises-and-limits-of-online-higher-education/

Zimbardo, P. G., Butler, L. D., & Wolfe, V. A. (2003). Cooperative college examinations: More gain, less pain when students share information and grades. The Journal of Experimental Education, 71(2), 101–125. https://doi.org/10.1080/00220970309602059 DOI: https://doi.org/10.1080/00220970309602059

Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81(3), 329–339. https://doi.org/10.1037/0022-0663.81.3.329 DOI: https://doi.org/10.1037//0022-0663.81.3.329