Designing an Online Collaborative Exam:
Infusing Opportunities for Regulation of Collaboration
DOI:
https://doi.org/10.18357/otessaj.2024.4.1.64Keywords:
collaborative exams, online assessment, computer-supported collaborative learning, online collaboration, self-regulated learning, socially shared regulation of learning, collaborative testingAbstract
The ability to collaborate online is an essential skill that allows learners to bring together diverse perspectives to deepen understanding regardless of physical location. As online learning and hybrid workplaces have become more prevalent since the COVID-19 global pandemic, the critical nature of this skill has become particularly relevant for post-secondary graduates. While exams are traditionally viewed as solitary endeavors, collaborative exams offer invaluable opportunities for learners to develop these skills. However, this form of collaborative assessment can be challenging for both learners and educators, and few studies offer guidance for the effective design of collaborative exams. As such, in this paper, we report on the design and implementation of a synchronous collaborative midterm exam in a large first-year undergraduate course. Specifically, we describe how we drew on a theoretical framework of self- and socially shared regulation of learning to design a three-phase exam fostering learners’ engagement in key processes of planning, strategic enactment, and reflection on collaboration processes and products. Finally, we discuss key considerations that arose during the design and implementation of the exam, including ensuring an emphasis on process and authenticity, ethical use of video, and equity of access.
References
Bakhtiar, A., Webster, E. A., & Hadwin, A. F. (2018). Regulation and socio-emotional interactions in a positive and a negative group climate. Metacognition and Learning, 13, 57–90. https://doi.org/10.1007/s11409-017-9178-x DOI: https://doi.org/10.1007/s11409-017-9178-x
Bannert, M., Reimann, P., & Sonnenberg, C. (2014). Process mining techniques for analysing patterns and strategies in students’ self-regulated learning. Metacognition and Learning, 9, 161–185. https://doi.org/10.1007/s11409-013-9107-6 DOI: https://doi.org/10.1007/s11409-013-9107-6
Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences, 12(3), 307–359. https://doi.org/10.1207/s15327809jls1203_1 DOI: https://doi.org/10.1207/S15327809JLS1203_1
Biggs, J., & Tang, C. (2011). Teaching for quality learning at university: What the student does. Open University Press.
Bloom, D. (2009). Collaborative test taking: Benefits for learning and retention. College Teaching, 57(4), 216–220. https://doi.org/10.1080/87567550903218646 DOI: https://doi.org/10.3200/CTCH.57.4.216-220
Bruno, B., Engels, J., Ito, G., Gillis-Davis, J., Dulai, H., Carter, G., Fletcher, C. R., & Böttjer-Wilson, D. (2017). Two-stage exams: a powerful tool for reducing the achievement gap in undergraduate oceanography and geology classes. Oceanography, 30(2), 198–208. https://doi.org/10.5670/oceanog.2017.241 DOI: https://doi.org/10.5670/oceanog.2017.241
Cao, Y., & Porter, L. (2017, March). Evaluating student learning from collaborative group tests in introductory computing. In Proceedings of the ACM SIGCSE Technical Symposium on Computer Science Education (pp. 99–104). https://doi.org/10.1145/3017680.3017729 DOI: https://doi.org/10.1145/3017680.3017729
Chen, J., Wang, M., Kirschner, P. A., & Tsai, C. C. (2018). The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis. Review of Educational Research, 88(6), 799–843. https://doi.org/10.3102/0034654318791584 DOI: https://doi.org/10.3102/0034654318791584
Cooper, B., & Cowie, B. (2010). Collaborative research for assessment for learning. Teaching and Teacher Education, 26(4), 979–986. https://doi.org/10.1016/j.tate.2009.10.040 DOI: https://doi.org/10.1016/j.tate.2009.10.040
Cooper, S. (2017). A collaborative assessment of students’ placement learning. Assessment & Evaluation in Higher Education, 42(1), 61–76. https://doi.org/10.1080/02602938.2015.1083093 DOI: https://doi.org/10.1080/02602938.2015.1083093
Cortázar, C., Nussbaum, M., Alario-Hoyos, C., Goñi, J., & Álvares, D. (2022). The impacts of scaffolding socially shared regulation on teamwork in an online project-based course. The Internet and Higher Education, 55, 100877. https://doi.org/10.1016/j.iheduc.2022.100877 DOI: https://doi.org/10.1016/j.iheduc.2022.100877
Cortright, R.N., Collins, H.L., Rodenbaugh, D.W. & Dicarlo, S.E. (2003). Student Retention of Course Content Is Improved by Collaborative-Group Testing. Advances in Physiology Education, 27,102-8. https://doi.org/10.1152/advan.00041.2002 DOI: https://doi.org/10.1152/advan.00041.2002
De Backer, L., Van Keer, H., & Valcke, M. (2022). The functions of shared metacognitive regulation and their differential relation with collaborative learners’ understanding of the learning content. Learning and Instruction, 77, 101527. https://doi.org/10.1016/j.learninstruc.2021.101527 DOI: https://doi.org/10.1016/j.learninstruc.2021.101527
Dignath, C., & Büttner, G. (2008). Components of fostering self-regulated learning among students. A meta-analysis on intervention studies at primary and secondary school level. Metacognition and Learning, 3, 231–264. https://doi.org/10.1007/s11409-008-9029-x DOI: https://doi.org/10.1007/s11409-008-9029-x
Dillenbourg, P. (1999). What do you mean by collaborative learning? Collaborative-Learning: Cognitive and Computational Approaches, 1–19. https://telearn.hal.science/hal-00190240/document
Dhawan, S. (2020). Online learning: A panacea in the time of COVID-19 crisis. Journal of Educational Technology Systems, 49(1), 5–22. https://doi.org/10.1177/0047239520934018 DOI: https://doi.org/10.1177/0047239520934018
Dobie, T. E., & MacArthur, K. (2022). Exploring shifts in student attitudes toward group exams in college calculus: The case of Dane. International Journal of Research in Undergraduate Mathematics Education, 8(1), 149–175. https://doi.org/10.1007/s40753-021-00148-7 DOI: https://doi.org/10.1007/s40753-021-00148-7
Dohms, M. C., Collares, C. F., & Tibério, I. C. (2020). Video-based feedback using real consultations for a formative assessment in communication skills. BMC Medical Education, 20, Article 57. https://doi.org/10.1186/s12909-020-1955-6 DOI: https://doi.org/10.1186/s12909-020-1955-6
Eaton, T. T. (2009). Engaging students and evaluating learning progress using collaborative exams in introductory courses. Journal of Geoscience Education, 57(2), 113–120. https://doi.org/10.5408/1.3544241 DOI: https://doi.org/10.5408/1.3544241
Eastridge, J. A., & Benson, W. L. (2019). Comparing two models of collaborative testing for teaching statistics. Teaching of Psychology, 47(1), 68–73. https://doi.org/10.1177/0098628319888113 DOI: https://doi.org/10.1177/0098628319888113
Eastwood, J. L., Kleinberg, K. A., & Rodenbaugh, D. W. (2020). Collaborative testing in medical education: student perceptions and long-term knowledge retention. Medical Science Educator, 30(2), 737–747. https://doi.org/10.1007/s40670-020-00944-x DOI: https://doi.org/10.1007/s40670-020-00944-x
Edwards, M., & Clinton, M. (2018). A study exploring the impact of lecture capture availability and lecture capture usage on student attendance and attainment. Higher Education, 77, 403–421. https://doi.org/10.1007/S10734-018-0275-9. DOI: https://doi.org/10.1007/s10734-018-0275-9
Fjermestad, J. (2004). An analysis of communication mode in group support systems research. Decision Support Systems, 37, 239–263. https://doi.org/10.1016/S0167-9236(03)00021-6 DOI: https://doi.org/10.1016/S0167-9236(03)00021-6
Garaschuk, K. and Cytrynbaum, E. N. (2019). Feasibility and effectiveness of group exams in mathematics courses. Primus, 29(10), 1061–1079. https://doi.org/10.1080/10511970.2018.1472684 DOI: https://doi.org/10.1080/10511970.2018.1472684
Gilley, B. H., & Clarkston, B. (2014). Collaborative testing: Evidence of learning in a controlled in-class study of undergraduate students. Journal of College Science Teaching, 43(3), 83–91. https://doi.org/10.2505/4/jcst14_043_03_83 DOI: https://doi.org/10.2505/4/jcst14_043_03_83
Goodman, J., Melkers, J., & Pallais, A. (2019). Can online delivery increase access to education? Journal of Labor Economics, 37(1), 1–34. https://doi.org/10.1086/698895 DOI: https://doi.org/10.1086/698895
Hadwin, A. F., Bakhtiar, A., & Miller, M. (2018). Challenges in online collaboration: Effects of scripting shared task perceptions. International Journal of Computer-Supported Collaborative Learning, 13, 301–329. https://doi.org/10.1007/s11412-018-9279-9 DOI: https://doi.org/10.1007/s11412-018-9279-9
Hadwin, A. F., Järvelä, S., & Miller, M. (2011). Self-regulated, co-regulated, and socially shared regulation of learning. Handbook of Self-Regulation of Learning and Performance, 30, 65–84. https://doi.org/10.4324/9780203839010.ch5 DOI: https://doi.org/10.4324/9780203839010.ch5
Hadwin, A., Järvelä, S., & Miller, M. (2017). Self-regulation, co-regulation, and shared regulation in collaborative learning environments. In D. H. Schunk & DOI: https://doi.org/10.4324/9781315697048-6
J. A. Greene (Eds.), Handbook of Self-Regulation of Learning and Performance (2nd ed., pp. 83–106). Routledge/Taylor & Francis Group. https://doi-org.ezproxy.library.uvic.ca/10.4324/9781315697048-6
Hall, D., & Buzwell, S (2012). The problem of free-riding in group projects: Looking beyond social loafing as reason for non-contribution. Active Learning in Higher Education 14(1), 37–49. https://doi.org/10.1177/1469787412467123 DOI: https://doi.org/10.1177/1469787412467123
Han, F., & Ellis, R. A. (2021). Patterns of student collaborative learning in blended course designs based on their learning orientations: a student approaches to learning perspective. International Journal of Educational Technology in Higher Education, 18(1). https://doi.org/10.1186/s41239-021-00303-9 DOI: https://doi.org/10.1186/s41239-021-00303-9
Helmericks, S. G. (1993). Collaborative testing in social statistics: Toward gemeinstat. Teaching Sociology, 21(3), 287–297. https://doi.org/10.2307/1319027 DOI: https://doi.org/10.2307/1319027
Homer-Dixon, T., Renn, O., Rockstrom, J., Donges, J. F., & Janzwood, S. (2021). A call for an international research program on the risk of a global polycrisis. Cascadia Institute. Technical Paper #2022-3. https://cascadeinstitute.org/technical-paper/a-call-for-an-international-research-program-on-the-risk-of-a-global-polycrisis/ DOI: https://doi.org/10.2139/ssrn.4058592
Isohätälä, J., Järvenoja, H., & Järvelä, S. (2017). Socially shared regulation of learning and participation in social interaction in collaborative learning. International Journal of Educational Research, 81, 11–24. https://doi.org/10.1016/J.IJER.2016.10.006. DOI: https://doi.org/10.1016/j.ijer.2016.10.006
Janssen, J., Erkens, G., Kirschner, P., & Kanselaar, G. (2009). Influence of group member familiarity on online collaborative learning. Computers in Human Behavior, 25, 161–170. https://doi.org/10.1016/j.chb.2008.08.010 DOI: https://doi.org/10.1016/j.chb.2008.08.010
Järvelä, S., Hadwin, A., Malmberg, J., & Miller, M. (2018). Contemporary perspectives of regulated learning in collaboration. In International Handbook of the Learning Sciences (pp. 127–136). Routledge. https://doi.org/10.4324/9781315617572-13 DOI: https://doi.org/10.4324/9781315617572-13
Järvelä, S., Kirschner, P. A., Hadwin, A., Järvenoja, H., Malmberg, J., Miller, M., & Laru, J. (2016). Socially shared regulation of learning in CSCL: Understanding and prompting individual-and group-level shared regulatory activities. International Journal of Computer-Supported Collaborative Learning, 11, 263–280. https://doi.org/10.1007/s11412-016-9238-2 DOI: https://doi.org/10.1007/s11412-016-9238-2
Johnson, D. W., Johnson, R. T., & Smith, K. (2007). The state of cooperative learning in postsecondary and professional settings. Educational psychology review, 19, 15–29. https://doi.org/10.1007/s10648-006-9038-8 DOI: https://doi.org/10.1007/s10648-006-9038-8
Kapitanoff, S. H. (2009). Collaborative testing: Cognitive and interpersonal processes related to enhanced test performance. Active Learning in Higher Education, 10(1), 56–70. https://doi.org/10.1177/1469787408100195 DOI: https://doi.org/10.1177/1469787408100195
Kinnear, G. (2021). Two-stage collaborative exams have little impact on subsequent exam performance in undergraduate mathematics. International Journal of Research in Undergraduate Mathematics Education, 7(1), 33–60. https://doi.org/10.1007/s40753-020-00121-w DOI: https://doi.org/10.1007/s40753-020-00121-w
Kofoed, M., Gebhart, L., Gilmore, D., & Moschitto, R. (2021). Zooming to class? Experimental evidence on college students’ online learning during COVID-19. IZA Discussion Paper No. 14356. https://doi.org/10.2139/ssrn.3846700 DOI: https://doi.org/10.2139/ssrn.3846700
Koivuniemi, M., Järvenoja, H., & Järvelä, S. (2018). Teacher education students' strategic activities in challenging collaborative learning situations. Learning, Culture and Social interaction, 19, 109–123. https://doi.org/10.1016/j.lcsi.2018.05.002 DOI: https://doi.org/10.1016/j.lcsi.2018.05.002
LaCaille, R. A., LaCaille, L. J., & Maslowski, A. K. (2019). Metacognition, course performance, and perceived competence for learning: An examination of quiz and exam wrappers. Scholarship of Teaching and Learning in Psychology, 5(3), 209–222. https://doi.org/10.1037/stl0000114 DOI: https://doi.org/10.1037/stl0000114
Lavric, A. (2024). Self-regulation of student learning and teamwork: The role of video feedback, self-reflection and lecturer feedback. INTED2024 Proceedings, pp. 133–137. https://doi.org/10.21125/inted.2024.0073 DOI: https://doi.org/10.21125/inted.2024.0073
Le, H. T. T., Janssen, J., & Wubbels, T. (2017). Collaborative learning practices: teacher and student perceived obstacles to effective student collaboration. Cambridge Journal of Education, 48(1), 103–122. https://doi.org/10.1080/0305764x.2016.1259389 DOI: https://doi.org/10.1080/0305764X.2016.1259389
Lee, A., O’Donnell, A. M., & Rogat, T. K. (2015). Exploration of the cognitive regulatory sub-processes employed by groups characterized by socially shared and other-regulation in a CSCL context. Computers in Human Behavior, 52, 617–627. https://doi.org/10.1016/j.chb.2014.11.072 DOI: https://doi.org/10.1016/j.chb.2014.11.072
Levine, R. E., Borges, N. J., Roman, B. J., Carchedi, L. R., Townsend, M. H., Cluver, J. S., Frank, J., Morey, O., Haidet, P., & Thompson, B. M. (2018). High-stakes collaborative testing: Why not? Teaching and Learning in Medicine, 30(2), 133–140. https://doi.org/10.1080/10401334.2017.1365719 DOI: https://doi.org/10.1080/10401334.2017.1365719
Lobczowski, N. G., Lyons, K., Greene, J. A., & McLaughlin, J. E. (2021). Socially shared metacognition in a project-based learning environment: A comparative case study. Learning, Culture and Social Interaction, 30, 100543. https://doi.org/10.1016/j.lcsi.2021.100543 DOI: https://doi.org/10.1016/j.lcsi.2021.100543
LoGiudice, A. B., Heisz, J. J., & Kim, J. A. (2023). Does collaborative testing in the classroom enhance delayed transfer of knowledge? Scholarship of Teaching and Learning in Psychology, 9(3), 312–325. https://doi.org/10.1037/stl0000267 DOI: https://doi.org/10.1037/stl0000267
Mahoney, J. W., & Harris-Reeves, B. (2019). The effects of collaborative testing on higher order thinking: Do the bright get brighter? Active Learning in Higher Education, 20(1), 25–37. https://doi.org/10.1177/1469787417723243 DOI: https://doi.org/10.1177/1469787417723243
Malmberg, J., Järvelä, S., & Järvenoja, H. (2017). Capturing temporal and sequential patterns of self-, co-, and socially shared regulation in the context of collaborative learning. Contemporary Educational Psychology, 49, 160–174. https://doi.org/10.1016/j.cedpsych.2017.01.009 DOI: https://doi.org/10.1016/j.cedpsych.2017.01.009
Michalsky, T., & Cohen, A. (2021). Prompting socially shared regulation of learning and creativity in solving STEM problems. Frontiers in Psychology, 12, 722535. https://doi.org/10.3389/fpsyg.2021.722535 DOI: https://doi.org/10.3389/fpsyg.2021.722535
Miller, M., & Hadwin, A. (2015). Scripting and awareness tools for regulating collaborative learning: Changing the landscape of support in CSCL. Computers in Human Behavior, 52, 573–588. https://doi.org/10.1016/j.chb.2015.01.050 DOI: https://doi.org/10.1016/j.chb.2015.01.050
Miller, M., & Hadwin, A. (in press). Comparing the effectiveness of CSCL scripts for shared task perceptions in socially shared regulation of collaborative learning. International Journal of Computer-Supported Collaborative Learning.
Moore L (2010). Students’ attitudes and perceptions about the use of cooperative exams in an introductory leadership class. Journal of Leadership Education, 9(2), 72–85. https://doi.org/10.12806/V9/I2/RF6 DOI: https://doi.org/10.12806/V9/I2/RF6
Mort, J. R., & Hansen, D. J. (2010). First-year pharmacy students' self-assessment of communication skills and the impact of video review. American Journal of Pharmaceutical Education, 74(5), Article 78. https://doi.org/10.5688/aj740578 DOI: https://doi.org/10.5688/aj740578
Nokes-Malach, T., Richey, J., & Gadgil, S. (2015). When Is It Better to Learn Together? Insights from Research on Collaborative Learning. Educational Psychology Review, 27, 645–656. https://doi.org/10.1007/S10648-015-9312-8. DOI: https://doi.org/10.1007/s10648-015-9312-8
Organization for Economic Co-Operation and Development. (2019). An OECD Learning Framework 2030.In G. Bast, E. G. Carayannis, & D. F. J. Campbell (Eds.), The Future of Education and Labor (pp. 23–35). Springer. https://doi.org/10.1007/978-3-030-26068-2_3 DOI: https://doi.org/10.1007/978-3-030-26068-2_3
Pandey, C., & Kapitanoff, S. (2011). The influence of anxiety and quality of interaction on collaborative test performance. Active Learning in Higher Education, 12(3), 163–174. https://doi.org/10.1177/1469787411415077 DOI: https://doi.org/10.1177/1469787411415077
Paterson, T., & Prideaux, M. (2020). Exploring collaboration in online group based assessment contexts: Undergraduate Business Program. Journal of University Teaching & Learning Practice, 17(3), Article 3. https://doi.org/10.53761/1.17.3.3 DOI: https://doi.org/10.53761/1.17.3.3
Rao, S. P., Collins, H. L. & Dicarlo, S. E. (2002). Collaborative testing enhances student learning. Advances in Physiology Education, 26, 37–41. https://doi.org/10.1152/advan.00032.2001 DOI: https://doi.org/10.1152/advan.00032.2001
Rennstich, J. K. (2019). Creative online collaboration: A special challenge for co-creation. Education and Information Technologies, 24, 1835–1836. https://doi.org/10.1007/s10639-019-09875-6 DOI: https://doi.org/10.1007/s10639-019-09875-6
Roediger III, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255. https://doi.org/10.1111/j.1467-9280.2006.01693.x DOI: https://doi.org/10.1111/j.1467-9280.2006.01693.x
Roschelle, J., & Teasley, S. D. (1995). The construction of shared knowledge in collaborative problem solving. In C. O’Malley (Ed.), Computer supported collaborative learning (pp. 69–97). Springer. https://doi.org/10.1007/978-3-642-85098-1_5 DOI: https://doi.org/10.1007/978-3-642-85098-1_5
Rogat, T. K., & Linnenbrink-Garcia, L. (2011). Socially shared regulation in collaborative groups: An analysis of the interplay between quality of social regulation and group processes. Cognition and Instruction, 29(4), 375–415. https://doi.org/10.1080/07370008.2011.607930 DOI: https://doi.org/10.1080/07370008.2011.607930
Sandahl, S. S. (2010). Collaborative testing as a learning strategy in nursing education. Nursing Education Perspectives, 31(3), 142–147.
Straus, S. G. (1997). Technology, group process, and group outcomes: Testing the connections in computer-mediated and face-to-face groups. Human–Computer Interaction, 12(3), 227–266. https://doi.org/10.1207/s15327051hci1203_1 DOI: https://doi.org/10.1207/s15327051hci1203_1
Strijbos, J. W., Kirschner, P. A., & Martens, R. L. (Eds.). (2004). What we know about CSCL: And implementing it in higher education. Kluwer Academic. DOI: https://doi.org/10.1007/1-4020-7921-4
Vuorenmaa, E., Järvelä, S., Dindar, M., & Järvenoja, H. (2022). Sequential patterns in social interaction states for regulation in collaborative learning. Small Group Research, 54(4), 512–550. https://doi.org/10.1177/10464964221137524 DOI: https://doi.org/10.1177/10464964221137524
Webb, N. M., & Palincsar, A. S. (1996). Group processes in the classroom. Prentice Hall International.
Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D.J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277–304). Lawrence Erlbaum Associates Publishers.
Xu, D., & Xu, Y. (2019). The promises and limits of online higher education: Understanding how distance education affects access, cost, and quality. American Enterprise Institute. https://www.aei.org/research-products/report/the-promises-and-limits-of-online-higher-education/
Zimbardo, P. G., Butler, L. D., & Wolfe, V. A. (2003). Cooperative college examinations: More gain, less pain when students share information and grades. The Journal of Experimental Education, 71(2), 101–125. https://doi.org/10.1080/00220970309602059 DOI: https://doi.org/10.1080/00220970309602059
Zimmerman, B. J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81(3), 329–339. https://doi.org/10.1037/0022-0663.81.3.329 DOI: https://doi.org/10.1037//0022-0663.81.3.329
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Mariel Miller, Safoura Askari
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors contributing to the OTESSA Journal agree to release their articles under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. This licence allows this work to be copied, distributed, remixed, transformed, and built upon for any purpose provided that appropriate attribution is given, a link is provided to the license, and changes made were indicated.
Authors retain copyright of their work and grant the OTESSA Journal right of first publication.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in the OTESSA Journal.