Abstract

Collaborative problem-solving has emerged as a critical skill in the 21st century, as it is essential for addressing complex and multifaceted challenges inherent in modern work environments (Graesser et al., 2018). In this thesis collaborative problem-solving skills are defined as the capacity of an individual to effectively engage in a process, where two or more agents with different knowledge bases attempt to solve complex tasks. The process of col-laborative problem-solving involves active interaction with the problem, decision-making under uncertainty, and the integration of knowledge and skills to create a shared problem representation needed to reach a solution. Medicine is domain where it is of critical importance to reduce diagnostic errors and thus ensure high quality patient care. Collaborative diagnostic reasoning, a form of collaborative problem-solving in knowledge-rich domains like medical diagnosing, describes the critical role of collaboration when solving diagnostic problems in order to achieve accurate, well-reasoned and efficient diagnoses. Building upon research on collaborative problem-solving and diagnostic reasoning the collaborative diagnostic reasoning model (CDR-M; Radkowitsch et al., 2022) proposes a joint perspective in solving diagnostic problems in a collaborative effort. While this thesis focuses primarily on medical contexts, the insights and methods developed are expected to be applicable across disciplines. To support the development of expertise in collaborative problem-solving and collabora-tive diagnostic reasoning, it is important to provide authentic situations allowing for knowledge application and schema acquisition. Through repeated exposure to diagnostic problems and experience with cases, knowledge gets encapsulated and a data-base of already seen cases is created or updated. This leads to prototypical abstract case representations ena-bling greater accuracy and efficiency when solving diagnostic problems (Boshuizen et al., 2020). The educational implications are straightforward: For the restructuring and reorgani-zation of biomedical knowledge, the early exposure to patient cases is considered essential. However, the opportunity to engage in real-life problem-solving is limited and relevant sit-uation to learn may arise less often or are too critical to be approached by novices. One way to overcome this issue and also facilitate the assessment of collaborative prob-lem-solving skills, is the use of technology-based assessments and simulation-based learning environments. Simulation-based learning environments offer authentic situations for learners to practice collaborative diagnostic reasoning without the risks associated with real patient cases (Chernikova et al., 2020). The use of computerized agents as collaboration partners allows to create a standardized and controlled setting that is hard to establish in collabora-tions among humans. However, Although the use of simulation-based learning environments and the integration of technology-based assessments presents opportunities it also entails challenges in assessing and supporting collaborative problem-solving skills. The develop-ment of technology-based interactive tasks and simulation-based learning using computer-ized tasks enables a closer approximation of real-world scenarios. These tasks allow for mon-itoring the process through observable problem-solving behaviors, which are stored as com-puter-generated log-file data and can be accessed to provide valuable additional insights. Hence, process data can not only be used to examine what has been achieved, but also how it was achieved, and to make inferences about the cognitive processes involved in collabora-tive problem-solving. These inferences are implications for assessing performance differ-ences, developing predictive models, and providing personalized support (Ulitzsch et al., 2023). However, there are also a number of challenges associated with its use: Starting with ethical considerations before and during data collection, through to the complexities of ana-lyzing the data and the need for theory in interpreting the results. The goal of this thesis is to improve the use of process data for assessing and supporting collaborative problem-solving, specifically in the context of collaborative diagnostic reason-ing in medical education. To do so, this thesis compromises three papers having different foci on the usage of process data. The first paper takes a meta-perspective and elaborates recent developments in leveraging process data through technology-based assessments for creating new knowledge, improving learning and instruction, and providing actionable ad-vice to policy stakeholders. Building on these considerations, two empirical studies illustrate how process data can be used for theoretical advancements and to improve instruction. The second paper and first empirical study validates the CDR-M using process data. The third paper and second empirical study then demonstrates how the combination of process data and theory can be used to predict outcomes that can inform instruction in simulation-based learning of collaborative diagnostic reasoning. The first paper, a theoretical paper, analyzes the impact of process data from interactive tasks in large-scale assessments. The paper highlights necessary changes that need to be un-dertaken at the scientific level in how we analyze process data to foster sustainable changes at the practical and policy levels. Firstly, linking process data to educational theory is crucial for enhancing the generalizability of our findings and hence facilitate theoretical advance-ments. Secondly, the design of assessment should be aligned with instructional design to in-form learning and instruction. Paper 2 employs process data to empirically test and refine the CDR-M and thus demon-strates how process data can be harnessed to generate new insights and advance theoretical frameworks in education. By analyzing data from three studies in a simulation-based envi-ronment the aim of the study was to better understand the collaborative diagnostic reasoning and the processes involved using a structural equation model including indirect effects. Results identified various stable relations between individual characteristics and collaborative diagnostic activities, and between collaborative diagnostic activities and diagnostic outcome, highlighting the multidimensional nature of collaborative diagnostic reasoning. In summary, the second paper found that for successful collaborative problem-solving in knowledge-rich domains, knowledge about the domain of the collaboration partner and collaborative diag-nostic activities play a crucial role in addition to content knowledge, which is traditionally in the focus of expertise research. The third paper focuses on enhancing simulation-based learning by predicting diagnostic accuracy in collaborative diagnostic reasoning using process data. This study developed a random forest classification model based on theoretically derived process indicators to pre-dict success in a simulated learning environment. Results showed a satisfactory prediction rate for collaborative diagnostic reasoning performance, indicated by diagnostic accuracy. The model predicted accurate and inaccurate diagnoses and was therefore suitable for making statements about the performance by only using process data of collaborative diagnostic reasoning. Hence, Paper 3 showed that using prediction models enables researchers to provide practical solutions such as identifying learners at risk to show inadequate performance in need of adaptive instructional support. In a nutshell, in terms of theoretical advancements, the papers presented indicate support for four assumptions proposed in the CDR-M, as well as adding two new assumptions to the CDR-M. Firstly, unique contribution of collaborative diagnostic activities to collaborative diagnostic reasoning and secondly, the need to investigate complex non-linear interactions between collaborative diagnostic activities. With respect to supporting the development of collaborative diagnostic reasoning skills, practical implications are to focus on collaboration knowledge and collaborative diagnostic activities and turn the measurement of processes like collaborative diagnostic activities into a design factor. In addition, a strategy for providing adaptive instructional support is proposed. Lastly, the findings in this thesis also reveal several insights into how the usage of process data analyses can be enhanced when assessing and supporting collaborative problem-solving skills. Most importantly, by leveraging theory-based frameworks to describe collaborative problem-solving processes, we can create a common ground for assessing and enhancing collaborative problem-solving skills across dif-ferent domains and thus further improve the use of process data analyses. Overall, findings of the three papers illustrate how process data can be used to advance theoretical models, as shown by the CDR-M, to support learning of collaborative diagnostic reasoning skills and, thus, ultimately enhance the usage of process data of collaborative problem solving. In conclusion, this thesis highlights the need of leveraging theory-based frameworks to describe collaborative problem-solving processes. This will lead to more pro-ficient collaborators in the future, not only in the medical domain.