Abstract

This study proposes an application of cognitive maps in the representation of cognitive structures of the experts and assessment of their development/modification as a result of a (computer or expert system-assisted) learning process. It strives to identify information needed for the guidance of the process of creation and management of expert knowledge by formal modeling tools. Changes in experts’ cognitive structures are assumed to stem from individual and collaborative (group-level) learning. The novel approach to assessing the outcomes of learning reflected as changes in the cognitive structures of experts or groups of experts, modeled by cognitive maps, does not assume any correct or desired outcome of the learning process to be known in advance. Instead, it identifies and analyzes the changes in (or robustness of) the constituents of the cognitive maps from different points of view and allows for quantifying and visualizing the actual effect of the learning. The proposed methodology can identify changes in cognitive diversity, causal structures in terms of causal relations and concepts, and the perceived importance of strategic issues over the learning period. It can also detect which cause–effect relationships have appeared/disappeared considering the pre-/post-mapping design. Thus, it provides an exploratory account on the changes in the cognitive structures of the expert(s) as a result of learning. The applicability of the proposed methods is illustrated in the assessment of the learning outcomes of a group of 71 graduate students who participated in an eight-week business simulation task. The results of the empirical analysis confirm the viability of the proposed methodology and indicate that the students’ understanding of the utilized concepts and associated relationships in the decision-making process improved throughout the learning activity, ultimately showing that the course learning has considerably improved students’ perception and knowledge. Based on the results, it can be concluded that the proposed approach has the potential to be effective in assessing learning outcomes in teaching–learning activities.

Introduction

Expert knowledge development and its efficient management are necessary in all areas of human activity and as such it is studied in various fields, especially in education, andragogy, sociology, and also in a field-specific context of professional training and expertise development, with potentially increasing importance in the design of expert systems and artificial intelligence solutions. Understanding how individuals learn and acquire and enhance their knowledge leads to efficient educational practices and knowledge management. Even though a lot of attention is being paid to the actual processes of learning and teaching (‘how’ the knowledge is being constructed), their results, that is the outcomes of learning (‘what’ has been learned), still remain difficult to quantify and thus analyze systematically. Individual perceptions influence the teaching–learning process and are characterized by personal knowledge, experience, and other aspects, such as beliefs, interests, and expectations (Robbins, 2005). Consequently, a situation, a problem, or a concept can be perceived from various angles and perspectives, which might also differ from one person to another. This makes the operationalization of the effects and outcomes of learning demanding. To gain a comprehensive understanding of the effects and outcomes of the teaching–learning process and allow for robust analyses, the concept of cognitive mapping as a participatory method has recently received interest among researchers and scholars (Gray et al., 2015). A cognitive map that acts as a cause-and-effect network of qualitative aspects (Tolman, 1948) supports individuals in visually representing their beliefs, arguments, and understanding of a situation or problem (Kumbure et al., 2022) and as such it can serve as a representation of cognitive structures representing (expert) knowledge in various domains and areas of activities. It is also interesting to note that it can represent the knowledge and various cognitive structures in various stages of the knowledge creation process. As such cognitive maps represent a tool for the assessment of changes and development (a flow-type measure) of expert knowledge. This very aspect of cognitive maps is going to be explored in this paper and methods for their use in the assessment of the outcomes of the teaching–learning process will be proposed. Cognitive maps remain a subject of ongoing research interest in various applications involving social science (e.g., Son et al., 2021), education research (e.g., Nesbit and Adesope, 2006, Sun et al., 2019, Wang et al., 2018), business and management research (e.g., Bergman et al., 2016, Kumbure et al., 2020), healthcare (e.g., Ottink et al., 2022), engineering and technology (e.g., Mendonca et al., 2013, Motlagh et al., 2012), environment research (e.g., Aledo et al., 2015), and computer science (e.g., Budak and Çoban, 2021, Kwon, 2011), to mention a few.

Conclusion, limitations, and future work

In conclusion, the present study demonstrated the effectiveness of the proposed cognitive mapping-based approach for assessing learning in the context of CSCL in terms of the students’ academic success and teacher’s perspectives. We proposed an unsupervised approach to the assessment of learning outcomes that is explorative and allows for the identification of the actual effects of the teaching event on the students — both on individual and group levels. Table 12 summarizes the empirical results obtained through each analysis for each research question and their implications.

The results from all analyses implied that the students have had a positive learning environment, and also teacher’s guidance and instructions were successful to some extent. Overall, this study has made several significant contributions to the existing literature, which can be summarized as follows: