International Journal of Information and Education Technology, Vol. 10, No. 8, August 2020 Utilizing Graphical Elements for Concept Map Analysis to Design Teaching and Learning Assessment Suparat Chuechote and Parames Laosinchai assessment [6]-[10]. Yin et al.‟s research offers the comparison between two models of a concept map construction; one model is a concept map construction assignment with only node terms restricted—students self-create linking phrases—and another is just a map assembling task where both node terms and linking phrases are provided [10]. From this work, it suggests that the first model is better for knowledge capturing, whereas the latter fits better for large scale scoring. Clearly seen in the map assembly task, the scoring could be bipolar; matching is either right or wrong. On the other hand, in the open-ended approach, the constructed concept maps can diversify and generate more complication in scoring. Since this study emphasizes on learning assessment, we therefore have embraced the first model of concept map construction with node terms about polynomials provided. To tackle with the complication of scoring, we have explored the graphical elements and features, such as, nodes, edges, diameters, cliques, travelling paths and structures, for potential use in map scoring scheme. With awareness of reasoning behind the map construction process, the collection of concept maps from students will be compared with the teacher‟s map to make a better vision of how the teacher has expected and what students have achieved. Analyzing graphical data, we have considered how these graphs are formed and how the elements are linked. The graphical result and its interpretation provide interesting angle in learning and teaching assessment. Remarkably, concept map similarity among students engaging in the same lesson can mirror prior knowledge of students and the effectiveness of the teaching approach. This research gives another purposeful use of the concept maps and suggests diagnostic scheme that could be beneficial to both learning and teaching assessment.  Abstract—The graphical elements as parts of concept map construction are employed to assess both learning and teaching. Augmenting the use of concept maps, this study examines the graphical elements, such as, nodes, edges, cliques, diameters, travelling paths and structures of the graphs to relate to ones’ understanding to a topic, in this case, polynomials for middle school. In the aspect of teaching assessment, the teacher’ concept map drawn according to the lesson plan is served as the master map, which echoes the teacher’s expectation of students’ learning. On the other hand, students’ maps also reveal their understanding through the nodal relationship, which can be the definitions of terms, related examples, graph representation and algebraic manipulation. Data collection includes a focus group of 10 students and 1 teacher undergoing the concept map assessment task with restricted node terms. Graphically analyzed, students’ concept maps reveal some common elements as in the teacher’s map. In addition, the interview with the teacher also suggests that concept map as the assessment tool is an effective teaching reflection for which the teacher can see what to fulfill for future classes. Index Terms—Concept mapping, learning assessment, teaching assessment, graphical information, polynomials. I. INTRODUCTION Concept maps have received a great deal of attention in science education as a learning strategy since 1990 [1]. The continuing research about this further suggests that the concept map construction is based on the epistemological assumption that the concept-concept relationship is the building block of knowledge [2]. However, structuring knowledge can be diverse, yet hard to visualize, particularly for abstract mathematical concepts. Therefore, use of concept mapping becomes handy to extract learners‟ knowledge construction and to reflect their understanding. In accordance with this argument, McGowen and Tall claims that a concept map is a diagram representing the conceptual structure of a subject discipline as a graph in which nodes represent concepts and lines connecting them represent cognitive links [3]. With the beneficial structures, some research extends the application of concept maps as an assessment tool, for which the comparison between a teacher and students‟ concept maps can be considered as a form of a lesson evaluation to assess the lesson‟s objectives [4], [5]. According to multiple reviews, the previous work about concept mapping assessment has involved around concept recognition, organization of concept in branching structure, graph alignment, similarity, and scoring scheme for learning Manuscript received March 17, 2020; revised June 10, 2020. The authors are with Mahidol University, Thailand chuechote@gmail.com). doi: 10.18178/ijiet.2020.10.8.1424 II. CONCEPT MAP CONSTRUCTION Since knowledge construction process is the integral part for this study, we designed the experiment to include 3-hour training of concept map construction following the adapted scheme from Malone & Dekkers [11]. The protocol is described as follows. - First task: List the key terms and find all possible terms that can be associated with the key terms. - Second task: Rank the strength in association of the key terms and other node terms. Arrange the key terms on top and the closest or strongly associated node terms with the key terms are one level below. If node terms have the same relationship with the key terms, put them on the same level. - Third task: Add edges or linking phrases according to the (e-mail: 557 International Journal of Information and Education Technology, Vol. 10, No. 8, August 2020 construction. After being familiar with this, the participants attended their regular mathematics class which covered a polynomials topic and were back to do concept mapping assessment as a major part of data collection. By the end of this process, we obtained 11 concept maps from both teacher and student participants and the teacher interview on her teaching approach, teaching media, styles and difficulties she experienced. The concept map built by the teacher served as the master map. It implied the expected leaning outcomes. Hence, it was used for reference in the graph alignment, reflecting how well the teacher delivered the domain content and concept. relationship. - Fourth task: Look for relationship between key terms or between node terms and add edges. III. CHARACTERISTICS OF A POLYNOMIAL CONCEPT MAP Fig. 1. List of node terms on polynomials topic for concept assessment. In this study, we have explored concept mapping on polynomials topic in middle schools. This topic has a potential for capturing students‟ geometrical and algebraic reasoning as well as reflecting teaching approach and media used in the lesson. Therefore, to teach this topic, the teacher should prepare the plan that give students enough experience for relevant knowledge. To design the set of node terms, we considered possible objectives that the teacher aimed to achieve for her teaching. The node terms were deduced from the teacher‟s lesson plans, learning objectives and pre-lesson interview. The declared objectives were to understand the meaning of terms, monomial, polynomial, similar monomial; to be able to manipulate, add, subtract, multiply and divide polynomials; to be able to apply it with algebraic reasoning, and to understand graphical interpretation. Fig. 1 shows the node terms used in the concept mapping assessment. The 7 categories of node terms are denoted as; “A” for definite terminologies, “B” for algebraic expressions, “C” for short description, “L” for graphs, “H” for area and volume, “K” for algebra tiles representation, and “Q” for polynomial factorization. To avoid bias in concept mapping caused by the teacher execution, these terms were not shown to the participants until the day of assessment. A concept map is a diagram that represents conceptual structure of a subject discipline as a graph G = (V, E) as a pair of a set of vertices (or nodes) V and a set of edges E. The edges represent relationships between the nodes. They can be oriented or not, depending on the nature of relations represented. Every graph is described by connecting nodes, which can be written in the form of an adjacency matrix A, a binary n × n matrix with entry aij = 1 if node vi is adjacent to node vj, and aij = 0 otherwise. The number of nodes in a graph is usually denoted by n while the number of edges is usually denoted m. The diameter of a graph is the longest shortest path between any pairs of nodes. In other words, a diameter is the largest number of edges which must be traversed in order to achieve shortest distance travel from one node to another node. Fig. 3. The adjacency matrix for the undirected graph shown in Fig. 2. According to the graph as in Fig. 2, we have the adjacency matrix A as 8 × 8 matrix, shown in Fig. 3 for the undirected graph. With diameter of the graph equal to 5, the longest shortest distance of the graph is the path travelling from “8x2+5x” to “-x3”. In this matrix form, we can easily manage graphical comparison. When each matrix entry represents node terms‟ relation, the frequency of edge links, cliques, common subgraphs or travelling paths can be computed with Fig. 2. Example of a polynomial concept map according to the construction protocol. Clearly illustrated this protocol, Fig. 2 shows how one works on concept map construction following the protocol. The components on cognitive development, such as concepts recognition, grouping concepts and organizing concepts in branching are visible via the process of concept maps 558 International Journal of Information and Education Technology, Vol. 10, No. 8, August 2020 of any intervention, it is likely to see different maps explaining the same concept but with different structure representation. A teacher could find it difficult to check whether the students have understood and have met the objectives of the lesson. Therefore, we consider the graphical elements and features in association with the learning domain. In this case, it is polynomials topic for middle school mathematics. the aid of R programing and its functions for matrices. A subgraph of a graph G is a graph whose nodes and edges are contained in G. A graph in which all nodes are adjacent to all others is said to be complete. A clique is a maximal complete subgraph of G. In other words, a clique of an undirected graph is a subgraph whose every two distinct nodes in the clique are adjacent. These graph elements and features play the important role in concept map analysis. With the assumption that students undergo the same process of the lesson, they likely generate common meaningful subgraphs. The common subgraphs can be in several forms, a net, a tree, a string or a clique. With the unique property that any nodes are adjacent, a clique then has potential to present a knowledge preposition. The investigation of knowledge formation representing through the graph will be delineated in the next section. However, with the master map from the teacher, a student‟s map will then be compared with map alignment for a similarity score, the comparison of adjacency matrices between teacher‟s and a student‟s map. Map similarity has been widely researched in computer network communities as they have similar structural features, for example, web and internet uses, semantic network, and accessibility. Likewise, the collection of concept maps as teaching and learning assessment can also be organized in the ontology framework using graphical features and hence become useful for the analysis of the domain of interest in the similar fashion [12], [13]. A. Key Terms and Their Association Concept mapping as a learning assessment has a key term where other terms are built around. At this section, we capture the core understanding of the topic indicated by the key terms and their neighbors. According to section III, the concept map construction protocol, a key term is supposed to get listed first with potential of being the hub. A sketch of knowledge prepositions can be deduced from the neighbors of a key term. To capture this individual latent understanding, we therefore associate the linking edges according to the cognitive links into 5 types; (1) defining, (2) containing, (3) example, (4) graphing, and (5) algebraic manipulating, as shown in Table I. Fig. 4 shows the possible concept map whose edges between nodes are categorized based on the types of nodal association that could infer the comprehension and ability to think around the key terms of the topic. IV. METHODOLOGY The aim of the research was to explore concept maps drawn by a teacher and students with the assumption that the maps‟ elements and features would show relatedness to students and teacher‟s understanding and hence navigate to graphical interpretation that reflects teaching and learning. The research involved the focus group containing 11 members, 1 teacher and 10 middle-school students, from Phitsanulok province, Thailand. There was a training phase to make sure that all participants were able to draw concept maps when node terms were provided. Note that the training phase we worked on the primary geometry topic to avoid pre-test manipulation bias. In addition to students‟ concept maps drawn after the lesson finished, the data collection included pre- and post-lesson interview with the teacher, a teacher‟s lesson plan for polynomials topic, a teacher‟s concept map drawn after lesson implementation. The interview questions were designed to go into details of the teacher‟s lesson plan, teaching approach, expectation, reflection and difficulties after lesson implementation. The analysis principally involved the graphical data, pattern extraction and graph alignment for class comparison and teacher-student comparison. The teacher‟s lesson plan and interview were to support the graphical interpretation and self-reflection on the lesson execution. Fig. 4. Example of a concept map on polynomials topic with edges categorized. B. Path Depth with Cognitive Reasoning Investigating semantic graph in the aspect of knowledge formation, researchers in graph mining community extend a graph depth of each entity to be a means of similarity measurement but with corresponding weight according to their importance in the domain [11]. According to the learning objectives of the polynomials lesson, we have investigated the cognitive reasoning through the paths that pass the key terms. Therefore, meaningfulness of a concept map in learning must consider the graphical path depth and the combination of linking edge types. In this graphical dataset, the diameter of a student‟s concept map contains the longest path of linking node terms. This path in Fig. 5 shows how one student represented his reasons through the nodal relationship. Constructing a concept-centered map with a key term put at the top level, the participants associated neighbor terms ranked by closeness according to their understanding. The diameter, which is defined as the longest path in each graph, may have diverse linking phrases and edge types. This echoes how ones can understand a concept, can define, can V. GRAPHICAL ANALYSIS When students are asked to draw concept maps regardless 559 International Journal of Information and Education Technology, Vol. 10, No. 8, August 2020 give examples, can apply, and/or can communicate via graphical representation. Fig. 5 illustrates an example of student‟s concept map. By reading the map, we could see that this student understands that a polynomial (A4) can contain variables (A1) and can be defined as addition of monomials (C4), which can be exemplified as in (B18), algebraically manipulated to be equal to (B7). The diameter in Fig. 4 passing through the key term, „polynomial‟ contain 4 out of 5 edge types. This information signifies the cognitive reasoning revolving around the key conceptual term. The path depth with various edge types about the key terms identifies the understanding in various dimension that could eventually satisfy the learning objectives. D. Graph Alignment Graph matching takes all the attributes of nodes and edges arranged in the form of an adjacency matrix. We first used matrix comparison to see the match of each entry of the compared matrices. Fig. 6 shows the union of 10 students‟ concept maps with yellow-highlighted nodes, in comparison to the master map constructed by the teacher. Therefore, the non-highlighted node terms represent the areas that teacher over-expected; no students shows the relation of terms such as “area (H1)”, “volume (H2)”, “a2+b2+2ab (Q2)”. On the other hand, Fig. 7 reveals the most common students‟ concept maps, i.e. 5 out of 10 students has got this subgraph. Also compared with Fig. 6, there is some room of improvement where students need to practice and learn more to achieve the understanding of the white terms‟ concepts. In this case, the node terms of categories “H” for area and volume, “K” for algebra tiles representation, and “Q” for polynomial factorization were left undone by the students. The teacher reflected herself that she should have used algebra tiles to be the manipulative for this lesson so that students could visualize the factorization of polynomials and better understand how polynomials related to areas and volumes and other applications. TABLE I: EXAMPLE OF EDGE TYPES OF A CONCEPT MAP Type of Edge Node From To (1) Defining Variables (2) Containing Monomial Symbols that represent values Variables (3) Example Monomial -16x (4) Graphing Graph L1 passing (0,0) with negative slope 12x3-8x3 -16x (5) Algebraic manipulating -8𝑥3 C. Cliques and Concept Formation Definition of a clique already suggests that any two node terms are related. For a polynomials topic, the node terms can be linked to form a clique and that clique could represent a knowledge preposition. Fig. 5 shows that magnified clique of the graph contains 4 nodes, “polynomial (A4)”, “addition of monomials (C4)”, “addition of similar monomials (C5)”, and “5a2b3+b3a2 (B18)”. Considering this 4-clique, we see that the formation that links the definition of the term “polynomial”, its characteristics, and its example, which is extended to reveal the ability of algebraic manipulation and understanding of “similar monomial”. This graph also shows interesting 3-clique, containing 3 nodes “45 (B4)”, “Coefficient (A2)”, the graph of constant in the xy-plane (L3). The interesting point here is that this student shows his knowledge indicating the misconception, and possibly the area that the teacher has neglected while teaching this topic. He shows that he understands graphical interpretation of a constant but has a misconception about coefficient. The coefficient multiplying a variable can produce various graphs, not just a horizontal line. From this evidence, the teacher can see the misconception and can correct it in the future. Fig. 6. The union concept maps (highlighted nodes) of 10 students in comparison to the master map. Fig. 7. The most common concept maps (highlighted nodes) of 10 students in comparison to the master map. However, with the maps reflecting conceptual understanding, graphing alignment alone is insufficient to Fig. 5. Concept map with highlighted diameter, the longest connecting path in the map, and circled clique. 560 International Journal of Information and Education Technology, Vol. 10, No. 8, August 2020 assess conceptualization, particularly, thinking about polynomial and factorization concepts. The alignment might give a high score if a student selects a key term as a nodal hub with its neighbor nodes as exemplifying terms, for example, “monomial (A3)” acting as a hub with 9 neighbor terms of its examples, such as, “9x2”, “-16x”, etc. This matching subgraph does not show other cognitive reasonings, such as, algebraic manipulation, concept connection or graph interpretation. Therefore, graph alignment can only measure similarity to the teacher‟s master map but not the growth of learning as an individual reveals the connection formed with various reasonings to show how he or she conceptualizes. with the topic. However, relating polynomials with graphs, starting from polynomials of degree 1 or linear relation will benefit the students in graphical interpretation and hence lead to spatial skills for higher-order problems. Nonetheless, the teacher agreed that having the algebra tiles for the manipulative of this topic would be appropriate. Some students would learn better with visualization and hands-on. If she had done this, the edge types of graph and algebraic manipulation found would become close to the master map. For the advantage of students, hands-on activity, specially the algebra tiles as for algebraic manipulation, would cultivate the sense of algebra and geometry linked and future concepts for factorization VI. RESULTS AND DISCUSSION We started the work with a small focus group to make sure that all participants could draw the map. Hence, it eliminated the interpretation of not drawing because of how-to issue. With the in-depth interview with the teacher prior to the polynomial lesson, we opened her to implement the lesson in the way she was confident. She rated herself as an active questioning teacher, who rarely used thinking manipulatives but the effective questions to motivate students‟ thinking. She believed that math exercise would help students understand. Therefore, in her lesson plans, she put the assignments corresponding to her lesson objectives. Considering concept maps of students, she then reflected herself that it would be better to blend in different approaches for her class. Fig. 8 shows the edge types found in the teacher‟s map and average of students‟ maps. From this evidence, it shows that most students can only obtain 41.67% of the master map. In other words, the average map from the students aligned 41.67% of all components of the master map, so called it similarity score. The lowest similarity score is about 20% and the highest similarity score is 85.19%, shown in Fig. 9. Fig. 8. The edge types found in the teacher‟s map and average of students‟ maps. Fig. 9. The similarity score and percentage of cliques found in concept maps of 10 students. According to the master map constructed by the teacher, a similarity score is the percentage of nodes and edges from a student‟s map aligned with the master map. As mentioned earlier, this score is not the best representation of conceptual understanding. Some students may have high similarity score but fail to capture the core concept. As Fig. 9 shows the student coded FG1-03 has 40.74% as his similarity score but only discovered 2 cliques. On the other hand, the student coded FG1-04 has lower similarity score 37.04% but dis covered 3 cliques, counted as 25% of the master map‟s cliques. Having cliques in the concept maps suggests the understanding in more complex relationship than the linear links or maps. As for example, a clique of nodes “polynomial (A4)”, “addition of monomials (C4)”, “addition of similar monomials (C5)”, and “5a2b3+b3a2 (B18)” shows the links between any pairs with different justification levels. This net-like structure of a clique incorporates a conceptual meaning with the map integrity. It infers the builder‟s understanding around the domain concept with his/her ability to employ technical terminology [9], [14]. VII. CONCLUSION The comments of the teacher after seeing the result of concept map analysis marked the significant self-reflection that led to the promising action plan. Therefore, we have a good start and would like to move this study forward for concept maps collection and graph mining. In addition, developing the electronic version for concept map construction will be useful. It will aid students to draw concept maps; pasting nodes and making links can be done more easily. Similarly, teachers can choose the significant Interestingly, Fig. 8 also tells that the edge types that students correctly linked were defining and containing. Agreeing with the graphical analysis, the teacher recorded in her lesson plan that she assigned exercise for the students to describe the definition of the polynomial and monomial terms. Not being in the major part of the plan, the graphing and algebraic manipulating were the areas that students failed to achieve. The teacher mentioned that she did not emphasize on graph representation because it was not quite associated 561 International Journal of Information and Education Technology, Vol. 10, No. 8, August 2020 [3] node terms corresponding to their lesson objectives to assess students‟ learning. Considering the graphical elements and feature, we have seen the promising relation of a map‟s structure and knowledge construction. Therefore, the scoring scheme of a concept map should incorporate the interpretation of the elements and features like diameters, cliques, and graph structures to the similarity score. As Kinchin and Hay suggested, concept mapping can help us see student understanding more explicitly. A student with a net-like map structure or with many cliques tends to show more various levels of thinking than the one with a linear map [9], [14]. This study serves as the initial phase to understand graphical data as for psychological and cognitive measurement. The research team will explore further for the best fit scoring scheme and graphical database for concept mapping. The reflection from the teacher encourages us to investigate more with various learning topics, various students‟ backgrounds and various learning objectives as we have been aware that teaching approach plays role in students‟ concept map construction. When it comes to learning and teaching, it needs to be locally customized. The ideal assessment for learning should be a means that can extract students‟ knowledge construction and reasoning trajectories. Whereas, the assessment for teaching should feedback teachers for their past actions and should shape or navigate to the teaching improvement to achieve students‟ learning. This research has achieved making the participating teacher realized that. [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] CONFLICT OF INTEREST The authors declare no conflict of interest. Copyright © 2020 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0). AUTHOR CONTRIBUTIONS The first author, Suparat Chuechote, conducted the research, data analysis and paper writing. The second author, Parames Laosinchai reviewed and revised the methodology. All authors had approved the final version. Suparat Chuechote received the BSc degree in mathematics at McGill University, Canada 2006. She received the MSc degrees in applied mathematics at Case Western Reserve University, USA, in 2010. She is currently a faculty member in the Department of Education at Naresuan University and in parallel continuing her study as being a PhD candidate in science and technology education at Institute for Innovative Learning, Mahidol University, Thailand. Her research area includes educational data mining, educational data ‟ management, mathematics education, STEM education, sustainable education and computational thinking. ACKNOWLEDGMENT The research is supported by Faculty of Graduate Studies and Institute for Innovative Learning, Mahidol University. The authors would like to express our gratitude toward the Faculty of Education, Naresuan University for the opportunity to explore mathematics class of middle schools in Phitsanulok province as well as the school that involved in this research and all participants that willingly learnt and participated throughout the period of this study. Parames Laosinchai is currently a faculty member at Institute for Innovative Learning, Mahidol University, Thailand. He received his BEng in computer at Chulalongkorn University, MBA in finance and investment at City University of New York, MSc in finance at Washington University in St. Louis, USA, and PhD in science and technology education at Mahidol University in 2011. His research area includes mathematics education, computer science education, physics education, applications of computer science and mathematics and phylogenetics. ‟ REFERENCES [1] [2] M. McGowen and D. Tall, “Concept maps & schematic diagrams as devices for documenting the growth of mathematical knowledge,” in Proc. Annual Conference of the International Group for the Psychology of Mathematics Education, 1999. M. A. Ruiz‐Primo and R. J. Shavelson, “Problems and issues in the use of concept maps in science assessment,” Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, vol. 33, no. 6, pp. 569-600, 1996. M. A. Ruiz-Primo, “Examining concept maps as an assessment tool,” in Proc. of the First Int. Conference on Concept Mapping, 2004. J. R. McClure, B. Sonak, and H. K. Suen, “Concept map assessment of classroom learning: Reliability, validity, and logistical practicality,” Journal of Research in Science Teaching, vol. 36, no. 4, pp. 475-492, 1999. D. B. Leake, A. G. Maguitman, and A. J. Cañas, “Assessing conceptual similarity to support concept mapping,” in Proc. FLAIRS Conference, 2002. J. A. Rye and P. A. Rubba, “Scoring concept maps: An expert map‐based scheme weighted for relationships,” School Science and Mathematics, vol. 102, no. 1, pp. 33-44, 2002. D. B. Hay, “Using concept maps to measure deep, surface and non-learning outcomes,” Studies in Higher Education, vol. 32, no. 1, pp. 39-57, 2007. Y. Yin et al., “Comparison of two concept-mapping techniques: Implications for scoring, interpretation, and use,” Journal of Research in Science Teaching, vol. 42, no. 2, pp. 166-184, 2005. J. Malone and J. Dekkers, “The concept map as an aid to instruction in science and mathematics,” School Science and Mathematics, vol. 84, no. 3, pp. 220-231, 1984. R. Ichise, “Machine learning approach for ontology mapping using multiple concept similarity measures,” in Proc. Seventh IEEE/ACIS International Conference on Computer and Information Science, 2008. P. D. H. Zadeh and M. Z. Reformat, “Context-aware similarity assessment within semantic space formed in linked data,” Journal of Ambient Intelligence and Humanized Computing, vol. 4, no. 4, pp. 515-532, 2013. I. M. Kinchin, D. B. Hay, and A. Adams, “How a qualitative approach to concept map analysis can be used to aid learning by illustrating patterns of conceptual development,” Educational Research, vol. 42, no. 1, pp. 43-57, 2000. J. D. Novak, “Concept mapping: A useful tool for science education,” Journal of Research in Science Teaching, vol. 27, no. 10, pp. 937-949, 1990. B. D. Pendley, R. L. Bretz, and J. D. Novak, “Concept maps as a tool to assess learning in chemistry,” Journal of Chemical Education, vol. 71, no. 1, p. 9, 1994. 562