Our graphic organizer audiobook has four main parts: (1) an introduction where we explain what graphic organizers are, how and why they came to be, and the basis for their use in cognition theory, followed by (2) a discussion of concept maps as one type of graphic organizer and (3) a discussion of story maps as another type, ending with (4) a conclusion that attempts to tie together the types of graphic organizers, their uses and critiques, and our understanding of human cognition. The audiobook also includes a PDF with concept maps for parts 1 and 2 and example story maps for part 3.
One article mentioned in the review is Kim’s meta-analysis of research on the use of graphic organizers with learning disabled students. The full APA citation is given below for those who wish to read the analysis in more detail.
For those who are particularly interested in the use of graphic organizers in science and math, Shannon recommends reading his review (or better yet, the original article) of Mautone and Mayer’s 2007 Cognitive aids for guiding graph comprehension since that article gives clear examples of different types of graphic organizers that affect what appear to be different cognitive functions.
Audiobook Bibliography
Kim, A., Vaughn, S., Wanzek, J., & Wei, S. (2004). Graphic organizers and their effects on the reading comprehension of students with LD: a synthesis of research, Journal of Learning Disabilities, 37, 105-118.
Mautone, P. D. & Mayer, R. (2007). Cognitive aids for guiding graph comprehension, Journal of Educational Psychology, 99, 640-652.
graphic organizers audiobook download
Tuesday, May 5, 2009
Saturday, May 2, 2009
Bibliography
The Graphic Organizers Bibliography contains a wide variety of articles highlighting not only the different types of graphic organizers but the versatility and importance of graphic organizers in different educational settings. The articles chosen represent topics ranging from concept mapping protocols in medical school to mind mapping activities in elementary school science settings. For the novice reader, the articles by Joseph Novak, the researcher credited with the modern day development of concept mapping, provide a thorough introduction to the development of concept mapping. For more diverse interests, the studies reviewed by individual members of our group demonstrate the impressive range of graphic organizers: Bera & Robinson (2008) pair graphic organizers with text summaries in an attempt to increase knowledge retention; Gonzalez et al. (2008) use a concept mapping strategy paired with a mediation protocol to enhance meaningful learning; Carlson, Chandler, & Sweller (2003) studied how graphic organizers may reduce cognitive load. In general, this bibliography spans am impressive range of topics and contains some of the most prominent names doing research in the area of graphic organizers today.
graphic organizer bibliography download
graphic organizer bibliography download
Implications
The Mautone and Mayer (2007) study suggests the potential to use structural and concrete graphic organizers to promote students’ comprehension of scientific graphs. Structural organizers should help students to understand how variables are related. In addition, there is the possibility that repeated use of concrete graphic organizers may promote causal understanding and integration of new knowledge into existing schemata. Theoretical implications suggest that signaling graphic organizers that focus on structure may help students to better understand relations in graphs. Structural graphic organizers can help students to better understand more complex relations between variables, such as curvilinear relationships.
The Ozmen (2009) study suggests the potential to use laboratory activities and concept mapping together to modify alternative conceptions that tend to be stable and resistant to correction by traditional teaching methods. Theoretical implications of this article involve possible synergistic effects between concept mapping and laboratory activities (and possibly other forms of experiential learning, group work, and project-based learning).
The Chiou, C. (2008) study indicates that concept mapping can be an effective meta-cognitive strategy to enhance meaningful learning in accounting and other related business courses. In domains that stress the incremental acquisition of knowledge (in particular math and science), concept mapping might be an effective instructional and summative tool by allowing corrective feedback during the course of instruction
The Gonzalez, H., Palencia, A., Umana, L., Galindo, L., Villafrade M,L., (2008)study suggests that concept mapping, combined with a mediated learning experience, can be an effective meta-cognitive strategy to enhance meaningful learning in a medical school setting.
One of the implications the Bahr, S & Dasereau,D. (2005) study includes using both BiK and monolingual knowledge maps in all kinds of academic settings in order to promote a shift toward a more beneficial top-down processing strategy among the learners.
The Bera and Robinson (2004) study suggests that future GO research further explore relations and interdependence between encoding efficiency and encoding distinctiveness, the two factors that can produce the efficient and effective comprehension of main ideas.
Practical implication for this study are likely to include the incorporation of Bera and Robinson efforts which call on educators to capitalize on students’ active involvement in the learning process into everyday in-class instruction and may lead to more content-appropriate handouts and displays in schools in all levels.
The Gardill and Jitendra(1999) study suggests that direct instruction in the use of story maps can lead to gains in comprehension with middle school students with learning disabilities. The number of instruction and practice sessions needed before mastery was achieved was up to 23, so repeated practice would be needed to effectively make use of this instructional strategy.
The Abi-El-Mona study lends support to the efficacy of using mind maps to support student learning in the domain of science instruction for middle school students. Furthermore, the gains were not mediated by students’ prior achievement.The results from the study contradict the theory of iconography’s connection with mind-mapping proposed by Buzan.
The Ozmen (2009) study suggests the potential to use laboratory activities and concept mapping together to modify alternative conceptions that tend to be stable and resistant to correction by traditional teaching methods. Theoretical implications of this article involve possible synergistic effects between concept mapping and laboratory activities (and possibly other forms of experiential learning, group work, and project-based learning).
The Chiou, C. (2008) study indicates that concept mapping can be an effective meta-cognitive strategy to enhance meaningful learning in accounting and other related business courses. In domains that stress the incremental acquisition of knowledge (in particular math and science), concept mapping might be an effective instructional and summative tool by allowing corrective feedback during the course of instruction
The Gonzalez, H., Palencia, A., Umana, L., Galindo, L., Villafrade M,L., (2008)study suggests that concept mapping, combined with a mediated learning experience, can be an effective meta-cognitive strategy to enhance meaningful learning in a medical school setting.
One of the implications the Bahr, S & Dasereau,D. (2005) study includes using both BiK and monolingual knowledge maps in all kinds of academic settings in order to promote a shift toward a more beneficial top-down processing strategy among the learners.
The Bera and Robinson (2004) study suggests that future GO research further explore relations and interdependence between encoding efficiency and encoding distinctiveness, the two factors that can produce the efficient and effective comprehension of main ideas.
Practical implication for this study are likely to include the incorporation of Bera and Robinson efforts which call on educators to capitalize on students’ active involvement in the learning process into everyday in-class instruction and may lead to more content-appropriate handouts and displays in schools in all levels.
The Gardill and Jitendra(1999) study suggests that direct instruction in the use of story maps can lead to gains in comprehension with middle school students with learning disabilities. The number of instruction and practice sessions needed before mastery was achieved was up to 23, so repeated practice would be needed to effectively make use of this instructional strategy.
The Abi-El-Mona study lends support to the efficacy of using mind maps to support student learning in the domain of science instruction for middle school students. Furthermore, the gains were not mediated by students’ prior achievement.The results from the study contradict the theory of iconography’s connection with mind-mapping proposed by Buzan.
Selection
These research articles selected and reviewed by the contributors of the Graphic Organizer Blog represents both a broad and a holistic overview of this particular segment of cognition research. With the aim to give the readers sufficient information and insight into the world of graphic organizers (GOs), we chose articles that examined the use of GOs in both hard and social science settings. As the titles of the articles are good indicator of the setting were the studies were performed, a decision was made to omit specific references hereafter.
We decided to highlight research that not only measured the effects of various types of GOs but also performed comparative analysis with other instructional methods currently in use in a typical classroom.
We also included research that either aimed to replicate or to confirm previous studies on GOs, something that is, in our opinion, not done with enough frequency in educational research.
Finally, we felt strongly about the need to show the versatility and diversity GOs can provide in terms of statement generation, mediated learning, and scaffolding for experts and learners alike in any instructional settings.
In our opinion, GOs offer enormous possibilities to improve human learning throughout the life stages by influencing instruction. We invite the reader to read additional articles from our bibliography to witness the amazing range of fields such as special education, science classrooms, medical schools, cancer patient advocacy and foreign language training where GO has been making a difference.
We decided to highlight research that not only measured the effects of various types of GOs but also performed comparative analysis with other instructional methods currently in use in a typical classroom.
We also included research that either aimed to replicate or to confirm previous studies on GOs, something that is, in our opinion, not done with enough frequency in educational research.
Finally, we felt strongly about the need to show the versatility and diversity GOs can provide in terms of statement generation, mediated learning, and scaffolding for experts and learners alike in any instructional settings.
In our opinion, GOs offer enormous possibilities to improve human learning throughout the life stages by influencing instruction. We invite the reader to read additional articles from our bibliography to witness the amazing range of fields such as special education, science classrooms, medical schools, cancer patient advocacy and foreign language training where GO has been making a difference.
Gardill, M. C., & Jitendra, A. K. (1999)
Gardill, M. C., & Jitendra, A. K. (1999). Advanced Story Map Instruction: Effects on the Reading Comprehension of Students with Learning Disabilities. Journal of Special Education, 33(1), 2-17,28.
Problem, Purpose and Research Questions:
The study being reviewed is entitled Advanced Story Map Instruction: Effects on the Reading Comprehension of Students with Learning Disabilities (Gardill & Jitendra, 1999). This study is a replication of work that was done with similarly disabled students (Gurrney et al. 1990). However, the researchers endeavored to improve on the research design and add an additional phase where generalization of the treatment was assessed.
Methods
The multiple baseline study design used assessments prior to the intervention, post measures, a time lapse and then a maintenance measure to see if gains in story comprehension continued after direct instruction and practice in story map construction was discontinued.
The study participants consisted of five males and one female. All students were Caucasian. One student also had a documented neurological disability. Pseudonyms were used throughout the study to relate details about each student’s performance throughout the study.
The researchers provided a table which included participant’s demographic and test data which included: age, grade, years in special education, performance on the full scale IQ assessment (WISC-R), and two subscale scores on the Woodcock Reading Mastery Test-Word Identification and Passage Comprehension.
The study researchers limited the participants to students who did not have severe reading delays so that reading ability would not be a confounding variable. The reading level criteria set up were that students had to score at least the fourth grade level on the word identification subtest of the Woodcock Reading Mastery Test – Revised (WRMT-Revised). In addition, to prove that they had difficulties with comprehension they had to have been so identified by their special education teacher as well as scoring at least two grades below their grade level on the reading comprehension subtest of the WRMT-Revised.
For the dependent variable, the researchers developed two comprehension quiz instruments. The first one focused on correctly identifying story grammar elements. These questions closely correlated with story map elements that the students complete with the story map form. The second, basal assessment instrument contained comprehension questions that combined literal and inferential questions. All questions required a production response of more than one word or phrase. Acceptable answers to these questions were developed so a scoring rubric was developed. Students also completed a basal story map with no teacher intervention except for “do your best.”
Students were given instruction on completing story map forms in a separate room in dyads, not in their classrooms. Teacher scripts were developed to control for some aspects of teacher interaction effects. The direct instruction sessions taught story map completion which included ascertaining the main idea, conflict and attempts at resolution. Students were first given sessions of direct instruction, then guided practice, and finally independent practice using a story map form that was developed by the researchers as a more advanced version of simple story maps. The intervention period for the dyads lasted from 14- to 20- weeks. The total time for the intervention phase lasted 21 to 27 hours (M=23.6 hr).
The story map form contained questions that were posted in a vertical portrait style format with horizontal lines provided for writing the answers to the following prompts: Student name, date, Story (Title), Name the problems or conflict. Identify the main characters and describe them. Where does the story take place? Tell how the characters try to solve the problem. Is there an added twist or complication in the story? Tell how the problem is or is not solved. What is the theme of the story? What is the author trying to say?
After students received instruction in story map completion, they participated in practice sessions until they reached the criteria for mastery- 80% on two out of three story grammar tests. A maintenance assessment of reading and comprehension questions was performed using another reading text after two weeks.
A verbal assessment of comprehension was incorporated into the study as well. It was in the form of story retelling where a teacher read a story to the student and then asked the student to retell the story in their own words. The rationale was put forward that students with learning disabilities often have problems expressing themselves via writing, so may not show their full subject knowledge through this means of communication. A rubric was set up to assess their verbal retelling of the text. Assessments were made before and after the direct instruction of story maps had completed. Each student was assessed separately, and retelling was audio taped and transcribed for later analysis.
Results
The study authors provided bar graphs showing pre, post, and maintenance scores for each student in the study.
The students also completed a student questionnaire in the form of a customer satisfaction survey. They were asked how they rated the story map procedure and whether or not they would advocate its use with their peers.
At baseline, the mean scores for story grammar quiz (story map completion) and basal comprehension were 30% and 39% respectively. During the teacher modeling phase, scores rose to a mean of 51% (range = 37% - 59%). For the independent phase, an average increase of 55% (range 40% - 67%) was made compared to baseline for the story grammar quizzes. A mean increase of 24% (range 5%- 26%) was made on the basal comprehension tests. For the generalization phase, the story grammar quiz mean score was 86% correct - an improvement of 56% over baseline. Whereas a mean increase of 16% was noted for the basal comprehension. In the maintenance phase, a mean increase over baseline of 19% (range 6% - 38%) was made on the story grammar quiz. A mean gain of 19% was also made on the basal comprehension test. With the exception of two students, literal and inferential comprehension scores rose for all other students.
For the oral retells, mean scores for the six students were not reported. Instead results were broken down by student. In comparing the number of words, the number of words increased for only two students. When analyzing the audio for story grammar elements, it was found that 5 of the 6 students showed increases in the number of story elements mentioned.
For the student questionnaire about the effectiveness of using story maps, the majority of the students chose the category often to describe how often they would use the technique (mean = 3 corresponding to “quite strongly”).
Discussion and Implications
The study researchers stated that using story maps helped students develop a framework to organize and remember story elements which in turn would help them develop greater comprehension. They point out that the mean performance of all six students improved from the baseline condition in basal comprehension. They noted the one female student’s performance decreasing after initial gains, though overall higher than baseline. She was also the student who was identified as having a neurological disorder. The researchers hypothesized that she became fatigued over the course of the many sessions.
The researchers listed some limitations that they saw in their study. First, they mentioned the small group of students in the study limits generalization. They listed the fact that intervention was provided in pairs where more individualized instruction could take place. Thirdly, they mentioned that students were given the intervention outside the classroom in a self contained environment, and suggested that studies performed in an inclusive environment are needed.
Critique
The two major problems with this study were two possible confounding variables the dyad instruction, which of course meant a much smaller teacher to student ratio, and the second being quiet environment with the concomitant lack of noise and distractions. These two variables could have accounted for comprehension gain and competed with the effectiveness of the independent variable under study – direct instruction in story map completion and all the sub-skills necessary for that independent skill.
Also, it would have been better if the results section listed quiz scores or z scores instead of improvement over baseline. The improvement over baseline, gave a “rosier” picture of gains made than would z scores. However the researchers did provide a plot graph for each student showing data points at each stage of the study for both story grammar quizzes and baseline comprehension tests. This graphic was useful in interpreting the results dispassionately.
Problem, Purpose and Research Questions:
The study being reviewed is entitled Advanced Story Map Instruction: Effects on the Reading Comprehension of Students with Learning Disabilities (Gardill & Jitendra, 1999). This study is a replication of work that was done with similarly disabled students (Gurrney et al. 1990). However, the researchers endeavored to improve on the research design and add an additional phase where generalization of the treatment was assessed.
Methods
The multiple baseline study design used assessments prior to the intervention, post measures, a time lapse and then a maintenance measure to see if gains in story comprehension continued after direct instruction and practice in story map construction was discontinued.
The study participants consisted of five males and one female. All students were Caucasian. One student also had a documented neurological disability. Pseudonyms were used throughout the study to relate details about each student’s performance throughout the study.
The researchers provided a table which included participant’s demographic and test data which included: age, grade, years in special education, performance on the full scale IQ assessment (WISC-R), and two subscale scores on the Woodcock Reading Mastery Test-Word Identification and Passage Comprehension.
The study researchers limited the participants to students who did not have severe reading delays so that reading ability would not be a confounding variable. The reading level criteria set up were that students had to score at least the fourth grade level on the word identification subtest of the Woodcock Reading Mastery Test – Revised (WRMT-Revised). In addition, to prove that they had difficulties with comprehension they had to have been so identified by their special education teacher as well as scoring at least two grades below their grade level on the reading comprehension subtest of the WRMT-Revised.
For the dependent variable, the researchers developed two comprehension quiz instruments. The first one focused on correctly identifying story grammar elements. These questions closely correlated with story map elements that the students complete with the story map form. The second, basal assessment instrument contained comprehension questions that combined literal and inferential questions. All questions required a production response of more than one word or phrase. Acceptable answers to these questions were developed so a scoring rubric was developed. Students also completed a basal story map with no teacher intervention except for “do your best.”
Students were given instruction on completing story map forms in a separate room in dyads, not in their classrooms. Teacher scripts were developed to control for some aspects of teacher interaction effects. The direct instruction sessions taught story map completion which included ascertaining the main idea, conflict and attempts at resolution. Students were first given sessions of direct instruction, then guided practice, and finally independent practice using a story map form that was developed by the researchers as a more advanced version of simple story maps. The intervention period for the dyads lasted from 14- to 20- weeks. The total time for the intervention phase lasted 21 to 27 hours (M=23.6 hr).
The story map form contained questions that were posted in a vertical portrait style format with horizontal lines provided for writing the answers to the following prompts: Student name, date, Story (Title), Name the problems or conflict. Identify the main characters and describe them. Where does the story take place? Tell how the characters try to solve the problem. Is there an added twist or complication in the story? Tell how the problem is or is not solved. What is the theme of the story? What is the author trying to say?
After students received instruction in story map completion, they participated in practice sessions until they reached the criteria for mastery- 80% on two out of three story grammar tests. A maintenance assessment of reading and comprehension questions was performed using another reading text after two weeks.
A verbal assessment of comprehension was incorporated into the study as well. It was in the form of story retelling where a teacher read a story to the student and then asked the student to retell the story in their own words. The rationale was put forward that students with learning disabilities often have problems expressing themselves via writing, so may not show their full subject knowledge through this means of communication. A rubric was set up to assess their verbal retelling of the text. Assessments were made before and after the direct instruction of story maps had completed. Each student was assessed separately, and retelling was audio taped and transcribed for later analysis.
Results
The study authors provided bar graphs showing pre, post, and maintenance scores for each student in the study.
The students also completed a student questionnaire in the form of a customer satisfaction survey. They were asked how they rated the story map procedure and whether or not they would advocate its use with their peers.
At baseline, the mean scores for story grammar quiz (story map completion) and basal comprehension were 30% and 39% respectively. During the teacher modeling phase, scores rose to a mean of 51% (range = 37% - 59%). For the independent phase, an average increase of 55% (range 40% - 67%) was made compared to baseline for the story grammar quizzes. A mean increase of 24% (range 5%- 26%) was made on the basal comprehension tests. For the generalization phase, the story grammar quiz mean score was 86% correct - an improvement of 56% over baseline. Whereas a mean increase of 16% was noted for the basal comprehension. In the maintenance phase, a mean increase over baseline of 19% (range 6% - 38%) was made on the story grammar quiz. A mean gain of 19% was also made on the basal comprehension test. With the exception of two students, literal and inferential comprehension scores rose for all other students.
For the oral retells, mean scores for the six students were not reported. Instead results were broken down by student. In comparing the number of words, the number of words increased for only two students. When analyzing the audio for story grammar elements, it was found that 5 of the 6 students showed increases in the number of story elements mentioned.
For the student questionnaire about the effectiveness of using story maps, the majority of the students chose the category often to describe how often they would use the technique (mean = 3 corresponding to “quite strongly”).
Discussion and Implications
The study researchers stated that using story maps helped students develop a framework to organize and remember story elements which in turn would help them develop greater comprehension. They point out that the mean performance of all six students improved from the baseline condition in basal comprehension. They noted the one female student’s performance decreasing after initial gains, though overall higher than baseline. She was also the student who was identified as having a neurological disorder. The researchers hypothesized that she became fatigued over the course of the many sessions.
The researchers listed some limitations that they saw in their study. First, they mentioned the small group of students in the study limits generalization. They listed the fact that intervention was provided in pairs where more individualized instruction could take place. Thirdly, they mentioned that students were given the intervention outside the classroom in a self contained environment, and suggested that studies performed in an inclusive environment are needed.
Critique
The two major problems with this study were two possible confounding variables the dyad instruction, which of course meant a much smaller teacher to student ratio, and the second being quiet environment with the concomitant lack of noise and distractions. These two variables could have accounted for comprehension gain and competed with the effectiveness of the independent variable under study – direct instruction in story map completion and all the sub-skills necessary for that independent skill.
Also, it would have been better if the results section listed quiz scores or z scores instead of improvement over baseline. The improvement over baseline, gave a “rosier” picture of gains made than would z scores. However the researchers did provide a plot graph for each student showing data points at each stage of the study for both story grammar quizzes and baseline comprehension tests. This graphic was useful in interpreting the results dispassionately.
Abi-El-Mona, I., & Adb-El-Khalick, F. (2008)
Abi-El-Mona, I., & Adb-El-Khalick, F. (2008). The Influence of Mind Mapping on Eighth Graders' Science Achievement. [Feature]. School Science and Mathematics, 108(7), 298-312.
Purpose, Problem and Research Questions
The problem was not directly stated in the article, however, it can be inferred. The integration of science concepts including reasoning and clear concept understanding can be improved in science instruction. To this end, the purpose of this study was to ascertain the effectiveness of students generating their own mind maps after a lecture and seeing if it increased their concept knowledge. Students with different initial achievement levels were used to see if the effectiveness was differentiated by different levels of prior knowledge or achievement.
Mind maps were defined as different from concept maps in that although they both share many attributes , ie both are graphic organizers with a web-like structure and conceptual links in a non-linear fashion, mind maps inspire more personal connections, are not necessarily hierarchical, use color and drawings or icons, and sometimes add a legend.
Methods
The study design was a 2x3 factorial study with post test only. The independent variable was the use of mind mapping in a classroom setting. There were two levels: use and non-use. The second factor was a moderator variable. It was prior academic achievement in science with three levels (high, medium, and low). The dependent variable was post instruction achievement.
The population for the study was a private middle school in a Middle Eastern country. All classes were taught in English. The study participants were 62 eighth grade students placed in four intact classrooms. The teacher/instructor for the both the control group and the intervention group was the same (one of the researchers). The instructional method during the lecture portion remained constant as well. Two class sections were randomly assigned to each experimental condition.
For the intervention group, mind mapping instruction took place over the period of one month. The control group was instructed in a note summarization procedure so that time of intervention for both conditions would be kept constant. During the intervention phase the classrooms either spent the last 10 minutes forming mind maps or doing note summarization .
Post instruction achievement was performed using a test instrument developed by both researchers. The test items were 30 multiple choice questions, which targeted NAEP science standards. These same standards were used in the curriculum development for the lecture part of the course. Domains fell across two categories- conceptual understanding and practical reasoning. The test item pool was examined by four secondary science teachers to ensure validity of the instrument. Some items were changed, added, or deleted based on the feedback that was collected.
Results
Two multivariate analyses of variance (MANOVA) were performed on the test results, with the factors under consideration, use/non use of the mind maps, and the three levels or prior science achievement. Another MANOVA was conducted with two categories of achievement –conceptual understanding and practical reasoning. When results were significant , a univariate analysis followed the MANOVA.
In addition a rubric was developed by the two researchers to analyze the mind maps that the students had created. Items such as number of links, the map geography, the use of colors, the use of drawings, the use of icons, presence of a central theme, and links to major concepts.
Mind maps were also analyzed in a historical fashion. Mind maps created at the beginning of the semester were compared to mind maps created later,
The test using the 3 levels of prior achievement, - (PSA) was statiscally significant for T (p>.05). In addition. Students scored higher on both conceptual understanding test items and practical reasoning items(M-exp = 72.9, M – comp=57.53, p <001) .
Discussion
The researchers listed one limitation of the study. That this is only one study and needs to be replicated in other settings. The researcher spent time going into the differences in design of mind maps, comparing maps high scoring, medium scoring and lower scoring students. High scoring students individualized their maps, used color, seldom used icons (contradicting another mind map researcher) and had few but correct links to other major topics.
Implications
The main implication is that creating mind maps to consolidate and organize knowledge may be a useful tool for students to enhance their learning. More research should be done in this promising endeavor, especially because it helps to personalize information for the learner in meaningful ways,and in turn, raising student achievement.
Critique
This article, despite the statistical complexities, was very well organized and presented. Ideas were clear, even though few diagrams were used. The only exception is the Vee maps.
Effect strength was not computed for the significant statistics. This would have been helpful for interpretation of the results importance to other researchers
In terms of ethical issues, the researchers don't indicate how securely they kept the testing materials nor the mind maps that they collected at the end of each session. Also, since the authors ascertained that mind maps were beneficial to student learning, they should have offered some way to offer that treatment/method of instruction to the students who missed out - the control group. Since each mind mapping exercise occurred directly after instruction, either the instruction would need to be repeated which is unlikely, or from that point on, all students would be given assignments with mind mapping after instruction.
Purpose, Problem and Research Questions
The problem was not directly stated in the article, however, it can be inferred. The integration of science concepts including reasoning and clear concept understanding can be improved in science instruction. To this end, the purpose of this study was to ascertain the effectiveness of students generating their own mind maps after a lecture and seeing if it increased their concept knowledge. Students with different initial achievement levels were used to see if the effectiveness was differentiated by different levels of prior knowledge or achievement.
Mind maps were defined as different from concept maps in that although they both share many attributes , ie both are graphic organizers with a web-like structure and conceptual links in a non-linear fashion, mind maps inspire more personal connections, are not necessarily hierarchical, use color and drawings or icons, and sometimes add a legend.
Methods
The study design was a 2x3 factorial study with post test only. The independent variable was the use of mind mapping in a classroom setting. There were two levels: use and non-use. The second factor was a moderator variable. It was prior academic achievement in science with three levels (high, medium, and low). The dependent variable was post instruction achievement.
The population for the study was a private middle school in a Middle Eastern country. All classes were taught in English. The study participants were 62 eighth grade students placed in four intact classrooms. The teacher/instructor for the both the control group and the intervention group was the same (one of the researchers). The instructional method during the lecture portion remained constant as well. Two class sections were randomly assigned to each experimental condition.
For the intervention group, mind mapping instruction took place over the period of one month. The control group was instructed in a note summarization procedure so that time of intervention for both conditions would be kept constant. During the intervention phase the classrooms either spent the last 10 minutes forming mind maps or doing note summarization .
Post instruction achievement was performed using a test instrument developed by both researchers. The test items were 30 multiple choice questions, which targeted NAEP science standards. These same standards were used in the curriculum development for the lecture part of the course. Domains fell across two categories- conceptual understanding and practical reasoning. The test item pool was examined by four secondary science teachers to ensure validity of the instrument. Some items were changed, added, or deleted based on the feedback that was collected.
Results
Two multivariate analyses of variance (MANOVA) were performed on the test results, with the factors under consideration, use/non use of the mind maps, and the three levels or prior science achievement. Another MANOVA was conducted with two categories of achievement –conceptual understanding and practical reasoning. When results were significant , a univariate analysis followed the MANOVA.
In addition a rubric was developed by the two researchers to analyze the mind maps that the students had created. Items such as number of links, the map geography, the use of colors, the use of drawings, the use of icons, presence of a central theme, and links to major concepts.
Mind maps were also analyzed in a historical fashion. Mind maps created at the beginning of the semester were compared to mind maps created later,
The test using the 3 levels of prior achievement, - (PSA) was statiscally significant for T (p>.05). In addition. Students scored higher on both conceptual understanding test items and practical reasoning items(M-exp = 72.9, M – comp=57.53, p <001) .
Discussion
The researchers listed one limitation of the study. That this is only one study and needs to be replicated in other settings. The researcher spent time going into the differences in design of mind maps, comparing maps high scoring, medium scoring and lower scoring students. High scoring students individualized their maps, used color, seldom used icons (contradicting another mind map researcher) and had few but correct links to other major topics.
Implications
The main implication is that creating mind maps to consolidate and organize knowledge may be a useful tool for students to enhance their learning. More research should be done in this promising endeavor, especially because it helps to personalize information for the learner in meaningful ways,and in turn, raising student achievement.
Critique
This article, despite the statistical complexities, was very well organized and presented. Ideas were clear, even though few diagrams were used. The only exception is the Vee maps.
Effect strength was not computed for the significant statistics. This would have been helpful for interpretation of the results importance to other researchers
In terms of ethical issues, the researchers don't indicate how securely they kept the testing materials nor the mind maps that they collected at the end of each session. Also, since the authors ascertained that mind maps were beneficial to student learning, they should have offered some way to offer that treatment/method of instruction to the students who missed out - the control group. Since each mind mapping exercise occurred directly after instruction, either the instruction would need to be repeated which is unlikely, or from that point on, all students would be given assignments with mind mapping after instruction.
Gonzalez et al. (2008)
Gonzalez, H., Palencia, A., Umana, L., Galindo, L., Villafrade M, L., (2008) Mediated learning experience and concept maps: A pedagogical tool for achieving meaningful learning in medical physiology students. Adv Physiol Educ. 32, 312-316.
Purpose/Problem
The research problem addressed in this experimental study is whether a concept mapping methodology, combined with a mediated learning experience, increases meaningful learning in students attending a cardiovascular module of a medical physiology course. Gonzalez, Palencia, Umana, Galindo and Villafrade define meaningful learning as 1) students being able to predict or explain the responses of the physiological system when stressed, and 2) students being able to solve quantitative problems related to the physiological system. Gonzalez et al. define mediated learning as the interaction between the learner and a mediator where the mediator is primarily concerned with how the learner approaches problem solving. Specifically, Gonzalez et al. utilized the model of mediation described by R. Feurstein. In this model, the mediator involves herself in the student learning process not as a traditional teacher, but as a type of learning coach. Gonzalez et al. describe four qualities that must be present in the mediator-student learner interaction: 1) intentionality, where the mediator concentrates on helping the learner understand how he or she is using their cognitive faculties; 2) reciprocity, where the mediator and the learner consider one another as being co-equals in the learning process; 3) mediation of meaning, where the mediator interprets for the learner the significance of what the learner has accomplished; 4) transcendence, where the mediator helps the learner apply the lessons learned to new situations.
This study is noteworthy because the use of concept mapping, combined with a meditated learning experience, could prove to be a powerful pedagogical tool in a medical physiology class as well as other medical school courses. Gonzalez et al. ground their study by citing well-established research establishing the efficacy of concept mapping to promote meaningful learning as well as research describing the effectiveness of the mediated learning experience. The researchers note that the literature points out the need for more effective pedagogical models to help medical students select, integrate, and transfer new learning, and therefore, reach a meaningful learning threshold.
Methods
The participants in this study were 121 medical students in their third semester of medical school in Columbia. At the beginning of the study, each student was assigned an index score based on: 1) their total score on a national high school graduate exam; 2) their cumulative average medical school grades from the previous two semesters, and 3) their grade obtained on a construction of a conceptual mentifact – a graphical instrument that represents thoughts and values – used to define how knowledge is organized and represented in the human mind. Depending on the index score, the students were divided into quartile groups: quartile one were students in the lowest 25 percent of the index score while quartile four were students who obtained the highest index scores or the top 25 percent. Students in quartile four were deemed the most competent in their medical studies while students in quartile one was deemed the least competent. Students in each quartile were randomly assigned to the control group or to the intervention group. In order to increase the possibility of finding significant differences between groups, Gonzalez et al. established a 2:1 ratio of intervention students to control students.
Students in the experimental group attended four 2-hour mediated sessions where mediators helped the students develop individualized approaches to studying the cardiovascular module. Students in the experimental group also received reading material from the cardiovascular module one week prior to the first session. At the beginning of each mediated session, the mediators, who were professors at the medical school, briefly discussed the information contained in the reading material. The mediators also explained concept mapping construction. After the explanations, each subgroup was given two sets of cards: one with propositions reflecting concepts; and the other with definitions of the concepts. Students in the experimental group were then asked to prepare concept maps linking propositions and definitions contained in the reading material. Mediators observed the cognitive performance of the students during the concept mapping activity and helped them to analyze and correct any difficulties that arose in organizing the concepts. The students in the experimental group then corrected their own maps.
Students in the control group also attended four 2-hour sessions conducted by the medical school professors and received the same reading material from the cardiovascular module. In explaining the reading material, the professors did not act as mediators but assumed more traditional roles as teachers. The pedagogical methods used with the control group were ones that were traditionally used at the medical school; teachers asked questions and students answered.
During each session, two researchers acted as observers and recorded the academic, cognitive, attitudinal, and behavioral performances of students in both the control and experimental groups, the mediators in the experimental group, and the professors in the control group. After completion of the four sessions of the cardiovascular module, the control and experimental groups took two exams to evaluate student learning and achievement. One exam consisted of multiple choice questions assessing the student recall of the information in the cardiovascular module. The second exam consisted of problem-solving scenarios assessing the students’ ability to transfer information learned during the module to new situations. Additionally, at the end of the cardiovascular module, both groups were given self-evaluation instruments requiring them to assess any perceived changes in their cognitive processes.
The experimental and control groups were compared using the following two variables: 1) average grades on the two post-sessions exams; and 2) the percentage of students who failed the exams (on a scale from 1.0 to 5.0, any student scoring below 3.0). Averages (means) and SDs were used for the continuous variables and percentages were used for discrete variables. Differences between the control and experimental groups were determined by a t-test (a comparison of the two means) and a Mann-Whitney test (p < 0.05).
Results
Gonzalez et al. report that mean scores from the problem-solving exam were higher in the experimental group (N = 83, M = 3.68) than the control group (N = 39, M = 2.79) achieving a statistical significance of p = 0.0013. When student performance was compared using the distribution by quartiles, the average scores on the problem-solving exam were higher for the experimental group than for the control group but the difference in scores were only statistically significant for the students in quartile one (p = 0.008). The average score on the multiple-choice exam was higher for the experimental group in all quartiles with the exception of quartile one.
Qualitative data detailing the effectiveness of mediation was gathered from the self-evaluations of students from both groups and from the researchers who acted as observers during concept mapping activity. Students in the experimental group reported that mediation during the concept mapping activity helped them develop more effective learning strategies by being made more aware of their strengths and weaknesses while organizing information. The experimental group also reported that being given a more active role in their learning was very motivating. These types of remarks were reported less frequently by the control group students.
Discussion
The main purpose of this study was to examine whether a combination of mediation and a concept mapping strategy could increase meaningful learning in medical students in a medical school setting. The results of the study showed that students in the experimental group improved their learning achievement more than the students in the control group. Gonzalez et al. note that these findings are in agreement with findings in the literature.
Gonzalez et al. also note that the statistically significant differences found between the experimental and control groups on the problem-solving exams and not on the multiple-choice exams are not unexpected. Gonzalez et al. posit that multiple-choice exams to not challenge student metacognitive abilities in the same way as problem-solving exams.
Gonzalez et al. also observe that although the experimental group globally benefitted from the implemented strategy, results were greatest in students in the lowest quartile. Gonzalez et al. speculate that the intervention may have stimulated development of cognitive processes that were previously underutilized.
Gonzalez et al. suggest that the qualitative analysis shows that the combination of mediation and concept mapping increases student motivation and stimulates students to participate more actively in their own learning.
Critique
In general, this study employs sound methodological and statistical techniques and represents a well-thought out and controlled study. The writing is clear, appropriately formal, direct, well organized, and factual. The study is well grounded in previous research and makes an important contribution to the literature. Sample size is sufficient (N = 122) to yield generalizable results.
The study could have been strengthened by inclusion of a third experimental group using mediation without the concept mapping strategy, or the concept mapping strategy without the mediation intervention. Since the experimental group showed improvement over the control group by a combination of mediation and concept mapping protocols, it is difficult to determine whether the combination of the interventions is effective or if one intervention contributed to the majority of the improved effect.
Theoretical, Methodological, and Practical Implications
This study indicates that concept mapping, combined with a mediated learning experience, can be an effective meta-cognitive strategy to enhance meaningful learning in a medical school setting. In domains that stress problem-solving and analysis, the combination of concept mapping and mediation might prove to be a powerful pedagogical tool allowing students to maximize meaningful learning. Given the unique context of combining mediation and concept mapping, this study makes an important contribution to the corpus of literature.
In addition, this study indicates that concept mapping combined with mediation encourages more efficient learning skills by allowing these medical students to reflect on their own cognitive processes and adjusting practices to maximize meaningful learning. This change in emphasis from traditional teaching methodologies to more mediated learning experiences seem to enhance meaningful learning in complex domains.
Purpose/Problem
The research problem addressed in this experimental study is whether a concept mapping methodology, combined with a mediated learning experience, increases meaningful learning in students attending a cardiovascular module of a medical physiology course. Gonzalez, Palencia, Umana, Galindo and Villafrade define meaningful learning as 1) students being able to predict or explain the responses of the physiological system when stressed, and 2) students being able to solve quantitative problems related to the physiological system. Gonzalez et al. define mediated learning as the interaction between the learner and a mediator where the mediator is primarily concerned with how the learner approaches problem solving. Specifically, Gonzalez et al. utilized the model of mediation described by R. Feurstein. In this model, the mediator involves herself in the student learning process not as a traditional teacher, but as a type of learning coach. Gonzalez et al. describe four qualities that must be present in the mediator-student learner interaction: 1) intentionality, where the mediator concentrates on helping the learner understand how he or she is using their cognitive faculties; 2) reciprocity, where the mediator and the learner consider one another as being co-equals in the learning process; 3) mediation of meaning, where the mediator interprets for the learner the significance of what the learner has accomplished; 4) transcendence, where the mediator helps the learner apply the lessons learned to new situations.
This study is noteworthy because the use of concept mapping, combined with a meditated learning experience, could prove to be a powerful pedagogical tool in a medical physiology class as well as other medical school courses. Gonzalez et al. ground their study by citing well-established research establishing the efficacy of concept mapping to promote meaningful learning as well as research describing the effectiveness of the mediated learning experience. The researchers note that the literature points out the need for more effective pedagogical models to help medical students select, integrate, and transfer new learning, and therefore, reach a meaningful learning threshold.
Methods
The participants in this study were 121 medical students in their third semester of medical school in Columbia. At the beginning of the study, each student was assigned an index score based on: 1) their total score on a national high school graduate exam; 2) their cumulative average medical school grades from the previous two semesters, and 3) their grade obtained on a construction of a conceptual mentifact – a graphical instrument that represents thoughts and values – used to define how knowledge is organized and represented in the human mind. Depending on the index score, the students were divided into quartile groups: quartile one were students in the lowest 25 percent of the index score while quartile four were students who obtained the highest index scores or the top 25 percent. Students in quartile four were deemed the most competent in their medical studies while students in quartile one was deemed the least competent. Students in each quartile were randomly assigned to the control group or to the intervention group. In order to increase the possibility of finding significant differences between groups, Gonzalez et al. established a 2:1 ratio of intervention students to control students.
Students in the experimental group attended four 2-hour mediated sessions where mediators helped the students develop individualized approaches to studying the cardiovascular module. Students in the experimental group also received reading material from the cardiovascular module one week prior to the first session. At the beginning of each mediated session, the mediators, who were professors at the medical school, briefly discussed the information contained in the reading material. The mediators also explained concept mapping construction. After the explanations, each subgroup was given two sets of cards: one with propositions reflecting concepts; and the other with definitions of the concepts. Students in the experimental group were then asked to prepare concept maps linking propositions and definitions contained in the reading material. Mediators observed the cognitive performance of the students during the concept mapping activity and helped them to analyze and correct any difficulties that arose in organizing the concepts. The students in the experimental group then corrected their own maps.
Students in the control group also attended four 2-hour sessions conducted by the medical school professors and received the same reading material from the cardiovascular module. In explaining the reading material, the professors did not act as mediators but assumed more traditional roles as teachers. The pedagogical methods used with the control group were ones that were traditionally used at the medical school; teachers asked questions and students answered.
During each session, two researchers acted as observers and recorded the academic, cognitive, attitudinal, and behavioral performances of students in both the control and experimental groups, the mediators in the experimental group, and the professors in the control group. After completion of the four sessions of the cardiovascular module, the control and experimental groups took two exams to evaluate student learning and achievement. One exam consisted of multiple choice questions assessing the student recall of the information in the cardiovascular module. The second exam consisted of problem-solving scenarios assessing the students’ ability to transfer information learned during the module to new situations. Additionally, at the end of the cardiovascular module, both groups were given self-evaluation instruments requiring them to assess any perceived changes in their cognitive processes.
The experimental and control groups were compared using the following two variables: 1) average grades on the two post-sessions exams; and 2) the percentage of students who failed the exams (on a scale from 1.0 to 5.0, any student scoring below 3.0). Averages (means) and SDs were used for the continuous variables and percentages were used for discrete variables. Differences between the control and experimental groups were determined by a t-test (a comparison of the two means) and a Mann-Whitney test (p < 0.05).
Results
Gonzalez et al. report that mean scores from the problem-solving exam were higher in the experimental group (N = 83, M = 3.68) than the control group (N = 39, M = 2.79) achieving a statistical significance of p = 0.0013. When student performance was compared using the distribution by quartiles, the average scores on the problem-solving exam were higher for the experimental group than for the control group but the difference in scores were only statistically significant for the students in quartile one (p = 0.008). The average score on the multiple-choice exam was higher for the experimental group in all quartiles with the exception of quartile one.
Qualitative data detailing the effectiveness of mediation was gathered from the self-evaluations of students from both groups and from the researchers who acted as observers during concept mapping activity. Students in the experimental group reported that mediation during the concept mapping activity helped them develop more effective learning strategies by being made more aware of their strengths and weaknesses while organizing information. The experimental group also reported that being given a more active role in their learning was very motivating. These types of remarks were reported less frequently by the control group students.
Discussion
The main purpose of this study was to examine whether a combination of mediation and a concept mapping strategy could increase meaningful learning in medical students in a medical school setting. The results of the study showed that students in the experimental group improved their learning achievement more than the students in the control group. Gonzalez et al. note that these findings are in agreement with findings in the literature.
Gonzalez et al. also note that the statistically significant differences found between the experimental and control groups on the problem-solving exams and not on the multiple-choice exams are not unexpected. Gonzalez et al. posit that multiple-choice exams to not challenge student metacognitive abilities in the same way as problem-solving exams.
Gonzalez et al. also observe that although the experimental group globally benefitted from the implemented strategy, results were greatest in students in the lowest quartile. Gonzalez et al. speculate that the intervention may have stimulated development of cognitive processes that were previously underutilized.
Gonzalez et al. suggest that the qualitative analysis shows that the combination of mediation and concept mapping increases student motivation and stimulates students to participate more actively in their own learning.
Critique
In general, this study employs sound methodological and statistical techniques and represents a well-thought out and controlled study. The writing is clear, appropriately formal, direct, well organized, and factual. The study is well grounded in previous research and makes an important contribution to the literature. Sample size is sufficient (N = 122) to yield generalizable results.
The study could have been strengthened by inclusion of a third experimental group using mediation without the concept mapping strategy, or the concept mapping strategy without the mediation intervention. Since the experimental group showed improvement over the control group by a combination of mediation and concept mapping protocols, it is difficult to determine whether the combination of the interventions is effective or if one intervention contributed to the majority of the improved effect.
Theoretical, Methodological, and Practical Implications
This study indicates that concept mapping, combined with a mediated learning experience, can be an effective meta-cognitive strategy to enhance meaningful learning in a medical school setting. In domains that stress problem-solving and analysis, the combination of concept mapping and mediation might prove to be a powerful pedagogical tool allowing students to maximize meaningful learning. Given the unique context of combining mediation and concept mapping, this study makes an important contribution to the corpus of literature.
In addition, this study indicates that concept mapping combined with mediation encourages more efficient learning skills by allowing these medical students to reflect on their own cognitive processes and adjusting practices to maximize meaningful learning. This change in emphasis from traditional teaching methodologies to more mediated learning experiences seem to enhance meaningful learning in complex domains.
Friday, May 1, 2009
Mautone and Mayer (2007)
Mautone, P. D. & Mayer, R. (2007). Cognitive aids for guiding graph comprehension. Journal of Educational Psychology, 99, 640-652.
Problem/Purpose
The researchers in this study examined how three different types of graphic organizers improved student comprehension of scientific graphs compared to students that were not exposed to graphic organizers at all. Mautone and Mayer examined the effects of signaling, concrete, and structural graphic organizers on students’ generation of causal and relational statements. Signaling organizers help students to understand the structure of graphs by using highlights, breakdowns, and other tools to convey graph structure. Concrete organizers help students to integrate graphical information through the use of images to connect variables and data in graphs to students’ existing schemata. Structural organizers help students to understand how patterns between new variables work by comparing them to similar patterns followed by other more familiar variables.
This study is of interest because it expands the understanding of how graphic organizers affect students’ comprehension and more specifically, how different kinds of graphic organizers affect different aspects of comprehension in different ways. From a theoretical perspective, this expansion provides data that may help construct a more detailed understanding of cognition. From a practical perspective, the expansion and application of graphic organizers can provide means for improving students’ comprehension of scientific graphs, especially in understanding causal and relational relationships between and among variables
Methods
Mautone and Mayer conducted three experiments.
In the first experiment, 60 students were divided into a control group and a cognitive aids group. Both groups were allowed to view self-paced PowerPoint presentations focused on four graphs derived from introductory college geography texts. The four graphs were each different in type, but all relatively complex and all depicted different aspects of common fluvial processes associated with rivers and streams and the deposits and landforms created by them. After studying the graph slide at their own pace, the control group students were prompted to write a summary of each slide for five minutes. This process was repeated for each of the four slides. The cognitive aids group in experiment one was given a modified version of the same PowerPoint presentation where the graphs were broken apart into multiple slides containing highlights, illustrations, and breakdowns that signaled the structure of the graphs. Students paged through the multiple slides for each graph at their own pace. After viewing the full graph and its preceding cognitive aids, students were then prompted to write a summary of the slide for five minutes. Student responses for both groups were scored based on the number of relational and causal statements. Separate scores were recorded for the number of relational statements and the number of causal statements. Inter-rate reliability was measured to be r=0.91 and r=0.82 and deemed to be high.
Because students in the cognitive aids group had more study time than the control group, Mautone and Mayer conducted multiple regression analyses with time and condition, finding no interaction. Analyses were also conducted using time as a covariate and no differences were found.
To separate the effects of signaling and graphic organizers, Mautone and Mayer conducted a second experiment with a 2x2 design with signaling and concrete organizers present and not present. The procedure for the second experiment was similar to experiment one except that there were four groups: control, signaling, concrete, and signaling with concrete. Scoring was done similarly to experiment one including follow-up analyses to determine that time spent studying was not a critical factor in between group differences.
Mautone and Mayer followed up with a third experiment to test the effects of structural graphic organizers that are intended to prime cognitive organizing and should increase the number of students’ relational statements and not increase the number of students’ causal statements. Again, students were divided into two groups: control and structural. The same graphs were used and the structural group viewed a self-paced PowerPoint embedded with structural cognitive aids. Student statements in the graph summaries were scored as before and time spent studying was analyzed to verify again that time was not a critical factor in the between group differences.
Results
In experiment one, Mautone and Mayer found statistically and practically significant increases in the number of causal and relational statements generated by students in the cognitive aids group. Cognitive aids group students generated an average of 5.3 relational statements (SD=2.19) while control group students generated an average of 3.6 relational statements (SD=1.99). The difference between groups was t(58)=2.96, p=0.004 and Cohen’s d was a medium-to-large 0.76. Cognitive aids group students generated an average of 1.1 causal statements (SD=0.88) while control group students generated an average of 0.4 relational statements (SD=0.62). The difference between groups was t(58)=3.55, p=0.001 and Cohen’s d was a large 0.93. Effects of time spent studying were not statistically significant.
In experiment two, Mautone and Mayer found a significant main effect of signaling graphic organizers on relational statements. Students who were in signaling groups generated an average of 5.14 relational statements (SD=1.90) while students in the other groups generated an average of 4.08 relational statements (SD=1.64). The difference between groups was F(1,98)=9.02, p=0.03 and Cohen’s d was a medium 0.60. There was no main effect for the presence of concrete graphic organizers on relational statements and there was no interaction effect between the presence of concrete and signaling graphic organizers on relational statements. Effects of time spent studying were not statistically significant.
Also in experiment two, Mautone and Mayer found a significant main effect of concrete graphic organizers on causal statements. Students who were in concrete groups generated an average of 1.02 causal statements (SD=0.96) while students in the other groups generated an average of 0.65 causal statements (SD=0.90). The difference between groups was F(1,98)=3.86, p=0.05 and Cohen’s d was a small-to-medium 0.39. There was no main effect for the presence of signaling graphic organizers on causal statements and there was no interaction effect between the presence of concrete and signaling graphic organizers on causal statements. Effects of time spent studying were not statistically significant.
In experiment three, Mautone and Mayer found statistically and practically significant increases in the number of relational statements generated by students in the structural graphic organizers group. Structural group students generated an average of 4.92 relational statements (SD=1.78) while control group students generated an average of 3.00 relational statements (SD=1.69). The difference between groups was t(45)=3.78, p<0.001 and Cohen’s d was a large 1.10. There was no significant difference between groups in the number of causal statements generated. Effects of time spent studying were not statistically significant.
Discussion
Mautone and Mayer conducted three experiments that demonstrate how different types of graphic organizers can improve students’ understanding of scientific graphs in different ways.
The differences in the three types of graphic organizers are not entirely obvious at first, but Mautone and Mayer provide clear descriptions and clarifying examples to explain the types and how they are intended to work. The signaling organizers help students to understand how variables are represented on graphs and how the different pieces of graphs are put together. Especially valuable in understanding signaling organizers is the idea of presenting partial graphs so that students can understand how fundamental parts of a graph are structured relative to each other before additional graph elements are added. Concrete organizers help students related the information on graphs to concepts and knowledge they already possess by using pictures, photographs, and diagrams to illustrate variables and their different values.
Structural organizers are perhaps the most sophisticated because they help students to understand the relationships between new variables by using other relationships the students already understand to help students understand the new relationship. Mautone and Mayer described a clever way to explain curvilinear relationships by using age and driving ability. Extremely young and old drivers have higher rates of accidents that those drivers who are closer to middle age. This structural organizer is used to then explain the curvilinear relationship between river water velocity and particle size with respect to erosion. Brilliant!
Mautone and Mayer addressed the major limitation of this study in their post-experiment analyses on the effects of time. In all three experiments, time was found to not be a statistically significant factor in between group differences.
Critique
Mautone and Mayer use clear language and provide concrete examples that are especially useful for distinguishing the three types of graphic organizers. Tables are clear, well organized and well placed within the text. The descriptions of methodology are detailed and thorough without being repetitive or redundant. Further, the follow-up analyses of time as an independent variable are thoughtful and are summarized clearly and concisely. All pertinent statistical data (means, standard deviations, effect sizes, p values and statistics) are provided clearly. The use of three experiments also provides evidence that the results are replicable, a commonly overlooked aspect of education research studies.
Theoretical, Methodological, and Practical Implications
A major practical implication of this article is the potential to use structural and concrete graphic organizers to promote students’ comprehension of scientific graphs. Structural organizers should help students to understand how variables are related. It would be interesting to conduct further studies to see if extended use of structural graphic organizers led to successful completion of far transfer tasks where new variables that have similar relationships can be understood without introduction using the graphic organizers.
As well, there is a possibility that repeated use of concrete graphic organizers may promote causal understanding and integration of new knowledge into existing schemata. Again, there may be a way that repeated use of concrete organizers could lead to far transfer ability and more generalized use of integration in novel situations without explicit use of concrete organizers. More research on these possible outcomes is needed.
Mautone and Mayer’s study extends and adds to theoretical understanding of cognition and graphic organizers by providing additional levels of detail. Signaling graphic organizers that focus on structure may help students to better understand relations in graphs and between variables. Concrete graphic organizers may help students to better understand causal relations in graphs and between variables. Structural graphic organizers can help students to better understand more complex relations between variables, such as curvilinear relationships. Graphic organizers may include both concrete and signaling design principles and thereby statistically increase students understanding of relations and integration of new knowledge as shown by increased numbers of accurate causal and relational statements. Mautone and Mayer’s work indicates that structure and integration may be independent cognitive processes.
Mautone and Mayer used a three-experiment approach with self-paced study time and timed responses. The use of replication and variation within and between the three experiments allows their method to demonstrate that their results with graphic organizers are reproducible and to examine details of graphic organizer and cognition theory through variations in the graphic organizers studied. Other research studies may benefit by following similar replication and variation strategies where multiple experiments are combined into one study. Pilot tests are commonly used in studies, but Mautone and Mayer went much further by using three full-scale experiments and not a mini-experiment as an antecedent to the main experiment.
Problem/Purpose
The researchers in this study examined how three different types of graphic organizers improved student comprehension of scientific graphs compared to students that were not exposed to graphic organizers at all. Mautone and Mayer examined the effects of signaling, concrete, and structural graphic organizers on students’ generation of causal and relational statements. Signaling organizers help students to understand the structure of graphs by using highlights, breakdowns, and other tools to convey graph structure. Concrete organizers help students to integrate graphical information through the use of images to connect variables and data in graphs to students’ existing schemata. Structural organizers help students to understand how patterns between new variables work by comparing them to similar patterns followed by other more familiar variables.
This study is of interest because it expands the understanding of how graphic organizers affect students’ comprehension and more specifically, how different kinds of graphic organizers affect different aspects of comprehension in different ways. From a theoretical perspective, this expansion provides data that may help construct a more detailed understanding of cognition. From a practical perspective, the expansion and application of graphic organizers can provide means for improving students’ comprehension of scientific graphs, especially in understanding causal and relational relationships between and among variables
Methods
Mautone and Mayer conducted three experiments.
In the first experiment, 60 students were divided into a control group and a cognitive aids group. Both groups were allowed to view self-paced PowerPoint presentations focused on four graphs derived from introductory college geography texts. The four graphs were each different in type, but all relatively complex and all depicted different aspects of common fluvial processes associated with rivers and streams and the deposits and landforms created by them. After studying the graph slide at their own pace, the control group students were prompted to write a summary of each slide for five minutes. This process was repeated for each of the four slides. The cognitive aids group in experiment one was given a modified version of the same PowerPoint presentation where the graphs were broken apart into multiple slides containing highlights, illustrations, and breakdowns that signaled the structure of the graphs. Students paged through the multiple slides for each graph at their own pace. After viewing the full graph and its preceding cognitive aids, students were then prompted to write a summary of the slide for five minutes. Student responses for both groups were scored based on the number of relational and causal statements. Separate scores were recorded for the number of relational statements and the number of causal statements. Inter-rate reliability was measured to be r=0.91 and r=0.82 and deemed to be high.
Because students in the cognitive aids group had more study time than the control group, Mautone and Mayer conducted multiple regression analyses with time and condition, finding no interaction. Analyses were also conducted using time as a covariate and no differences were found.
To separate the effects of signaling and graphic organizers, Mautone and Mayer conducted a second experiment with a 2x2 design with signaling and concrete organizers present and not present. The procedure for the second experiment was similar to experiment one except that there were four groups: control, signaling, concrete, and signaling with concrete. Scoring was done similarly to experiment one including follow-up analyses to determine that time spent studying was not a critical factor in between group differences.
Mautone and Mayer followed up with a third experiment to test the effects of structural graphic organizers that are intended to prime cognitive organizing and should increase the number of students’ relational statements and not increase the number of students’ causal statements. Again, students were divided into two groups: control and structural. The same graphs were used and the structural group viewed a self-paced PowerPoint embedded with structural cognitive aids. Student statements in the graph summaries were scored as before and time spent studying was analyzed to verify again that time was not a critical factor in the between group differences.
Results
In experiment one, Mautone and Mayer found statistically and practically significant increases in the number of causal and relational statements generated by students in the cognitive aids group. Cognitive aids group students generated an average of 5.3 relational statements (SD=2.19) while control group students generated an average of 3.6 relational statements (SD=1.99). The difference between groups was t(58)=2.96, p=0.004 and Cohen’s d was a medium-to-large 0.76. Cognitive aids group students generated an average of 1.1 causal statements (SD=0.88) while control group students generated an average of 0.4 relational statements (SD=0.62). The difference between groups was t(58)=3.55, p=0.001 and Cohen’s d was a large 0.93. Effects of time spent studying were not statistically significant.
In experiment two, Mautone and Mayer found a significant main effect of signaling graphic organizers on relational statements. Students who were in signaling groups generated an average of 5.14 relational statements (SD=1.90) while students in the other groups generated an average of 4.08 relational statements (SD=1.64). The difference between groups was F(1,98)=9.02, p=0.03 and Cohen’s d was a medium 0.60. There was no main effect for the presence of concrete graphic organizers on relational statements and there was no interaction effect between the presence of concrete and signaling graphic organizers on relational statements. Effects of time spent studying were not statistically significant.
Also in experiment two, Mautone and Mayer found a significant main effect of concrete graphic organizers on causal statements. Students who were in concrete groups generated an average of 1.02 causal statements (SD=0.96) while students in the other groups generated an average of 0.65 causal statements (SD=0.90). The difference between groups was F(1,98)=3.86, p=0.05 and Cohen’s d was a small-to-medium 0.39. There was no main effect for the presence of signaling graphic organizers on causal statements and there was no interaction effect between the presence of concrete and signaling graphic organizers on causal statements. Effects of time spent studying were not statistically significant.
In experiment three, Mautone and Mayer found statistically and practically significant increases in the number of relational statements generated by students in the structural graphic organizers group. Structural group students generated an average of 4.92 relational statements (SD=1.78) while control group students generated an average of 3.00 relational statements (SD=1.69). The difference between groups was t(45)=3.78, p<0.001 and Cohen’s d was a large 1.10. There was no significant difference between groups in the number of causal statements generated. Effects of time spent studying were not statistically significant.
Discussion
Mautone and Mayer conducted three experiments that demonstrate how different types of graphic organizers can improve students’ understanding of scientific graphs in different ways.
The differences in the three types of graphic organizers are not entirely obvious at first, but Mautone and Mayer provide clear descriptions and clarifying examples to explain the types and how they are intended to work. The signaling organizers help students to understand how variables are represented on graphs and how the different pieces of graphs are put together. Especially valuable in understanding signaling organizers is the idea of presenting partial graphs so that students can understand how fundamental parts of a graph are structured relative to each other before additional graph elements are added. Concrete organizers help students related the information on graphs to concepts and knowledge they already possess by using pictures, photographs, and diagrams to illustrate variables and their different values.
Structural organizers are perhaps the most sophisticated because they help students to understand the relationships between new variables by using other relationships the students already understand to help students understand the new relationship. Mautone and Mayer described a clever way to explain curvilinear relationships by using age and driving ability. Extremely young and old drivers have higher rates of accidents that those drivers who are closer to middle age. This structural organizer is used to then explain the curvilinear relationship between river water velocity and particle size with respect to erosion. Brilliant!
Mautone and Mayer addressed the major limitation of this study in their post-experiment analyses on the effects of time. In all three experiments, time was found to not be a statistically significant factor in between group differences.
Critique
Mautone and Mayer use clear language and provide concrete examples that are especially useful for distinguishing the three types of graphic organizers. Tables are clear, well organized and well placed within the text. The descriptions of methodology are detailed and thorough without being repetitive or redundant. Further, the follow-up analyses of time as an independent variable are thoughtful and are summarized clearly and concisely. All pertinent statistical data (means, standard deviations, effect sizes, p values and statistics) are provided clearly. The use of three experiments also provides evidence that the results are replicable, a commonly overlooked aspect of education research studies.
Theoretical, Methodological, and Practical Implications
A major practical implication of this article is the potential to use structural and concrete graphic organizers to promote students’ comprehension of scientific graphs. Structural organizers should help students to understand how variables are related. It would be interesting to conduct further studies to see if extended use of structural graphic organizers led to successful completion of far transfer tasks where new variables that have similar relationships can be understood without introduction using the graphic organizers.
As well, there is a possibility that repeated use of concrete graphic organizers may promote causal understanding and integration of new knowledge into existing schemata. Again, there may be a way that repeated use of concrete organizers could lead to far transfer ability and more generalized use of integration in novel situations without explicit use of concrete organizers. More research on these possible outcomes is needed.
Mautone and Mayer’s study extends and adds to theoretical understanding of cognition and graphic organizers by providing additional levels of detail. Signaling graphic organizers that focus on structure may help students to better understand relations in graphs and between variables. Concrete graphic organizers may help students to better understand causal relations in graphs and between variables. Structural graphic organizers can help students to better understand more complex relations between variables, such as curvilinear relationships. Graphic organizers may include both concrete and signaling design principles and thereby statistically increase students understanding of relations and integration of new knowledge as shown by increased numbers of accurate causal and relational statements. Mautone and Mayer’s work indicates that structure and integration may be independent cognitive processes.
Mautone and Mayer used a three-experiment approach with self-paced study time and timed responses. The use of replication and variation within and between the three experiments allows their method to demonstrate that their results with graphic organizers are reproducible and to examine details of graphic organizer and cognition theory through variations in the graphic organizers studied. Other research studies may benefit by following similar replication and variation strategies where multiple experiments are combined into one study. Pilot tests are commonly used in studies, but Mautone and Mayer went much further by using three full-scale experiments and not a mini-experiment as an antecedent to the main experiment.
Monday, April 27, 2009
Bera and Robinson (2004)
Bera and Robinson (2004) Exploring the boundary conditions of the delay
hypothesis with adjunct displays. Journal of Educational Psychology. Vol. 96, No. 2, 381–388.
Problem/Purpose
This article was selected due to its dual merit within the graphic organizer domain. First, it highlights the enduring debate surrounding the use of graphic organizers (GOs) and outlines within the instructional domain. Second, Bera and Robinson makes a significant contribution to the current knowledge base by not aiming to discount one or the other presentation format, but rather by attempting to identify individual context specific strengths of both formats by using the delay hypothesis as their theoretical anchor.
The present study was conducted in order to explain the contradictory results of two previous seminal research studies, by exploring the boundary conditions of the delay hypothesis with adjunct displays. That is, the authors wanted to determine what types of adjunct displays would result in optimal retention when different lengths of text and numbers of displays are used. To explore the possibility that certain types of adjunct displays lead to more distinctive encoding, they tested students who viewed outlines or GOs either immediately following acquisition or after a short delay.
Methods
Bera and Robinson performed two experiments. In the first experiment they wanted to replicate the findings of Robinson and Schraw (1994) but in a more realistic setting. Bera and Robinson made a total of two modifications; first, they used a modified longer text (1,000 vs. 200 words) in order to create more realistic classroom reading material; second, they reduced the testing delay (5 versus 25 minutes) to determine whether differences in retention could be observed when the interval between study and testing was similar to those experienced in traditional classrooms . In the second experiment, Bera and Robinson wanted to replicate the findings of Robinson and Kiewra (1995) using the original 6500 words text with multiple displays but with a reduced delay (5 minutes versus two days).
In the first experiment, the participants were 124 undergraduates all enrolled in educational psychology courses at the University of Texas at Austin. The design was an A2 (display: text plus outline vs. text plus GO) 2 (testing: immediate vs. 5-min delay) factorial. Students were randomly assigned to one of the four between-subjects conditions and were presented with a 956-word text, which was adapted from an undergraduate developmental psychology text focusing on different explanations of human development, then the students received two different displays , an outline and a GO. Assessment consisted of ten multiple-choice questions which were constructed to measure the knowledge of hierarchical and coordinate concept relations( See Appendix A,B).
In the second experiment both the 72 participants and the design were identical to those used in Experiment 1. The text of 6,500 words was taken from two widely used undergraduate introductory psychology textbooks. In total, seven outlines and seven GOs were constructed on two sets of displays, both showing equivalent information. The authors constructed two tests; the first, a relation-test contained 10 four-option, multiple-choice items that required knowledge of hierarchical and coordinate concept relations that were presented in both the text and the displays. The second, an application-test, contained 10 matching items that required the identification of disorders by giving novel examples of symptoms and behavior.
Results
In experiment 1, all of the statistical tests were conducted using an alpha of .05 and an A2 (display) X 2 (testing occasion) factorial analysis of variance (ANOVA) was conducted on the test scores. Here Bera and Robinson found that the main effect of display was not statistically significant and thus could not be ruled different from zero, F(1, 120)=1.00, MSE =2.87; meanwhile the main effect of testing occasion F(1, 120)=5.06, indicated that students scored higher when tested immediately than when tested after 5 min. This testing occasion main effect was qualified by the Display X Testing interaction, F(1, 120)=7.29. Tests of the simple effect of testing occasion within both of the display conditions indicated that the outline groups’ performance did not change from immediate to delayed testing, whereas the GO groups’ performance decreased from immediate to delayed testing.
In experiment 2, an A2 (testing occasion) X 2 (display) X 2 (test) repeated measures ANOVA was conducted using the relations and application scores as repeated measures. Here Bera and Robinson found that only two of the seven main and interaction effects were significant. The F values for the other five effects (all with 1 and 68 degrees of freedom), including the main effects of testing (1.25), test (1.32), and the interaction effects of Display X Test, (1.39), Testing Occasion X Test, (1.08), and Display X Testing Occasion X Test (0.01), were not statistically significant and could not be ruled different from zero. In addition the main effect of display, F(1, 68)=17.47, MSE=3.41, indicated that students who viewed GOs (M=12.00) scored higher than those who viewed outlines (M =9.41). This display main effect was compromised, however, by a Display X Testing Occasion interaction, F(1, 68) =4.95. And finally, tests of the simple effect of testing occasion within each of the display conditions revealed that for the outline groups performance did not change from immediate to delayed testing, but for the GO group, delayed testing was superior to immediate testing.
Discussion
Bera and Robinson state that the results of both experiments are consistent with the findings of Robinson and Schraw (1994) and Robinson and Kiewra (1995) but indicate that because the length of the texts and number of adjunct displays differed in both the previous two experiments and in the present two experiments, the inconsistency in results may be due to these differences. They describe the possibility that the length of the learning materials may explain these inconsistencies and posit that the distinctiveness of encoding hypothesis may explain why outlines are optimal with short text and a single display, whereas GOs are optimal when text is longer and multiple displays are used.
In other words, when the study process becomes too difficult with a long text and multiple outlines because GOs allow readers to grasp these relations more quickly and easily, the difficulty is lessened as the GOs are now permitting readers to encode the information more distinctively. This results in a more durable encoding compared with what is remembered from the more difficult-to-comprehend outlines.
Finally, the authors suggested future research that would investigate other populations in order to test whether differences between outlines and GOs in terms of the delay hypothesis do occur with elementary or high school students.
Critique
This paper was a very well researched, organized and written piece where the authors used clear and easy to understand language. One critique that was not acknowledged by Bera and Robinson was the less than optimal number of participants for factorial data analysis. All other shortcomings/ limitations were either addressed by the authors or were likely to be left out due to common editorial constraints.
Theoretical and practical implications
One theoretical implication of the Bera and Robinson study is that the results are likely to produce or at least demand that future GO research further explore relations and interdependence between encoding efficiency and encoding distinctiveness, the two factors that can produce the efficient and effective comprehension of main ideas.
Practical implication are likely to include the incorporation of Bera and Robinson efforts which call on educators to capitalize on students’ active involvement in the learning process into everyday in-class instruction and may lead to more content-appropriate handouts and displays in schools in all levels.
hypothesis with adjunct displays. Journal of Educational Psychology. Vol. 96, No. 2, 381–388.
Problem/Purpose
This article was selected due to its dual merit within the graphic organizer domain. First, it highlights the enduring debate surrounding the use of graphic organizers (GOs) and outlines within the instructional domain. Second, Bera and Robinson makes a significant contribution to the current knowledge base by not aiming to discount one or the other presentation format, but rather by attempting to identify individual context specific strengths of both formats by using the delay hypothesis as their theoretical anchor.
The present study was conducted in order to explain the contradictory results of two previous seminal research studies, by exploring the boundary conditions of the delay hypothesis with adjunct displays. That is, the authors wanted to determine what types of adjunct displays would result in optimal retention when different lengths of text and numbers of displays are used. To explore the possibility that certain types of adjunct displays lead to more distinctive encoding, they tested students who viewed outlines or GOs either immediately following acquisition or after a short delay.
Methods
Bera and Robinson performed two experiments. In the first experiment they wanted to replicate the findings of Robinson and Schraw (1994) but in a more realistic setting. Bera and Robinson made a total of two modifications; first, they used a modified longer text (1,000 vs. 200 words) in order to create more realistic classroom reading material; second, they reduced the testing delay (5 versus 25 minutes) to determine whether differences in retention could be observed when the interval between study and testing was similar to those experienced in traditional classrooms . In the second experiment, Bera and Robinson wanted to replicate the findings of Robinson and Kiewra (1995) using the original 6500 words text with multiple displays but with a reduced delay (5 minutes versus two days).
In the first experiment, the participants were 124 undergraduates all enrolled in educational psychology courses at the University of Texas at Austin. The design was an A2 (display: text plus outline vs. text plus GO) 2 (testing: immediate vs. 5-min delay) factorial. Students were randomly assigned to one of the four between-subjects conditions and were presented with a 956-word text, which was adapted from an undergraduate developmental psychology text focusing on different explanations of human development, then the students received two different displays , an outline and a GO. Assessment consisted of ten multiple-choice questions which were constructed to measure the knowledge of hierarchical and coordinate concept relations( See Appendix A,B).
In the second experiment both the 72 participants and the design were identical to those used in Experiment 1. The text of 6,500 words was taken from two widely used undergraduate introductory psychology textbooks. In total, seven outlines and seven GOs were constructed on two sets of displays, both showing equivalent information. The authors constructed two tests; the first, a relation-test contained 10 four-option, multiple-choice items that required knowledge of hierarchical and coordinate concept relations that were presented in both the text and the displays. The second, an application-test, contained 10 matching items that required the identification of disorders by giving novel examples of symptoms and behavior.
Results
In experiment 1, all of the statistical tests were conducted using an alpha of .05 and an A2 (display) X 2 (testing occasion) factorial analysis of variance (ANOVA) was conducted on the test scores. Here Bera and Robinson found that the main effect of display was not statistically significant and thus could not be ruled different from zero, F(1, 120)=1.00, MSE =2.87; meanwhile the main effect of testing occasion F(1, 120)=5.06, indicated that students scored higher when tested immediately than when tested after 5 min. This testing occasion main effect was qualified by the Display X Testing interaction, F(1, 120)=7.29. Tests of the simple effect of testing occasion within both of the display conditions indicated that the outline groups’ performance did not change from immediate to delayed testing, whereas the GO groups’ performance decreased from immediate to delayed testing.
In experiment 2, an A2 (testing occasion) X 2 (display) X 2 (test) repeated measures ANOVA was conducted using the relations and application scores as repeated measures. Here Bera and Robinson found that only two of the seven main and interaction effects were significant. The F values for the other five effects (all with 1 and 68 degrees of freedom), including the main effects of testing (1.25), test (1.32), and the interaction effects of Display X Test, (1.39), Testing Occasion X Test, (1.08), and Display X Testing Occasion X Test (0.01), were not statistically significant and could not be ruled different from zero. In addition the main effect of display, F(1, 68)=17.47, MSE=3.41, indicated that students who viewed GOs (M=12.00) scored higher than those who viewed outlines (M =9.41). This display main effect was compromised, however, by a Display X Testing Occasion interaction, F(1, 68) =4.95. And finally, tests of the simple effect of testing occasion within each of the display conditions revealed that for the outline groups performance did not change from immediate to delayed testing, but for the GO group, delayed testing was superior to immediate testing.
Discussion
Bera and Robinson state that the results of both experiments are consistent with the findings of Robinson and Schraw (1994) and Robinson and Kiewra (1995) but indicate that because the length of the texts and number of adjunct displays differed in both the previous two experiments and in the present two experiments, the inconsistency in results may be due to these differences. They describe the possibility that the length of the learning materials may explain these inconsistencies and posit that the distinctiveness of encoding hypothesis may explain why outlines are optimal with short text and a single display, whereas GOs are optimal when text is longer and multiple displays are used.
In other words, when the study process becomes too difficult with a long text and multiple outlines because GOs allow readers to grasp these relations more quickly and easily, the difficulty is lessened as the GOs are now permitting readers to encode the information more distinctively. This results in a more durable encoding compared with what is remembered from the more difficult-to-comprehend outlines.
Finally, the authors suggested future research that would investigate other populations in order to test whether differences between outlines and GOs in terms of the delay hypothesis do occur with elementary or high school students.
Critique
This paper was a very well researched, organized and written piece where the authors used clear and easy to understand language. One critique that was not acknowledged by Bera and Robinson was the less than optimal number of participants for factorial data analysis. All other shortcomings/ limitations were either addressed by the authors or were likely to be left out due to common editorial constraints.
Theoretical and practical implications
One theoretical implication of the Bera and Robinson study is that the results are likely to produce or at least demand that future GO research further explore relations and interdependence between encoding efficiency and encoding distinctiveness, the two factors that can produce the efficient and effective comprehension of main ideas.
Practical implication are likely to include the incorporation of Bera and Robinson efforts which call on educators to capitalize on students’ active involvement in the learning process into everyday in-class instruction and may lead to more content-appropriate handouts and displays in schools in all levels.
Wednesday, April 22, 2009
Bahr and Dasereau (2005)
Bahr, S & Dasereau, D. (2005). Bilingual knowledge maps (BiK Maps) as a presentation format: delayed recall and training effects. Journal of experimental Education, 73(2), 101-118.
Problem/Purpose
This article was selected because it represents the latest study in an almost decade long bilingual knowledge map research by the same authors. The focus of research within the bilingual language map domain is two-fold. The first is to develop new strategies that help learners of foreign language form the so-called lexical association between native and foreign language words. The second is to make semantic foreign language concepts more accessible to novice learners with highly limited vocabulary and almost nonexistent knowledge of grammar.
Previous research on monolingual as well as BiK maps suggest that BiK maps are likely to create a very positive learning context by allowing for additional tolerance for the lack of basic foreign language vocabulary and grammar and by adding both multidimensional pictorial and visio-spatial information.
The purpose of this study was to investigate two factors related to BiK map research, the presentation format, meaning the use of either BIK maps or standard word pair list, and training on how to learn using BIK maps or word-pair lists. Within these two factors, based on previous studies by the authors, the following three objectives were explored: (1) to test the prediction that when learners’ recall s is tested following a two day delay without prior immediate recall, BiK map learners will show greater recall than word-list learners (2) to asses recall strategies and the use of cues using processing inventory of vocabulary recall (PI-VOR), and (3) to asses the training effects on both BiK map and word-list trainees performance using both PI-VOR and processing inventory of behavioral intentions for textbook study (PI-BIT).
Methods
Bahr and Dansereau recruited 82 undergraduates from several psychology classes from an unknown university to participate in this study in exchange for academic credit. The participants were treated according to guidelines of the American Psychological Association and were randomly assigned to one of the four groups. The study consisted of three sessions ranging from 35 minutes to 2 hours and was conducted within a seven day period on Monday, Wednesday and Friday. In session 1 participants were randomly assigned to conditions where their previous German language knowledge, their motivations were assessed, then received either BiK map or traditional word-list training.
In session 2, both groups warmed up by using Hindi and English word-pairs and then were trained with 16 word sets of German English word-pairs with an average of 6.65 letters per foreign word reflecting tourist vocabulary. In session 3, the authors collected memory measures on both free and cued recall, conducted two additional personal inventory post-tests and ended the session with a debriefing of the participants.
Results
Bahr and Dansereau conducted four 2 X 2 multiple analyses of covariance (MANCOVA) primary analyses on four sets of dependent measures. The first set consisted of the memory measures, which included the free recall scores
FR-wp and FR-add) and cued recall scores. The second, third, and fourth sets consisted of the personal inventory (PI) factors retained from the PI-VOR, PI-BIV, and PI-BIT, respectively.
In the memory measure (Training X Presentation) format MANCOVA on the memory measures revealed a significant effect for study presentation format, F(3, 75) = 5.16, p < .002. The authors found the univariate effects on all memory measures: cued recall, F(1, 77) = 4.78, p < .031; FR-wp, F(1, 77) = 6.61, p < .012; and FR-add, F(1, 77) = 13.57, p < .001.
The adjusted means of participants who studied BiK maps were higher than those of word-list students: cued recall, 9.03 vs. 6.47, respectively; FRwp, 3.47 vs. 1.87, respectively; FR-add, 5.55 vs. 2.90, respectively ( see Table 5 below). The standardized mean differences were dCuedRecall = .497, dFR-wp = .565, and dFR-add = .826. Here the authors found that no other effects, including set effects, were significant.
In the PI-VOR. (Training X Presentation) format MANCOVA on the two PIVOR measures revealed a significant effect for presentation format, F(2, 76 ) = 7.43, p < .001, which was reflected at the univariate level as a main effect for
the Breadth Processing factor, F(1, 77) = 14.40, p < .001. Here the authors discovered that the adjusted mean score was greater for BiK-map students than for the word- list students (4.51 vs. 3.49, respectively; d = .847).
MANCOVA analysis on the two PI-BIV measures revealed a main effect for training approaching significance, F(2, 76 ) = 3.08, p < .051. At the univariate level, this effect was apparent in the Surface Processing factor, F(1, 77) = 5.20,
p < .025. Here the authors found word-list trainees indicated greater intentions than BiK map trainees to engage in this mode of processing (5.63 vs. 5.08, respectively; d = .495).
MANCOVA analysis on the PI-BIT measures revealed no significant differences. However, given the exploratory nature of the study, a univariate finding for training is reported with the Theme (Breadth) Processing factor, F(3, 75) = 4.33,
p < .040. Here the authors found that BiK map trainees indicated greater intent than word-list trainees (5.57 vs. 5.13, respectively; d = .453).
Discussion
Bahr and Dansereau state that the general results of the study are comparable with others obtained by previous foreign language acquisition researchers. This means that large standard deviations and relatively low recall scores and correlation coefficients ( in this study Cronbach’s alpha is ranging from .613 to .969) characterize it. In this respect the authors concluded that their experimental data provided support for the argument that BIK map-based learning does have significant advantage over word-list learning, in particular within short terms. However, Bahr and Dansereau also acknowledged that students’ performance in the present study was significantly lower (ranging from 10.8% to 28.8% in the BiK group and 5.8% to 20.2% in the list group) than those of students from earlier studies by the same authors in 2001.
The authors state that the difference in the students’ performance in the present study is likely due to the fact that delayed recall in previous experiments was preceded by immediate recall, which apparently caused a rehearsal effect. Although their objective in the current study was to eliminate the possibility of differential rehearsal effects, it seems that, without rehearsal, a consequent suppression of memory in general is unavoidable when testing for BiK-map effects.
The authors also discussed the possibilities of taking BiK map out of the academic realm in order test the durability of findings in realistic settings and expressed hope That future investigation into foreign language learning connectivity and multilingual memory will lead to the development of more sophisticated learning materials.
Critique
Bahr and Dansereau’s research paper, although relatively well written, is presented in a nontraditional format that can be hard to follow as both the result and discussion sections are found at the beginning of the paper. The article could have been improved by providing additional information about the location of the study and by limiting references to the authors themselves as their work has been replicated by others on numerous occasions. Other areas of weakness include the very low Cronbach’s alpha values in the processing inventory section on page 107-108 and the very low number of participants (well below the usual 300+ that is recommend for factor analysis). In spite of these critiques and in light of the experimental nature of this investigation I conclude that the Bahr and Dansereau article is both clear written and its aim is well directed.
Theoretical and practical implications
One of several theoretical implications of this study is that it highlighted the need for additional investigation to better understand the connections between of BiK maps and the foreign and the native language lexica. Although there is previous research (see page 102) that seems to support the hypothesis that a strong semantic or conceptual association exists between foreign language words and their meaning, future research should not only confirm these hypothesis’ but should test new presentation formats that would facilitate these connections in novice learners.
Practical implications of this study include using both BiK and monolingual knowledge maps in all kinds of academic settings in order to promote a shift toward a more beneficial top-down processing strategy among the learners. The use of both bi and monolingual knowledge maps in various settings also raises the possibilities that the textbook publishers would include BiK supplements in their new editions of foreign language textbooks and that BiK map creation as a subject would be added to teachers of foreign language degree course offerings.
Problem/Purpose
This article was selected because it represents the latest study in an almost decade long bilingual knowledge map research by the same authors. The focus of research within the bilingual language map domain is two-fold. The first is to develop new strategies that help learners of foreign language form the so-called lexical association between native and foreign language words. The second is to make semantic foreign language concepts more accessible to novice learners with highly limited vocabulary and almost nonexistent knowledge of grammar.
Previous research on monolingual as well as BiK maps suggest that BiK maps are likely to create a very positive learning context by allowing for additional tolerance for the lack of basic foreign language vocabulary and grammar and by adding both multidimensional pictorial and visio-spatial information.
The purpose of this study was to investigate two factors related to BiK map research, the presentation format, meaning the use of either BIK maps or standard word pair list, and training on how to learn using BIK maps or word-pair lists. Within these two factors, based on previous studies by the authors, the following three objectives were explored: (1) to test the prediction that when learners’ recall s is tested following a two day delay without prior immediate recall, BiK map learners will show greater recall than word-list learners (2) to asses recall strategies and the use of cues using processing inventory of vocabulary recall (PI-VOR), and (3) to asses the training effects on both BiK map and word-list trainees performance using both PI-VOR and processing inventory of behavioral intentions for textbook study (PI-BIT).
Methods
Bahr and Dansereau recruited 82 undergraduates from several psychology classes from an unknown university to participate in this study in exchange for academic credit. The participants were treated according to guidelines of the American Psychological Association and were randomly assigned to one of the four groups. The study consisted of three sessions ranging from 35 minutes to 2 hours and was conducted within a seven day period on Monday, Wednesday and Friday. In session 1 participants were randomly assigned to conditions where their previous German language knowledge, their motivations were assessed, then received either BiK map or traditional word-list training.
In session 2, both groups warmed up by using Hindi and English word-pairs and then were trained with 16 word sets of German English word-pairs with an average of 6.65 letters per foreign word reflecting tourist vocabulary. In session 3, the authors collected memory measures on both free and cued recall, conducted two additional personal inventory post-tests and ended the session with a debriefing of the participants.
Results
Bahr and Dansereau conducted four 2 X 2 multiple analyses of covariance (MANCOVA) primary analyses on four sets of dependent measures. The first set consisted of the memory measures, which included the free recall scores
FR-wp and FR-add) and cued recall scores. The second, third, and fourth sets consisted of the personal inventory (PI) factors retained from the PI-VOR, PI-BIV, and PI-BIT, respectively.
In the memory measure (Training X Presentation) format MANCOVA on the memory measures revealed a significant effect for study presentation format, F(3, 75) = 5.16, p < .002. The authors found the univariate effects on all memory measures: cued recall, F(1, 77) = 4.78, p < .031; FR-wp, F(1, 77) = 6.61, p < .012; and FR-add, F(1, 77) = 13.57, p < .001.
The adjusted means of participants who studied BiK maps were higher than those of word-list students: cued recall, 9.03 vs. 6.47, respectively; FRwp, 3.47 vs. 1.87, respectively; FR-add, 5.55 vs. 2.90, respectively ( see Table 5 below). The standardized mean differences were dCuedRecall = .497, dFR-wp = .565, and dFR-add = .826. Here the authors found that no other effects, including set effects, were significant.
In the PI-VOR. (Training X Presentation) format MANCOVA on the two PIVOR measures revealed a significant effect for presentation format, F(2, 76 ) = 7.43, p < .001, which was reflected at the univariate level as a main effect for
the Breadth Processing factor, F(1, 77) = 14.40, p < .001. Here the authors discovered that the adjusted mean score was greater for BiK-map students than for the word- list students (4.51 vs. 3.49, respectively; d = .847).
MANCOVA analysis on the two PI-BIV measures revealed a main effect for training approaching significance, F(2, 76 ) = 3.08, p < .051. At the univariate level, this effect was apparent in the Surface Processing factor, F(1, 77) = 5.20,
p < .025. Here the authors found word-list trainees indicated greater intentions than BiK map trainees to engage in this mode of processing (5.63 vs. 5.08, respectively; d = .495).
MANCOVA analysis on the PI-BIT measures revealed no significant differences. However, given the exploratory nature of the study, a univariate finding for training is reported with the Theme (Breadth) Processing factor, F(3, 75) = 4.33,
p < .040. Here the authors found that BiK map trainees indicated greater intent than word-list trainees (5.57 vs. 5.13, respectively; d = .453).
Discussion
Bahr and Dansereau state that the general results of the study are comparable with others obtained by previous foreign language acquisition researchers. This means that large standard deviations and relatively low recall scores and correlation coefficients ( in this study Cronbach’s alpha is ranging from .613 to .969) characterize it. In this respect the authors concluded that their experimental data provided support for the argument that BIK map-based learning does have significant advantage over word-list learning, in particular within short terms. However, Bahr and Dansereau also acknowledged that students’ performance in the present study was significantly lower (ranging from 10.8% to 28.8% in the BiK group and 5.8% to 20.2% in the list group) than those of students from earlier studies by the same authors in 2001.
The authors state that the difference in the students’ performance in the present study is likely due to the fact that delayed recall in previous experiments was preceded by immediate recall, which apparently caused a rehearsal effect. Although their objective in the current study was to eliminate the possibility of differential rehearsal effects, it seems that, without rehearsal, a consequent suppression of memory in general is unavoidable when testing for BiK-map effects.
The authors also discussed the possibilities of taking BiK map out of the academic realm in order test the durability of findings in realistic settings and expressed hope That future investigation into foreign language learning connectivity and multilingual memory will lead to the development of more sophisticated learning materials.
Critique
Bahr and Dansereau’s research paper, although relatively well written, is presented in a nontraditional format that can be hard to follow as both the result and discussion sections are found at the beginning of the paper. The article could have been improved by providing additional information about the location of the study and by limiting references to the authors themselves as their work has been replicated by others on numerous occasions. Other areas of weakness include the very low Cronbach’s alpha values in the processing inventory section on page 107-108 and the very low number of participants (well below the usual 300+ that is recommend for factor analysis). In spite of these critiques and in light of the experimental nature of this investigation I conclude that the Bahr and Dansereau article is both clear written and its aim is well directed.
Theoretical and practical implications
One of several theoretical implications of this study is that it highlighted the need for additional investigation to better understand the connections between of BiK maps and the foreign and the native language lexica. Although there is previous research (see page 102) that seems to support the hypothesis that a strong semantic or conceptual association exists between foreign language words and their meaning, future research should not only confirm these hypothesis’ but should test new presentation formats that would facilitate these connections in novice learners.
Practical implications of this study include using both BiK and monolingual knowledge maps in all kinds of academic settings in order to promote a shift toward a more beneficial top-down processing strategy among the learners. The use of both bi and monolingual knowledge maps in various settings also raises the possibilities that the textbook publishers would include BiK supplements in their new editions of foreign language textbooks and that BiK map creation as a subject would be added to teachers of foreign language degree course offerings.
Monday, April 6, 2009
Özmen et al. (2009)
Özmen, H, Gökhan, D., & Coll, R. K. (2009). A comparative study of the effects of a concept mapping enhanced laboratory experience on Turkish high school students’ understanding of acid-base chemistry. Journal of Science and Mathematics Education, 7, 1-24.
Problem/Purpose
The researchers in this study examined how the combined effect of laboratory activities and concept mapping improved student learning of acid-base chemistry relative to traditional lecture and problem set teaching methods in chemistry classes. Student learning included test measures through quantitative scales with multiple choice problems as well as student interviews to identify qualitative differences, especially alternative conceptions. The expression “alternative conceptions” refers to student conceptions that fail to align with accepted scientific models and conceptions. In addition to examining student learning, the researchers set out to measure student affect in the form of attitude toward chemistry.
This study is of interest because it expands the understanding of concept maps and laboratory activities in science teaching by examining the effects of using the two methods concurrently. By investigating more qualitative measures like alternative conceptions, the study promotes the identification and correction of incomplete or incorrect student understandings. Özmen et al. further relate that alternative conceptions are pervasive, stable, and resistant to change.
Methods
Özmen et al. conducted a quasi-experiment using one teacher, two groups, and two teaching methods. The teacher taught a unit on acids and bases in the traditional lecture-based format he had used in the past for one group. For the intervention group, he taught using concept maps and laboratory activities. The researchers provided two 45-minute training sessions on concept maps and laboratory activities for the teacher. Both groups received five 45-minute lessons per week for a total of four weeks.
For the traditional group, the teaching observed was primarily whole-group instruction consisting of teacher lectures explaining concepts with examples chosen from the textbook. The lectures were followed by teacher-driven explanations and seatwork with the teacher moving about the room, helping students to complete the seatwork.
For the intervention group, the pre-test administered to identify student alternative conceptions at the beginning of the study was used as part of the instructional design. Students were provided with instruction about alternative conceptions at the beginning of the unit. Later in the unit alternative conceptions were presented again and discussed at times when students would likely develop cognitive conflict and potentially be replace the alternative conceptions with accepted scientific conceptions.
Students in the intervention completed two laboratory activities per week (eight total). Within the intervention group, students were assigned to smaller study groups. The study groups were matched based on student pre-test performance.
Students in the intervention group also received instruction with respect to concept mapping and practice in constructing concept maps. These included partially constructed maps at the beginning, and unassisted maps later. Students prepared concept maps for seven different topics within the acid-base unit.
With both groups, the researchers used pretests and posttests. In addition to the pretest at the beginning of the unit, the researchers conducted 30-40 minute interviews with 15 students to evaluate student understanding and to identify students’ alternative conceptions. The 15 students were selected to represent a range of achievement levels based on past chemistry examination grades. The post-test used with both groups was different from the pre-test, which Özmen et al. describe as non-equivalent.
For the pretests and posttests, student understanding was measured using the Concept Achievement Test (CAT). The CAT contains 25 items, 15 of which are multiple choice and 10 of which are multiple choice with required explanations. Response choices included the scientific explanation, one common alternative conception, and three plausible distracters. Three chemistry educators validated the content of the CAT and reliability was assessed via item analysis using the Kuder-Richardson 20 statistic, which was found to be 0.81 and considered acceptable by the researchers.
Results
The researchers compared pretest scores for the two groups using an independent samples t-test (t=0.40, df=57, p=0.82), and found no statistically significant difference between the groups (p>0.05). For the posttest scores, Özmen et al. found that the intervention group had higher scores and that the difference was statistically significant (t=5.58, df=57, p<0.01). Özmen et al. did not report Cohen’s d, but the author of this review found it to be d=1.45, a rather large and noteworthy effect size.
The researchers also examined students’ alternative conceptions to determine if there were differences in stability and change of these conceptions with respect to the different instructional methods. Özmen et al. present these results in a table (Table III). For the seventeen alternative conceptions, there were larger changes for the intervention group than for the traditional group. Özmen et al. do not present any statistics to evaluate statistical significance or effect sizes for these differences.
Discussion
Özmen et al. state that their findings indicate that students in the intervention group who participated in laboratory and concept map construction activities performed better on the posttest and showed greater changes in correcting alternative conceptions with scientific explanations. Students in the traditional group improved in these ways as well, but the posttest difference between groups was statistically significant with a large effect size. Statistics and effect sizes were not calculated for the corrections of alternative conceptions.
Because Özmen et al. studied the combined effects of two treatments, it is not possible to determine if both treatments are required to accomplish the outcomes observed. The change may be due to one component, the other, or the combination of the two. As Özmen et al. state, “A simpler intervention may be equally effective.”
Some of the changes are likely due to testing of alternative conceptions in the laboratory. Özmen et al. describe having students test the conception that acids melt metals by having students treat metals with strong acids to observe the effects. The rejection of the conception from contradictory laboratory experience is expected, but this effect was not isolated and tested specifically. A similar experience was created for students in a demonstration to controvert the alternative conception that “the only way to test a sample whether it is an acid or a base is to see if it eats something away, for example metal, plastic, animal, and us.” Again, the rejection of the conception from contradictory laboratory experience is expected, but this effect was not isolated and tested specifically.
Özmen et al. state that the role of concept maps in the intervention was threefold. First the students used concept maps to defend their ideas from laboratory work. Second the students used concept maps to understand the results obtained in the laboratory. Third the concept maps allowed the teacher to identify alternative conceptions and gaps in understanding. Concept maps were used to augment understanding and processing of laboratory activities and data
Özmen et al. indicate that the literature is mixed with respect to the effect of laboratory activities alone in improving student scores on post-instruction tests and in correcting alternative conceptions. Likewise Özmen et al. indicate that the literature is mixed with respect to the effect of concept mapping alone in improving student scores on post-instruction tests and in correcting alternative conceptions. However, Özmen et al. state that a limited literature on the effect of using both strategies together is consistent with the findings of this paper. Özmen et al. additionally state the sentiment that while laboratory work may be the more critical element of the two parts of the intervention, that concept maps are crucial in helping students to integrate the laboratory work within their conceptual structure.
The limitations of this study include geographic and temporal specificity. Furthermore, the use of one teacher to teach both methods and groups does not guarantee against teacher effects. The use of two components only in combination and not in isolation does not allow for empirically based evaluations of the contributions of each component or for any synergistic effects of the components used together.
Critique
Özmen et al. use language that is understandable and jargon is avoided when possible and explained where necessary. The absence of effect size calculations for the posttest results is troubling, especially when the effect is large and worthy of inclusion. Likewise, the lack of statistical significance testing on the results of Table III for correction of students’ alternative conceptions fails to provide empirical data analysis to support the author’s contentions. Calculation and inclusion of significance tests and effect sizes for these corrections should not be difficult. Özmen et al. could also have included two treatments with only laboratory investigations included in one and concept mapping included in the other to determine component and synergy effects. This last critique can still be addressed through future studies.
Theoretical, Methodological, and Practical Implications
A major implication of this article is the potential to use laboratory activities and concept mapping together to modify alternative conceptions that tend to be stable and resistant to correction by traditional teaching methods. In addition to correcting these major misunderstandings, the significant improvement in posttest scores using the combination of concept mapping and laboratory activities in equal time is an important practical implication. This may be even more notable given that some instructional time normally devoted to content was replaced with instructional time for learning to construct and use concept maps, thereby reducing the time spent directly on content in the intervention group.
This study used a specially designed test to assess students’ use of alternative conceptions based on alternative conceptions detected at the outset of the study through student interviews. This method allowed the researchers to focus on student understanding. Future research based on this method could be improved by having more than one choice being an alternative conception and by analyzing the distracters and the alternative assessment using item response theory to ensure that the distracter and alternative conceptions are equally attractive responses.
Theoretical implications of this article involve possible synergistic effects between concept mapping and laboratory activities (and possibly other forms of experiential learning, group work, and project-based learning). As well, there is the potential for theoretical work to connect concept mapping and laboratory activities in new ways to visual and kinesthetic processing and phonological loops via the multi-sensory input of laboratory activities and the use of concept maps to connect the sensory input with phonological loops and learners’ internalized concept structures.
Problem/Purpose
The researchers in this study examined how the combined effect of laboratory activities and concept mapping improved student learning of acid-base chemistry relative to traditional lecture and problem set teaching methods in chemistry classes. Student learning included test measures through quantitative scales with multiple choice problems as well as student interviews to identify qualitative differences, especially alternative conceptions. The expression “alternative conceptions” refers to student conceptions that fail to align with accepted scientific models and conceptions. In addition to examining student learning, the researchers set out to measure student affect in the form of attitude toward chemistry.
This study is of interest because it expands the understanding of concept maps and laboratory activities in science teaching by examining the effects of using the two methods concurrently. By investigating more qualitative measures like alternative conceptions, the study promotes the identification and correction of incomplete or incorrect student understandings. Özmen et al. further relate that alternative conceptions are pervasive, stable, and resistant to change.
Methods
Özmen et al. conducted a quasi-experiment using one teacher, two groups, and two teaching methods. The teacher taught a unit on acids and bases in the traditional lecture-based format he had used in the past for one group. For the intervention group, he taught using concept maps and laboratory activities. The researchers provided two 45-minute training sessions on concept maps and laboratory activities for the teacher. Both groups received five 45-minute lessons per week for a total of four weeks.
For the traditional group, the teaching observed was primarily whole-group instruction consisting of teacher lectures explaining concepts with examples chosen from the textbook. The lectures were followed by teacher-driven explanations and seatwork with the teacher moving about the room, helping students to complete the seatwork.
For the intervention group, the pre-test administered to identify student alternative conceptions at the beginning of the study was used as part of the instructional design. Students were provided with instruction about alternative conceptions at the beginning of the unit. Later in the unit alternative conceptions were presented again and discussed at times when students would likely develop cognitive conflict and potentially be replace the alternative conceptions with accepted scientific conceptions.
Students in the intervention completed two laboratory activities per week (eight total). Within the intervention group, students were assigned to smaller study groups. The study groups were matched based on student pre-test performance.
Students in the intervention group also received instruction with respect to concept mapping and practice in constructing concept maps. These included partially constructed maps at the beginning, and unassisted maps later. Students prepared concept maps for seven different topics within the acid-base unit.
With both groups, the researchers used pretests and posttests. In addition to the pretest at the beginning of the unit, the researchers conducted 30-40 minute interviews with 15 students to evaluate student understanding and to identify students’ alternative conceptions. The 15 students were selected to represent a range of achievement levels based on past chemistry examination grades. The post-test used with both groups was different from the pre-test, which Özmen et al. describe as non-equivalent.
For the pretests and posttests, student understanding was measured using the Concept Achievement Test (CAT). The CAT contains 25 items, 15 of which are multiple choice and 10 of which are multiple choice with required explanations. Response choices included the scientific explanation, one common alternative conception, and three plausible distracters. Three chemistry educators validated the content of the CAT and reliability was assessed via item analysis using the Kuder-Richardson 20 statistic, which was found to be 0.81 and considered acceptable by the researchers.
Results
The researchers compared pretest scores for the two groups using an independent samples t-test (t=0.40, df=57, p=0.82), and found no statistically significant difference between the groups (p>0.05). For the posttest scores, Özmen et al. found that the intervention group had higher scores and that the difference was statistically significant (t=5.58, df=57, p<0.01). Özmen et al. did not report Cohen’s d, but the author of this review found it to be d=1.45, a rather large and noteworthy effect size.
The researchers also examined students’ alternative conceptions to determine if there were differences in stability and change of these conceptions with respect to the different instructional methods. Özmen et al. present these results in a table (Table III). For the seventeen alternative conceptions, there were larger changes for the intervention group than for the traditional group. Özmen et al. do not present any statistics to evaluate statistical significance or effect sizes for these differences.
Discussion
Özmen et al. state that their findings indicate that students in the intervention group who participated in laboratory and concept map construction activities performed better on the posttest and showed greater changes in correcting alternative conceptions with scientific explanations. Students in the traditional group improved in these ways as well, but the posttest difference between groups was statistically significant with a large effect size. Statistics and effect sizes were not calculated for the corrections of alternative conceptions.
Because Özmen et al. studied the combined effects of two treatments, it is not possible to determine if both treatments are required to accomplish the outcomes observed. The change may be due to one component, the other, or the combination of the two. As Özmen et al. state, “A simpler intervention may be equally effective.”
Some of the changes are likely due to testing of alternative conceptions in the laboratory. Özmen et al. describe having students test the conception that acids melt metals by having students treat metals with strong acids to observe the effects. The rejection of the conception from contradictory laboratory experience is expected, but this effect was not isolated and tested specifically. A similar experience was created for students in a demonstration to controvert the alternative conception that “the only way to test a sample whether it is an acid or a base is to see if it eats something away, for example metal, plastic, animal, and us.” Again, the rejection of the conception from contradictory laboratory experience is expected, but this effect was not isolated and tested specifically.
Özmen et al. state that the role of concept maps in the intervention was threefold. First the students used concept maps to defend their ideas from laboratory work. Second the students used concept maps to understand the results obtained in the laboratory. Third the concept maps allowed the teacher to identify alternative conceptions and gaps in understanding. Concept maps were used to augment understanding and processing of laboratory activities and data
Özmen et al. indicate that the literature is mixed with respect to the effect of laboratory activities alone in improving student scores on post-instruction tests and in correcting alternative conceptions. Likewise Özmen et al. indicate that the literature is mixed with respect to the effect of concept mapping alone in improving student scores on post-instruction tests and in correcting alternative conceptions. However, Özmen et al. state that a limited literature on the effect of using both strategies together is consistent with the findings of this paper. Özmen et al. additionally state the sentiment that while laboratory work may be the more critical element of the two parts of the intervention, that concept maps are crucial in helping students to integrate the laboratory work within their conceptual structure.
The limitations of this study include geographic and temporal specificity. Furthermore, the use of one teacher to teach both methods and groups does not guarantee against teacher effects. The use of two components only in combination and not in isolation does not allow for empirically based evaluations of the contributions of each component or for any synergistic effects of the components used together.
Critique
Özmen et al. use language that is understandable and jargon is avoided when possible and explained where necessary. The absence of effect size calculations for the posttest results is troubling, especially when the effect is large and worthy of inclusion. Likewise, the lack of statistical significance testing on the results of Table III for correction of students’ alternative conceptions fails to provide empirical data analysis to support the author’s contentions. Calculation and inclusion of significance tests and effect sizes for these corrections should not be difficult. Özmen et al. could also have included two treatments with only laboratory investigations included in one and concept mapping included in the other to determine component and synergy effects. This last critique can still be addressed through future studies.
Theoretical, Methodological, and Practical Implications
A major implication of this article is the potential to use laboratory activities and concept mapping together to modify alternative conceptions that tend to be stable and resistant to correction by traditional teaching methods. In addition to correcting these major misunderstandings, the significant improvement in posttest scores using the combination of concept mapping and laboratory activities in equal time is an important practical implication. This may be even more notable given that some instructional time normally devoted to content was replaced with instructional time for learning to construct and use concept maps, thereby reducing the time spent directly on content in the intervention group.
This study used a specially designed test to assess students’ use of alternative conceptions based on alternative conceptions detected at the outset of the study through student interviews. This method allowed the researchers to focus on student understanding. Future research based on this method could be improved by having more than one choice being an alternative conception and by analyzing the distracters and the alternative assessment using item response theory to ensure that the distracter and alternative conceptions are equally attractive responses.
Theoretical implications of this article involve possible synergistic effects between concept mapping and laboratory activities (and possibly other forms of experiential learning, group work, and project-based learning). As well, there is the potential for theoretical work to connect concept mapping and laboratory activities in new ways to visual and kinesthetic processing and phonological loops via the multi-sensory input of laboratory activities and the use of concept maps to connect the sensory input with phonological loops and learners’ internalized concept structures.
Chiou (2008)
Chiou, C. (2008). The effect of concept mapping on students’ learning achievements and interests. Innovations in Education and Teaching International. (45)4, 375-387
Purpose/Problem
The purpose of this experimental study was to examine whether concept mapping could be used to improve accounting education in an advanced accounting course in a university setting in Taiwan. The researcher of the study examined two areas of interest: whether concept mapping improved students’ learning achievement in an advanced accounting course; and the attitudes of students toward concept mapping as a learning tool. Student learning achievement was measured by accounting achievement scores in pre- and post-test instruments as well an independent CPA-level exam measuring student knowledge of the domain area. Student attitude toward concept mapping was measured pre- and post-implementation of the concept mapping intervention by a satisfaction questionnaire developed by the researcher.
This study is noteworthy because the use of concept mapping as an instructional and summative tool is contrasted with traditional teaching methodologies in a domain that examines complex mathematical concepts. In addition, the study attempts to measure the effect of student attitudes toward concept mapping both before and after implementation of the concept mapping protocol. The researcher grounded his study by citing well-established research indicating the efficacy of concept mapping as an effective tool to improve student independent learning. Further, the researcher pinpoints a lack of research in the literature in this domain area which this study fills.
Methods
The participants in this study were 124 students from two classes in advanced accounting courses and one instructor in a university setting in Taiwan. Half the students were randomly assigned as an experimental group and the remaining 62 students were assigned as a control group. One instructor taught both classes. The instructor and textbooks for both groups were identical to avoid key confounding effects on the study. None of the participants reported using concept mapping in prior classes.
Several stages were used to accomplish the purpose of the study. First, the researcher and teacher spent two months prior to the teaching of the classes discussing overall experimental design and procedures. Second, grades from a prior intermediate accounting class were combined with an accounting achievement pre-test score and evaluated in order to ensure participants had similar backgrounds in accounting. Third, the experimental group was given three hours of training on concept mapping and its rationale. The instructor then taught from the text book using teacher-generated concept maps. Students from the experimental group were asked to generate their own maps after the conclusion of a chapter. The instructor and researcher provided corrective feedback on each student map enabling the students to correct their own efforts. The students in the experimental group were able to use the modified maps as a review tool for later study. This procedure was repeated until the end of chapter six of the textbook for a total implementation period of 12 weeks. The control group was taught in a more traditional manner with the instructor giving introductory lectures and teacher-generated abstracts. Students in the control group were asked to work on individual problems to which the instructor provided corrective feedback. The identical procedure was repeated until the end of chapter six in the textbook. The implementation period was the same as the experimental group. A post-test was administered to both groups after completion of the chapters to measure learning achievement. Student attitude surveys were administered to the experimental group to measure student satisfaction of concept maps as a learning tool.
Results
T-tests were conducted to compare two factors: 1) the experimental and control groups’ scores in intermediate accounting in the two semesters preceding the study; and 2) the scores in the pre-study accounting pre-test. The differences between the two groups were not significant (t = - 0.65, p > 0.05; t = 0.89, p > 0.05) implying that both groups had similar levels of accounting knowledge and ability.
A post-test t-test was conducted to examine whether the concept mapping technique contributed to student learning achievement. Results show a significant difference (t = 2.96, p < 0.001) between the two classes. The experimental group had a mean score of 73.24 (SD = 15.314) while the control group had a mean score of 63.31 (SD = 21.561) indicating the experimental group outperformed the control group by a significant margin. A Cohen’s d was not reported.
To insure that pre-test scores did not influence the experimental effect, a one-way analysis of covariance was applied with pre-test scores as covariates and the post-test scores as dependent variables. The results attained significance (F = 8.65, p < 0.01; F = 7.99, p < 0.01) indicating that the experimental group who utilized the concept map technique had significantly higher achievement than the control group who were taught using traditional teaching methods.
The researcher also examined student responses on a 10-question Likert-style satisfaction survey to determine student attitudes toward concept mapping. Agree and Strongly Agree responses were converted into “Agree” and Disagree and Strongly Disagree responses were converted to “Disagree” statements then converted to percentages. The first four questions on the survey were designed to reveal whether concept mapping improved learning. Ninety-seven percent of the participants agreed that concept mapping aided their learning of accounting principles. Ninety-five percent of the participants agreed that concept mapping helped them to learn and think independently. Eighty-nine percent of the participants expressed the opinion that concept mapping helped them to reduce barriers to learning accounting.
The next six questions were designed to test student acceptance of concept mapping as a learning tool. Ninety-five percent of the participants indicated that concept mapping could be used as an effective teaching and learning method and easily applied to other curriculum areas. Eighty-four percent of the participants reported liking the concept mapping technique in their study of accounting. Overall, ninety percent of the participants were satisfied with using concept mapping to learn accounting. However, only fifty-eight percent of the students indicated that they “could soon adapt to concept mapping.” The researcher notes that this result indicates that probably more time is needed to train students effectively in the technique of concept mapping.
Discussion
The main purpose of this study was to examine whether concept mapping could be used to help advanced accounting students in a university setting improve learning achievement. The results of the study showed that students in the experimental group improved in their learning achievement more than the students in the control group. Chiou notes that his findings are in agreement with a number of other studies’ findings in other disciplines.
Chiou also notes that the second research area – positive student perceptions of concept mapping – also reinforced previous research. The researcher does note, however, that “nearly half the students indicated that they could not quickly adapt to the approach of concept mapping” (p. 383). While this result is reported in a number of other studies, the results from this study adds to the growing consensus that a lack of familiarity with the technique of concept mapping can be a barrier to students individually adopting concept mapping as a personal learning tool.
Chiou extends his findings to promote concept mapping as “the only approach to address both what students know and how students organize their knowledge” (p. 383) and thus “is a useful meta-learning strategy in relation to helping students ‘learn how to learn’ (p. 384). He concludes with the statement that “concept mapping is an effective meta-cognitive strategy and this study is a pioneering detailed investigation in the usefulness of concept mapping in university-level business accounting courses” (p. 385).
Critique
In general, this study employs sound methodological and statistical techniques and represents a well-thought out and controlled study. The writing is clear, appropriately formal, direct, concise, well organized, and factual. The study is well grounded in previous research and makes an important contribution to the literature. Sample size is sufficient (N = 124) to yield generalizable results. Ethical considerations seem to be at least partially addressed with student anonymity being assured in the student attitude response post-study. Confounding effects are somewhat anticipated and addressed by ensuring homogeneity of the participants’ backgrounds and abilities strengthening the validity of the experimental results. No mention is made of the students’ cultural background as a possible barrier to the generalizability of the study results.
The study could have been strengthened by a more detailed description and analysis of the Likert-style questionnaire given post-study to the experimental group. As described, Chiou grouped all affirmative responses and all negative responses into two categories and converted to percentages. The strength of these responses could have been recalibrated given a more detailed Likert response questionnaire.
Theoretical, Methodological, and Practical Implications
This study indicates that concept mapping can be an effective meta-cognitive strategy to enhance meaningful learning in accounting and other related business courses. Given the dearth of studies in this area, this study makes an important contribution to the corpus of literature. In domains that stress the incremental acquisition of knowledge (in particular math and science), concept mapping might be an effective instructional and summative tool by allowing corrective feedback during the course of instruction.
In addition, this study seems to indicate that concept mapping encourages more independent learning skills by allowing these accounting students to make their own connections between different concept areas. This change in emphasis from more traditional teaching methods seems to enhance more meaningful learning in complex domains.
Finally, concept mapping could be an effective meta-cognitive strategy in making better transitional connections between courses in a particular domain.
Purpose/Problem
The purpose of this experimental study was to examine whether concept mapping could be used to improve accounting education in an advanced accounting course in a university setting in Taiwan. The researcher of the study examined two areas of interest: whether concept mapping improved students’ learning achievement in an advanced accounting course; and the attitudes of students toward concept mapping as a learning tool. Student learning achievement was measured by accounting achievement scores in pre- and post-test instruments as well an independent CPA-level exam measuring student knowledge of the domain area. Student attitude toward concept mapping was measured pre- and post-implementation of the concept mapping intervention by a satisfaction questionnaire developed by the researcher.
This study is noteworthy because the use of concept mapping as an instructional and summative tool is contrasted with traditional teaching methodologies in a domain that examines complex mathematical concepts. In addition, the study attempts to measure the effect of student attitudes toward concept mapping both before and after implementation of the concept mapping protocol. The researcher grounded his study by citing well-established research indicating the efficacy of concept mapping as an effective tool to improve student independent learning. Further, the researcher pinpoints a lack of research in the literature in this domain area which this study fills.
Methods
The participants in this study were 124 students from two classes in advanced accounting courses and one instructor in a university setting in Taiwan. Half the students were randomly assigned as an experimental group and the remaining 62 students were assigned as a control group. One instructor taught both classes. The instructor and textbooks for both groups were identical to avoid key confounding effects on the study. None of the participants reported using concept mapping in prior classes.
Several stages were used to accomplish the purpose of the study. First, the researcher and teacher spent two months prior to the teaching of the classes discussing overall experimental design and procedures. Second, grades from a prior intermediate accounting class were combined with an accounting achievement pre-test score and evaluated in order to ensure participants had similar backgrounds in accounting. Third, the experimental group was given three hours of training on concept mapping and its rationale. The instructor then taught from the text book using teacher-generated concept maps. Students from the experimental group were asked to generate their own maps after the conclusion of a chapter. The instructor and researcher provided corrective feedback on each student map enabling the students to correct their own efforts. The students in the experimental group were able to use the modified maps as a review tool for later study. This procedure was repeated until the end of chapter six of the textbook for a total implementation period of 12 weeks. The control group was taught in a more traditional manner with the instructor giving introductory lectures and teacher-generated abstracts. Students in the control group were asked to work on individual problems to which the instructor provided corrective feedback. The identical procedure was repeated until the end of chapter six in the textbook. The implementation period was the same as the experimental group. A post-test was administered to both groups after completion of the chapters to measure learning achievement. Student attitude surveys were administered to the experimental group to measure student satisfaction of concept maps as a learning tool.
Results
T-tests were conducted to compare two factors: 1) the experimental and control groups’ scores in intermediate accounting in the two semesters preceding the study; and 2) the scores in the pre-study accounting pre-test. The differences between the two groups were not significant (t = - 0.65, p > 0.05; t = 0.89, p > 0.05) implying that both groups had similar levels of accounting knowledge and ability.
A post-test t-test was conducted to examine whether the concept mapping technique contributed to student learning achievement. Results show a significant difference (t = 2.96, p < 0.001) between the two classes. The experimental group had a mean score of 73.24 (SD = 15.314) while the control group had a mean score of 63.31 (SD = 21.561) indicating the experimental group outperformed the control group by a significant margin. A Cohen’s d was not reported.
To insure that pre-test scores did not influence the experimental effect, a one-way analysis of covariance was applied with pre-test scores as covariates and the post-test scores as dependent variables. The results attained significance (F = 8.65, p < 0.01; F = 7.99, p < 0.01) indicating that the experimental group who utilized the concept map technique had significantly higher achievement than the control group who were taught using traditional teaching methods.
The researcher also examined student responses on a 10-question Likert-style satisfaction survey to determine student attitudes toward concept mapping. Agree and Strongly Agree responses were converted into “Agree” and Disagree and Strongly Disagree responses were converted to “Disagree” statements then converted to percentages. The first four questions on the survey were designed to reveal whether concept mapping improved learning. Ninety-seven percent of the participants agreed that concept mapping aided their learning of accounting principles. Ninety-five percent of the participants agreed that concept mapping helped them to learn and think independently. Eighty-nine percent of the participants expressed the opinion that concept mapping helped them to reduce barriers to learning accounting.
The next six questions were designed to test student acceptance of concept mapping as a learning tool. Ninety-five percent of the participants indicated that concept mapping could be used as an effective teaching and learning method and easily applied to other curriculum areas. Eighty-four percent of the participants reported liking the concept mapping technique in their study of accounting. Overall, ninety percent of the participants were satisfied with using concept mapping to learn accounting. However, only fifty-eight percent of the students indicated that they “could soon adapt to concept mapping.” The researcher notes that this result indicates that probably more time is needed to train students effectively in the technique of concept mapping.
Discussion
The main purpose of this study was to examine whether concept mapping could be used to help advanced accounting students in a university setting improve learning achievement. The results of the study showed that students in the experimental group improved in their learning achievement more than the students in the control group. Chiou notes that his findings are in agreement with a number of other studies’ findings in other disciplines.
Chiou also notes that the second research area – positive student perceptions of concept mapping – also reinforced previous research. The researcher does note, however, that “nearly half the students indicated that they could not quickly adapt to the approach of concept mapping” (p. 383). While this result is reported in a number of other studies, the results from this study adds to the growing consensus that a lack of familiarity with the technique of concept mapping can be a barrier to students individually adopting concept mapping as a personal learning tool.
Chiou extends his findings to promote concept mapping as “the only approach to address both what students know and how students organize their knowledge” (p. 383) and thus “is a useful meta-learning strategy in relation to helping students ‘learn how to learn’ (p. 384). He concludes with the statement that “concept mapping is an effective meta-cognitive strategy and this study is a pioneering detailed investigation in the usefulness of concept mapping in university-level business accounting courses” (p. 385).
Critique
In general, this study employs sound methodological and statistical techniques and represents a well-thought out and controlled study. The writing is clear, appropriately formal, direct, concise, well organized, and factual. The study is well grounded in previous research and makes an important contribution to the literature. Sample size is sufficient (N = 124) to yield generalizable results. Ethical considerations seem to be at least partially addressed with student anonymity being assured in the student attitude response post-study. Confounding effects are somewhat anticipated and addressed by ensuring homogeneity of the participants’ backgrounds and abilities strengthening the validity of the experimental results. No mention is made of the students’ cultural background as a possible barrier to the generalizability of the study results.
The study could have been strengthened by a more detailed description and analysis of the Likert-style questionnaire given post-study to the experimental group. As described, Chiou grouped all affirmative responses and all negative responses into two categories and converted to percentages. The strength of these responses could have been recalibrated given a more detailed Likert response questionnaire.
Theoretical, Methodological, and Practical Implications
This study indicates that concept mapping can be an effective meta-cognitive strategy to enhance meaningful learning in accounting and other related business courses. Given the dearth of studies in this area, this study makes an important contribution to the corpus of literature. In domains that stress the incremental acquisition of knowledge (in particular math and science), concept mapping might be an effective instructional and summative tool by allowing corrective feedback during the course of instruction.
In addition, this study seems to indicate that concept mapping encourages more independent learning skills by allowing these accounting students to make their own connections between different concept areas. This change in emphasis from more traditional teaching methods seems to enhance more meaningful learning in complex domains.
Finally, concept mapping could be an effective meta-cognitive strategy in making better transitional connections between courses in a particular domain.
Subscribe to:
Posts (Atom)