Assessing student’s ability to control variables: The impact of test format and sub-skills

The ability to design controlled experiments and to interpret their outcomes is a core scientific reasoning skill and a prominent object of science curricula and standards. Skills related to the design and interpretation of controlled experiments such as planning controlled experiments (PL), identifying controlled experiments (ID), interpreting the outcome of a controlled experiment (IN) and understanding the indeterminacy of confounded experiments (UN) are summarized under the term “control of variables strategy” (CVS)”.  A meta-analysis of 72 CVS intervention studies yields that the effect of intervention studies depends on the format of the utilized test instrument. Studies utilizing multiple-choice instruments to assess student CVS achievement have significantly smaller effect sizes than studies utilizing other instrument formats (e.g. open response, hands-on experimental tasks). However, a detailed comparison of these instruments shows that instruments with different formats in fact measure different sub-skills (planning, interpreting) of the broader CVS construct. Thus, instrument format and measured CVS sub-skill are confounded in existing CVS instruments so that the isolated effect of the utilized CVS sub-skill on CVS measures is unknown. This multi-trait-multi-method study separates the effect of test format and sub-skills on student measures. Data from 142 students who worked on hands-on and multiple-choice CVS tests that operationalize the same CVS sub-skills were analyzed with structure equation modelling. The results show that the format and the sub-skills have an impact on students CVS measures. Implications for further research and the interpretation of existing research findings are discussed.

Download
Assessing student’s ability to control variables: The impact of test format and sub-skills
Slides from my presentation of SIG meanting in Jerusalem in 2018
sig_inqury_learning_2018_slides_schwicho
Adobe Acrobat Dokument 2.0 MB


Professionsorientierung durch einen Inverted Classroom Ansatz in der Physikdidaktik

Fachdidaktisches Wissen ist neben Fachwissen und pädagogischem Wissen ein zentrales Element des professionellen Wissens von Lehrkräften (Schulman, 1987). Um ein fundiertes fachdidaktisches Wissen aufzubauen, das angehende Lehrkräfte bei der Planung, Durchführung und Reflexion von Unterricht anwenden können, sollten sich Didaktikveranstaltungen sowohl an Forschungsbefunden als auch an der Unterrichtspraxis orientieren. In traditionellen Vorlesungen, die nur auf die Vermittlung von Konzepten und Wissen ausgerichtet sind, kann dies jedoch kaum realisiert werden (Borko & Putnam, 1996). Ferner erwarten Studierende von Didaktikveranstaltungen, dass vielmehr die Konzipierung konkreter Unterrichtssituationen und weniger Forschungsbefunde behandelt werden. Damit stehen derartige Lehrveranstaltungen in einem Spannungsfeld zwischen einer fundierten Einführung in die Physikdidaktik als empirische Bildungsforschung und einer von den Studierenden geforderten, stärkeren Fokussierung auf die Unterrichtspraxis. Um dieser Unzufriedenheit der Studierenden  (siehe Albrecht & Nordmeier, 2011) zu begegnen, um dem vorwiegend rezeptiven Lernen traditioneller Didaktikvorlesungen entgegenzuwirken und um mehr Lernsituationen mit Praxisbezug zu ermöglichen, wurde eine Lehrkonzeption entwickelt und erprobt, welche auf eine stärkere Theorie-Praxis-Verzahnung fokussiert. Durch diese explizite Theorie-Praxis-Verzahnung soll die oft nicht wahrgenommene Relevanz fachdidaktischen Wissens bei der Planung, Durchführung und Reflexion von Unterricht zunehmen. Dementsprechend lautet die Fragestellungen der Studie: Inwieweit verändert sich die Wahrnehmung der Studierenden hinsichtlich der Bedeutsamkeit zentraler fachdidaktischer Konzepte für ihren späteren Berufsalltag? Inwiefern schlägt sich die gesteigerte Motivation der Studierenden auf individueller Ebene in einem erhöhten Zuwachs fachdidaktischen Wissens nieder?

 

Download
poster_gfd.pdf
Adobe Acrobat Dokument 1.7 MB


The development of different facets of the control-of-variables strategy

Poster presented at the 2015 CDS conference in Columbus, OH, USA.

The control-of-variables strategy (CVS) incorporates the important scientific process skills of designing and interpreting controlled experiments. Therefore, CVS is a prominent concept in numerous science standards. CVS consists of four sub-skills: (a) understanding the indeterminacy of uncontrolled experiments, (b) planning, (c), identifying and (d) interpreting controlled experiments. This quasi-longitudinal study with N = 1289 German high school students investigates the development of the four CVS sub-skills from 5th to 13th grade (ages 10 to 18). The raw data are transformed in Rasch-measures before they are analyzed. The results show that students in the 5th grade can already plan, identify, and interpret controlled experiments but that they do not understand the indeterminacy of uncontrolled experiments before 10th grade. In conclusion, even young students can utilize science process skills but do not understand why they have to use these skills. Implications for science instruction and further research are discussed.


Download
Poster_cds_2015.pdf
Adobe Acrobat Dokument 753.0 KB

What students learn from hands-on experimental tasks

Presentation at the ESERA conference 2015 in Helsinki, Finland.

The ability to design and interpret controlled experiments is a core scientific method and a prominent object of science standards. Numerous intervention studies investigate how the so called control-of-variables-strategy [CVS] can be introduced to students. However, a meta-analysis of 72 intervention studies found out that the opportunity to train CVS skills on hands-on tasks seems to hinder an effective acquisition of CVS. A reason for this might be that hands-on tasks are complex as they require additional non-CVS related skills like making measurements, writing lab books or arranging materials. A resulting cognitive overload is known to hinder learning. Therefore, this intervention study with 161 eight graders investigated the differential effects of hands-on and less complex paper-and-pencil training tasks on students’ achievement regarding CVS. First, CVS is introduced to all students before they were grouped into a hands-on or paper-and-pencil training condition based on their pretest results. In both training conditions students had to design and interpret experiments about variables that influence the force of electromagnets. Students in the hands-on group interacted with experimental equipment while students in the paper-and-pencil group planed experiments theoretically and interpreted the outcome of experiments that were presented on a photo. Both groups did equally well on a multiple-choice CVS instrument after instruction. However, on a hands-on CVS test we found a differential effect. Both groups are similar on a transfer task but the hands-on group outperforms the paper-and pencil group on a task identical to the training task. In summary, it seems that students learn particularly task-specific procedural knowledge when interacting with hands-on materials, but they do not acquire a deeper conceptual understanding of CVS. Implications for instruction and assessment will be discussed.


Download
presentation_esera2015.pdf
Adobe Acrobat Dokument 1.5 MB

Teaching the control of variables strategy: A research-synthesis

Presentation at the NARST conference 2014 in Pthsburgh, PA, USA.

Designing controlled experiments and drawing valid inferences from such experiments is a crucial requirement for learning from inquiry. However, students do not show an adequate understanding of the control of variables strategy [CVS] without instruction. The presented meta-analysis identifies features of effective instruction from 67 intervention studies. Direct instruction in combination with cognitive conflicts seems to have a positive impact on student achievement concerning the CVS. Discovery learning is found to be less effective. Therefore, the CVS should be taught by direct instruction particularly when learning through inquiry is the focus of the curriculum. If students should learn from their own investigations, then the crucial inquiry skills should be taught as effectively as possible. The missing evidence of an age-dependency of student achievement offers the possibility to integrate the CVS into science curricula from K to 12. Besides the intervention aspects, a significant impact of the features of the tests on the intervention outcome was found. For example, multiple choice tests lead to higher intervention effects than open response formats. Therefore, standards for measuring student achievement in school relevant intervention studies are discussed. Further research could investigate the interaction of test and instruction aspects.

Download
presentation_narst2014.pdf
Adobe Acrobat Dokument 701.6 KB