Test instruments matter
The results of our meta-analysis on CVS intervention studies and an older meat-analysis (Ross, 1988) show consistently that the test instrument influence the results of intervention studies. This finding is not just a mythological interesting fact. It scrutinizes everything we know about scientific reasoning as it doubts the validity of test instruments. Also there has been a lot of research on scientific reasoning and particular on controlling variables the validety of instruments utilized in this studies are not well investigated as a review by Opitz and colleagues shows. Further research should take validity of instruments more seriously.
Control of Variables Strategy Inventory (CVSI)
The ability to design controlled experiments and interpret experimental outcomes is a core scientific reasoning skill and a prominent object of science curricula and standards. Hence, appropriate assessment instruments are needed in order to 1) measure this core scientific reasoning skill and 2) evaluate the impact of specific science instruction on the development of those skills. In the literature skills related to controlling variables have often been referred to as “isolation of variables” (Inhelder & Piaget, 1958), “vary-one-thing-at-a-time” [VOTAT] (Tschirgi, 1980) or “control of variables strategy” [CVS]” (Chen & Klahr, 1999). We created a new test instrument that measures multipel sub-skills of the boader CVS construct. The control of variables strategy inventory (CVSI) was tested with more then 1400 students from 5th to 13th grade. The instrument is aviable in Englisch and German.
Please refer to the instrument as:
CVS Hands-on test instrument
In this CVS hands-on test students have to plan interpret and justify experiments to test a given hypothesis. Student scores are based on their answers in a test booklet and a coding of photos of their experiments. The contexts of the experiments are (1) forces of electromagnets, (2) resistance of wires, (3) melting ice cubes and (4) heat radiation. All tasks have three independent variables with two values each. Students have to choose appropriate materials from a box to design proper experiments. After planning the experiments, but before carrying out the experiments, a research assistant takes a photograph of the students’ experimental designs. After the students conducted the experiment, they had to choose, based on their findings, whether the hypothesis under consideration was supported or not. The test instrument has been used in two studies and is sensitive to intervention effects.
orcid.org/0000-0001-9694-7183