When can you say ‘something’ works in education?

  • Print

Does ‘it’ work?

When can you say ‘something’ works in education?

Willem de Vlaming, February 2020

 

Can only be said that 'something works' if a defined effect of a pedagogical or didactic intervention (re)occurs: (1) with repetition over time; (2) when applied in different contexts; (3) when applied by different educational professionals; (4) with different, but comparable, target groups and effect-groups?

Or can also be said that 'something works' if it is plausible in a specific context that: (1) a combination of interventions; (2) through collaboration of professionals and non-professionals; (3) in a complex system: (4) has made a difficulty to measure contribution to (5) achieving a desirable effect; (6) in which the cause and effect relation and the size of the effect can be only be determined (inter) subjectively?

 

Three Dutch educational researchers promote a thorough evidence based ‘spring cleaning’ and have had a critical look at four domains of educational myths: (1) myths about learning; (2) neuromyths; (3) myths about technology in education; and (3) myths in education policy.1) But what are criteria to determine if something works in educational practice? What is a damaging myth, and what is a useful principle or rule of thumb? Some types of bias that make it hard to have a critical look at educational myths

  • patternicity bias: searching for patterns in random events;
  • confirmation bias: seeking evidence for what we already believe;
  • hindsight bias: attributing meaning to past events from the knowledge we have today. 2)

But, from a different perspective, Biesta states that, 'education can only be successful through weak connections between communication and interpretation, of interruption and response.' Professional practice is not a slavish application of recipes - based on what has happened or has been proven in the past. The core of professionalism in education consists of virtuoso substantiated judgment and acting in a complex practice — focused on a future worth pursuing.3)Evidence-based positivism fails to recognize that educational practice is colored by the views of those involved - both in terms of goals and ways of achieving them. And evidence-based positivism focuses on 'powerful' relationships between 'inputs' and revenues. Something that can be called 'replication and causality bias'.4)

 

Perspectives on 'Does' IT 'work?'

For one school of thought, generalizability through stripping out contextual variables is fundamental, whilst, for another generalizations that say little about the context have little that is useful to say about human behavior.’ Cohen, Manion & Morisson.5)

 

For an objectivist, external validity of knowledge is essential, for a subjectivist internal validity matters most. To draw generally valid (externally valid) conclusions, research is designed in such a way that situational factors can be filtered out so that something general can be said about a cause - effect relation. Teachers however have to deal with a complex situational reality, with specific school characteristics, teacher qualities, and student characteristics. These are not factors to filter out, this is the reality to deal with.

 

Does it work?

When can you say something works in #education?

An intervention, that is proven to be effective in replicable research, may not work in a specific educational practice. And what works in a specific practice (by subjectivist standards)  may be impossible to prove in replication (by objectivist standards).

 

Alternative bases for interpreting social reality

Partly based on: Cohen; Manion; & Morrison: Research Methods in Education. p.8

Objectivism

The world exists and is knowable as it really is.

Science must identify universal social laws and formulate explanatory models for human behavior.

Science performs (quasi) experimental research into (single) causal relationships.

Subjectivism

The world exists but different people construe it in different ways.

Science must identify how people interpret, give meaning and shape the world in which they live

Science investigates complex relationships, patterns, systems and meaning.

 

A

THEORY ORIENTED

Prove

B

O

B

J

E

C

T

I

V

I

S

M

R

E

D

U

C

T

I

O

N

S

M

validity, reliability, generalizability 

(Developing theory)

H

O

L

I

S

M

S

U

B

J

E

C

T

I

V

I

S

M

What works (almost) always in comparable practices?

(General valid knowledge, theory, concepts, insights, explanatory models)

Causality  & replication

Traceability  & plausibility

Separate details

Relations between variables.

EXTERNAL VALIDITY

The whole, patterns, structures. Relations between patterns.

INTERNAL VALIDITY

Diagnostics — Intervention

Understanding & Developing

What works (probably) in this practice here, now and in the future?

(Developing practice)

validation process, traceability, usability

C

Improve

PRACTICE ORIENTED

D

 

To do or not to do, that is the question.

In the objectivistic assessment of an intervention four outcomes are possible, it is: (1) proven that ‘it’ works; (2) not proven that ‘it’ works; (3) proven that ‘it’ does not work; (4) proven that ‘it’ is harmful. From a subjectivist perspective, the main distinction is, whether or not it is plausible (plausible) that ‘it’ will be effective.

If there are elements in an intervention that are objectively proven to work, it is still relevant to answer the question, whether it is plausible in the specific situation that 'it works'. If that is not the case, it does not make much sense to apply that intervention in the specific situation. 

 

To do or not to do, that is the question.

 

SUBJECTIVISTIC - HOLISTIC

OBJECTIVISTIC

REDUCTIONISTIC

Effectivity

 is 

plausible

Effectivity 

is NOT

plausible

It is proven that ‘it’ works

DO

explain why DO or Don’t  DO

It is not proven that ‘it’ works

explain why DO or Don’t  DO

Don’t  DO

It is proven that ‘it' does not ‘work’

explain why DO or Don’t  DO

Don’t  DO

It is proven that ‘it’  is harmful

Don’t  DO

Don’t  DO

 

What do you do with interventions that are not (yet) proven objectivistically, but are plausible to be effective in your specific practice? What makes it plausible that an intervention that does not work from an objectivist perspective, will work in your practice? 

Be clear when taking positions with regard to 'what works' and 'what is allowed' and supporting this. From which paradigm (mix of paradigms) do you build your argumentation? Which reasoning supports your position (s)? Which criticism is possible from other perspectives? What makes your reasoning relevant to your position (s)? On which facts, theories, concepts, principles, and beliefs are your views based?

Make sure that the ambition and goals of interventions are clearly defined, that relevance is substantiated and the justification is substantiated. Make vision, norms and values explicit. Make it plausible that a strategy or intervention will be effective in the intended situation and context.

 

 

The beautiful risk of education demands a lot of explanation and substantiation of choices and the quality and effectiveness of practice — not the blind following of objectivist proven recipes.

 

Notes

  1. De Bruyckere, Kirschner en Hulshof: Jongens zijn slimmer dan meisjes.. Some of the myths they address are: 'People have different learning styles,' 'Discovering is better than getting an explanation,' 'The left hemisphere is analytical, the right is creative,' 'Men have a different kind of brain than women ',' A small class is better ',' More money ensures better education '
  2. De Bruyckere, Kirschner en Hulshof: Jongens zijn slimmer dan meisjes. p.12
  3. Biesta: Het prachtige risico van onderwijs. p.199-200
  4. Biesta: Het prachtige risico van onderwijs. (The beautiful risk of education) p.177
  5. Cohen; Manion; & Morrison: Research Methods in Education. p.186

 

Literature

Biesta, G. J. (2015). Het prachtige risico van onderwijs. Culemborg: Phronese. 

Cohen, L.; Manion, L. & Morrison, K. (2011). Research Methods in Education. Seventh edition. London, New York: Routledge. p. 179

De Bruyckere, P.; Kirschner, P. & Hulshof, C. (2018, 2016 ed). Jongens zijn slimmer dan meisjes. 35 mythes over leren en onderwijs. Culemborg: Lannoo |Anderz

Vlaming, W. de (2018). De kunst van het (op ontwikkeling gericht) keuren van het eigen vlees. Aalsmeer: WdV-Advies. [op: www.wdv-advies.com ] via http://www.wdv-advies.com/index.php/resources

Vlaming, W. de (2019). Samen bouwen aan een professionele leergemeenschap. TIBtool 29. Dordrecht: GARANT info via http://www.wdv-advies.com/index.php/blogs/37-blog/115-plgbouwen 

Vlaming, W. de (2019). Werkt ‘HET’? Wanneer zeg je in het onderwijs dat ‘iets’ werkt, en dat ‘het’ goed is om te doen? Op: PlatformPraktijkontwikkeling.nl. Utrecht: WOSO