The detailed design process of a high-quality randomized controlled stuttering trial has started

Hilde, Linn, Åse and I are about to arrive in Stavanger to participate in a one-week course about randomized controlled trials (RCT) in social science at the University of Stavanger. The course is led by Eric Perry Bettinger from Standford School of Education, and we will actually be students for five days from 9:00 a.m. to 4:00 p.m. daily and have a reading list of seven pages full of references.

I am still a bit old fashioned when it comes to reading; I like paper versions of articles best so that I can make notes (e.g., inferences, comments, associations, questions) with a pen. I am aware of the latest technology, such as the Apple pencil that can be used to make comments by hand on the iPad, but I still prefer jotting down notes the old way. I do not know if it is because of my bad eyesight or the fact that it gives me a better overview of the text’s spatial landmarks and thereby reduces the cognitive load required to orient myself to the text. Or maybe it’s the depth of reading that is possible (e.g. effectively revisiting parts). It may simply be due to habit (reading rigidity). Luckily, the three others in the EST-team also prefer paper, so I am comforted by the feeling that I am not so outdated (read ‘old’) and it make me optimistic about my potential for learning the course materials. A systematic review of the research on the effects of reading media on reading comprehension by Delgadoa, Vargasb, Ackermanc and Salmeróna (2018) actually concluded that paper-based reading yields better comprehension outcomes than digital-based reading. So at least I could say that my preferred strategy is based on results from international research.

My thanks go to Linn and Åse for printing all the articles on the reading list and sending them to the Repro Central at the University for binding as books. They became no less than five “large books”, so then I knew what to do in the weekend. On Saturday, I opened the first book of articles and started to read enthusiastically Understanding and Misunderstanding Randomized Controlled Trials, by Deaton and Cartwright (2017). The content reminded me very much of the views on RCTs of a colleague at the University of Oslo, Tone Kvernbekk. She underlined RCTs pitfalls and the need for a combination of methods to determine what works and why.

On the one hand, the Deaton and Cartwright (2017) article critically highlights the pitfalls of the design of RCTs, such as potential confounders; on the other hand, it stresses the fact that other designs also have the same challenges in addition to their own. Therefore, RCT is still broadly considered to be the gold standard of designs. Neveertheless, the authors highlight the importance of having a realistic understanding of the strengths of RCT design and of its limitations and pointed to three common misunderstandings of it’s quality:

1)    Randomization ensures that the treatment and control groups differ only with regards to the treatment variable

2)    Randomization provides precise and unbiased estimates of average treatment effects

3)    Standard significant tests of RCTs are reliable

In addition, the athors underline the need for replication studies and complementary study designs to gain insights into the results of different samples and contexts and why the target intervention is effective and for whom (individual variations).

An update of our course experiences will follow shortly.

All the best,

Kari-Anne

-----
References:

Delgadoa, P., Vargasb, C., Ackermanc, R., & Salmeróna, L. (2018). Don't throw away your printed books: A meta-analysis on the effects of reading media on reading comprehension. Educational Research Review, 25, 23-38.

Deaton, A. & Cartwright, N. (2017). Understanding and misunderstanding randomized controlled trials. NBER working paper 22595. Cambridge: National bureau of economic research.