Dec 20, 2021
In this “Ask Me Anything” (AMA)
episode, Peter and Bob dive deep into all things related to
studying studies to help one sift through all the noise to find the
signal. They define the various types of studies, how a study
progresses from idea to execution, and how to identify study
strengths and limitations. They explain how clinical trials work,
as well as the potential for bias and common pitfalls to watch out
for. They dig into key factors that contribute to the rigor (or
lack thereof) of an experiment, and they discuss how to measure
effect size, differentiate relative risk from absolute risk, and
what it really means when a study is statistically significant.
Finally, Peter lays out his personal process when reading through
scientific papers.
If you’re not a subscriber and listening on a
podcast player, you’ll only be able to hear a preview of the AMA.
If you’re a subscriber, you can now listen to this full episode on
your private
RSS feed or on our website at the AMA
#30 show notes page. If you are not a subscriber, you can learn
more about the subscriber benefits here.
We discuss:
- The ever changing landscape of
scientific literature [2:15];
- The process for a study
to progress from idea to design to execution
[4:15];
- The various types of studies and
how they differ [7:30];
- The different phases of a
clinical trial [19:15];
- Observational studies and the
potential for bias [26:30];
- Experimental studies:
Randomization, blinding, and other factors that make or break a
study [44:00];
- Power, p-values, and statistical
significance [56:15];
- Measuring effect size: Relative
risk vs. absolute risk, hazard ratios, and “Number Needed to Treat”
[1:07:45];
- How to interpret confidence
intervals [1:17:30];
- Why a study might be stopped
before its completion [1:23:45];
- Why only a fraction
of studies are ever published and how to combat
publication bias [1:31:30];
- Why certain journals are more
respected than others [1:40:30];
- Peter’s process when reading a
scientific paper [1:43:45]; and
- More.