Scientific Inquiry and Reasoning Skills 3 and 4
Research design and statistics are only 20 percent of a science section, and they are the 20 percent that needs no memorising at all.
15 topics · 64 questions
Each topic is a path, not a menu: work down it in order and every question builds on the one above. Click any of them to watch the answer draw itself.
Skill 3: Reasoning About Research Design and Execution
- How it worksHow does a Skill 3 question make you do science by picking apart how a study was actually built?
- Worked exampleWalk me through spotting the design flaw when a memory study tests every subject at eight in the morning.
- Common mistakeWhy isn't a study with an enormous sample automatically a well-designed study?
- Going furtherWhat changes about a design critique when a study is observational and nothing could be manipulated?
Your turnmedium1 of 3
A memory study tests all 151 participants at eight in the morning and reports that recall for word lists is 26 percent better than an earlier study found. Question: what is the design flaw?Independent and Dependent Variables
- How it worksHow do you find the independent and the dependent variable in a study description with no labels in it?
- Worked exampleWalk me through naming the variables when researchers vary room temperature and record reaction times.
- Going furtherWhat changes when a study has two independent variables crossed with each other?
Your turneasy1 of 3
Researchers set room temperature at 17, 22 and 30 degrees Celsius in turn and record how quickly participants press a key when a light appears. Question: which variable is which?Controls: The One Variable Held Still
- How it worksHow does holding every variable but one still turn a messy comparison into a clean one?
- How it worksHow does a control group differ from a control variable, and why does a study usually need both?
- Worked exampleWalk me through what a plant growth study proves without a pot that received no fertiliser at all.
- Common mistakeWhy isn't a before-and-after comparison within the same subjects a proper control?
- Going furtherWhat changes when the control condition itself turns out to have an active effect, as a placebo does?
Your turneasy1 of 3
Fertiliser is applied to 31 pots of seedlings, which grow to 24 centimetres over 10 weeks. No pot was left unfertilised. Question: what does the study establish?Samples, Populations and Generalization
- How it worksHow does the way participants were recruited set a hard ceiling on who the results can apply to?
- Worked exampleWalk me through what a sleep study advertised only inside a campus gym can and cannot claim about adults.
- Common mistakeWhy doesn't a larger sample fix a sample that was drawn from the wrong population?
- Going furtherWhat changes about generalisation when a sample is genuinely representative but very small?
Your turnmedium1 of 3
A sleep study recruits 233 volunteers from notices posted inside a campus gym and reports that adults sleeping under 7 hours show worse glucose control. Question: what may the study claim?Appropriateness, Precision and Accuracy of a Design
- How it worksHow do you judge whether a tool is appropriate for the question a study is actually trying to answer?
- How it worksHow do reliability and validity come apart, so a measure can be perfectly consistent and measure nothing?
- Worked exampleWalk me through evaluating a self-report questionnaire as a way to measure implicit racial bias.
- Common mistakeWhy doesn't reporting a measurement to three decimal places make a study more accurate?
Your turneasy1 of 3
A balance reads the same mass to within 4 thousandths of a gram on repeated weighings, but every reading is 7 hundredths of a gram above the certified value of the reference mass. Question: how should the instrument be described?The Change That Removes a Confound
- How it worksHow does a confound act as a second arrow into your outcome that nobody ever drew on the diagram?
- How it worksHow does randomly assigning subjects break the link between a confound and the treatment they receive?
- Worked exampleWalk me through the single change that fixes a trial where the drug group was also the younger group.
- Common mistakeWhy isn't every variable that differs between two groups a confound?
- Going furtherWhat changes when a confound cannot be removed and can only be measured and adjusted for afterwards?
Your turneasy1 of 3
In a trial the drug arm has mean age 44 and the placebo arm mean age 64, and the outcome is one that improves with youth. Question: what single change fixes this?Research Ethics: Consent, Confidentiality and Risk
- How it worksHow does informed consent require a participant to understand the risks before anything is done to them?
- How it worksHow does confidentiality differ from anonymity when a dataset still holds birth dates and postal codes?
- Common mistakeWhy isn't a study ethical just because every single participant signed a consent form?
- Worked exampleWalk me through what went wrong ethically when subjects were told a study was about vision, not obedience.
- Going furtherWhat changes about the risk calculation when the participants are children or prisoners?
Your turnmedium1 of 3
Participants were told a study concerned visual perception. It in fact measured whether they would continue delivering what they believed were shocks of up to 404 volts on instruction from an experimenter over 33 trials. Question: what was the ethical failure?Skill 4: Data-Based and Statistical Reasoning
- How it worksHow do you read a figure for its pattern first and for its individual numbers only second?
- Worked exampleWalk me through reading a bar chart where the error bars on two groups overlap by half their length.
- Common mistakeWhy doesn't a dramatic-looking rise on a plotted curve mean the effect itself is large?
- Going furtherWhat changes about a figure when its vertical axis does not start at zero?
Your turnhard1 of 3
A bar chart shows group A at 17 units and group B at 30 units, with error bars that overlap by about half their length. Question: what follows about the difference?Central Tendency and Dispersion
- How it worksHow do the mean, the median and the mode pull apart the moment a distribution grows a long tail?
- Worked exampleFind the median and the mean of 2, 3, 3, 4 and 88, and show why the two land so far apart.
- How it worksHow do range, interquartile range and standard deviation each measure spread in a different way?
- Common mistakeWhy isn't the mean always the right summary, and why is household income reported as a median?
- Going furtherWhat changes about a standard deviation when a single extreme outlier is added to the sample?
Your turneasy1 of 3
A sample contains the values 2, 3, 3, 4 and 96. By how much does the mean exceed the median?Random vs Systematic Error
- How it worksHow do random and systematic error look different as darts on a board, scattered against clustered off-centre?
- How it worksHow does averaging more measurements shrink random error while leaving systematic error untouched?
- Worked exampleWalk me through classifying the error when a thermometer reads half a degree high in every single trial.
- Common mistakeWhy doesn't repeating an experiment more times fix a miscalibrated instrument?
- Ties togetherHow does the accuracy-versus-precision split map onto systematic and random error in a study design?
Your turneasy1 of 3
A thermometer reads 7 tenths of a degree high on every one of 72 trials. Question: how should this error be classified, and what will repetition do to it?Significance Levels and Confidence Intervals
- How it worksHow do two overlapping distributions separate as a sample grows until the difference counts as real?
- How it worksHow does a 95 percent confidence interval that contains zero tell you an effect might not exist?
- Worked exampleWalk me through what a p-value of 0.03 against an alpha of 0.05 licenses you to conclude, and what it does not.
- Common mistakeWhy doesn't a p-value of 0.04 mean there is a 96 percent chance the hypothesis is true?
- Going furtherWhat changes about significance when one dataset is tested twenty different ways?
Your turnhard1 of 3
A trial of 151 thousand participants reports a difference of 5 tenths of a point on a 97-point scale, with p less than 0.001. Question: what does that combination license?Association vs Causation
- How it worksHow does random assignment convert a bare association into evidence for a cause?
- Worked exampleWalk me through the three explanations available when helmet use correlates with fewer head injuries.
- Common mistakeWhy doesn't a long-running longitudinal study establish causation on its own?
- Going furtherWhat changes when the only available evidence is cross-sectional and nothing can be randomised?
Your turnmedium1 of 3
Across 31 regions, cyclists who wear helmets suffer 49 percent fewer head injuries than those who do not. Question: which explanations are available?Statements That Go Beyond the Evidence
- How it worksHow do you draw the line where the evidence stops and mark everything past it as unsupported?
- Worked exampleWalk me through why the drug cures depression overreaches a 12-week trial of mood scores in 40 adults.
- Common mistakeWhy isn't a conclusion supported just because the researchers printed it in their own abstract?
Your turneasy1 of 3
A 14-week trial in 49 adults measures mood scores and reports a mean improvement of 9 points. The abstract concludes that the drug cures depression. Question: what is wrong with the conclusion?Alternative Explanations for the Same Data
- How it worksHow do you generate rival explanations for one set of data by changing what caused what?
- Worked exampleWalk me through three explanations for students who study with music scoring lower on a recall test.
- Going furtherWhat changes when an alternative explanation would require a variable that nobody measured?
- Common mistakeWhy isn't the researchers' preferred explanation the default answer on a data question?
Your turnmedium1 of 3
Students who chose to study with music scored 10 points lower on a recall test than those who chose silence, across 151 participants. Question: which set of explanations is complete?Whether a Graph Fits the Observations
- How it worksHow do you check whether a chosen chart type can even express the relationship hiding in the data?
- Worked exampleWalk me through why a pie chart is the wrong picture for a measurement taken once an hour for a day.
- Common mistakeWhy isn't a line drawn through scattered points evidence that the relationship is linear?
- Going furtherWhat changes about a figure's honesty when its axis is truncated or its scale is logarithmic?
Your turneasy1 of 3
A concentration is measured once an hour for 20 hours and the results are drawn as a pie chart with 20 wedges sized by the measured value. Question: what is wrong with the representation?AP®, SAT® and PSAT/NMSQT® are trademarks registered by the College Board (PSAT/NMSQT® jointly with the National Merit Scholarship Corporation). ACT® is a registered trademark of ACT Education Corp. GRE® is a registered trademark of ETS. LSAT® is a registered trademark of the Law School Admission Council. MCAT® is a registered trademark of the Association of American Medical Colleges. ISEE® and SSAT® are registered trademarks of their owners. None of these organizations is affiliated with, or endorses, Jentoo.