Postgraduate Readiness practice
Free Postgraduate Readiness practice with 75 original tasks, worked guidance and links to timed practice and a full mock. These tasks are not live-bank items.
What you will learn
- Practise analytical reasoning in Postgraduate Readiness contexts.
- Practise quantitative data literacy in Postgraduate Readiness contexts.
- Practise communication in Postgraduate Readiness contexts.
- Practise ai digital literacy in Postgraduate Readiness contexts.
- Practise ethical judgement in Postgraduate Readiness contexts.
These 75 tasks teach the Postgraduate Readiness reasoning process. They are public preparation, not a scored sitting, and they do not reuse protected assessment items.
Before you begin
- Use only the information given. State what remains unknown.
- Attempt a task before opening the guidance.
- Write the evidence for the choice, not only the final answer.
- Use timed practice after an untimed pass, then the full mock.
Practice activities
Analytical Reasoning · foundational
Which assumption must hold for that conclusion to follow? Situation: A methods section that does not pre-register outcomes. Four outcomes were collected; the paper highlights the only one with p<0.05. A side count of 4 used a different definition from 9.
Review answer guidance
Name the missing premise. If it is unchecked, the conclusion is not shown. The claim in play is that the highlighted outcome was the planned test. selecting after seeing p-values is not the same as a planned outcome The extra figure (A side count of 4 used a different definition from 9) is part of what you must not ignore.
Analytical Reasoning · moderate
Which inference is actually supported by the stated facts? Situation: Two studies of the same intervention with opposite signs. Study A (n=400) finds harm; study B (n=24) finds benefit; a blog cites only B. The working copy is dated 9 days later than a 20-page cited copy.
Review answer guidance
Keep the conclusion inside the evidence. Do not import a cause that was not given. The claim in play is that the intervention helps. the larger study and the omitted sign are part of the evidence The extra figure (The working copy is dated 9 days later than a 20-page cited copy) is part of what you must not ignore.
Analytical Reasoning · challenging
What alternative explanation still accounts for the difference? Situation: A systematic review mixing RCTs and blog posts. The abstract says 'the literature shows'; 6 of 11 included items are unreviewed posts. 14 of 31 rows were added after the first total was announced.
Review answer guidance
A change that arrives with another change is not a cause until those extras are ruled out. The claim in play is that the literature has been reviewed. unreviewed posts are not the same class of evidence as RCTs The extra figure (14 of 31 rows were added after the first total was announced) is part of what you must not ignore.
Analytical Reasoning · foundational
Which of the two arguments is better supported, and why? Situation: A dataset with 18 percent missing follow-up. The paper reports complete-case effects and does not describe the missingness. A second observer recorded 19 where the first sheet shows 42.
Review answer guidance
Score each argument by the evidence it actually cites, not by how confident it sounds. The claim in play is that the reported effect is the population effect. 18 percent missing needs a sensitivity check The extra figure (A second observer recorded 19 where the first sheet shows 42) is part of what you must not ignore.
Analytical Reasoning · moderate
What counterexample would be enough to refute the general rule? Situation: A qualitative codebook with collapsing categories. Twelve codes were merged into two after seeing which story looked cleaner. The window used for 5 days is not the window used for 16 days.
Review answer guidance
One well-formed contrary case is enough to show that an unrestricted rule fails. The claim in play is that the two themes were discovered in the data. merging after seeing the story is a design choice that must be reported The extra figure (The window used for 5 days is not the window used for 16 days) is part of what you must not ignore.
Analytical Reasoning · challenging
Which constraint is being ignored in the recommended action? Situation: A grant proposal claiming certainty from n=12. The proposal says the pilot 'establishes efficacy'. A note in the margin says 10 cases were excluded without a rule, against 27 kept.
Review answer guidance
List the limits that still bind. An option that needs a resource you do not have is not available. The claim in play is that n=12 establishes efficacy. a pilot can estimate feasibility, not establish efficacy The extra figure (A note in the margin says 10 cases were excluded without a rule, against 27 kept) is part of what you must not ignore.
Analytical Reasoning · foundational
How far can the claim fairly be extended? Situation: A replication preprint that cannot share data. The authors cite commercial sensitivity; the claim is a public-health effect. The same label covers 15 events in one table and 38 in the other.
Review answer guidance
Match the claim's scope to the sample, time window and population that were actually observed. The claim in play is that the effect can be taken as checked. an unreplicable public-health claim is weakly checked The extra figure (The same label covers 15 events in one table and 38 in the other) is part of what you must not ignore.
Analytical Reasoning · moderate
What evidence is required before the claim can be treated as shown? Situation: A meta-analysis without a search strategy. Nine papers, all from one lab, all positive. An appendix lists 20 exceptions that the summary of 12 does not mention.
Review answer guidance
Identify the comparison, baseline or source that is absent, and withhold the stronger wording until it arrives. The claim in play is that the meta-analysis represents the field. one lab and no search strategy is a convenience collection The extra figure (An appendix lists 20 exceptions that the summary of 12 does not mention) is part of what you must not ignore.
Analytical Reasoning · challenging
Which wording is stronger than the evidence permits? Situation: An ethics form that waives consent for convenience. The reason given is 'students are used to being studied'. A side count of 6 used a different definition from 23.
Review answer guidance
Replace 'proves' and 'always' with what the observations actually cover. The claim in play is that convenience is a waiver ground. habit is not a lawful or ethical substitute for consent where consent is required The extra figure (A side count of 6 used a different definition from 23) is part of what you must not ignore.
Analytical Reasoning · foundational
Which disputed definition would change the result? Situation: A model that treats correlation as a mechanism. Ice cream sales and incident counts rise together; the model includes sales as a cause of incidents. The working copy is dated 11 days later than a 34-page cited copy.
Review answer guidance
If two sides count different things, agree the definition before arguing about the number. The claim in play is that the coefficient is a mechanism. a seasonal confounder can produce both The extra figure (The working copy is dated 11 days later than a 34-page cited copy) is part of what you must not ignore.
Analytical Reasoning · moderate
How did the way the cases were chosen affect the conclusion? Situation: A thesis chapter that synthesises conflicting theories. The chapter lists both theories and concludes they 'complement each other' without stating where they disagree. 16 of 45 rows were added after the first total was announced.
Review answer guidance
A volunteer, leftover or convenience sample cannot stand in for the group the claim names. The claim in play is that the conflict is resolved. complementary is not a synthesis if the contradiction is unnamed The extra figure (16 of 45 rows were added after the first total was announced) is part of what you must not ignore.
Analytical Reasoning · challenging
Which other variable moved at the same time as the supposed cause? Situation: A journal review asking for robustness checks. The author replies 'results were similar' with no table. A second observer recorded 21 where the first sheet shows 19.
Review answer guidance
Name the co-occurring change and say what comparison would separate the two. The claim in play is that robustness was shown. similar is not a check you can audit without the numbers The extra figure (A second observer recorded 21 where the first sheet shows 19) is part of what you must not ignore.
Analytical Reasoning · foundational
What comparison is required before calling one option better? Situation: A p-hacking pattern. Twenty tests, one reported; the reported test used a covariate chosen after looking at residuals. The window used for 7 days is not the window used for 30 days.
Review answer guidance
Better than what, for whom, over which period, on which measure. The claim in play is that the reported test is confirmatory. covariate choice after residuals is exploratory The extra figure (The window used for 7 days is not the window used for 30 days) is part of what you must not ignore.
Analytical Reasoning · moderate
When would the recommended rule not apply? Situation: A HARKing suspicion. The introduction reads as if the surprising finding was predicted; the lab notebook shows it was not. A note in the margin says 12 cases were excluded without a rule, against 41 kept.
Review answer guidance
State the boundary condition. A rule without a stated exception is being oversold. The claim in play is that the narrative is the hypothesis. a hypothesis written after the result is not a prediction The extra figure (A note in the margin says 12 cases were excluded without a rule, against 41 kept) is part of what you must not ignore.
Analytical Reasoning · challenging
Who has the burden of proof for the disputed claim? Situation: A forest plot with one huge study. The pooled effect tracks that study; smaller studies go the other way. The same label covers 17 events in one table and 15 in the other.
Review answer guidance
The party asserting a general effect must produce the comparison; the other party need not disprove a slogan. The claim in play is that the pool is a consensus. the pool may be one study in costume The extra figure (The same label covers 17 events in one table and 15 in the other) is part of what you must not ignore.
Analytical Reasoning · foundational
What mechanism is asserted, and what would count against it? Situation: A qualitative quote used to prove prevalence. One vivid interview line is introduced with 'participants felt'. An appendix lists 22 exceptions that the summary of 26 does not mention.
Review answer guidance
A story about how an effect happens is not evidence that it happened. The claim in play is that the line is a prevalence claim. one quote is existence, not prevalence The extra figure (An appendix lists 22 exceptions that the summary of 26 does not mention) is part of what you must not ignore.
Analytical Reasoning · moderate
Which two statements in the account cannot both be true? Situation: A data-availability statement. It says 'available on request'; three requests in a year were refused. A side count of 8 used a different definition from 37.
Review answer guidance
Put the statements side by side. If they conflict, at least one needs a source or a revision. The claim in play is that the data are available. refused requests contradict the statement The extra figure (A side count of 8 used a different definition from 37) is part of what you must not ignore.
Analytical Reasoning · challenging
Why is the measured proxy a poor stand-in for the stated outcome? Situation: A multiple-comparisons problem. Forty questionnaire items, two 'significant' at 0.05, no correction, no pre-specified items. The working copy is dated 13 days later than a 11-page cited copy.
Review answer guidance
Say what the proxy actually captures and what part of the outcome it misses. The claim in play is that two findings were shown. two of forty at 0.05 is close to chance The extra figure (The working copy is dated 13 days later than a 11-page cited copy) is part of what you must not ignore.
Analytical Reasoning · foundational
What time-order problem undermines the causal claim? Situation: A translator who is also a co-author and advocate. Interview translations were done by a campaigner for the intervention. 18 of 22 rows were added after the first total was announced.
Review answer guidance
A cause must precede its effect. If the 'result' was already trending, the intervention is not shown to be responsible. The claim in play is that the translations are a neutral record. advocacy plus translation is a bias risk that needs independent checking The extra figure (18 of 22 rows were added after the first total was announced) is part of what you must not ignore.
Analytical Reasoning · moderate
From this case, which generalisation is unwarranted? Situation: A power analysis written after data collection. The analysis uses the observed effect to say the study was 'adequately powered'. A second observer recorded 4 where the first sheet shows 33.
Review answer guidance
A single setting does not licence a rule about all settings of that type. The claim in play is that this is a valid power calculation. post-hoc power using the observed effect is circular The extra figure (A second observer recorded 4 where the first sheet shows 33) is part of what you must not ignore.
Analytical Reasoning · challenging
Which unstated premise does the author need? Situation: A figure without a sample size in the legend. Error bars are shown; n is only in a supplement the journal PDF omits. The window used for 9 days is not the window used for 44 days.
Review answer guidance
Write the suppressed step. If that step is false, the argument collapses. The claim in play is that the figure is interpretable alone. error bars without n are under-specified The extra figure (The window used for 9 days is not the window used for 44 days) is part of what you must not ignore.
Analytical Reasoning · foundational
Which rival hypothesis is most worth testing next? Situation: A secondary analysis of open data. The new paper does not cite the original collection protocol's exclusion rules and silently includes the excluded rows. A note in the margin says 14 cases were excluded without a rule, against 18 kept.
Review answer guidance
Pick the alternative that would produce the same observations, then name the test that could split them. The claim in play is that the open data can be used as-is. the protocol's exclusions are part of the dataset's meaning The extra figure (A note in the margin says 14 cases were excluded without a rule, against 18 kept) is part of what you must not ignore.
Analytical Reasoning · moderate
Which quantifier in the claim is unjustified? Situation: A 'nationally representative' panel. Weights are described in a sentence; the unweighted n in a key cell is 6. The same label covers 19 events in one table and 29 in the other.
Review answer guidance
All, most, some and none are different claims. Keep the one the data can support. The claim in play is that the cell supports a national estimate. a weighted story on n=6 is still a thin cell The extra figure (The same label covers 19 events in one table and 29 in the other) is part of what you must not ignore.
Analytical Reasoning · challenging
Where does the analogy break? Situation: A theory paper that cites only supporting cases. Known counterexamples exist in a neighbouring literature and are uncited. An appendix lists 5 exceptions that the summary of 40 does not mention.
Review answer guidance
Name the property that holds in the source case but not in the target case. The claim in play is that the theory has been stress-tested. uncited counterexamples are a selection of evidence The extra figure (An appendix lists 5 exceptions that the summary of 40 does not mention) is part of what you must not ignore.
Analytical Reasoning · foundational
What decision rule is being used, and is it stated? Situation: A lab's internal replication. The same team, same code, same data produce the same number. A side count of 10 used a different definition from 14.
Review answer guidance
If the threshold is implicit, make it explicit before treating the verdict as fair. The claim in play is that the finding is replicated. that is a rerun, not an independent replication The extra figure (A side count of 10 used a different definition from 14) is part of what you must not ignore.
Analytical Reasoning · moderate
Which source is weakest, and why does that matter here? Situation: A mixed-methods claim. The qualitative theme is used to explain a quantitative null as 'really positive'. The working copy is dated 15 days later than a 25-page cited copy.
Review answer guidance
Match the source type to the claim type. An opinion column cannot carry a prevalence claim. The claim in play is that the null is overturned by quotes. quotes cannot reverse a null without a stated mixed-methods design for doing so The extra figure (The working copy is dated 15 days later than a 25-page cited copy) is part of what you must not ignore.
Analytical Reasoning · challenging
What was left out that would most change the decision? Situation: An effect size reported only as 'significant'. p=0.049, n=1,200, effect 0.04 on a 100-point scale. 20 of 36 rows were added after the first total was announced.
Review answer guidance
Identify the missing group, time period or cost, and say how it would move the recommendation. The claim in play is that significance is the finding. the effect is tiny; p is not the practical result The extra figure (20 of 36 rows were added after the first total was announced) is part of what you must not ignore.
Analytical Reasoning · foundational
What would you need to see to reverse the current conclusion? Situation: A preprint cited as 'the established evidence'. The preprint appeared last week and has no reviews. A second observer recorded 6 where the first sheet shows 10.
Review answer guidance
State a specific observation that would count the other way. If none exists, the claim is unfalsifiable. The claim in play is that established is fair. unreviewed last week is not established The extra figure (A second observer recorded 6 where the first sheet shows 10) is part of what you must not ignore.
Analytical Reasoning · moderate
How does aggregation hide an important subgroup difference? Situation: A supervisor's preferred measure replacing the registered one. The registered primary outcome is dropped because it 'did not work'. The window used for 11 days is not the window used for 21 days.
Review answer guidance
A total can be right while every important subgroup moves the other way. Ask for the split. The claim in play is that the replacement is still primary. the registered outcome remains primary; the replacement is additional and must be labelled The extra figure (The window used for 11 days is not the window used for 21 days) is part of what you must not ignore.
Analytical Reasoning · challenging
What is the cost of doing nothing, given only the stated facts? Situation: A consent form that bundles optional future contact with required study consent. One tick box covers both. A note in the margin says 16 cases were excluded without a rule, against 32 kept.
Review answer guidance
Inaction is an option. Compare it with the proposal using the same outcome and the same unknowns. The claim in play is that the tick is valid consent for both. bundling optional contact with required consent is not freely given for the optional part The extra figure (A note in the margin says 16 cases were excluded without a rule, against 32 kept) is part of what you must not ignore.
Quantitative Data Literacy · foundational
What percentage change is actually shown, and what is still unknown? Situation: A methods section that does not pre-register outcomes. Four outcomes were collected; the paper highlights the only one with p<0.05. The same label covers 21 events in one table and 43 in the other.
Review answer guidance
Compute the change from the stated baseline. Percent change without a base, period or definition is incomplete. The claim in play is that the highlighted outcome was the planned test. selecting after seeing p-values is not the same as a planned outcome The extra figure (The same label covers 21 events in one table and 43 in the other) is part of what you must not ignore.
Quantitative Data Literacy · moderate
Which ratio should be compared, rather than the raw totals? Situation: Two studies of the same intervention with opposite signs. Study A (n=400) finds harm; study B (n=24) finds benefit; a blog cites only B. An appendix lists 7 exceptions that the summary of 17 does not mention.
Review answer guidance
Divide each count by its own eligible population and period before ranking. The claim in play is that the intervention helps. the larger study and the omitted sign are part of the evidence The extra figure (An appendix lists 7 exceptions that the summary of 17 does not mention) is part of what you must not ignore.
Quantitative Data Literacy · challenging
How should the base rate change the reading of the count? Situation: A systematic review mixing RCTs and blog posts. The abstract says 'the literature shows'; 6 of 11 included items are unreviewed posts. A side count of 12 used a different definition from 28.
Review answer guidance
A large count from a large group can be a lower rate than a small count from a small group. The claim in play is that the literature has been reviewed. unreviewed posts are not the same class of evidence as RCTs The extra figure (A side count of 12 used a different definition from 28) is part of what you must not ignore.
Quantitative Data Literacy · foundational
The chart's axis does not start at zero. What is the responsible reading? Situation: A dataset with 18 percent missing follow-up. The paper reports complete-case effects and does not describe the missingness. The working copy is dated 17 days later than a 39-page cited copy.
Review answer guidance
Describe the difference in units, not the visual height. A truncated axis exaggerates change. The claim in play is that the reported effect is the population effect. 18 percent missing needs a sensitivity check The extra figure (The working copy is dated 17 days later than a 39-page cited copy) is part of what you must not ignore.
Quantitative Data Literacy · moderate
What is the chance of the stated event on the information given? Situation: A qualitative codebook with collapsing categories. Twelve codes were merged into two after seeing which story looked cleaner. 22 of 13 rows were added after the first total was announced.
Review answer guidance
Use the given counts as the sample space. Do not import an outside probability. The claim in play is that the two themes were discovered in the data. merging after seeing the story is a design choice that must be reported The extra figure (22 of 13 rows were added after the first total was announced) is part of what you must not ignore.
Quantitative Data Literacy · challenging
How should missing values change the reported rate? Situation: A grant proposal claiming certainty from n=12. The proposal says the pilot 'establishes efficacy'. A second observer recorded 8 where the first sheet shows 24.
Review answer guidance
Recalculate under more than one assumption for the missing cases and say whether the ranking survives. The claim in play is that n=12 establishes efficacy. a pilot can estimate feasibility, not establish efficacy The extra figure (A second observer recorded 8 where the first sheet shows 24) is part of what you must not ignore.
Quantitative Data Literacy · foundational
Which weighted comparison is the fair one? Situation: A replication preprint that cannot share data. The authors cite commercial sensitivity; the claim is a public-health effect. The window used for 13 days is not the window used for 35 days.
Review answer guidance
Weight by the size of each group, not by the number of groups. The claim in play is that the effect can be taken as checked. an unreplicable public-health claim is weakly checked The extra figure (The window used for 13 days is not the window used for 35 days) is part of what you must not ignore.
Quantitative Data Literacy · moderate
What order-of-magnitude estimate is defensible from the figures given? Situation: A meta-analysis without a search strategy. Nine papers, all from one lab, all positive. A note in the margin says 18 cases were excluded without a rule, against 9 kept.
Review answer guidance
Round to the precision the data support. A precise-looking figure from rough inputs is a false exactness. The claim in play is that the meta-analysis represents the field. one lab and no search strategy is a convenience collection The extra figure (A note in the margin says 18 cases were excluded without a rule, against 9 kept) is part of what you must not ignore.
Quantitative Data Literacy · challenging
What per-person figure is required before comparing the two groups? Situation: An ethics form that waives consent for convenience. The reason given is 'students are used to being studied'. The same label covers 4 events in one table and 20 in the other.
Review answer guidance
Totals are not comparable when the groups differ in size. Divide by headcount. The claim in play is that convenience is a waiver ground. habit is not a lawful or ethical substitute for consent where consent is required The extra figure (The same label covers 4 events in one table and 20 in the other) is part of what you must not ignore.
Quantitative Data Literacy · foundational
The two percentages cover different time windows. What must you do first? Situation: A model that treats correlation as a mechanism. Ice cream sales and incident counts rise together; the model includes sales as a cause of incidents. An appendix lists 9 exceptions that the summary of 31 does not mention.
Review answer guidance
Put both rates on the same period, or refuse the comparison. The claim in play is that the coefficient is a mechanism. a seasonal confounder can produce both The extra figure (An appendix lists 9 exceptions that the summary of 31 does not mention) is part of what you must not ignore.
Quantitative Data Literacy · moderate
Which part-whole mistake is being made? Situation: A thesis chapter that synthesises conflicting theories. The chapter lists both theories and concludes they 'complement each other' without stating where they disagree. A side count of 14 used a different definition from 42.
Review answer guidance
A percentage of a subset is not the same as a percentage of the whole. The claim in play is that the conflict is resolved. complementary is not a synthesis if the contradiction is unnamed The extra figure (A side count of 14 used a different definition from 42) is part of what you must not ignore.
Quantitative Data Literacy · challenging
What is the conditional rate, not the overall rate? Situation: A journal review asking for robustness checks. The author replies 'results were similar' with no table. The working copy is dated 19 days later than a 16-page cited copy.
Review answer guidance
Restrict the denominator to the group named in the question. The claim in play is that robustness was shown. similar is not a check you can audit without the numbers The extra figure (The working copy is dated 19 days later than a 16-page cited copy) is part of what you must not ignore.
Quantitative Data Literacy · foundational
What expected value follows from the stated chances and amounts? Situation: A p-hacking pattern. Twenty tests, one reported; the reported test used a covariate chosen after looking at residuals. 5 of 27 rows were added after the first total was announced.
Review answer guidance
Multiply each amount by its probability and add. Do not pick the most vivid outcome. The claim in play is that the reported test is confirmatory. covariate choice after residuals is exploratory The extra figure (5 of 27 rows were added after the first total was announced) is part of what you must not ignore.
Quantitative Data Literacy · moderate
Why might the mean mislead relative to the median here? Situation: A HARKing suspicion. The introduction reads as if the surprising finding was predicted; the lab notebook shows it was not. A second observer recorded 10 where the first sheet shows 38.
Review answer guidance
A single extreme value pulls the mean. Report both, or use the median when the tail matters. The claim in play is that the narrative is the hypothesis. a hypothesis written after the result is not a prediction The extra figure (A second observer recorded 10 where the first sheet shows 38) is part of what you must not ignore.
Quantitative Data Literacy · challenging
The figure is an index, not a count. What does a one-point change mean? Situation: A forest plot with one huge study. The pooled effect tracks that study; smaller studies go the other way. The window used for 15 days is not the window used for 12 days.
Review answer guidance
Read the index definition. A point on an index is not a person, a pound or a percent unless the documentation says so. The claim in play is that the pool is a consensus. the pool may be one study in costume The extra figure (The window used for 15 days is not the window used for 12 days) is part of what you must not ignore.
Quantitative Data Literacy · foundational
Where did rounding create a false exactness? Situation: A qualitative quote used to prove prevalence. One vivid interview line is introduced with 'participants felt'. A note in the margin says 20 cases were excluded without a rule, against 23 kept.
Review answer guidance
Keep the uncertainty that rounding removed. Do not treat a rounded input as a precise fact. The claim in play is that the line is a prevalence claim. one quote is existence, not prevalence The extra figure (A note in the margin says 20 cases were excluded without a rule, against 23 kept) is part of what you must not ignore.
Quantitative Data Literacy · moderate
Which unit conversion is required before the comparison is valid? Situation: A data-availability statement. It says 'available on request'; three requests in a year were refused. The same label covers 6 events in one table and 34 in the other.
Review answer guidance
Convert to one unit. Mixing hours with minutes, or local currency with another, is not a comparison. The claim in play is that the data are available. refused requests contradict the statement The extra figure (The same label covers 6 events in one table and 34 in the other) is part of what you must not ignore.
Quantitative Data Literacy · challenging
How does double-counting affect the total? Situation: A multiple-comparisons problem. Forty questionnaire items, two 'significant' at 0.05, no correction, no pre-specified items. An appendix lists 11 exceptions that the summary of 45 does not mention.
Review answer guidance
If categories overlap, adding them inflates the whole. Use a unique-count. The claim in play is that two findings were shown. two of forty at 0.05 is close to chance The extra figure (An appendix lists 11 exceptions that the summary of 45 does not mention) is part of what you must not ignore.
Quantitative Data Literacy · foundational
Why is the percentage unstable at this sample size? Situation: A translator who is also a co-author and advocate. Interview translations were done by a campaigner for the intervention. A side count of 16 used a different definition from 19.
Review answer guidance
A few observations swinging the other way would reverse a large-looking percentage. The claim in play is that the translations are a neutral record. advocacy plus translation is a bias risk that needs independent checking The extra figure (A side count of 16 used a different definition from 19) is part of what you must not ignore.
Quantitative Data Literacy · moderate
A total is reported without a denominator. What must you obtain? Situation: A power analysis written after data collection. The analysis uses the observed effect to say the study was 'adequately powered'. The working copy is dated 21 days later than a 30-page cited copy.
Review answer guidance
Ask for the eligible population and period. Rank rates, not raw totals. The claim in play is that this is a valid power calculation. post-hoc power using the observed effect is circular The extra figure (The working copy is dated 21 days later than a 30-page cited copy) is part of what you must not ignore.
Quantitative Data Literacy · challenging
Two points are not a trend. What extra series is required? Situation: A figure without a sample size in the legend. Error bars are shown; n is only in a supplement the journal PDF omits. 7 of 41 rows were added after the first total was announced.
Review answer guidance
Ask for the values before and after the highlighted pair, and for any other change in that window. The claim in play is that the figure is interpretable alone. error bars without n are under-specified The extra figure (7 of 41 rows were added after the first total was announced) is part of what you must not ignore.
Quantitative Data Literacy · foundational
The share rose while the total fell. What does that mean? Situation: A secondary analysis of open data. The new paper does not cite the original collection protocol's exclusion rules and silently includes the excluded rows. A second observer recorded 12 where the first sheet shows 15.
Review answer guidance
A larger share of a smaller whole can be fewer people. Report both the share and the count. The claim in play is that the open data can be used as-is. the protocol's exclusions are part of the dataset's meaning The extra figure (A second observer recorded 12 where the first sheet shows 15) is part of what you must not ignore.
Quantitative Data Literacy · moderate
The gap is reported in percentage points. How is that different from percent? Situation: A 'nationally representative' panel. Weights are described in a sentence; the unweighted n in a key cell is 6. The window used for 17 days is not the window used for 26 days.
Review answer guidance
Percentage points are a subtraction of two rates. Percent of a rate is a different number. The claim in play is that the cell supports a national estimate. a weighted story on n=6 is still a thin cell The extra figure (The window used for 17 days is not the window used for 26 days) is part of what you must not ignore.
Quantitative Data Literacy · challenging
How should you normalise before ranking the two sites? Situation: A theory paper that cites only supporting cases. Known counterexamples exist in a neighbouring literature and are uncited. A note in the margin says 22 cases were excluded without a rule, against 37 kept.
Review answer guidance
Use the same definition, the same hours, and the same eligible group. The claim in play is that the theory has been stress-tested. uncited counterexamples are a selection of evidence The extra figure (A note in the margin says 22 cases were excluded without a rule, against 37 kept) is part of what you must not ignore.
Quantitative Data Literacy · foundational
What range of rates is consistent with the missing data? Situation: A lab's internal replication. The same team, same code, same data produce the same number. The same label covers 8 events in one table and 11 in the other.
Review answer guidance
Compute the rate if all missing cases are successes, and if all are failures. Report that interval. The claim in play is that the finding is replicated. that is a rerun, not an independent replication The extra figure (The same label covers 8 events in one table and 11 in the other) is part of what you must not ignore.
Quantitative Data Literacy · moderate
At what point do the two options cost the same? Situation: A mixed-methods claim. The qualitative theme is used to explain a quantitative null as 'really positive'. An appendix lists 13 exceptions that the summary of 22 does not mention.
Review answer guidance
Set the two cost expressions equal and solve. That break-even is the decision boundary given these figures. The claim in play is that the null is overturned by quotes. quotes cannot reverse a null without a stated mixed-methods design for doing so The extra figure (An appendix lists 13 exceptions that the summary of 22 does not mention) is part of what you must not ignore.
Quantitative Data Literacy · challenging
After the stated adjustment, what residual difference remains? Situation: An effect size reported only as 'significant'. p=0.049, n=1,200, effect 0.04 on a 100-point scale. A side count of 18 used a different definition from 33.
Review answer guidance
Subtract the explained portion. Do not treat the remainder as noise without saying so. The claim in play is that significance is the finding. the effect is tiny; p is not the practical result The extra figure (A side count of 18 used a different definition from 33) is part of what you must not ignore.
Quantitative Data Literacy · foundational
What happens to the headline figure if the extreme observation is set aside? Situation: A preprint cited as 'the established evidence'. The preprint appeared last week and has no reviews. The working copy is dated 4 days later than a 44-page cited copy.
Review answer guidance
Recompute without it. If the story depends on one point, say so. The claim in play is that established is fair. unreviewed last week is not established The extra figure (The working copy is dated 4 days later than a 44-page cited copy) is part of what you must not ignore.
Quantitative Data Literacy · moderate
What share of the eligible group is actually in the dataset? Situation: A supervisor's preferred measure replacing the registered one. The registered primary outcome is dropped because it 'did not work'. 9 of 18 rows were added after the first total was announced.
Review answer guidance
Coverage is counted over the group the claim names, not over the files you happened to receive. The claim in play is that the replacement is still primary. the registered outcome remains primary; the replacement is additional and must be labelled The extra figure (9 of 18 rows were added after the first total was announced) is part of what you must not ignore.
Quantitative Data Literacy · challenging
How many significant figures are justified? Situation: A consent form that bundles optional future contact with required study consent. One tick box covers both. A second observer recorded 14 where the first sheet shows 29.
Review answer guidance
Match the output precision to the roughest input. Extra decimals are not extra knowledge. The claim in play is that the tick is valid consent for both. bundling optional contact with required consent is not freely given for the optional part The extra figure (A second observer recorded 14 where the first sheet shows 29) is part of what you must not ignore.
Communication · foundational
Rewrite the key message for a reader who has two minutes. Situation: A methods section that does not pre-register outcomes. Four outcomes were collected; the paper highlights the only one with p<0.05. The window used for 19 days is not the window used for 40 days.
Review answer guidance
Lead with the decision, then the evidence, then the unknown. Cut decoration. The claim in play is that the highlighted outcome was the planned test. selecting after seeing p-values is not the same as a planned outcome The extra figure (The window used for 19 days is not the window used for 40 days) is part of what you must not ignore.
Communication · moderate
What must change so the same facts work for this audience? Situation: Two studies of the same intervention with opposite signs. Study A (n=400) finds harm; study B (n=24) finds benefit; a blog cites only B. A note in the margin says 5 cases were excluded without a rule, against 14 kept.
Review answer guidance
Name what the reader must do, by when, and which detail they do not need. The claim in play is that the intervention helps. the larger study and the omitted sign are part of the evidence The extra figure (A note in the margin says 5 cases were excluded without a rule, against 14 kept) is part of what you must not ignore.
Communication · challenging
Which synthesis is faithful to both sources? Situation: A systematic review mixing RCTs and blog posts. The abstract says 'the literature shows'; 6 of 11 included items are unreviewed posts. The same label covers 10 events in one table and 25 in the other.
Review answer guidance
Keep agreements, flag conflicts, and do not average two incompatible claims into one smooth sentence. The claim in play is that the literature has been reviewed. unreviewed posts are not the same class of evidence as RCTs The extra figure (The same label covers 10 events in one table and 25 in the other) is part of what you must not ignore.
Communication · foundational
Which sentence should open the note, given the decision that is needed? Situation: A dataset with 18 percent missing follow-up. The paper reports complete-case effects and does not describe the missingness. An appendix lists 15 exceptions that the summary of 36 does not mention.
Review answer guidance
Open with the request or the recommendation. Context follows; it does not lead. The claim in play is that the reported effect is the population effect. 18 percent missing needs a sensitivity check The extra figure (An appendix lists 15 exceptions that the summary of 36 does not mention) is part of what you must not ignore.
Communication · moderate
Which ambiguity would most likely cause the wrong action? Situation: A qualitative codebook with collapsing categories. Twelve codes were merged into two after seeing which story looked cleaner. A side count of 20 used a different definition from 10.
Review answer guidance
Find the word that could mean two things to two readers, and replace it with a specific noun and a time. The claim in play is that the two themes were discovered in the data. merging after seeing the story is a design choice that must be reported The extra figure (A side count of 20 used a different definition from 10) is part of what you must not ignore.
Communication · challenging
Which recommendation is actually backed by the stated evidence? Situation: A grant proposal claiming certainty from n=12. The proposal says the pilot 'establishes efficacy'. The working copy is dated 6 days later than a 21-page cited copy.
Review answer guidance
A recommendation needs a stated option, a reason from the facts, and a named unknown. The claim in play is that n=12 establishes efficacy. a pilot can estimate feasibility, not establish efficacy The extra figure (The working copy is dated 6 days later than a 21-page cited copy) is part of what you must not ignore.
Communication · foundational
The tone may be read as blame. What change preserves the facts? Situation: A replication preprint that cannot share data. The authors cite commercial sensitivity; the claim is a public-health effect. 11 of 32 rows were added after the first total was announced.
Review answer guidance
Describe the behaviour and the impact, not the person. Keep a specific request. The claim in play is that the effect can be taken as checked. an unreplicable public-health claim is weakly checked The extra figure (11 of 32 rows were added after the first total was announced) is part of what you must not ignore.
Communication · moderate
Which structure best reports this to a mixed audience? Situation: A meta-analysis without a search strategy. Nine papers, all from one lab, all positive. A second observer recorded 16 where the first sheet shows 43.
Review answer guidance
Recommendation, evidence, limits. Put technical detail after the decision. The claim in play is that the meta-analysis represents the field. one lab and no search strategy is a convenience collection The extra figure (A second observer recorded 16 where the first sheet shows 43) is part of what you must not ignore.
Communication · challenging
A summary omits who is affected. What must the next sentence add? Situation: An ethics form that waives consent for convenience. The reason given is 'students are used to being studied'. The window used for 21 days is not the window used for 17 days.
Review answer guidance
Name the audience, the action asked of them, and the deadline or uncertainty. The claim in play is that convenience is a waiver ground. habit is not a lawful or ethical substitute for consent where consent is required The extra figure (The window used for 21 days is not the window used for 17 days) is part of what you must not ignore.
Communication · foundational
Which subject line would let the right person act without opening every paragraph? Situation: A model that treats correlation as a mechanism. Ice cream sales and incident counts rise together; the model includes sales as a cause of incidents. A note in the margin says 7 cases were excluded without a rule, against 28 kept.
Review answer guidance
Put the decision and the date in the subject. Curiosity is not a professional purpose. The claim in play is that the coefficient is a mechanism. a seasonal confounder can produce both The extra figure (A note in the margin says 7 cases were excluded without a rule, against 28 kept) is part of what you must not ignore.
Communication · moderate
Where is the hedge hiding a decision that has already been made? Situation: A thesis chapter that synthesises conflicting theories. The chapter lists both theories and concludes they 'complement each other' without stating where they disagree. The same label covers 12 events in one table and 39 in the other.
Review answer guidance
If a choice was made, say it. Softening language after the fact is not caution; it is fog. The claim in play is that the conflict is resolved. complementary is not a synthesis if the contradiction is unnamed The extra figure (The same label covers 12 events in one table and 39 in the other) is part of what you must not ignore.
Communication · challenging
Which term should be replaced so a non-specialist can act? Situation: A journal review asking for robustness checks. The author replies 'results were similar' with no table. An appendix lists 17 exceptions that the summary of 13 does not mention.
Review answer guidance
Swap the in-group word for the ordinary description of the action required. The claim in play is that robustness was shown. similar is not a check you can audit without the numbers The extra figure (An appendix lists 17 exceptions that the summary of 13 does not mention) is part of what you must not ignore.
Communication · foundational
The facts are in chronological order. What order would help the reader decide? Situation: A p-hacking pattern. Twenty tests, one reported; the reported test used a covariate chosen after looking at residuals. A side count of 22 used a different definition from 24.
Review answer guidance
Decision order beats diary order. Time-stamps can sit in an appendix. The claim in play is that the reported test is confirmatory. covariate choice after residuals is exploratory The extra figure (A side count of 22 used a different definition from 24) is part of what you must not ignore.
Communication · moderate
What should be cut first so the note stays under one screen? Situation: A HARKing suspicion. The introduction reads as if the surprising finding was predicted; the lab notebook shows it was not. The working copy is dated 8 days later than a 35-page cited copy.
Review answer guidance
Cut repeated background and keep the numbers with units. Length is not thoroughness. The claim in play is that the narrative is the hypothesis. a hypothesis written after the result is not a prediction The extra figure (The working copy is dated 8 days later than a 35-page cited copy) is part of what you must not ignore.
Communication · challenging
Which claim needs an attribution before it can be sent? Situation: A forest plot with one huge study. The pooled effect tracks that study; smaller studies go the other way. 13 of 9 rows were added after the first total was announced.
Review answer guidance
A figure without a source is an assertion. Name who measured it, or drop the figure. The claim in play is that the pool is a consensus. the pool may be one study in costume The extra figure (13 of 9 rows were added after the first total was announced) is part of what you must not ignore.