Paper itself above. Need a deeper reading with my notes but on the surface the stats are so-so. They check normality, but don’t confirm linearity (use of pmcc will not be valid without - there are also a few other conditions to check for hypothesis testing with PMCC if memory serves), use of a continuous test (PMCC, ANOVA, unpaired t’s) for discrete (likert) data is also little controversial, but generally condoned.
As for the conclusion, not a psych phd so I’ll assume they know their stuff!
my personal rule of thumb is that if it’s published in Nature, Cell, or another well-regarded journal, the statistical and experimental methodologies are almost certainly solid. Do you think I should adjust that rule going forward?
Honestly, I always poke the stats no matter how good the journal. The best way to read any article is as a skeptic (the onus is on the writer to prove their point), and any small irregularity is something to be queried.
No matter how good the journal, it’s only as good as the reviewers, and reviewers are humans too. Odds are a paper in nature is all above board, but I’m somewhat of a pedant when it comes to checking test conditions.
i do that to, i also try to find most recent research, anything older than 5+years is suspect, because they always come with revised papers in newer studies/research eventually.
In some fields (e.g. mathematics) old papers hold up well. However, in fields like psychology where the landscape shifts a lot that’s probably a good shout!
sometimes, but they have retracted quite a few papers based on misleading papers, or even AI rgenerated. also because it can mislead readers into thinking “oh this is the sole cause and effect” but not potential alternative scenarios.
Here’s the link to the actual article.
https://www.nature.com/articles/s41598-025-19418-4.pdf
Paper itself above. Need a deeper reading with my notes but on the surface the stats are so-so. They check normality, but don’t confirm linearity (use of pmcc will not be valid without - there are also a few other conditions to check for hypothesis testing with PMCC if memory serves), use of a continuous test (PMCC, ANOVA, unpaired t’s) for discrete (likert) data is also little controversial, but generally condoned.
As for the conclusion, not a psych phd so I’ll assume they know their stuff!
my personal rule of thumb is that if it’s published in Nature, Cell, or another well-regarded journal, the statistical and experimental methodologies are almost certainly solid. Do you think I should adjust that rule going forward?
Honestly, I always poke the stats no matter how good the journal. The best way to read any article is as a skeptic (the onus is on the writer to prove their point), and any small irregularity is something to be queried.
No matter how good the journal, it’s only as good as the reviewers, and reviewers are humans too. Odds are a paper in nature is all above board, but I’m somewhat of a pedant when it comes to checking test conditions.
i do that to, i also try to find most recent research, anything older than 5+years is suspect, because they always come with revised papers in newer studies/research eventually.
In some fields (e.g. mathematics) old papers hold up well. However, in fields like psychology where the landscape shifts a lot that’s probably a good shout!
sometimes, but they have retracted quite a few papers based on misleading papers, or even AI rgenerated. also because it can mislead readers into thinking “oh this is the sole cause and effect” but not potential alternative scenarios.
Thank you!