March 18, 2006
So I'm reading about mediation and mechanisms in theory, and thinking that if you don't find what you are looking for when you experiment and manipulate and control and test, then supposedly you suck. Several (Pfeffer most loudly) have protested that science is about the knowledge of what works and DOESN'T work, and not just of what works. I agree with him, but I have an addendum: I think the real skill is in when one doesn't find what one expects, then what is the story that one CAN tell...Are you still capable of telling a story of what's going on here, given that you thought to examine a particular situation for one reason or another? You tell me...
Posted by jpsteph at March 18, 2006 08:52 PM
this is a major problem. i totally agree with pfeffer (though I can't pronounce his name). the issue is connected to the problem of publication bias, whereby null findings are rarely published. not such a big deal except for the fact that people's livelihoods (medications, treatment interventions, public and health policy, etc.) are affected by what gets published!! if a drug or intervention has no effect but those studies are never published, what is our ethical responsibility as scientists to make this information readily available? further compounding the problem is the pressure to publish or perish. junior faculty trying to get tenure are not going to make a name for themselves (or get tenure) publishing null findings. after all like you said, the convention is to find what you are looking for (everyone's doing it), and if you don't, you suck, won't get your research published, and won't have a job in about 5-7 years after busting your a**. i think we need to move in the direction where unexpected and even null findings get more time and play. if all of the studies published are mostly what we expect to find with a customary finding or two that were unexpected or "surprising," it doesn't seem like explaining these findings is a skill that's valued. typically a sentence or two in the discussion section explaining the surprising finding will do. okay, so i'm being a little cynical and exaggerated to make my point, but i agree it's definitely a skill to explain what you don't expect to find. unfortunately, i don't think it's a skill that people pay a whole of attention to (unless they're looking for shortcomings in someone else's research) because the system isn't set up to encourage it... just my .02.
Posted by: eneblett at March 20, 2006 01:20 AM
wow...if that's $0.02, I'd hate to see you throw around some benjamins!!!! It IS all about the system, and it's pretty hard to shake the foundations of the ivory tower....I see you've been thinking about this a lot...DR. Neblett!!
Posted by: jpsteph at March 20, 2006 09:14 PMLogin to leave a comment. Create a new account.