IRB opinions on dumb little internet polls
May 6, 2009
Hey peeps, do me a favor would you? Hunt around on your University’s website for the Human Subjects office or Institutional Review Board website. Find the email contact for the IRB coordinator or, failing that, the chair. Dash them off a note and ask if posting a simple internet poll such as you see on blogs now and again would require IRB consideration? Is a matter of explicit exemption or is something that they cannot conceive of why you would be asking and of course it doesn’t require consideration.
If you get a ruling either summarize in the comments (I guess I’d prefer no explicit details as to where unless said IRB person says it is okay) or drop me an email.
thxkbai!
DrugMonkey 
May 6, 2009 at 6:30 pm
These days, I don’t do any IRB-eligble research, but I’d think internet polls should be exempt according to the rules. There are no risks involved, and the ‘subjects’ can’t be individually linked to their responses.
I’d have to pull up the regs to be positive, but I’m pretty sure that meets the definition of exempt research (assuming one even considers it human research).
LikeLike
May 6, 2009 at 11:27 pm
Depends. If your poll is part of sponsored research, could cause psychological distress, is targeted toward minors, or answers can in any way be linked to respondents (perhaps via automatic IP address collection), then you should probably have IRB approval. You also should have approval if you plan to draw conclusions based on the results and disseminate them in any authoritative way. Note that human subjects research always requires informed consent (e.g. respondents need to know what the poll results will be used for).
If the poll is basically for observational purposes (e.g. ‘fun’) where no attempt will be made to interpret and disseminate scientific conclusions based on the research, then probably you don’t need IRB approval.
It also comes down to whether the institution has a stake in the outcome (e.g. could get heat). Basically, DM, if you are posting a poll as a blogger, then you’re probably OK. But if you are posting a poll as whomever you are IRL, or might use the results IRL, then you should seek IRB approval.
It’s quite an interesting question. Personally, I think Internet polls are so obviously scientifically flawed that if I were on IRB I’d fail the approval on experimental design regardless of anything else. But given that internet polls are so obviously flawed, that then makes them almost by definition not serious research. Which means IRB approval is not needed. So either way there’s no IRB approval, but for conflicting reasons that still never really settle the matter of whether it’s needed. In the end, I guess it’s like asking whether some firecrackers wrapped in duct tape violate nuclear non-proliferation treaties. If you think you have made a nuclear bomb, then I guess they do. But yet they don’t.
In summary: Poll away, dude. But don’t draw any conclusions from the results.
LikeLike
May 7, 2009 at 12:14 am
You know how I feel about this DM……grrrrrrrrrrrr.
LikeLike
May 7, 2009 at 9:23 am
Luigi, I know it’s traditional on the internet to spout off at length on topics about which you have no actual knowledge, but you could try some restraint on this one.
First, if you’re doing the poll as a private citizen, on your own non-funded time, go for it.
If you’re doing the poll in a professional capacity, then it matters. If the research is federally funded, you are required by federal law to comply with IRB requirements. In particular, you may also need to comply with HIPAA regulations and if you fail to do this, you can be help personally liable. Your institution can tell you if this is necessary. If your work is not federally funded, the formal rules are less strict but your institution will generally insist that you comply anyway. It upsets the feds when an institution has different standards for funded vs unfunded research.
You need to determine two things: 1) is it research? 2) does it involve human subjects?
These are defined here: http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.htm
For an internet poll, the answer to both of these questions is “yes” and you will need IRB approval. If your subjects are unidentified, and if there’s no greater than minimal risk to the subjects, your research probably qualifies for “exempt” status. That makes the IRB review relatively painless and quick, but it still has to be done. You will need to have documented evidence that you have completed human subjects research training, usually in the form of an online course. Then the IRB has to certify your study as “exempt”. When this happens, the IRB will issue an “exemption certification” letter to you. The letter will contain language similar to this:
The good news is that unlike IACUC, the IRB does not generally have an adversarial relationship with researchers: they will work with you to make sure you are in compliance with any relevant federal regulations.
Note that human subjects research always requires informed consent.
This is wrong. If you don’t want to consent your human subjects to research, you can look into whether you can get (again from the IRB) a waiver of informed consent.
Non-compliance with human subject research regulations can significantly imperil you both personally and professionally. As usual, ignorance of the regulations, whether willful or otherwise is not a defense. You should absolutely run your internet poll concept by an IRB representative at your institution and get a definitive answer on what you need to do.
LikeLike
May 7, 2009 at 9:31 am
Whimple,
One place I disagree is the notion that one can conveniently define leisure activity to avoid oversight. I would argue that if your professional capacity would oblige you to have oversight, merely doing it at home is no excuse. If I set up operant boxes in my garage to run rodent experiments at home would this exempt me from iacuc responsibility? I think not.
LikeLike
May 7, 2009 at 10:17 am
DM, remind me again what the “I” in IACUC stands for?
LikeLike
May 7, 2009 at 10:38 am
whimple,
CFR 46.101(b) states (in part):
I’d think many internet polls would meet condition (i), as long as respondents aren’t asked to provide identifying data. Even if IP addresses are logged, that alone doesn’t make the individuals identifiable, does it?
Assuming an internet poll does meet the above, the language indicates it would be exempt from the entire policy, which implies exemption from any IRB review at all. I looked for some indication that IRB’s must approve such exemptions, but couldn’t find one. If I missed it, could you please provide the section or link?
Of course, I could see individual institutions requiring IRB review of supposedly exempt research. Institutions probably wouldn’t want investigators to make their own determinations on exemptions, given the potential problems if an investigator decides wrongly. But I don’t see such a requirement codified in the federal regs.
Am I missing something?
LikeLike
May 7, 2009 at 10:54 am
Say I’m drunk off my fucking ass at Murphy’s Tavern, and I shout at the top of my lungs, “Red Sock fans–you shitty motherfucking assholes–raise your hands! OK, now Yankees fans–you wonderful human beings–raise your hands!”, do I need IRB approval?
LikeLike
May 7, 2009 at 11:04 am
Our IRB requires certification of exemption. Your individual institutional mileage may vary. I recommend getting formal IRB clearance for the research; this is for your own protection. For studies that really are exempt, it’s not particularly onerous.
Notice the “Unless otherwise required by department or agency heads” language that begins subparagraph b) on whether the research is exempt or not.
LikeLike
May 7, 2009 at 11:06 am
OK, now Yankees fans–you wonderful human beings–raise your hands!”, do I need IRB approval?
No, because this isn’t institutional research.
LikeLike
May 7, 2009 at 11:10 am
OK, I’m not in any sort of clinical area, but my understanding has always been that institutions want you to send stuff to the IRB even if it doesn’t need IRB approval, just so they can stamp “EXEMPT” on it, to be safe. A completely understandable stance. In the undergraduate institution where I teach, the place where this comes up most often is pedagogical research, where people want to try something new in their classes and, if successful, report on it to colleagues outside the institution. We generally send these things to the IRB, just to be safe, and they send it back stamped “EXEMPT.” I should probably be better informed about the intricacies.
I just participated in a professional society poll regarding a possible new journal. Since the society keeps records of who responds (so they can auction off a prize to respondents, as an incentive to respond) should they have gotten IRB approval before asking questions about where we publish and what we look for in a journal?
LikeLike
May 7, 2009 at 11:19 am
whimple,
I am all for spouting on the internet, but in this case I actually do know what I’m talking about. Furthermore, it comes from actually checking into this stuff with our IRB like DM requested and not just surfing the internet and copy/pasting.
OK, so I admit that I didn’t actually go to my IRB in response to DM’s request. I didn’t need to. I checked into essentially the exact same thing with some IRB members a couple years ago, out of a combination of potential need and curiosity. In (extreme) brief: I regularly teach a large class, and rely almost exclusively on multiple choice tests designed to match the difficulty of certain national pre-professional exams. Early on, I started having the secretary photocopy exams in several different colors, purely because I like colors and thought it would be pretty. Interestingly, students automatically assumed the different exam colors had some meaning (different versions or something). I have never disabused them of this notion, and since then have ‘experimented’ with all sorts of different exam styles and formats. Eventually, I learned that I was compiling huge amounts of data that could potentially be used to address weird questions about whether exam color or certain formats make a difference to performance etc. I also started including ‘poll’ questions on my exams, in an effort to gauge student’s background knowledge, interest re certain topics, etc. The answers to these questions did not affect one’s grade, of course, but the computer scoring of the exams made data compilation easy. I did this for about 3 years and had craploads of data that were very helpful for teaching better and in more innovative ways (I have several teaching awards now). Sounds cool, right? I thought so too. But I was talking about it with some colleagues at a college party and one said: “Um, don’t you need human subjects approval for that?”. And I was like: “Oh. I never thought of that. Do I?” I checked into it, and since I as already in the grey zone (at least to me) my queries expanded into exactly the sorts of things DM asks about. And that formed the basis of my reply. As it turned out, I did need IRB approval for my ‘studies’, so dumped all the data and haven’t pursued it any more.
There are some key points to be made:
1) What made my polls on the exam require IRB approval vs polls where I simply ask people a question in class and ask them to respond by raising their hands (which don’t require IRB approval)?
-The written polls collect and archive data, the answers are linked to identities, and may be construed to have taken place under stressful situations. Hand polls are also more obviously ‘voluntary’ and the purpose of collecting the data is usually laid out explicitly during the poll.
2) Why did my exam color study need IRB approval?
– For the reasons above, and because there was absolutely no informed consent. The students didn’t even know they were taking part in a ‘study’. Plus, I had an intention to analyze and possibly eventually report the results.
3) Why can I have online (or written) quizzes without IRB approval but need IRB approval for online (or written) polls?
-Informed consent is already implied for the quizzes. Students know the purpose of quizzes, and essentially agree to be ‘studied’ in that way when they enroll in a graded class. This isn’t necessarily true for polls, even though polls are more obviously voluntary.
Basically, it all comes down to the purpose for which the data is being collected and the context in which the data is collected. It’s all about protecting the subjects.
I stand by my summary to DM: Poll away. The key is that I’m assuming that DM is doing it as a blogger and won’t be using the results in ways typical of a professional scientific study. On the other hand, if DM wants to find out whether readers of his blog on average tend to use more MDMA than he does personally and then stick that information into an entertaining seminar slide, then he needs to talk to his IRB.
[And no, I can’t tell you whether exam colors have an effect, or whether there might be apparent sex-specific differences. Telling you would be reporting the data as a representative of my institution. To do that, I would have needed IRB approval, which I never sought. Of course, over beer some time I could relate interesting observations I’ve made in the course of teaching over several years.]
LikeLike
May 7, 2009 at 11:21 am
The fact that various activities are not “institutional” is an absolutely crucial point as a matter of law, given what the “I” in “IRB” stands for. However, I think that DM is hinting at a bigger ethical issue: Whether or not a poll on a blog counts as institutional, if it is a research activity (albeit a not particularly systematic one) that involves human subjects, should he just be going off on his own and doing it, or should he be running his practices by somebody else?
The bigger point behind the IRB is that individuals don’t get to make the tough judgment calls on their own projects. They don’t even get to sign off that the judgment calls are easy and there’s no need for somebody else to step in. They have to run it by somebody else. The “I” part of “IRB” might not apply to internet polls, but the principle is a sound one for “non-I” activities. I think that’s where DM is coming from.
Having said all that, an internet poll does seem particularly trivial. OTOH, people have suffered personal consequences for their internet activities. However, if an internet poll should be subject to some sort of authority because participation might be embarrassing, well, should there be an Internet Cop to police every forum where people might embarrass themselves or get into emotionally damaging conflicts? Should the Internet Review Board step in when somebody on some dumbass forum encourages people to post pics of them getting trashed at parties?
LikeLike
May 7, 2009 at 11:24 am
Whimple I am suggesting that some things become intrinsic responsibilities of the profession and do not depend solely on the narrowestinterpretation of the controlling legal authority. Do medical doctors abandon their ethical standards when offering services outside of the formally enrolled dr-pt relationship?. Subject protection oversight in research has similar obligation in my view.
LikeLike
May 7, 2009 at 11:49 am
It’s not that simple. You have to make sure you’re not inadvertently using institutional resources: Computers, software, internet access, email servers, etc. And that you don’t conduct the poll or report the results in a way that leverages your professional status in any way.
LikeLike
May 7, 2009 at 12:02 pm
For the reasons above, and because there was absolutely no informed consent. The students didn’t even know they were taking part in a ‘study’. Plus, I had an intention to analyze and possibly eventually report the results.
Had you looked into it, you could have easily obtained formal permission to conduct this research. You would need a waiver of informed consent, for which you could argue that the deception of not telling the participants that they were participating in research was a) essential to the research being done and b) caused no more than acceptably minimal harm.
Do medical doctors abandon their ethical standards when offering services outside of the formally enrolled dr-pt relationship?. Subject protection oversight in research has similar obligation in my view.
Excellent question. Clinical practice is by definition not “research” and therefore doctors have wide latitude to do whatever they want outside the purvey of the IRB. They still are responsible for HIPAA compliance however, in addition to a whole pile of other regulations, just not IRB involvement.
You should “do the right thing” because a) it’s the right thing to do, and b) because you are required to by statute. Where it gets fuzzy is that it has been demonstrated conclusively that people can convince themselves that essentially whatever they want is the right thing to do. This is the motivation for requiring independent oversight. Alex’s statement that, “the bigger point behind the IRB is that individuals don’t get to make the tough judgment calls on their own projects” is completely correct.
LikeLike
May 7, 2009 at 12:03 pm
@14: I tend to agree.
Suppose a researcher plans to conduct some protocol involving human subjects. Suppose further that the protocol would require IRB approval if NIH funding were involved, but because the researcher plans to use only private funding, 46CFR does not technically apply. (I don’t know if that would actually be the case; just suppose.) I’d argue the researcher would be ethically obligated to obtain IRB approval anyway. e.g. from a private IRB.
OTOH, suppose the reseacher is convinced his protocol would be exempt from IRB review under any circumstances (e.g. it’s an internet poll where responses can’t possibly be linked back to individuals). Should he still seek IRB review to confirm that the protocol is exempt? If he’s at an institution that requires it, sure, but what if he’s working independently? In that case, I’m not sure there’s a clear ethical obligation. If there’s any reasonable question as to whether the protocol would be exempt, ethics would argue for IRB review. If the researcher is quite certain the exemption would apply (e.g. based on prior experience where the exemption was confirmed), I wouldn’t fault him for not seeking IRB review.
LikeLike
May 7, 2009 at 12:11 pm
I think the idea what your responsibilities are as an ethical member of the profession is interesting. If you had rat boxes at your house, you couldn’t get IACUC approval, because there is no IACUC for your garage, there is no institution and no funding and no oversight. I think that is where you, as an ethical scientist, would know you have an obligation to do everything as an IACUC would require- you would know you have to provide appropriate housing, and appropriate food, and take measures to reduce pain or discomfort, and hire a vet to routinely check their health. That is what doctors do- maybe they don’t file notes about a patient in the hospital computer system at work if what they did was treat their neighbor at home, but they would still comply with all the ethical standards of medical care and if they’re good, would comply with the spirit of HIPAA confidentiality rules.
These internet polls seem to be the same thing. If it is on a blog, has no affiliation with an institution, and is done completely recreationally with no intent to share it scientifically, then which IRB could even approve it? But, that doesn’t mean you don’t have your own obligation to take care of the things the IRB would care about. Like, maybe you would need to have a pseudo consent form- like a little checkbox so that people could indicate that they know that their response will be used for XYZ and that because of how internet addresses work it would be possible for someone to trace their answer to them, but that you will do your best to maintain their confidentiality. Then when you ask the questions, maybe you have a moral obligation to think about their consequences.
It seems to me that the IACUC and IRB are things that happen at work, through an institution, to regulate institutional activities, create a common ethical standard in that place, and legally cover everyone’s tail. Maybe these legal entities don’t exist outside of work, but saying that isn’t the same thing as saying that there is no obligation to be ethical once you leave the lab.
LikeLike
May 7, 2009 at 1:05 pm
Is posting an Internet poll that asks if people think deadwood senior faculty suck ass institutional research?
LikeLike
May 7, 2009 at 1:16 pm
Is posting an Internet poll that asks if people think deadwood senior faculty suck ass institutional research?
I don’t know if it’s research, but if it were on FARK it would get “Obvious” tags.
LikeLike
May 7, 2009 at 1:25 pm
Invoking the image of ‘deadwood faculty sucking ass’ could cause permanent and unnecessary psychological harm to the participants.
LikeLike
May 7, 2009 at 1:27 pm
I meant “Evoking.”
English is my second language. E noi ragazzi paese non sono molto intelligenti.
LikeLike
May 7, 2009 at 1:51 pm
“Our IRB requires certification of exemption”
Mine, too. And, they’re reluctant to give out such exemptions — they’d rather you just get IRB approval, because that will result in greater oversight overall, and won’t put the onus on the researcher to “inform” when the study has “changed.”
I wouldn’t run an internet poll and plan on using it for research (rather than, for example, fun) unless I got IRB approval. An interesting frame is the “experiments” run at Cognitive Daily. If a cog scientist was running them and planning on publishing them, they’d almost certainly need approval. But, Munger is a journalist. So, his ethical obligations are different, presumably. But, what about a journalism professor? What if they set up a blog on internet polling? Then, do they need IRB approval? They probably wouldn’t think of getting it. And, Tierney, at the NY times, has no way, presumably, of getting an approval (since I doubt that the NYT has an IRB).
I think these issues are tricky, and some IRB’s have leaned in one direction while others have leaned in the other. Contrary to whimple’s experience, some IRBS are obstructionist. Some certainly wouldn’t approve Luigi’s study — because, for example, the students are signed up in a required class. On a related, but different, issue, I remember reading an exchange among ethnographers, who were worried that they were going to have to go through IRB approval in order to go out and interview people, with IRBS at med schools, who normally look at apps on drug intervention and brain implants having to handle the approvals.
(So, DM, practical reasons for this question? Are you planning on running internet “polls” “experiments” on us?)
LikeLike
May 7, 2009 at 1:54 pm
“Is posting an Internet poll that asks if people think deadwood senior faculty suck ass institutional research?”
I’m guessing some IRBs would want to call that institutional research, if you were actually going to publish the results. I think there are some that would call JNP’s evaluation of gender bias in its reviews institutional research. Any thoughts on whether Linden/Lane got approval for that one, first? The most obvious reason why an IRB would want to stick their nose in is the possibility of non-anonymity the results (in the JNP study).
LikeLike
May 7, 2009 at 2:07 pm
That part, at least, is easily addressed. There are scads of private, for-profit IRBs. If the NYT wanted to get IRB review, they could easily do so.
LikeLike
May 7, 2009 at 2:35 pm
neurolover@#23:
An interesting frame is the “experiments” run at Cognitive Daily. If a cog scientist was running them and planning on publishing them, they’d almost certainly need approval. But, Munger is a journalist. So, his ethical obligations are different, presumably.
The co-blogger at Cog Daily is a professor in a related field and the polls/analyses are frequently reported as having been done in the plural.
So, DM, practical reasons for this question? Are you planning on running internet “polls” “experiments” on us?
A discussion along these lines emerges now and again amongst a subset of bloggers. As I mentioned in a prior post, I went so far as to broach the subject with my IRB and get what I interpret as a casual no-prob exemption for that purpose. Were I to get more into polling I might seek additional rulings.
My larger purpose is, ideally, to get bloggers that do happen to have affiliations with institutions that have IRBs to get rulings. Particularly if noted in disclaimers on blog, this will go a long way toward laying down a precedent of established practice. Then, when some chump has a fearful t-crossing, i-dotting IRB, s/he would have at least some point of reference.
LikeLike
May 7, 2009 at 4:47 pm
No actual IRB responds to queries fast enough for this to give you any information before you forget about this blog post.
LikeLike
May 7, 2009 at 5:46 pm
Did you get IRB approval for this research?
LikeLike
May 7, 2009 at 6:25 pm
“The co-blogger at Cog Daily is a professor in a related field and the polls/analyses are frequently reported as having been done in the plural.’
Do they publish the results? As must be clear by now, there’s clearly variability in how different IRBs deal with things, but the cog daily experiments would not fly at my institution, if I were doing them, without IRB approval (I’ve asked, about similar experiments, ’cause I think that for some such experiments, i.e. one’s where selection bias doesn’t matter as much, the internet could be a fascinating place to run the experiments). But I do think that we have a particularly compulsive IRB, for a variety of different reasons (including potentially mis-steps in the past that bites everyone).
(Oh, and do the cog daily people have IRB review? They might. The experiments should clearly be allowed).
Other people (Perett’s face processing lab (St. Andrews; http://www.perceptionlab.com/)) have IRB approval to run their web-based experiments.
This story is an interesting read:
http://www.psychologicalscience.org/observer/getArticle.cfm?id=2339
It details Elizabeth Loftus’s battle with IRBs and the court system over interviews she conducted with people related to a woman who claimed “false memories”. The IRB at her institution became involved in the whole mess.
LikeLike
May 11, 2009 at 10:23 pm
You could pose your query directly to a good many IRB administrators by posting it at http://www.irbforum.org/
As some of the comments suggest, you might get more meaningful results if you flesh out the hypothetical beyond “a simple internet poll such as you see on blogs now and again.” Many administrators would indeed want to know who is doing the poll, what computers and servers are being used, what questions are being asked, how respondents are being identified, what is being done with the data, and so forth.
LikeLike
May 15, 2009 at 1:43 pm
This …. this thing you are reading now … is a form of internet poll. Don’t be fooled by the lack of radio buttons into thinking anything else. You are all going to IRB prison.
But on topic:
It is probably the case that you need to get the IRB exempt stamp (or equivalent) for wiping your ass, given the way that some rule systems are worded in many cases. I would say that the more realistic this is varies with the kind of researcher in question and the kind of institution, and mostly, the nature of the poll itself.
The rule of thumb is: If you do something, get IRB to tell you you don’t need IRB, don’t assume it. But, at the same time, there is a pragmatic and realistic approach one could and should consider. I would use the alignment test.
Consider the range of research done by your unit. Consider the kind of research done by you. Consider the research-related nature of the poll. Consider the privacy issues that may be be involved with the poll. Then consider all the IRB and HIPPAA and other compliance related policies. Consider the institutional wrapper of the poll itself. Then consider what possible uses the poll could be put to.
If you are an anthropologist interested in addiction, the poll asks about drug use, the poll also asks about some data that could be (even if only remotely) used to identify a person, and you have even the most remote possibility of using this information, and the poll is on a web site that identifies in any way with your professional life and research orientation, then all systems are go: Talk to your IRB contact. It is not hard.
If you are a muscle physiologist, you have a private web site on blogger or some similar place using your real name but that does not mention where you work, you are a car freak and have a poll on which classic car is hardest to maintain, and the poll does not reveal private information but does pop up the running count for each kind of car selected by poll takers, you would be insulting your IRB people and wasting resources to get this reviewed.
Which of these alignment factors (and there are others) is more important is not easy to say. Some may not be relevant at all (the above is a VERY provisional set of suggestions).
One could respond: Right … but where do you draw the line? WE WON’T KNOW WHERE TO DRAW THE LINE!!!!!111!!! WHAT WILL WE DO THEN111!!!
And the answer to that is very simple.
First, take it down a notch. Then, put your thinking cap on, if you can find it.
If you are anywhere near any conceivable line, do one of the following: a) redesign your approach so you are no where the line (and if you can’t do that forget the poll) or b) talk to your IRB contact.
The bottom line: Rules do not work as well as thinking. Try thinking.
LikeLike
May 15, 2009 at 1:52 pm
BTW: it may be worth mentioning that the two made up examples I gave are based on real cases. I had a student studying addiction (from an ethnographic perspective) with IRB questions fairly recently, and my wife studies muscle physiology but at one point in the past needed to design a poll for union related purposes. The union poll was for a high school, her research with muscle physiology is at the U. IRB was necessary in the first instance (the addiction research) but absurd in the second instance.
LikeLike