John Ioannidis: The role of bias in nutritional research

John Ioannidis: The role of bias in nutritional research


Thank you very much indeed. We have the great honor now of hearing from
right across the world from John Ioannidis, who is here. Hello, John. I don’t know if you can see us. We can see you. You’re in some very exotic place. Where are you? Stanford, if you can call that an exotic place. But the backdrop looks kind of like you’re
in some Mexican … I don’t know what the backdrop is. Anyways, very nice. Lovely to see you. John, thank you very much. We’re going to have a talk from you I think,
for sort of 15 or so minutes, and then some Q&A. Handing over to you now. Thank you very much for being with us. Thank you for the very kind invitation, and
apologies that I couldn’t join you in beautiful Switzerland. The role of bias in nutrition research, the
short answer is pretty easy. If we’re talking about nutrition in general,
probably 95% of that in bias. We’re talking about nutrition research, we
would expect to do a bit better. The typical recipe of nutrition research is
necessarily leading to a failure rate that is very close to the overall average person
impression about nutrition. Mostly, we’re dealing with non-randomized
designs, with impossible or very difficult to control confounding, also many low-quality
small trials. We have large measurement error. We have cherry-picking among multiple hypotheses,
post-hoc analyses, selective reporting, very lenient statistical tools, no registration,
limited data sharing, very strong beliefs driven by cultural, religious, personal views,
and what more … lots of white hat bias. A lot of us really feel that we can save the
world and our opinions might save the world. Finally, strong financial interest from Big
Food that may modulate the literature. Here’s a systematic cookbook review that I
published a few years ago. We used literally a cookbook, The Boston Cookbook. It has been out there since the 19th century. We randomly checked 50 ingredients from that
good cookbook, and we asked how many of those have been assessed for association with increased
or decreased cancer risk in the scientific literature. The result is what you see here, 40 out of
50 had scientific studies associating them with cancer risk. The other 10 didn’t come up, just because
of our search strategy. If you search for vanilla, we couldn’t find
any study, but if we had searched with vanillin, we could have found some studies associating
it with either increased or decreased risk, or both. In fact, for most of these foods and ingredients
there are studies claiming both increased, and decreased risk, with some exceptions. For example, bacon seems to be always bad,
but if you look at the relative risks in the bottom panel, they are really incompatible
with logic. I do believe that bacon probably does increase
cancer risk. Very, very small increase is possible, but
if you take literally these relative risks that have been reported in scientific studies
of 2, and 7, and 10, they are incompatible with common sense. Meta-analysis may help a little bit to get
rid of that excess significance. This is showing the distribution of Z-scores
or P-values, less than -2, or more than 2 means that you have “statistically significant
results”. If you look at the fine print, these are the
gray zone, the gray color measurements. There’s a few more non-significant results,
but most of the results that circulate in that literature are significant, even though
you would expect for so subtle and tiny effects, the vast majority of the studies alone not
being able to give significant results. If you look at meta-analysis, some of that
middle gap of emptiness in the non-significant space starts filling up, but still, there’s
lots of things missing. In nutritional epidemiology any result is
possible, practically. What you get is what I call the Janus phenomenon
from vibration of effects. The reason for that is that there are so many
things that you have to try to adjust for, because people who eat different things do
have very different profiles, and depending on what you adjust, and how many adjusting
factors you can consider, and how exactly you frame your analysis, you can get what
I show you here, which is one million different ways to analyze the same dataset. In this case, this is data from the National
Household Survey on nutrition assessments. The last panels on the right are alpha-Tocopherol. About 700,000 of these results suggest that
alpha-Tocopherol decreases the risk of death, and about 300,000 of these analyses suggest
that alpha-Tocopherol increases the risk of death. Depending on what you believe, you may really
select your analysis of choice. Some countries are even worse. We have shown that for behavioral and lifestyle
types of research US studies give very extreme results. Probably there’s more pressure in the US to
generate more extreme results, and to base that extreme environment of nutritional, or
behavioral, or other lifestyle findings to make claims for funding. Most of the nutritional variables are highly
correlated. This is data from the Singapore Prospective
Cohort, looking at 19 nutrition intake variables. If you see a link between two of them, it
means that they are correlated. Practically, there’s links among all of them. If someone wants to put emphasis on one of
these nutrients … Just because I have a very strong belief that that nutrient may
be very important, I can always make a case, because if it is some other nutrient that
is important, that nutrient would still be correlated with the one that I want to build
a case on. One might use the traditional criteria, like
the Bradford Hill Criteria, to sort out that conundrum. Unfortunately, they don’t really work. In nutritional science I think that they work
the least. Most of the time when I see very strong results
in nutrition, what I say is that this is evidence that there is very strong bias. Strength, which typically has been used to
select the most credible epidemiological associations, in the case of nutrition just makes me think
that there’s tremendous bias here. Consistency is nice, but obviously, we don’t
see much consistency. Specificity, probably not. Temporality, of course, but how exactly to
document it may not necessarily be easy in many of these studies. Biological gradient … I have no clue what
exactly is the dose range that you would see a biological gradient or dose response for
different nutrients and exposures. Plausibility, anyone can build a case. Especially if you’re biased, you can easily
make a case out of these thousands of studies that are circulating that would fit your needs. Coherence is very difficult to operationalize. Experiment … as I said, we have randomized
trials, but I would come to them, because they also have tremendous bias. Analogy, again, is just a recipe for more
detriment by building narratives that don’t really make sense. Observational epidemiology doesn’t seem to
square with randomized trials in nutrition. A number of years ago I looked at the most
highly cited claims across all medicine, and when it came to observational studies five
out of six of the most cited claims were refuted within a decade, typically by very large randomized
trials. Stan Young has a list of 52 major epidemiological
claims. None of them were validated in randomized
trials. Some highly contradicted findings have made
waves when they appeared. Vitamin E, flavonoids, low-fat diet, beta-carotene,
directly contradicted by randomized evidence or very large studies. Some others probably make some sense. Fruit intake I think is wonderful. It may have an impact on breast cancer risk,
but clearly not up to 90%, as some studies have suggested. When claims are refuted, there’s still a very
large segment of the literature that continues to cite them. There’s so strong bias in the field, that
even if you have five very large randomized trials, like in the case of beta-carotene,
people still continue to citing the original claim that beta-carotene could be an effective
chemopreventive agent. Actually, that original paper continues to
get more citations than the average paper that was published in the same journal in
the same year. In that case, it was Nature. If you were to read these papers, the citations
are not that beta-carotene has been refuted as a chemopreventive agent. They say that beta-carotene is a very promising
chemopreventive agent, and this is why we continue doing research on it. Now, randomized trials have been helpful in
the few situations where we have put together resources to run some very large trials with
long followup and with very strong methods, but the vast majority of them are not really
much better than observational information. They are equally biased, sometimes even more
biased. The current situation is that we have many
small fragmented studies. This is a list of what you will find listed
in ClinicalTrials.gov, which probably lists 50% or less of the trials that are being done
on diet or nutrition. We have several thousands of nutrition trials,
but we have learned very little out of them, and they’re still susceptible to the very
same conflicts, and biases, and selective reporting forces that you see in observational
epidemiology, maybe with a caveat of maybe that being a bit more limited because of registration,
and maybe having also the protection of randomization, which, nevertheless, is not guaranteed. Yesterday or two days ago New England Journal
of Medicine retracted PREDIMED, which is the largest nutrition trial that … one of the
big successes, supposedly, because the randomization process was entirely subverted. To be honest, after looking at the retracted
and republished paper, I’m still not happy that that was all that went wrong in that
trial. Big Data make things even worse, or even better. It gives more opportunities for data dredging,
and for also some reasonable and interesting analyses, but obviously the possibility of
instilling bias and disseminating the results that I like the most, the degrees of freedom
that I have once I have more data, just go up exponentially. This is the summary of the summary of the
evidence for different types of food. That’s a paper in the American Journal of
Clinical Nutrition, done by excellent scientists. They have put together all the studies that
they could find from prospective cohorts, and if you look down the list, practically
all foods seem to modulate substantially the risk of death, mortality. Many of them decrease the risk of death. Many others increase the risk of death. Just refined grains is the only one that doesn’t
seem to have an association. If you take their results literally, with
one hazelnut per day you will live one year longer. By extrapolation, if you can eat that many
hazelnuts every day your life expectancy will exceed 100 years by far. I love hazelnuts, and I do recommend that
you eat more of them, but I don’t do it because I expect to live 120 years if I eat that many
hazelnuts every day. Now, if you drink three cups of coffee every
day, you will live 12 years longer. Very solid “evidence”, repeatedly seen in
reported epidemiological studies. What is the chances of that being correct? I would say 0%, as close to 0% as anything
that I can imagine. Here’s some other possibilities to try to
disentangle from the selective approach and the bias in choosing what to report, and what
to highlight, and what to interpret, and what to communicate. Instead of cherry-picking, just analyze everything. This is what we have done here with Chirag
Patel. We have looked at the National Household Survey
in the US, and we have analyzed every exposure, including all exposures of nutrients with
death risk during followup, and there are a few surviving variables. Physical activity seems to survive, and smoking
seems to survive, and cadmium seems to survive for its association with death, either because
of its own harm, or because it is highly linked to smoking. There’s only one nutrient that is right at
the borderland of our false discovery rate, trans-lycopene, if I were to bet is this correct
or not, I would say it’s probably wrong, but the vast majority of nutrients are very, very
low in that scheme. As I showed you earlier, many of these variables
are highly correlated. Smoking does kill people. It will kill one billion of people, unless
we do something and discontinue thinking that just by eating a little bit more, or a little
bit less we will get rid of the smoking risk. But smoking, smoking behavior is correlated
with so many other things, including many nutritional exposures, that it’s very easy
to come up with associations and claims that some nutrients may be the key to longevity
or to getting rid of diseases on their own. If you look at cotinine, it has 37 strong
correlations in these data that we analyzed. If you look at, again, something like beta-carotene,
you will see 68 strong associations with other variables, some of which may be genuine associations
with disease risk, or even death risk. If you place the results in the context of
an entire field, nutrients tend to have very strong correlation with each other, and they
also tend to give zillions of associations, even if you try to look across all the possible
analyses. This is a heat map plot where we assess every
exposure in … (NOTE: speaker names may be different in each
section) I hid my plop where we assess every exposure
and every outcome and enhance with Sharad Patel and basically each one of
these points we could have published as a separate paper. Instead, we decided to run this analysis and
we haven’t been able to publish that paper yet. Although, I hope it will be coming out soon. If you look more carefully for nutrients,
nutrients seem to be associated with the most outcomes. Not very strongly so, but if you have very
lenience at these thresholds, you will get almost every nutrient to light some signal
in association with some outcome. What it means is just inerrant confounding
relevant gender associations. What could we do, at a minimum? I think that at a minimum we should clarify
which analyses are not pre-specified. This will remove some part of the bias, not
all the bias but I think it would be a good start. Use pre-registration when appropriate. That can also help. Perform additional large simple trials with
long follow-up. We have done a few. Almost all of them are negative. The private predimed was positive
but as I said, it was retracted two days ago. After the correction, the results remained
the same. I seriously doubt that they’re correct. Most of these large simple trials I think
will be negative. We need some more negative shock in nutrition
to be able to move forward into some new ideas. When you defeat our research agenda to small
and tiny effects, abandon design when noise overwhelms the signal, this might mean abandoning
most of the research that we currently do with food frequency questionnaires. We know that they are tremendously biased,
we know that they are tremendously inaccurate. What other methods will we replace them with? I think that there are tools that are being
piloted and actually more than piloted at the moment, that we can measure exactly the
exposure, like taking photographs of food and try to integrate what exactly is contained
within them. It’s not going to be easy. Maybe the noise will be more than the signal,
most of the time if not all the time. I feel very strongly that data should be shared
publicly by default. I think it’s unacceptable that predimed data are not shared by default to every investigator who wants to take a look
at them. It’s great that the investigators audited
them but I think that others should take a look. Share in advance prodigals when prodigals
exist. Very often prodigals do not exist. Prodigals are shaped at a lab meeting when
you have an investigator who is sitting with his fellows and the fellows present the tables
of the next paper to be published. The investigator just says, “I don’t like
this result because it doesn’t really fit. What I believe, come back next week some other
analysis, with some other adjustments.” That’s not prodigal. That’s not control of the science. It’s a superior’s control of the science. Very important, disclose financial and non-financial
conflicts. We have very strong interests in the fields. We’re talking about hundreds of billions of
dollars at stake from big food. We also have tremendous interest from people
who have very strong beliefs, including myself perhaps sometimes. I may have very strong beliefs. For example, that there is so much bias. I may be wrong. You know that I have written about that repeatedly. In other cases, maybe this is not so visible. Maybe people have agendas, non-profit organizations,
foundations. They have built hospitals that focus on promoting
nutrition and preventive medicine. They publish books, some of them are books
that are best sellers and they make millions of dollars out of sales of these books that
are mostly nonsense. We should know about that. Limit or control involvement of stakeholders
with conflicts. I think that there are so many stakeholders
with conflicts in that area that we have to think on who is going to do what, in order
to get reliable information. Registration can help. I don’t expect that every study has to be
registered. Most research is exploratory. This is perfectly fine. I just want to see some acknowledgement that
this is some study that I did that was entirely exploratory and that would be fine. What I worry is mostly studies that don’t
exist. I’m completely paranoid. I don’t worry about the studies that exist. These I can try to read, I can try to critique,
I can try to see whether they are biased. If I have more transparency with more data,
more data available and particles it would be nice. I worry about the potential of running studies
which is if you have a data set, if you have a number of variables, if you have a number
of participants that you have information on, then you can estimate what is the capacity
of generating x number of p-values that you can launch against humanity. I need to disclose that I have a database
in nutrition that I can launch five billion p-values against you. It’s like disclosing a nuclear arsenal. When people see a couple of p-values, they
know what the denominator is of how many others could have been produced. We have these levels of registration. No registration for entirely exploratory research,
registration for data sets. Just telling us about the nuclear arsenal. Registration of a prodigal, if a prodigal
exists. Registration of the analysis plan. Both the analysis plan and raw data and open
live streaming. Eventually, if we do things right, as I said,
we may gets lots of negative signals. This is results from the Alfred Cofferal
beta carotene trial that has provided us with results over twenty three years now. It’s still running. Some early signals that we’re seeing with
data up to 1993 for decreased risk of prostate cancer with Alfred Cofferal and
increased risk of lung cancer with beta carotene when we have more follow up, pretty much even
those have disappeared. It may be that most of these signals will
disappear. We need some more of these trials. We need also to learn to live with small and
tiny effects. We need to adjust our agenda with the fact
that bacon may be bad, but maybe the increased risk for over-mortality may be something like
1.01, or even less than that. That might still translate to a number of
deaths that possibly we could avoid but we have to be prepared to go after very tiny
effects and see whether the noise that we get in our designs and the bias that can be
introduced can be take care of. We need more reproducible science. We need more data sharing. We need more transparency, more openness. I think that nutritional epidemiology could
make a difference by being open. Air pollution epidemiology is very open. People who have contrarying views have re-analyzed
the data, they clearly also see that air pollution kills people. I’d like to see that done also for nutrition
epidemiology. Finally, disclosure. Disclosure of financial conflicts of course,
but also non-financial conflicts. These are important to know. If someone is a public advocate for a particular
diet, for a particular type of food, for a particular type of fasting of whatever, I
need to know that this is what they advocate. If they are a public advocate, disclosing
that information in their published work is an opportunity to show that this is what I
believe and then people can even feel more confident that here is someone who really
states what they also state in public. They don’t try to cheat. They don’t try to confuse us with telling
us about something and not telling us at the same time that they are in a campaign
to which having that adopted. To conclude, most current evidence in nutrition
at the population level is hopelessly biases and unreliable. Bias shouldn’t be taken for granted. It’s not 100% but it’s close to 100%. In the current environment, the literature
is shaped primarily by the biases of scientists, reviewers, editors who are often the same
people, and sponsors. Often heavily conflicted ones. Many things need to change in the nutrition
agenda before some minimal credibility can be claimed. Nutrition and diet is important. I think that people, yes, can die because
of their own nutrition. Of course, they can die if they don’t eat
enough food, or if they eat too much and they become obese. It is extremely important for health and this
means that nutrition science deserves better. Special thanks to a number of my colleagues
who have shared the trip of generating evidence on some of the issues that I shared with you
today. Without their help and contribution and bright
ideas, I think I would have shown you next to nothing. Special thanks to you for listening from a
distance. Thank you so much. I hope I’m given permission to extend for
ten minutes or so to get some questions. If you would like to raise your hands, the
microphones will come your way, and we have already one question here. Where are some other questions? One question there. Go ahead please, yes. Then one, here. Do stand and introduce yourself. Hello, my name is Sam Obsen
from Tempe Medical in Aberdeen. Just a very quick question, how did you randomly
choose fifty ingredients? John, how did you randomly … This is using a pseudo-random number generator
in a computer like you would do in any random choice. Then, choosing randomly pages, recipes and
ingredients within the recipes. Thank you, John. Walter? Yeah, this is Walter Willet and
first of all, I think you’re really misrepresent how nutritional epidemiology is conducted. It’s a little bit like telling a surgeon how
to conduct your surgery, or if I’m going into surgery, telling the surgeon how to conduct
the surgery. We really have validated the dietary assessment
methods in detail. We don’t just generate lots of p-values and
report the positive p-values, we write grants for an age with hypothesis
… … With pre-specified. Right. Not all the pre-specified. We do try to indicate which ones, not pre-specified. Sometimes you learn a lot from things you
didn’t anticipate. That’s not how we conduct nutritional epidemiology. It did cite that paper, I think it was in
fact published in BMJ saying that 52 out 52 observational studies were not reproduced
in randomized trials. Actually, the first 2 that were mentioned
in the commentary were type-A personality and I really do have a hard time understanding
how you do randomized trials of type-A personality. The other was, trans-fat. I’m quite sure there’s no randomized trial
of trans-fat in coronary heart disease. I never could find that paper that was cited
there about 52 out of 52. Maybe you can explain that a little bit better. John, over to you. Yes, I think that there’s different views
about nutritional epidemiology and I have great respect for your work and all that you
have done to try to standardize and validate nutritional epidemiology. I think that we have to acknowledge that despite
these efforts, with frequency questionnaires, their error rate, plus adding the component
of being self-reported information, the noise is just much larger than the signal. I think that we have seen that repeatedly. I think that there’s many papers that show
that some of these data are incompatible with physiology or incompatible with life. I’m not saying that this is the fault of anyone,
I think it’s an effort. I think that we have to do better. I think that we have to move on to different
tools that may be more accurate. I’m not trying to discredit also exploratory
research. I think that exploratory research is fantastic. I think most of what we have learned in science
comes from exploratory research but we need to be very clear about what is exploratory
and what is pre-specified. Very little in nutritional epidemiology is
pre-specified in a way that is fully determined how exactly the analysis and how exactly the
reporting is going to be done. The degrees of freedom are just tremendous
and just having funding from NIH means absolutely nothing in terms of whether there is pre-specification. It just means that people are very well connected
to get that funding even without trying that they’re doing. In terms of the 52 studies, this is a paper
by Stan Young. It’s not in the BMJ. It does include my paid personality, it includes
mostly lifestyle interventions, many of them are nutrition supplementations. I think that the record is really, very, very
strong that there is very important coordinates in the observational data and randomized trials. There is some importance between some other
observational data in nutrition and some small randomized trials with surrogate outcomes
with short follow up, with messy analysis that is not necessarily better than what has
been done in nutritional epidemiology as a whole. I think that nutritional epidemiology is a
great field. It’s very important to move forward. We need to move forward because people are
dying because of wrong choices and we’re not telling them much about what to do because
we really don’t know what to tell them or what to tell them is largely wrong. Thank you, very much. A question here. Hello, I’m Adele Hight. I’m a grad student at NC State North Carolina. My question is, we’re seeing a lot of discussion
of randomized control trials versus nutritional epidemiology and I remember reading a piece
of yours about evidence based medicine being hijacked. What I’m trying to think about this problem
from the other angle. What about practice based evidence? What are the things that we need to do as
clinicians to try and get what is done in clinics, everyday with real people … In clinics, every day, with real people into
the literature. I think that it’s important to have more input
from clinicians and from people who see patients on a daily basis in real life. And use that insight to design the research
agenda. Even if we had some solid evidence on nutrition,
we would still have very limited information on how exactly to implement that in real life
because there are so many voices, there are so many sources of information and misinformation. And unfortunately, clinicians gradually lose
also their credibility because as I said, there’s that hijacking of medicine by people
who are conflicted for various reasons. We still have an opportunity. I think that we need to sit at the same table,
have researchers in nutrition, have clinicians, and try to come up with some research agenda
that focuses on implementation on that patient and counter with the clinician
on a daily basis. I think we have very little of that. People get mostly misinformed about nutrition
and many other things. They get very little personal information
with their own physician. Thank you John. Discussions are coming There’s that, and there’s mike. Could we get a mike ready for the lady here
when there’s one free? So, yeah, okay. Ambyod Hayman from Copenhagen University. I want to thank you for bringing up the issue
with the strong advocates, the missionaries, the strong believers, because I do think in
many of our scientific meetings they tend to peculate a lot of these meeting. And that I think prohibits real science going
on and being promoted in an unbiased way. This is more a comment to thank you for that. John, hold the response to that because we’ll
just heat from Tim. Just introduce yourself. John, it’s Tim Specter. Hello that. You’ve commented before on the genetic epidemiology
in a rather critical way, particularly on laboratory studies and early candidate gene
studies. And the pharmaceutical studies, as well, have
come under scrutiny. So how to you rank nutrition compared to all
these because it’s in a way easier to pick holes in all these fields. Some have inherent problems with actually
measuring the inputs. Nutrition is notoriously had to measure. You can easily do it in grams, or a pill. So, how do you measure these up against each
other, the different areas? John? So, genetic epidemiology as compared to nutritional
epidemiology has made some major moves that make it far more critical. And, yes, the measurement component may be
part of the story, because of course we have very accurate measures in genetics while despite
valiant efforts, we don’t have equally accurate measurement in nutrition. But there’s far more than that. I think that there’s a tradition of sharing
of openness of transparency of commutative knowledge in genetic epidemiology that has
really transformed the field. That has not happened in nutritional epidemiology. The norm, in genetics, that you will have
a coalition of all the teams working on a particular type of phenotype or disease. They would join forces, they would perform
meta-analysis of their data, perspective meta-analysis. They will share all the information, they
would have three different teams of analysts analyzing the same dating, comparing notes
to make sure they get the same results. There’s full transparency, everybody can see
that. Everybody can make sure that what we get is
correct. Now, whether that information is going to
save lives? It hasn’t saved lives yet. Will it save lives in the future? With some exceptions. Will it save lives in the future? It may. I mean, I cannot exclude that possibility. Nutrition epidemiology has not used these
recipes for research methodology transparency although it can be done. It has been done in so many other fields. I mentioned air pollution epidemiology that
is highly credible and has many of the other problems that nutrition epidemiology faces,
but has used transparency and openness and specification and validation. And even contrarian re-analysis. So, I think that we can clearly use examples
from other fields to make things better. One field can learn from the other. One can focus more on the weaknesses, one
many focus more on the strengths and the positive lessons. Thank you John. Thank you very much John. I’d like to pick up- Just say who you are. Hi, Nita Falouhi from Cambridge
in the Epistemology unit. So you make some excellent suggestions for
how to improve the field, and set up good analytical practice of protocols, of precepecified
fields of various stages. Absolutely for that, and I think journal editors
and journals can help with that in reinforcing some of those. As for here and now, given that you agree
that nutritional epidemiology and nutrition science is so critical for human health, what
would you say we should do in the current situation? The FFQ has its limitations, but done correctly
with repeated measures and all the other things we get through it, on it. Many of us are working trying to develop nutritional
biomarkers. The digital technologies you mentioned are
a far cry from being ready for use and implementation in some of these studies we’re talking about. Trials also rely on self report in terms of
dietary assessment. So given the importance of nutrition, what’s
your suggestion for what we do now. And sorry, a second point I’d like to add
on is the random hundreds of dietary factors. Why is it that scientists study them, because
people I’m interested in: Will this individual nutrient or factor affect my cancer risk or
whatever risk? So what’s your solution for that, too. Thank you. John. So the solutions that I would propose are
the same as I mentioned during the talk. I would want to see more large, long term,
randomized trials. Even though they do have the problem of self-report,
they do study intention, which is really what matters, because if we’re going to make a
public health impact, we need to study what the impact of our intentions to change the
behavior in a particular way, or the nutrition or the diet in a particular way. Yes, there are people who will tell us that
they do that, but they don’t do that. And others who will switch, and others who
will not be able to adhere. But that’s still pragmatic. It’s telling us it would tell people to do
something, what would we get at the end of the day? And it could be that some of these trials
do show us some beneficial effects, especially if we’re talking about not single nutrients
but big changes in their diet profile. I’m very eager and very willing to see some
substantial effects and I would be the happiest person on Earth to see that. For existing epidemiological studies, I think
that we need openness, I think that data needs to become available. They need to be standardized, they need to
be combined in the same way as you see with genetic epidemiology as Tim suggested. And I think that we can still learn something
about removing some of the biases. Not all of the biases, but at least some of
the biases. It’s not very clear to me if you have five
biases that are major in that field, which is the relevant contribution of each one of
them. If we can get rid of those that we can’t control,
and we have the recipe to control. We can go down the path that has been followed
in genetic epidemiology, we will know whether the rest are so difficult or insurmountable,
or they are less important than the others. We need better measurement tools. I think that yes, digital technology is not
yet 100% ready, but I think we need more of that. I think that since we know that there’s such
huge complexity in nutrition, since there’s tens of thousands of people, tens of thousands
os different chemicals that people may digest and eat, we need to model that very carefully. We need not to ignore the complexity of the
system. If we continue doing what we do, it would
be the equivalent of just studying the genome with the genome scans of the 1980s and 90s,
when we had 200 micro-satellite markers, and we were trying to understand the genetic risk
of disease. And obviously we got absolutely nothing. Maybe with one exception That was it. Out of the tens of thousands of the associations
that we have validated at the moment very, very carefully, and with very strong evidence. So, we have lots of things to do. It’s not that it is a lost cause. And I think that we have lots of excellent
scientist who have worked on these data and know their limitations, and are ready to take
the next step. I just want to see that next step taken. Thank you, and we have the last question because
we must finish. It’s the lady that has been very patient. Mediterranean, and Mediterranean
for … I don’t know if that’s important. I’m just curious because you trashed the results
of the predimed Haven’t you read the 100 pages of the results
of new analysis? Also, the replications of the results, and
sharing the data with other colligues from Harvard, for example. Also, the new analysis for protocol analysis,
as well. Just, I don’t know, in my opinion I think
that you, with your free claim, you are throwing away a lot of work, honest work of more than
ten years of a lot of people. So just I’m curious. If anyone has any doubt about the new results
of the predimed, please visit predimed.es , and you can see also the explanations
and the Q and answers from the- And, you, you’re not involved in predimed? Were you part of the predimed team? A little bit. Tell us in what way. In? In what way were you involved, just briefly? In what way? Well, I’m just, collaborating, I’m just more
working with another, with observational studies, not the clinical trials. Another question- We haven’t got time for them, so sorry to
stop you there. Have you conducted clinical trials in an additional
field? John, that’s two questions. Then why do you doubt the reanalysis of predimed and have you, yourself conducted a clinical trial, is that what you’re asking? I’m with John, this is your final shot. Okay, so, first of all, disclosing my bias. I love the Mediterranean diet, okay, I love
olive oil, personally, I love nuts, personally. I have repeatedly stated in previous talks
that predimed was a- Carry on John, carry on. Wonderful trial. And it was a great opportunity to perform
a randomized trial with long term follow up, with clinical outcomes. So, I was one of the big funds of predimed, and I have repeatedly stated that both in talks, and in writing. However, I have looked at the data myself,
at least the published data, and if you just type predimet, there’s 273 publications. If you look at his data, my impression is
that much of that evidence that is reported is incompatible. What was reported that led to the retraction
of the paper was a problem. It was a statistical technique that was applied
to the baseline data. Then, there was an audit by your team that
showed that there were some major problems with the randomization. Some of them are really, very hard to believe. And If you have a situation where someone
is randomizing an entire village, and cause that randomization, in an individual randomized
trial, I have every right to believe that there’re more problems that may have happened
in the context and the analysis of the trial. I feel insecure. I feel even betrayed, because I have trusted
that predimed was really a great trial in the past. And I think that these corrections that were
in the reanalysis and in the republication of the trial do not really explain the problems
with the baseline characteristics being equal. Anyone with any statistical knowledge realizes
that the type of problem that was detected would not have led to this type of baseline
characteristics being equal. I think I have to be honest. I don’t trust the trial any longer. It’s not really a randomized trial. I wish it were. Thank you John. We’re getting, oh yes. I have published several randomized trials,
and I have been involved in dozens of randomized trials. And I hope that I have not randomized the
whose village as an individual. John, we’re going to call to close there. This conference is called Food Fully for Thought,
and you’ve given us a great deal of food for thought. Thank you very much indeed for joining us. Thank you.

Daniel Yohans

3 thoughts on “John Ioannidis: The role of bias in nutritional research

  1. larisa mazepina says:

    It sounds like situation in nutrition science is doom and gloom.

  2. Darius Teixeira says:

    Prof. Ioannidis, my Dr colleagues start to wake up thanks to you.

  3. Frank Kelly says:

    The questioner says he could never find the "52 of 52" paper – took me 5 seconds of Googling https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.1740-9713.2011.00506.x . If you can't google might be time to hang it up

Leave a Reply

Your email address will not be published. Required fields are marked *