How to Live Forever (or at least make it seem like you do!)
It’s September which means RFK Jr will soon be releasing his promised report on the ‘causes’ of autism. Until then, we're learning about important biases. Whatever his report says, we’ll need to make sure that none of these biases are present for his conclusions to be believable. Previously, I explained ‘confounding by indication’, competing events, and temporal bias. Today its time for my favorite bias of all: immortal time bias. Subscribe so you don’t miss future biases!
Today’s bias is my favorite research bias: immortal time bias. It happens way too often, both because it’s an easy mistake to make and because it’s a very difficult problem to avoid.
The basic idea is kind of hard to distill, so let’s start with an example instead.
Imagine you want to know whether surgery improves survival after a heart attack. If you use observational data, there will be a problem: anyone who has an immediately fatal heart attack is unable to subsequently have surgery. In order to get surgery, you must survive long enough to have the surgery. The group of people who did get surgery following a heart attack essentially has a chunk of pre-surgery time where they were never going to die (or they wouldn’t have ended up in that group). That chunk of time is immortal time. The group of people who did not get the surgery, on the other hand, are not required to survive their initial heart attack. This bias can make the surgery look good even if it does nothing at all. In the extreme case (when many many people die before surgery), it could make the surgery look good even if it is actively harmful!
Immortal time can happen when being eligible to receive the exposure requires survival (or event-free survival) for a period of time. Immortal time bias happens when that requirement for survival is different for people in your exposed and comparison groups.
This first scenario is an example of the first type of immortal time bias: improperly including time when one group cannot have the outcome of interest by definition.
There is also a second type of immortal time bias: improperly excluding time when one group cannot have the the outcome of interest by definition.
In our first example, the exposed group benefitted from the immortal time bias. But that doesn’t have to be the case. We can also have immortal time bias that makes the comparison group look better. This happens very frequently in studies where the comparison group is “never exposed” and in these studies, the problem is almost always the second type of immortal time bias.
Why? Because it’s very common to identify the “never exposed” people as the group of people who reach a certain threshold in time without having ever been exposed and without having yet had the outcome or died. We essentially discard any outcomes that happen in the ‘never’ group before our artificial start time. And that makes their time immortal!
This bias is very relevant for recent studies trying to assess vaccines and autism (like this one) because they often compare ‘ever vaccinated’ and ‘never vaccinated’ children. Typically, these studies set a threshold age (often 5 years old) after which they stop checking vaccinations and start checking for autism diagnoses. But, since everyone knows that causes have to happen before effects, they will also set another restriction on their data: anyone who is diagnosed with autism before they receive a vaccine is excluded from the data.
That means being in the ‘never vaccinated’ group requires being immortal (i.e., alive and not yet diagnosed) until your 5th birthday.
The bottom half of the figure above shows how this works in picture form: If you get any vaccine before your 5th birthday, the researchers will include any autism diagnosis that happens any time after that vaccine even if it happens before your 5th birthday. But if you did not get any vaccines before your 5th birthday, they will only include autism diagnoses that happen after your 5th birthday.
But the first scheduled vaccine in the US happens the day you are born, and the average age of autism diagnosis in the US is 5 years old. So half of all diagnoses are expected to happen before age 5, and these are simply thrown out of the data on the “never vaccinated” kids. Meanwhile, a good proportion of the ever vaccinated kids had their first vaccine either at birth or within their first 6 months (when autism can’t be diagnosed yet). So pretty much all of their autism diagnoses get counted. That essentially guarantees the rate of autism will be higher in the “ever vaccinated” group. Without requiring any actual causal role of vaccines at all!
So, that’s immortal time bias in a nutshell. But as you can see it’s a pretty tricky one to handle because you can run into trouble both by including and excluding the immortal time!
Worse, there isn’t a one-size-fits-all solution. There’s no easy thing I can suggest you look for when you’re reading a paper or doing your own research that will tell you if the researchers have handled this correctly.
But it is (a bit) easier to spot the cases where immortal time might be happening and has been ignored. And those cases are almost guaranteed to give misleading results:
First, check if they are using a ‘never exposed’ comparison group. If so, ask yourself “when does never start?” (i.e., at what point do we know that someone is never exposed?). Is this the same time that ‘ever’ starts (i.e., the point where we know someone is ever exposed)?
Or, if they’re using some other type of comparison group, ask whether the start time for both the exposed group and the comparison group is the same. Unless the start times are the same, there’s a risk of immortal time bias.
Second, ask yourself “when can the outcome happen?”. If the answer is “any time at all” for one group and “only after eligibility” for the other group, then there’s a risk of immortal time bias.
Finally, see if the authors mention immortal time or immortal time bias anywhere. If they don’t, then they definitely aren’t addressing it!
The only time there’s basically no risk of immortal time bias is when there is a very clear start time that is defined based on some external criteria and at which time the exposure or comparison status of every individual is known (e.g., randomization in an RCT).
But there are some signs we can look for that the researchers have thought about this problem and at least tried to avoid it:
They explicitly mention immortal time bias and why they think they don’t have it, or what they did to address it.
They talk about a “grace period” for exposure and they also include a discussion of statistical methods for accounting for the grace period. (These methods are complex, and so they could end up failing to fix the problem, but talking about them shows you that the researchers are aware and care about this bias)
If you’re curious about immortal time bias, and want to dive deeper into it here are some suggested methodological papers:
Lévesque et al. 2010. Problem of immortal time bias in cohort studies: example using statins for preventing progression of diabetes. BMJ. https://doi.org/10.1136/bmj.b5087
Hernán et al. 2017. Specifying a target trial prevents immortal time bias and other self-inflicted injuries in observational analyses. J Clin Epi. doi: 10.1016/j.jclinepi.2016.04.014
Caniglia et al. 2024. Emulating a sequence of target trials to avoid immortal time bias in pregnancy studies – an application to antibiotic initiation and preterm delivery. Epidemiology. doi: 10.1097/EDE.0000000000001601
As an independent epidemiologist, I make my living entirely through reader support. Readers like you make E is for Epi possible. If you find value in this work, consider becoming a paid subscriber.
My head hurts. Is that because it 11 o’clock or I’m an old fellow? I think I understand but will have to read again when I’m fresher.
This is my favorite bias too. I love spotting it and explaining it.