The risk of highly pathogenic avian influenza infection to humans is challenging to estimate because many human avian influenza virus (AIV) infections are undetected as they may be asymptomatic, symptomatic but not tested, and as contact tracing is difficult because human-to-human spread is rare. We derive equations that consider the evolutionary mechanisms that give rise to pandemics and are parameterized to be consistent with records of past pandemics. We estimate that thousands of human AIV infections occur worldwide in an average year and estimate the infection fatality ratio as 32 deaths per 10,000 infections (95% confidence interval: [9.6, 75]). We estimate that preventing 20% of animal-to-human influenza spillovers annually would delay pandemic emergence by an average of 9.4 years. There is a high level of uncertainty in our estimates due to the few records of past pandemics, but even so this infection fatality ratio is comparable to SARS-CoV-2 during the recent pandemic and is higher than seasonal human influenza. Preventing human infections with AIV is necessary given the high risk of severe outcomes to individuals and to reduce the risk of pandemics occurring in the future.