The discovery and characterization of broadly neutralizing human antibodies (bnAbs) to the highly conserved stem region of influenza hemagglutinin (HA) have contributed to considerations of a universal influenza vaccine. However, the potential for resistance to stem bnAbs also needs to be more thoroughly evaluated. Using deep mutational scanning, with a focus on epitope residues, we found that the genetic barrier to resistance to stem bnAbs is low for the H3 subtype but substantially higher for the H1 subtype owing to structural differences in the HA stem. Several strong resistance mutations in H3 can be observed in naturally circulating strains and do not reduce in vitro viral fitness and in vivo pathogenicity. This study highlights a potential challenge for development of a truly universal influenza vaccine.