H5N1 highly pathogenic avian influenza viruses have spread globally and pose a risk for a human pandemic. Prior studies suggest that early life exposures to group 1 influenza viruses (H1N1 and H2N2) prime antibodies that cross-react to the hemagglutinin of H5N1, which is also a group 1 virus. Less is known about how immune history affects antibody responses against the neuraminidase (NA) of H5N1 viruses. Here, we measured NA inhibition antibodies against multiple H5N1 viruses using sera from 155 individuals born between 1927 and 2016. We found that individuals primed in childhood with H1N1 viruses were more likely to possess higher levels of antibodies that cross-react with the NA of H5N1 viruses compared to individuals primed in childhood with H2N2 or H3N2 viruses. While young children rarely possessed cross-reactive NA antibodies, we found that childhood infections with contemporary H1N1, but not H3N2, viruses can elicit them. These data suggest that immune history greatly impacts the generation of cross-reactive NA antibodies that can inhibit H5N1 viruses.