An important characteristic of influenza A is its ability to escape host immunity through antigenic drift. A novel influenza A strain that causes a pandemic confers full immunity to infected individuals. Yet when the pandemic strain drifts, these individuals will have decreased immunity to drifted strains in the following seasonal epidemics. We compute the required decrease in immunity so that a recurrence is possible. Models for influenza A must make assumptions on the contact structure on which the disease spreads. By considering local stability of the disease free equilibrium via computation of the reproduction number, we show that the classical random mixing assumption predicts an unrealistically large decrease of immunity before a recurrence is possible. We improve over the classical random mixing assumption by incorporating a contact network structure. A complication of contact networks is correlations induced by the initial pandemic. We provide a novel analytic derivation of such correlations and show that contact networks may require a dramatically smaller loss of immunity before recurrence. Hence, the key new insight in our paper is that on contact networks the establishment of a new strain is possible for much higher immunity levels of previously infected individuals than predicted by the commonly used random mixing assumption. This suggests that stable contacts like classmates, coworkers and family members are a crucial path for the spread of influenza in human populations.