Interferon-inducible transmembrane protein 3 (IFITM3) inhibits influenza virus infection by blocking viral membrane fusion, but the exact mechanism remains elusive. Here, we investigated the function and key region of IFITM3 in blocking influenza virus entry mediated by hemagglutinin (HA). The restriction of IFITM3 on HA-mediated viral entry was confirmed by pseudovirus harboring HA protein from H5 and H7 influenza viruses. Subcellular co-localization and immunocoprecipitation analyses revealed that IFITM3 partially co-located with the full-length HA protein and could directly interact with HA2 subunit but not HA1 subunit of H5 and H7 virus. Truncated analyses showed that the transmembrane domain of the IFITM3 and HA2 subunit might play an important role in their interaction. Finally, this interaction of IFITM3 was also verified with HA2 subunits from other subtypes of influenza A virus and influenza B virus. Overall, our data demonstrate for the first time a direct interaction between IFITM3 and influenza HA protein via the transmembrane domain, providing a new perspective for further exploring the biological significance of IFITM3 restriction on influenza virus infection or HA-mediated mediated antagonism or escape.