The codon usage pattern can reveal the adaptive changes that allow virus survival and fitness adaptation to their particular host, as well as the external environment. Although still considered a novel influenza virus, there is an increasing number of influenza D viruses (IDV) reported. Considering the vital role of the hemagglutinin-esterase fusion (HEF) gene in in receptor binding, receptor degradation, and membrane fusion, we investigated the codon usage pattern of the IDV HEF gene to better understand its adaptive changes during evolution. Based on the HEF gene, three groups including, D/OK, D/660, and D/Japan were identified. We found a low codon usage bias, which allowed IDV to replicate in the corresponding hosts by reducing competition during evolution, that was mainly driven by natural selection and mutation pressure, with a profound role of natural selection. Furthermore, the interaction between the codon adaption index (CAI) and the relative codon deoptimization index (RCDI) revealed the adaption of IDV to multiple hosts, especially cattle which is currently considered its reservoir. Additionally, similarity index (SiD) analysis revealed that the swine exerted a stronger evolutionary pressure on IDV than cattle, though cattle is considered the primary reservoir. In addition, the conserved PB1 gene showed a similar pattern of codon usage compared to HEF. Therefore, we hypothesized that IDV has a preference to maintain infection in multiple hosts. The study aids the understanding of the evolutionary changes of IDV, which could assist this novel virus prevention and control.