The fusion peptide (FP), a highly conserved domain within the influenza A spike protein hemagglutinin, is the only portion of the virus that interacts directly with the target membrane. FP clusters within the membrane, is necessary for viral fusion to occur, and can induce membrane poration. Here, the aggregation of FP was investigated through molecular dynamics simulations in four different membrane compositions to examine the lipid-mediated clustering and poration free energy. The formation of highly stable antiparallel FP dimers was confirmed, found to be primarily stabilized by peptide–peptide interactions, and was largely insensitive to membrane composition. However, lipid sorting under the FP dimer was strongly dependent on the lipid spontaneous curvature, with a positive spontaneous curvature-generating lipid depleted and negative spontaneous curvature-generating lipids enriched under the dimer. In simulations of ten FP dimers, the cholesterol-containing membrane promoted higher-order clustering, seen as the formation of tightly bound tetramers and linear arrangements of dimers that were not observed in the other membrane compositions. These tightly bound tetramers were associated with reduced poration efficiency compared to more loosely associated configurations, consistent with earlier experimental work showing that cholesterol partially inhibits poration by FP. These findings demonstrate how lipids with different curvature-generating propensities modulate peptide aggregation and pore formation and may have broader implications for other enveloped viruses, such as SARS-CoV-2, where similar fusion peptide clustering has been observed.