Despite antigenic evolution at the global scale, positive selection of influenza virus antigenic variants is not readily observed within hosts. Here, we tested the extent to which fitness tradeoffs, the timing of immune pressure, and stochastic effects impede antigenic selection within pre-immune hosts. We used genetically barcoded influenza A/Texas/50/2012 (H3N2) viruses (Tx/12) in a guinea pig model to probe these dynamics. Positive selection of an antigenic variant was reliant on a high strength of immune pressure acting early in infection. However, when fitness tradeoffs of the antigenic change were lessened, a lower strength and later introduction of immune pressure favored the antigenic variant. In all conditions, barcode dynamics revealed moderate stochastic effects. Our results suggest that stochastic evolution does not impede selection during acute influenza virus infection. The rarity of antigenic escape may instead stem from low mutational supply, fitness tradeoffs, and the intrinsic delay between infection and antibody recall.