Antigenic drift in influenza A virus hemagglutinin (HA) limits humoral protective immunity. Here, we combine cell fate mapping with adoptive transfer of antigenic-site-specific antibodies (Abs) and memory B cells (MBCs) with moderately drifted HA vaccination in mice to better understand how this influences immune escape and protective responses. We demonstrate that drift in vaccine antigens affects MBC reactivation and naive B cell responses in germinal centers (GC). Strikingly, passively transferred monoclonal and polyclonal Abs suppress cognate epitope-specific GC B cell responses only when the vaccine antigen was multivalent while responses to monovalent recombinant trimeric HA remain unaffected. Using MBC and Abs co-transfer we unveil that antigenic site-specific suppression is more potent in blocking MBC rather than naive B cells entry into GC. In addition, we show that MBC hamper naive B cell recruitment to GC even in the absence of antibody transfer through local differentiation and Ab release in the responding lymph node. Altogether, our study reveals that serum Ab feedback depends on vaccine valency, while pre-existing MBC alone without Abs present can reshape immunodominance of naive B cells, with critical practical implications for rational universal influenza vaccine design.