Cell tropism, or the preference of a virus for particular cell types, has major implications for viral transmission, pathogenesis, and evolution. Historically, changes in cell tropism for particular viruses have been observed to correlate to changes in viral fitness, in the form of increased within-host replication and increased transmission between hosts. This is illustrated in the context of influenza, where adaptation to infect cells expressing α2-6 linked sialic acid receptors enhances human-to-human transmissibility. Target cell populations differ not only in abundance but also in intrinsic properties such as susceptibility, viral production, and interferon responses, rendering the relationship between tropism and viral fitness multi-faceted and complex. Understanding how different cell tropisms quantitatively change fitness remains an important open question in virology and quantitative biology. Here, we present a within-host mathematical model that incorporates distinct target cell types differing in key properties, and examine how cell tropism affects fitness-related summary statistics such as peak viral load, infection duration, or total virus produced. Our analysis reveals that tradeoffs may arise when cell types differ by multiple characteristics. We further demonstrate that model parameters describing heterogeneity between cell types can be more accurately inferred when cell type proportions are measured alongside viral load. Our findings provide a framework for assessing the links between viral evolution, cell tropism, and within-host fitness, and motivate the design of experiments to collect quantitative data on between-cell-type heterogeneity.