Sociologists of race and ethnicity have convincingly demonstrated that cultural categorizations are affected by racialization. For example, some music genres are clearly dominated by people of color (e.g., rap, soul, r&b), whereas others are particularly popular among whites (e.g., rock, country, classical). However, we know relatively little about how habitual such racialized categorizations are for people. We offer an integrated approach of cultural-sociological insights on the implicit underpinnings of categorization processes to understand how implicit, “nondeclarative” aspects of racial association lie at the heart of the—often, but not always, unintentional—(re)production of racialized cultural categories. We use an Implicit Association Test in combination with a survey (n = 920) to study the connection between rock music and whiteness, and rap music and blackness. First, we find a relationship between whiteness/blackness and having a preference for rock/rap. Second, we demonstrate that the racial associations regarding these genres are firmly grounded in non-declarative knowledge and that this is widely shared across racial groups, albeit somewhat more among people of color.