Background: Children with autism have difficulties in understanding relationships, yet little is known about the levels of autistic traits with regard to peer relationships. This study examined the association between autistic traits and peer relationships. Additionally, we examined whether the expected negative association is more pronounced in children with a lower non-verbal IQ and in those who exhibit more externalizing problems. Method: Data were collected in a large prospective birth cohort of the Generation R Study (Rotterdam, the Netherlands) for which nearly 10,000 pregnant mothers were recruited between 2002 and 2006. Follow up data collection is still currently ongoing. Information on peer relationships was collected with PEERS application, an interactive computerized task (M = 7.8 years). Autistic traits were assessed among general primary school children by using the Social Responsiveness Scale (M = 6.1 years). Information was available for 1580 children. Result: Higher levels of autistic traits predicted lower peer acceptance and higher peer rejection. The interaction of autistic traits with externalizing problems (but not with non-verbal IQ or sex) was significant: only among children with low externalizing problems, a higher level of autistic traits predicted less peer acceptance and more peer rejection. Among children exhibiting high externalizing problems, a poor peer acceptance and high level of rejection is seen independently of the level of autistic traits. Conclusion: We conclude that autistic traits—including traits that do not classify as severe enough for a clinical diagnosis—as well as externalizing problems negatively impact young children’s peer relationships. This suggests that children with these traits may benefit from careful monitoring and interventions focused at improving peer relationships.