Background: The objective was to estimate the heritability for height and weight during fetal life and early childhood in two independent studies, one including parent and singleton offsprings and one of mono- and dizygotic twins. Methods: This study was embedded in the Generation R Study (n = 3407, singletons) and the Netherlands Twin Register (n = 33694, twins). For the heritability estimates in Generation R, regression models as proposed by Galton were used. In the Twin Register we used genetic structural equation modelling. Parental height and weight were measured and fetal growth characteristics (femur length and estimated fetal weight) were measured by ultrasounds in 2nd and 3rd trimester (Generation R only). Height Results: Heritability estimates for length increased from 2nd to 3rd trimester from 13% to 28%. At birth, heritability estimates for length in singletons and twins were both 26% and 27%, respectively, and at 36 months, the estimates for height were 63% and 72%, respectively. Heritability estimates for fetal weight increased from 2nd to 3rd trimester from 17% to 27%. For birth weight, heritability estimates were 26% in singletons and 29% in twins. At 36 months, the estimate for twins was 71% and Conclusions: Heritability estimates for height and weight increase from second trimester to infancy. This increase in heritability is observed in singletons and twins. Longer follow-up studies are needed to examine how the heritability develops in later childhood and puberty.