Background: Phthalates are synthetic chemicals with endocrine-disrupting properties. They are reportedly associated with various neurotoxic outcomes. Studies on exposure to phthalates and children’s autistic traits have shown inconsistent results with respect to sex and susceptible time periods. We investigated the association of phthalate exposure during the prenatal period and childhood with autistic traits over time using a birth cohort in South Korea. Methods: Five phthalate metabolites were measured during mid-term pregnancy and children's follow-up at ages of 4, 6, and 8 years among a total of 547 mother–child pairs. The social communication questionnaire (SCQ) was used to assess autistic traits of children at each time point. The relationship between phthalate metabolites and SCQ scores were analyzed by exposure windows and sex. Results: A 2.7 fold increase in di-(2-ethylhexyl) phthalate metabolite levels, mono-(2-ethyl-5-hydroxyhexyl) phthalate (MEHHP) and mono-(2-ethyl-5-oxohexyl) phthalate (MEOHP) during pregnancy was associated with increased SCQ scores at 4 years by 8.5% (95% confidence intervals [CI]: 1.9%, 15.5%) and 7.4% (95% CI: 0.3%, 15.0%), respectively, but not at the age of 6 or 8 years. Moreover, MEHHP levels at ages of 4 and 8 years were associated with increased SCQ scores at 8 years by 9.9% (95% CI: 1.8%, 18.6%) and 9.6% (95% CI: 1.3%, 18.6%), respectively. Boys showed stronger associations between phthalate exposure and SCQ scores than girls. Conclusion: The study suggested different susceptible time windows of phthalate exposure: exposure during pregnancy is associated with autistic traits in young children, whereas exposure during early childhood years leads to autistic traits in school-aged children, particularly boys.