One potential source of heterogeneity within autism spectrum conditions (ASC) is language development and ability. In 80 high-functioning male adults with ASC, we tested if variations in developmental and current structural language are associated with current neuroanatomy. Groups with and without language delay differed behaviorally in early social reciprocity, current language, but not current autistic features. Language delay was associated with larger total gray matter (GM) volume, smaller relative volume at bilateral insula, ventral basal ganglia, and right superior, middle, and polar temporal structures, and larger relative volume at pons and medulla oblongata in adulthood. Despite this heterogeneity, those with and without language delay showed significant commonality in morphometric features when contrasted with matched neurotypical individuals (n = 57). In ASC, better current language was associated with increased GM volume in bilateral temporal pole, superior temporal regions, dorsolateral fronto-parietal and cerebellar structures, and increased white matter volume in distributed frontal and insular regions. Furthermore, current language-neuroanatomy correlation patterns were similar across subgroups with or without language delay. High-functioning adult males with ASC show neuroanatomical variations associated with both developmental and current language characteristics. This underscores the importance of including both developmental and current language as specifiers for ASC, to help clarify heterogeneity.