Small was born in Collinsville OK. He graduated with two Bachelors degrees from the University of Oklahoma, one in architecture and one in engineering. He later attended the Illinois Institute of Technology studying under Mies van der Rohe. In Chicago he worked for Perkins/Will and Hudgins Thompson Ball.
At the recommendation of Henry Kamphoefner, who was his professor at Oklahoma, Small relocated to North Carolina in 1948 to be Chief Designer for William Deitrick, at the time Raleigh's largest architectural firm and most committed to Modernist design. While there he designed the 1947 Carolina Country Club, the first such club in the country of Modernist design. Small left Deitrick in 1949 to start his own firm. In the 1950s, he partnered with Joseph Boaz, also from Oklahoma.
Small is noted for designing many buildings, including Carter-Finley Stadium, the NCSU Student Center, WRAL-TV studios (with Joseph Boaz), St. Stephen's Episcopal Church, Durham's Home Security Life Building, and his prolific modernist houses below. His son, G. Milton III, continues as principal at the firm, now Small Kane Architects.