128William Carter was the son of Andrew Jackson Carter Sr. and his wife Margaret Graves Carter. He married Ellen Harriet Mason on January 6, 1878, and they had four children: William Harvey, Alberta, Art Edgar, and Fannie.