Tracie thinks: might be son of John William and brother of John Wm and Mary Frances
But 1860 & 70 census there was a James H and Mary Stewart who were born ca 1820 and ca1830.
living next door to Richard A Stewart in 1880

James Hartwell Stewart103