Kelly J. Baker writes in an opinion piece in the New York Times (on the merits of which I’d rather not comment):
Madison Grant, a lawyer, eugenicist and the author of “The Passing of the Great Race,” wrote that the American “stock” would be jeopardized by these particular European immigrants…
…in “The Great Gatsby,” F. Scott Fitzgerald reflected the way the ideas of Grant and other scientific racists worked their way into mainstream thought.
“Have you read ‘The Rise of the Colored Empires’ by this man Goddard?” Tom Buchanan asks, in a thinly masked allusion to Grant.
By all signs, it’s an allusion to The Rising Tide of Color Against White World-Supremacy by Lothrop Stoddard, published in 1920 or 1921. The Great Gatsby, published in 1925, is set in 1922. Everything fits.
However, the “G” in “Goddard” might have come from Grant. His opus magnum, The Passing of the Great Race, appeared earlier, in 1916. Grant wrote a preface to Stoddard’s The Rising Tide of Color.