Here is a fascinating essay by Isaac Asimov authored back in the 1950’s as part of some consulting on a government funded research project. Assimov’s thoughts on where ideas come from, how to spur creativity, and the right team size and location to conduct brainstorming are very revealing. Fundamentally, Asimov describes the process as making connections between new facts and experiences with older ideas and insights … what he terms “the ability to make cross-connections.”
Undoubtedly in the first half of the 19th century, a great many naturalists had studied the manner in which species were differentiated among themselves. A great many people had read Malthus. Perhaps some both studied species and read Malthus. But what you needed was someone who studied species, read Malthus, and had the ability to make a cross-connection.
That is the crucial point that is the rare characteristic that must be found. Once the cross-connection is made, it becomes obvious. Thomas H. Huxley is supposed to have exclaimed after reading On the Origin of Species, “How stupid of me not to have thought of this.”
But why didn’t he think of it? The history of human thought would make it seem that there is difficulty in thinking of an idea even when all the facts are on the table. Making the cross-connection requires a certain daring. It must, for any cross-connection that does not require daring is performed at once by many and develops not as a “new idea,” but as a mere “corollary of an old idea.”
As they say, read the whole thing.