Stephen Jay Gould on Trends and Progress: a Caution for Trend Watchers

Stephen Jay Gould on Trends and Progress

Stephen Jay Gould, in is 1988 Stanford Presidential Address “TRENDS AS CHANGES IN VARIANCE: A NEW SLANT ON PROGRESS AND DIRECTIONALITY IN EVOLUTION” talks about trends in scientific understanding, his ideas apply to many areas where the term “trend” pops up. Consider this paragraph:

“The greatest impediments to scientific progress are often conceptual locks, not factual lacks. The missing pieces of a puzzle may well deprive us of an adequate solution, but a far more serious bar to understanding lies in mental frameworks that limit conceivable solutions to a restricted, and false, subset of larger possibilities. When we think that we proceed with absolute and comprehensive objectivity, we are even more likely to be lost, for then we unconsciously cloak our own disabling biases and sally forth down a primrose path masquerading as the straight and narrow road to final truth.”

Trends watching is a precarious occupation for those with a bias, because they are likely to follow trends right over a cliff. In addition, trends can be disrupted beyond being false, leaving those who hang their hat upon a trend stranded in an intellectual desert.  Luckily for most trend watchers, as Dr. Michio Kaku recently shared over breakfast in Austria, they “don’t predict things that will happen before you die.”

Beside disruption interrupting a trend, a disruption resets trends that surround the disruption much as an earthquake resets the strain on a fault. Unlike earthquake faults, a trend does not precipitate future versions of itself. Forecasting what they do precipitate is difficult, if not impossible, in terms of both direction and time, because when trends reset they become the pawn of forces larger than themselves. Disruption creates a new set of conditions. Some trends may continue unaltered, others may be deflected, slowing or waning, while new trends may appear as a result of the new conditions.

As we enjoy this early stage of the 2015 baseball season, I give you with Mr. Gould’s thoughtful analysis about trends in his most beloved of sports:

“I conclude, therefore, that .400 hitting has disappeared as an automatic consequence of symmetrically shrinking variation around a constant mean. This new depiction of an old observation implies a reversed interpretation as well. The old explanations wept and wailed, because they assumed that something precious had been lost–the obvious interpretation for removal of an entity. But I hold that the trend reflects increasing general excellence of play–and that symmetrically shrinking variance should occur in systems that stabilize as they improve. Pitching and hitting have both become substantially better as training of athletes intensifies, and as opportunities open for players of all races and nations. But the balance between hitting and pitching has been maintained as both improve–and we define that balance by the unchanged mean batting average of .260. As everyone gets better, the discrepancy between average and best must decrease (leading to the disappearance of .400 hitters), while poor batters once tolerated for excellence in fielding no longer make the grade as the pool of players who can both hit and field grows (leading to shrinkage of the left tail).

The game has become more precise and unfailingly correct in execution — as 100 years of trial and error distill the optimal procedures in all situations of fielding, hitting, and pitching. The best can no longer take advantage of sloppiness in a young system still regulating its subparts. Wee Willie Keeler could “hit ’em where they ain’t,” and bat .432 in 1897, because fielders didn’t yet know where they should be. Now every pitch and every hit is charted; the weaknesses and propensities of every batter are assessed in detail. Boggs and Carew were surely better hitters than Keeler, but neither has reached .400 in modem baseball. Increasing general excellence of play has eliminated .400 hitting, but we must first picture the phenomenon as a symmetrical reduction in variance before we can grasp this explanation.”

If businesses applied this type of analysis to problems, many of those problems might not seem so much problems as results of hard work paying off. The senior sales person on a major account not being able to increase quota is a good thing, if all things are equal, literarily. He or she can’t be expected to deliver the same growth targets as a colleague on a new account or in a number of accounts in an emerging market. For the senior sales person and the account, they have negotiated toward a symmetrical reduction in variance between the client expectations and the vendors portfolio of goods or services. Only if their is a major innovation or disruption that either creates a new customer need, or produces a product or service that fills a previously unmet need in the customer, can the share increase in a dramatic way, otherwise this movement toward equilibrium is itself a trend.

Good trend watchers see a swarm of interacting influences swirling around centers of change.  Like objects careening off of each other in a vacuum, trends intersect, change, and then retreat or merge.  The rising of deep analysis, and now Big Data, in baseball is as much an influence on the new mean as is the quality of overall improvement in play. They really can’t be separated as one intimately influences the other.

Strategists should be cautious about any nonchalance about trends. They need to create a context of uncertainty, map the swarm of trends, and then play the trends out under different models of the future. They still won’t be able to predict the future, but they will have a much better grasp on the elements in play and how they relate than someone looking at a “trend” in isolation. And as those trends under scrutiny alight, remember that they won’t all be pointing toward some ideal future that an individual bias wants to see occur, but toward an uncertain future where they may be creating but a niche, not a sea change.