It was recently discovered that the universe's expansion is accelerating, not slowing, as was previously thought. Light from distant exploding stars revealed that an unknown force (dubbed "dark energy") more than outweighs gravity on cosmological scales.
Unexpected by researchers, such a force had nevertheless been predicted in 1915 by a modification that Albert Einstein proposed to his own theory of gravity, the general theory of relativity. But he later dropped the modification, known as the "cosmological term", calling it the "biggest blunder" of his life.
So the headlines proclaim: "Einstein was right after all", as though scientists should be compared as one would clairvoyants: Who is distinguished from the common herd by knowing the unknowable - such as the outcome of experiments that have yet to be conceived, let alone conducted? Who, with hindsight, has prophesied correctly?
But science is not a competition between scientists; it is a contest of ideas - namely, explanations of what is out there in reality, how it behaves, and why. These explanations are initially tested not by experiment but by criteria of reason, logic, applicability, and uniqueness at solving the mysteries of nature that they address. Predictions are used to test only the tiny minority of explanations that survive these criteria.
The story of why Einstein proposed the cosmological term, why he dropped it, and why cosmologists today have reintroduced it illustrates this process. Einstein sought to avoid the implication of unmodified general relativity that the universe cannot be static - that it can expand (slowing down, against its own gravity), collapse, or be instantaneously at rest, but that it cannot hang unsupported.
This particular prediction cannot be tested (no observation could establish that the universe is at rest, even if it were), but it is impossible to change the equations of general relativity arbitrarily. They are tightly constrained by the explanatory substance of Einstein's theory, which holds that gravity is due to the curvature of spacetime, that light has the same speed for all observers, and so on.
The mystery of accelerating expansion
But Einstein realised that it is possible to add one particular term - the cosmological term - and adjust its magnitude to predict a static universe, without spoiling any other explanation. All other predictions based on the previous theory of gravity - that of Isaac Newton - that were testable at the time were good approximations to those of unmodified general relativity, with that single exception: Newton's space was an unmoving background against which objects move. There was no evidence yet, contradicting Newton's view - no mystery of expansion to explain. Moreover, anything beyond that traditional conception of space required a considerable conceptual leap, while the cosmological term made no measurable difference to other predictions. So Einstein added it.
Then, in 1929, Edwin Hubble discovered that the universe is expanding, consistently (within the observational accuracy of the day) with unmodified general relativity. So Einstein dropped the cosmological term. His doing so had nothing to do with Hubble being less blunder-prone; nor was Einstein deferring to Hubble's superior prophetic abilities. It was just that the problem that the term was intended to solve no longer existed.
The new observations did not refute the existence of a cosmological term. They merely made it a bad explanation. Then, in 1998, came those new observations of a universe whose expansion is accelerating. As a result, the cosmological term that has been "reinstated" to account for the new observations is not quite the one that Einstein proposed and retracted. It is larger, for it now has to explain not just why the universe isn't collapsing, but why its expansion is accelerating.
Einstein's remark about having "blundered" is as misleading as the idea that he is "right after all." The cosmological term is not something that should never have been proposed. Its introduction represented progress in understanding reality - as did its abandonment in light of Hubble's discovery and its reinstatement in revised form to account for the new observations.
Likewise, the mid-twentieth century "Bohr-Einstein debate" about quantum theory is often misinterpreted as a personal clash between wizards. So counter-intuitive are quantum theory's predictions that, under the leadership of one of its pioneers, Neils Bohr, a myth grew that there is no underlying reality that explains them.
Particles get from A to B without passing through the intervening space, where they have insufficient energy to exist; they briefly "borrow" the energy, because we are "uncertain" about what their energy is. Information gets from A to B without anything passing in between - what Einstein called "spooky action at a distance". And so on.
Abandonment of realism
What these paradoxical interpretations have in common is that they abandon realism, the doctrine that a physical world, existing in reality, accounts for all of our experience. Anti-realism remains popular and appears in various guises in textbooks and popular accounts of quantum theory. But Einstein insisted that physical phenomena have explanations in terms of what he called "elements of reality".
Fortunately, a minority of physicists, myself included, likewise side unequivocally with realism, by adopting Hugh Everett's multiple-universes interpretation of quantum theory. According to this view, no particles exist where they have insufficient energy to be; it is simply that in some universes they have more energy than average, and in others, less. All alleged "paradoxes" of quantum theory are similarly resolved.
So, while most accounts say that Bohr won the debate, my view is that Einstein, as usual, was seeking an explanation of reality, while his rivals were advocating nonsense. Everett's interpretation doesn't make Einstein a demigod. But it does make him right.
David Deutsch is a visiting professor of physics at the Centre for Quantum Computation, the Clarendon Laboratory, University of Oxford, and author of The Fabric of Reality and The Beginning of Infinity.
A version of this article was first published on the Project Syndicate.
