These Paradigms Won't Shift Themselves...

A.S. I've been 'reading' a book that I'm planning to write up a book club about, but I felt that I first needed to rant about a more general aspect of science writing as a preamble to discussing the book itself.

It's somewhat unfortunate that Thomas Khun has become the patron saint of quackery - at least that's the way that I feel given how often he's brought up by purveyors of non-science. I've read Khun's The Structure of Scientific Revolutions (1962) and found it to be quite interesting, if a tad simplistic. Its classic contribution to the philosophy of science is its argument that science doesn't progress only by the accumulation of facts that build upon existing theories, but rather more importantly, by the accumulation of facts that cannot be squared with current theories. Eventually, the weight of these unresolvable observations 'breaks' the discipline requiring a qualitatively different approach to explaining the data: a completely different set of theories, a.k.a. a new 'paradigm'.

This model works rather well in explicating the context and subsequent effects of landmark historical discoveries (relativity, evolution by natural selection, etc.1), however, it has been criticized in its ability to explain the sorts of 'revolutions' happening in modern science. The argument goes like this: Modern science is a much more well-funded and institutionalized pursuit than it was during these classical revolutions. New 'discoveries' are being made rather frequently and so, when it comes to most sciences, hypotheses are being tested, refuted, and reassessed on much smaller scales than previously. Furthermore, the number of independent observations supporting many fundamental hypotheses is much, much larger than in the past. All of this suggests that we're unlikely to see many 'paradigm shifts' on the level of the classical Khunian examples any time soon, at least for most sciences2.

Unfortunately for the quacks, a new hypothesis - the efficacy of homeopathy, for instance - must incorporate existing observations. And yet so many do not. I think that this is why many of us get frustrated when non-scientists, especially science writers, call for some form of 'middle ground' between practitioners of woo and mainstream science. 'But this or that anecdotal evidence suggests that there might be something to this,' they say. It's a Kuhnian revolution in the making... ungh.

Sadly for the overly brash, science necessitates conservatism. Working within a theoretical framework that may be incomplete, but is solid and works for most circumstances (such as Newtonian gravity), is certainly preferable to overindulging in research under frameworks that have little to no support to begin with. As with many things, ideas are important, but relatively cheap in comparison to being able to come up with methods through which to test them (the challenge of experimental design). Conservatism, as a reviewer of a manuscript once pointed out, also means that novel hypotheses that explain existing observations as well as existing hypotheses are insufficient. They must do better than existing hypotheses - that is to say that they must explain facts that are not explained by those hypotheses in current vogue.

There's a time commitment to science, and testing every random idea out there isn't necessarily an efficient way to make progress... unless you're the NCCAM? 

1The 'Copernican Revolution' is often cited among these, but I don't understand its relevance. As is discussed in detail in Owen Gingerich's The Book That Nobody Read, Ptolemaic epicycles and Copernical heliocentrism were equivalent models for explaining the available data and both made testable predictions about the position of planets (see here). In the absence of other theories that would have cinched the debate (e.g., gravity), the only real difference between the two theories was that Copernicus' made some of the calculations easier, which is why a preface was added to his book, De Revolutionibus Orbium Coelestium, which contained the following:

...it is the duty of an astronomer to compose the history of the celestial motions through careful and expert study. Then he must conceive and devise the causes of these motions or hypotheses about them. Since he cannot in any way attain to the true causes, he will adopt whatever suppositions enable the motions to be computed correctly ... The present author has performed both these duties excellently. For these hypotheses need not be true nor even probable. On the contrary, if they provide a calculus consistent with the observations, that alone is enough ... For this art, it is quite clear, is completely and absolutely ignorant of the causes of the apparent [movement of the heavens]. And if any causes are devised by the imagination, as indeed very many are, they are not put forward to convince anyone that they are true, but merely to provide a reliable basis for computation. 

2That's not to say that I don't think that there are any 'big shifts' on the horizon. Some 'common knowledge', ostensibly derived from science, doesn't have nearly as much evidence as we'd like to believe - especially in disciplines where controlled experimentation isn't always easy. One example that's being challenged in books and research is the idea that fat intake is a major cause of heart disease, something promoted extensively by the American Heart Association. There seems to be growing evidence that refined sugars are a much stronger culprit - and in particular intake of certain sugars in doses not typically found in nature (e.g., high-fructose corn syrup). None of this is my expertise though. While his views seem a bit extreme, I would like to read Gary Taubes' Why We Get Fat, which discusses the issue.