Be Less Wrong: Lessons In Science From SPARCS2014

Nothing to do with the text. I just like the cover.

The SPARCS conference was a unique opportunity for people to hear directly from researchers; not restrained by an interviewer, the PIs were able to present their research in whatever way they felt was important. Researchers don’t often get to talk directly to the public and I can’t think of a case where such a distinguished line-up has ever gotten to do so – and FREE!

I was glad the presenters spoke not just about their work but also about how science in general; how it is done and not done. Most people rarely get to peek behind the curtains, to witness the internal discussions, the disagreements, the problems that come up and the fights that take place.

So with the conference still fresh in mind, and before World Cup fever and Yellow Jersey fever combine to render me senseless and wipe everything from memory, I thought I’d re-iterate a few important points about science.

Hypothesis and Falsification

In 1861 Darwin wrote, “About thirty years ago there was much talk that geologists ought only to observe and not theorise; and I well remember some one saying that at this rate a man might as well go into a gravel-pit and count the pebbles and describe the colours. How odd it is that anyone should not see that all observation must be for or against some view if it is to be of any service!”

The importance of a working hypothesis hasn’t gone away, without that guidance one might as well count pebbles, but that’s not always the case. Ethologists might record and codify behavior into ethograms and only make sense of it after the fact, automated data mining, heuristic knowledge discovery and similar research can also proceed without a hypothesis. However, for the topic of canine behavior, the traditional format still generally applies.

“If it could be demonstrated that any complex organ existed which could not possibly have been formed by numerous, successive, slight modifications, my theory would absolutely break down.”- With this simple statement Darwin put forth a way for anyone to falsify his theory of evolution.

This sentiment is mostly true; falsification is how we know it is time to dump an idea for something better. It is important not to confuse a theory’s inability to explain X as an indication the hypothesis/theory is false. Incompleteness isn’t falsification – all scientific ideas are incomplete and there really is no way of ever knowing if any idea is completely true.

Methods

Science requires a systematic approach to observation and the system used is described by the method. The methodology describes what we observe, how we observe and how we measure it. This is different from anecdotes, folklore or casual observations; that’s the realm of Yeti, Cesar Millan and guilty looks.

The issue came up a couple of times during the presentations; bad methods lead to unreliable results. Moreover, it’s not only ‘bad’ methods that can lead to problems; validation is also important.

Various presenters emphasized the importance of validation, something that is an ongoing concern in behavioral biology.  It’s all well and good to say a dog attacked a rubber hand placed in its food bowl, it a whole other thing to claim this is a valid test for resource guarding. Conclusion based on non-validated assessments have life-and-death implications for shelter dogs who may be deemed unadoptable based on non-validated behavior tests.

Validation is how we know we are measuring what we think we are measuring.

Replication

RIKEN, a world-class research institute in Japan is currently in a deep crisis. Dr. Haruko Obokata, one of their researchers published an apparent breakthrough but there was a problem, no other lab could repeat her success. It led to an investigation, censure and finally her concession to retract the papers in question. [Personal note: I think Obokata was inexperienced and somewhat incompetent but her senior collaborators were also at fault yet they’ve been spared of any blame]

Dr. Gabois recounted the case with a “genius” dog whose performance dropped to chance after the departure of one of the students. The failure to replicate earlier results led them to take a closer look at the how they were conducting the experiment and they were able to find the source of the problem and tighten up their procedures.

If it only works once, it doesn’t work. And it’s not science.

Parsimony

The law of parsimony demands we adopt an explanation with the least assumptions over one with more assumptions. There is a mathematical justification for parsimony; every parameter introduced has a degree of uncertainty, which may be known, or unknown (the 9th circle of statistical hell is Bayesian) and the more complex the explanation the more uncertainty it carries. There is also a practical reason; simpler explanations are easier to test.

There no guarantee the simplest explanation is the correct explanation.  Parsimony is the go-to method because the alternative results in an ever-expanding number of possibilities, the complexity of which is only limited by imagination.

Criticism

The SPARCS2014 filler clip had Dr. Wynne making an obvious observation, scientists disagree. Unfortunately, the only time most people see it is under the contrived conditions fabricated by sensational media. It usually involves politicizing topics where they pit science against pretend science of climate deniers, anti-vaxxers, creationists, etc. Like Dr. Wynne, I am referring to scientific disagreements.

I often think that for scientists, the next best thing to coming up with a new discovery is demonstrating someone else is wrong. Criticism isn’t the product of pettiness or vindictiveness (mostly) but rather because when it comes to our own specialty, we are very fussy about details.

Cosmologists are currently battling the recent conclusions that came out of BICEP2 and whether the authors really found signals of cosmic inflation. Under peer-review and facing pressure from cosmologists, the authors changed their definitive statement to acknowledge the signal they detected may have been a background artefact. Other teams are working to resolve the problem.

Criticism –that means SCIENTIFIC CRITICISM, not sophistry, personal incredulity or ideological dogma – is vitally important to science and it cultivates the advancement of knowledge.

We Don’t Need “Ultimate” Answers

Mathematician Richard Courant wrote: “To renounce the goal of comprehending the “thing itself” of knowing the “ultimate truth,” of unraveling the innermost essence of the world, may be a psychological hardship for the naive enthusiasts, but in fact it was one of the most fruitful turns in modern thinking”

Naïve people often want/ expect science to provide ultimate (not Tinbergen’s ultimate) answers; science can’t do that. None of the presenters ever pretended they has solved or figured out the “essence” of dogs/behavior/emotion/whatever; that is because scientists are not essentialists. We don’t need to know the intrinsic, essential nature of a thing – gravity, behavior, genetics – to study it and understand some aspect of it. Science can only ask a specific question about a specific phenomenon which is why science don’t talk about ‘essence’ (how Aristotelian!) and focuses on specifics.

Share your Knowledge

Dr. Udell observed that science is a process; I would add that it is an open-ended and never-ending process. Science builds on previous knowledge, something long recognized by its practitioners; Newton made this clear over 300 years ago when he wrote, “If I have seen a little further it is by standing on the shoulders of Giants.”

Science advances because we share what we learn. Sharing also extends to providing a platform and assistance to the next generation of researchers. Science is hard, research is even harder; it is a skill that requires time to develop and best learned under senior guidance. The importance and impact of mentor was evident in the high quality presentations by the crop of new researchers.

In my mind, sharing should also extend to the data; something behavioral researchers haven’t yet embraced. I hope canine researchers take a cue from researchers who make their datasets freely available and those who study animal behavior get on board with the movement to make raw data available with all papers, not only for transparency to keep the data from disappearing.

“Be Less Wrong”

Galileo was less wrong than Kepler. Newton was less wrong than Galileo. Einstein was less wrong than Newton but Einstein was still wrong and there remains a discrepancy between his calculations and reality.

One of my first tweets during the #SPARCS2014 was a quote paraphrased from Churchill; I tweeted:

“Science is the worst method to learn about the universe, except for all the others we’ve tried.”

Scientists cannot provide us with the truth with a capital T, that’s the realm of philosophy and religion.   As a scientist I don’t have to be “right” to advance human knowledge, all I have to do is be less wrong than those who came before me.

Science is not a perfect method it’s just the best we have; it cannot provide perfect answers but we can still be less wrong if we stick to scientific principles.