Month: December 2018

Marcin Miłkowski

Our joint paper (written by myself, Mateusz Hohol, and Witold Hensel) on reproducibility of computational neuroscience has just been assigned to the December issue of the Journal of Computational Neuroscience (open access). In this paper, we argue that assuring replication of scientific results does not yield to a single solution. And this is still a problem, with few reproductions being published and low code availability in major journals of computational neuroscience, as our preliminary study shows:

Most importantly, repeating the model, or rerunning the same model by the same researchers, and replicating the model, or rerunning it by others, requires different set of best practices. They actually would benefit from minute documentation of the whole modelling process, including noting random seeds, versioning all the scripts, recording all intermediate results, etc. (a proposal of ensuring repeatability and replicability of computer science this way was recently defended by Sandve et al. 2013).

But for theoretical purposes of understanding and explanation, these could be detrimental. If one wishes to actually reproduce the results by building another model by following its theoretical description, all the minute details could be actually detrimental and distracting. As we argue in the paper, publications regarding models in computational neuroscience should therefore contain all and only information relevant to reproducing a model and evaluating its value. Alas, many papers currently publish fail in both respects: sometimes they include redundant introductions of the theoretical framework, for example, instead of describing how a particular model was produced, and sometimes they simply fail to make clear how theoretical understanding was operationalized in a model.

Our proposal is, as it turned out, similar in spirit to what Guest and Cooper proposed in their paper. They argue that what they call ‘specification of a model’ is not to be conflated with its implementation. Thus, a theory behind a model should be included in its specification, and implementation details, which sometimes include ad hoc assumptions required to make the model actually run, should not be confused with the theory. A good example of such a confusion is how Pinker and Prince (1988) criticize the influential model of past-tense learning by Rumelhart and McClelland (1986): they take the implementation detail to be theoretically important, while later connectionist work shows that the theoretical account is much more general than a particular choice of problem representation (such as the particular phonological representation in the first model).

Our approach is similar to what Guest and Cooper defend. Nonetheless, we stress that one should distinguish two practices – and ensure not only replication and repetition but also reproduction. The first is served better by open repositories, public code review and such, while the second is best ensured by good theoretical publication and subsequent attempted reproduction.

We recently published a paper From Wide Cognition to Mechanisms: A Silent Revolution in Frontiers in Psychology. The paper was six years in the making, which mirrors the slow, silent revolution in cognitive science. In the paper, we argue that several recent ‘wide’ perspectives on cognition—embodied, embedded, extended, enactive, and distributed—are now only partially relevant to the study of cognition.

The study of cognition has already progressed beyond these proposed perspectives toward building integrated explanations of the mechanisms involved, including not only internal submechanisms but also interactions with others, groups, cognitive artifacts, and their environment. Wide perspectives are essentially research heuristics for building mechanistic explanations. It’s a silent revolution; it happens without much fanfare. But it is real.

In some ways, what we say could be considered merely a description of what happens right now in cognitive (neuro)science: it does move away from pure methodological individualism to embrace a view of cognition that is no longer treated as autonomous from the brain, culture, or society. So, what is controversial about our paper?

First and foremost, we argue that wide perspectives on cognition are best understood mechanistically. This means we believe that people who stress embodiment or the role of the environment are best seen as proposing causal explanations of cognitive phenomena. Maybe controversial, if you deny that causal explanations exist, believe that there are general invariant laws of cognition, or consider boxes-and-arrows models of cognition to be more useful than causal models. We don’t think this is really controversial.

Another point is that we stress that wide perspectives offer only limited heuristics for the study of cognition. This is admittedly much more controversial, especially if you have spent a lot of time arguing that methodological solipsism is not a good idea. What, embodied cognition or the extended mind are not really complete theories of cognition?

Nope. Theories should offer detailed predictions for the phenomena we want to study. Take mind-reading: the ability to understand other minds. What in particular does the extended mind perspective predict and explain about it? Maybe that there will be some non-brain-based props that we use to understand others, such as linguistic utterances of others. So maybe some manipulations of linguistic utterances could influence mind-reading. But that’s it. Preciously little. Ditto for embodiment: all it points out is that there could be non-neural factors of mind-reading. Maybe we are much more effective in ascribing mental states to bodies that resemble ours (but again, we know that people can easily ascribe intentions to triangles and squares). Still, we remain quite in the dark about the mechanism that allows us to understand what others think. We need a more integrative story. In the paper, we describe a story based on the idea of mind-shaping as defended recently by Tad Zawidzki.

Thus, what we say is that instead of providing complete theories or iron principles of cognition, wide perspectives help researchers make informed choices of what causal factors to study when explaining cognitive phenomena. Wide perspectives provide fallible heuristics for the study of cognition and thus, they usually best work together.

I’m sure that there is a lot here to disagree and discuss further. What I hope to see is whether we could, thanks to this discussion, make some progress toward understanding what theory in cognitive science should be.

Thanks to all authors who helped to write this paper. 

Marcin Miłkowski