The goal of cognitive metascience is to unite cognitive science, philosophy of science, digital philosophy of science, and digital humanities to explore how researchers reflect on their scientific practice and provide normative guidance during the credibility crisis.

We’re using the insights of these large research traditions to study the actual scientific practice. This way, we can enrich considerations required to make valid normative claims that could pave the way towards solving the crisis, especially regarding theory development and assessment.

Here are some resources that we have made available recently.

Corpora

Peer Review corpora – open data sets

Here are some corpora we released:

– eLife Open Peer Review Corpus, https://doi.org/10.18150/FKPEQN

– PLoS Open Peer Review Corpus, https://doi.org/10.18150/KZHVGE

– MDPI Open Peer Review Corpus, https://doi.org/10.18150/D5L2EK

– MDPI Open Peer Review Corpus 2 (much bigger), https://doi.org/10.18150/SHKP7B

The MDPI data set in the version 2 is much bigger – the crawler we used was upgraded to find more data. It is probably the biggest open peer review corpus out there.

Software

A pure Python open source reimplementation of word sketches (collocation) for corpora of arbitrary size, just the kind you would expect in SketchEngine. However, by using our code, you can:

ensure reproducibility (anyone can run the same code using your data),

work with datasets of arbitrary size (even if your institution has very limited access to SketchEngine),

run word sketches in a Python Notebook (think of Constellate).

The project lacks full documentation etc., but we plan to add it in the next couple of weeks.

Papers and preprints

Hensel, W. M., Miłkowski, M., & Nowakowski, P. (2022). Without more theory, psychology will be a headless rider. Behavioral and Brain Sciences, 45, e20. https://doi.org/10.1017/S0140525X21000212

In this commentary to Yarkoni’s paper, we argue that psychology needs more explicit theory development, which can start from fairly mundane claims.

Miłkowski, M. (2022). Turing’s Conceptual Engineering. Philosophies, 7(3), 69. https://doi.org/10.3390/philosophies7030069

This paper describes how to intermingle close and distant reading, as applied to Turing’s influential 1950 paper on machine intelligence.

Miłkowski, M. (2022). Cognitive Artifacts and Their Virtues in Scientific Practice, Studies in Grammar, Logic, and Rhetoric 67(3), 219-246. https://doi.org/10.2478/slgr-2022-0012

This paper describes the conceptual foundations of cognitive metascience: scientific representations are cognitive artifacts whose function we should study across multiple dimensions. A short glimpse on the virtues of models through corpora analysis is given.

Miłkowski, M., Hensel, W. M., & Hohol, M. (2018). Replicability or reproducibility? On the replication crisis in computational neuroscience and sharing only relevant detail. Journal of Computational Neuroscience, 45(3), 163–172. https://doi.org/10.1007/s10827-018-0702-z

We claim that low model reproducibility poses more of a threat to long-term scientific progress than low model replicability and reports of computational studies should remain selective and include all and only relevant bits of code.

Talks

Hensel, W., & Miłkowski, M. (2022, October 22). What Makes Hypotheses Accurate? An Empirical Study. Presented at the Hypothesis in science. On the 550th anniversary of the birth of Nicolaus Copernicus, Uniwersytet im. Mikołaja Kopernika, Poland, Toruń.

Hvorecky, J., & Miłkowski, M. (2022, August 29). Theoretical Virtues of Cognitive Extension. Presented at the Mind and Technology Congress, Coimbra University, Portugal, Coimbra.

Miłkowski, M. (2021a, December 13). Putting it all together. A plea for theorizing in cognitive (neuro)science. Keynote Address presented at the AISC 2021, 7th Conference of the Italian Association for Cognitive Science, Noto, Italy.

Miłkowski, M. (2022, July 8). From theoretical neuroscience to theories in neuroscience. Presented at the Nencki School of Ideas in Neuroscience, Instytut Biologii Doświadczalnej im. M. Nenckiego PAN, Poland, Warszawa.

Miłkowski, M., Hohol, M., & Hensel, W. (2018, June 22). Replicability of Computational Models: Achilles Heel of Neuroscience. IACAP 2018, IACAP / IFiS PAN, Poland, Warszawa.

Miłkowski, M. (2018, November 25). Replicability or reproducibility? On the replication crisis in computational neuroscience and sharing only relevant detail. Aspects of Neuroscience, Neurobiology Scientific Student Association at the University of Warsaw, Poland, Warszawa.

Funded projects

Wiktor Rorot‘s project “Investigation of the use of the concept communication in biology and cognitive sciences” has been funded by the National Science Center. Under my supervision, he will study the notion of communication and related notions in life sciences, by relying on digital philosophy of science. The popular description of the project is also available on the funder site.

Student projects

Are you a student in the Cognitive Science program at the University of Warsaw looking for a new project? Work with me on projects related to cognitive metascience, using language technology and digital philosophy of science. Contact me if you’re interested.

Research stays for students in my lab are also available.