Tag: extrapolation

Chance & Necessity: Another Look

In everyday experience, we think of chance as what governs gambling, barring schemes to harness probability in their favor. Necessity, on the other hand, brings to mind the air we breathe, food and/or water. Previously, we discussed Jacques Monod as an early explorer of the collision of probability and chance in biology. In Laws of the Game: How the Principles of Nature Govern Chance (German: Das Spiel: Naturgesetze steuern den Zufall), Manfred Eigen & Ruthild Winkler critique Jacques Monod, stating that his innovation was that the substrate of all biology is chance and accident. You may note that the subtitle of Laws of the Game states the opposite.

This gets even more subtle if you consider Johan Huizinga’s Homo Ludens: A Study of the Play Element in Culture. He emphasizes the role of playful behavior as the university that educates us in the connection between chance and necessity.

Foreword

The lay reader interested in the natural sciences is exposed to a constantly rising flood of information. He may often feel himself cast in the role of a judge before whom different authors appear like competing litigants, each of whom hopes to be found in the right. If these authors are sufficiently clever in presenting their cases, the reader may be swayed by each in turn. But then a reviewer appears on the scene and objects that each cannot be as right as the other, particularly when one author’s claims are the exact opposite of another’s. The reader has no choice but to find the reviewer right, too.

This familiar anecdote, which we have slightly adapted to suit our purposes, contains a moral: Everyone could in fact be right if each one did not insist on being the only one to be right.

Everything that happens in our world resembles a vast game in which nothing is determined in advance but the rules, and only the rules are open to objective understanding. The game itself is not identical with either its rules or with the sequence of chance happenings that determine the course of play. It is neither the one nor the other because it is both at once. It has as many aspects as we project onto it in the form of questions.

We see this game as a natural phenomenon that, in its dichotomy of chance and necessity, underlies all events. Our interpretation of games and play thus goes far beyond Huizinga’s, which considers play only in its relation to human behavior. In applying the concept of play to art, we come much closer to Adorno’s views, which clearly disagree with Huizinga‘s identification of play and art.

The considerations that gave rise to this book go back to the molecular theory of evolution worked out a few years ago. It also draws on the models developed in connection with that theory, models for simulating phenomena like equilibrium, selection, and growth, all of which are based on natural laws. Although this book makes frequent use of examples drawn from the field of biology, its scope is much broader and includes general scientific, philosophical, sociological, and aesthetic perspectives. Our main purpose is to show play in its symbolic nature and in all its metamorphoses. We also make use of its different aspects to illuminate both our scientific understanding of the world and various philosophical positions. We are aware that this brings us into conflict with those who attribute validity to one and only one aspect.

The actual writing of the book was preceded by innumerable talks and discussions with friends and colleagues during our Engadin winter seminars, on hikes and ski tours in the mountains, and, occasionally, over a bottle of wine. We first thought a dialogue would be the appropriate form for our exchange of ideas, but we soon abandoned this concept. The classical dialogues are without exception artificially constructed. Simplicio and Salviati, for example, are both mouthpieces for Galileo. We finally decided that we would both take turns going over all the material point by point. We have, of course, ventured into areas in which we are only dilettantes. We hope the reader will forgive us this in the light of our efforts to uncover parallels and so to emphasize the unity of nature and mind.

We do not want to go any further into the substance of the book here than is necessary to make clear why we chose the title we did. Similarly, the introductions to the individual chapters are meant to lead the reader into the material treated, not to summarize it.

Chance and law (or principles) are the basic elements of games. The subtitle of the book suggests the interaction between them. And here we should add that it is the consequences of chance that are subject to regulation. Only if there is a large number of individual events does the element of chance come under the control of statistical law. This law is at work, for instance, when chance fluctuations undergo self-regulation in equilibrium or when they are amplified in the evolutionary process. The origin of the genetic code, the development of languages by which we transmit our thoughts, the intellectual play of artistic imagination are all based on the same fundamental principles of evolution, even though the results of individual games are determined by the caprice and variability of chance.

The manifestations of the game that matter plays in time and space and the effects this game has on human beings are the themes of the central chapters in the book. Much as we may agree with Jacques Monod in his view of molecular biology, we differ from him greatly in the conclusions he draws from it for mankind and society. We see Monod’s demand for an “existential attitude toward life and society” as an animistic inflation of the role of “chance.” Such a demand neglects the complementary role played by natural law. Criticism of the dialectical overemphasis on “necessity”—a criticism we find fully justified—should not move us to deny completely its obvious infiuence.

We are in complete agreement with Monod’s statement that ethics and knowledge cannot be divorced from each other. For us, however, this view does not condemn the great religions but rather sets them a new task. As we have occasion to point out in the present work, the natural sciences yield no proof of God’s existence, but neither do they claim that men “can do without faith in God.”

The unity of nature is expressed more in the laws governing the relationships between structures than in the structures themselves. Taking this “motto” as a point of departure for Part IV, we examine a number of problems that have recently come more and more under the purview of the natural sciences and that are still far from being solved. In these considerations, we are mainly concerned with the mechanisms of cognizance. In terms of the elementary processes involved, these mechanisms always amount to a falsification in Karl Popper’s sense. Still, we cannot fully accept the views of this great epistemological logician, particularly when he claims: “There is no induction.” Good enough! But there are different mechanisms of falsification that vary in their degree of inductive adaptation.

One of our main concerns in this book lies at the heart of Part III. There we raise the still unsolved question of the hubris of human knowledge. Friedrich Dürrenmatt’s categorical imperative “What concerns all can only be worked out by all” is as relevant for the physicist whose knowledge can unleash the forces of nature as it is for the biologist who is now in a position to manipulate genetic material and influence psychic behavior with drugs. And our appeal here is also directed at the economist and the politician, who have the task of creating, maintaining, and protecting the conditions necessary for a worthwhile human existence.

Pessimistic predictions based on simple extrapolation are widespread today. We will not be able to solve the problem of limited resources by frugality alone. A raw material that will be used up in about fifty years at present rates of consumption might last a hundred years if we cut down on its use. (Such a reduction might, of course, threaten the stability of our economy.) In other words, the catastrophe could be delayed for another fifty years. Would it not be more sensible to concentrate on ensuring unlimited resources, perhaps by reusing our raw materials in a recycling economy and by developing sufficiently productive sources of energy? However we solve this problem, the practicability of our solution will depend on stopping population growth, perhaps even on reducing the world’s population.

We must be clear about one thing: The human being is not a fluke of nature, nor does nature automatically guarantee his survival. The human being is one player in a huge game, the outcome of which is, for him, uncertain. He has to make full use of his capabilities to hold his own as a player and not become a plaything of chance.

Manfred Eigen / Ruthild Winkler-Oswatitsch
Göttingen, September 1975

ONE

The Taming of Chance

Research in physics has shown beyond the shadow of a doubt that in the overwhelming majority of phenomena whose regularity and invariability have led to the formulation of the postulate of causality, the common element underlying the consistency observed is chance.

Erwin Schrödinger, “What Is a Natural Law?” (Inaugural Address delivered at the University of Zurich, December 1922)

1

The Origin of Play

Play is a natural phenomenon that has guided the course of the world from its beginnings. It is evident in the shaping of matter, in the organization of matter into living structures, and in the social behavior of human beings.

The history of play goes back to the beginnings of time. The energy released in the “big bang” set everything in motion, set matter whirling in a maelstrom of activity that would never cease. The forces of order sought to bring this process under control, to tame chance. The result was not the rigid order of a crystal but the order of life. From the outset, chance has been the essential counterpart of the ordering forces.

Chance and rules are the elements that underlie games and play. Play began among the elementary particles, atoms, and molecules, and now our brain cells carry it on. Human beings did not invent play, but it is “play and only play that makes man complete.”1

All our capabilities arise from play. First, there is the play of limbs and muscles. The aimless grasping and kicking of an infant develop into carefully coordinated movements. Then there is the play of our senses. Playful curiosity sends us in search of profound knowledge. From play with colors, shapes, and sounds emerge immortal works of art. The first expressions of love take the form of play: the secret exchange of glances, dancing, the interplay of thoughts and emotions, the yielding of partners to each other. In Sanskrit, the union of lovers is called kridaratnam, “the jewel of games.”2

Every game has its rules that set it apart from the surrounding world of reality and establish its own standards of value. Anyone who wants to “play” has to follow those rules. In parlor games, rules established before the game begins determine the course the game will take and define the scale of values by which it is played. But the effects of a chance occurrence can change the constellation of the game and set it running in a totally new direction. This is how life initiated its first games, and this is how our thoughts and ideas continue to play those games.

From the terrifying realm of elemental forces and the holy realm of laws arises the aesthetic impulse, which creates a third, carefree realm. This is the realm of play and appearance. In it, man is freed from the bonds of all circumstance and liberated from compulsion, moral as well as physical.1

It is our intention in this book to trace the interplay of chance and law that has come down to us from the farthest reaches of time and found its most perfect expression in the inexhaustible creativity of our ideas. We hope to translate Hermann Hesse’s symbol of the glass bead game back into reality. Hesse described this game as one that incorporated all the values of our culture.

All the insights, noble thoughts, and works of art that the human race has produced in its creative eras, all that subsequent periods of scholarly study have reduced to concepts and converted into intellectual property—on all this immense body of intellectual values the Glass Bead Game player plays like the organist on an organ. And this organ has attained an almost unimaginable perfection; its manuals and pedals range over the entire intellectual cosmos; its stops are almost beyond number. Theoretically this instrument is capable of reproducing in the Game the entire intellectual content of the universe.3

There is something mysterious about glass beads. The reflection and refraction of light make them glow, and as we explore the idea of play in this book, Hesse’s glass beads will take on a new life. Part of their symbolic significance derives from the fact that they can exemplify the constant transformations and metamorphoses we observe all around us: atoms evolve into crystals, molecules into genes, living cells into cognitive beings, letters into words, notes into harmonies. The course the bead game takes is determined by the roll of the dice, yet at the same time it is also influenced by the rules of the game, just as chance in nature is subject to the laws of physics. The dice and the rules of the game these are our symbols for chance and natural law.

List of References

  1. Schiller, Friedrich von, “Über die ästhetische Erziehung des Menschen (On the Aesthetic Education of Mankind),” in a series of letters from the years 1793-94, printed in his Philosophical and Critical Writings (many eds.).
  2. Huizinga, Johan, Homo Ludens: A Study of the Play Element in Culture. Boston: Beacon Press, 1955.
  3. Hesse, Hermann, The Glass Bead Game: Magister Ludi. New York: Holt, Rinehart & Winston, 1969.
Manfred Eigen & Ruthild Winkler. Laws of the Game: How the Principles of Nature Govern Chance. Translated by Rita & Robert Kimber, Princeton University Press, 1993, pp. xi-5.

Theodor Adorno, a Marxist theoretician, believed the filter before any interpretation of the world was ever-present advertising and, by extension, capitalism. While Adorno thinks of capitalism as destiny, Yeats came to realize that “accident is destiny” (as we previously discussed). Amid these infinitely debatable opinions, the world is often a perplexity machine that may not be described by a single view.

Science-Watching: Forecasting New Diseases in Low-Data Settings Using Transfer Learning

[from London Mathematical Laboratory]

by Kirstin Roster, Colm Connaughton & Francisco A. Rodrigues

Abstract

Recent infectious disease outbreaks, such as the COVID-19 pandemic and the Zika epidemic in Brazil, have demonstrated both the importance and difficulty of accurately forecasting novel infectious diseases. When new diseases first emerge, we have little knowledge of the transmission process, the level and duration of immunity to reinfection, or other parameters required to build realistic epidemiological models. Time series forecasts and machine learning, while less reliant on assumptions about the disease, require large amounts of data that are also not available in early stages of an outbreak. In this study, we examine how knowledge of related diseases can help make predictions of new diseases in data-scarce environments using transfer learning. We implement both an empirical and a synthetic approach. Using data from Brazil, we compare how well different machine learning models transfer knowledge between two different dataset pairs: case counts of (i) dengue and Zika, and (ii) influenza and COVID-19. In the synthetic analysis, we generate data with an SIR model using different transmission and recovery rates, and then compare the effectiveness of different transfer learning methods. We find that transfer learning offers the potential to improve predictions, even beyond a model based on data from the target disease, though the appropriate source disease must be chosen carefully. While imperfect, these models offer an additional input for decision makers for pandemic response.

Introduction

Epidemic models can be divided into two broad categories: data-driven models aim to fit an epidemic curve to past data in order to make predictions about the future; mechanistic models simulate scenarios based on different underlying assumptions, such as varying contact rates or vaccine effectiveness. Both model types aid in the public health response: forecasts serve as an early warning system of an outbreak in the near future, while mechanistic models help us better understand the causes of spread and potential remedial interventions to prevent further infections. Many different data-driven and mechanistic models were proposed during the early stages of the COVID-19 pandemic and informed decision-making with varying levels of success. This range of predictive performance underscores both the difficulty and importance of epidemic forecasting, especially early in an outbreak. Yet the COVID-19 pandemic also led to unprecedented levels of data-sharing and collaboration across disciplines, so that several novel approaches to epidemic forecasting continue to be explored, including models that incorporate machine learning and real-time big data data streams. In addition to the COVID-19 pandemic, recent infectious disease outbreaks include Zika virus in Brazil in 2015, Ebola virus in West Africa in 2014–16, Middle East respiratory syndrome (MERS) in 2012, and coronavirus associated with severe acute respiratory syndrome (SARS-CoV) in 2003. This trajectory suggests that further improvements to epidemic forecasting will be important for global public health. Exploring the value of new methodologies can help broaden the modeler’s toolkit to prepare for the next outbreak. In this study, we consider the role of transfer learning for pandemic response.

Transfer learning refers to a collection of techniques that apply knowledge from one prediction problem to solve another, often using machine learning and with many recent applications in domains such as computer vision and natural language processing. Transfer learning leverages a model trained to execute a particular task in a particular domain, in order to perform a different task or extrapolate to a different domain. This allows the model to learn the new task with less data than would normally be required, and is therefore well-suited to data-scarce prediction problems. The underlying idea is that skills developed in one task, for example the features that are relevant to recognize human faces in images, may be useful in other situations, such as classification of emotions from facial expressions. Similarly, there may be shared features in the patterns of observed cases among similar diseases.

The value of transfer learning for the study of infectious diseases is relatively under-explored. The majority of existing studies on diseases remain in the domain of computer vision and leverage pre-trained neural networks to make diagnoses from medical images, such as retinal diseases, dental diseases, or COVID-19. Coelho and colleagues (2020) explore the potential of transfer learning for disease forecasts. They train a Long Short-Term Memory (LSTM) neural network on dengue fever time series and make forecasts directly for two other mosquito-borne diseases, Zika and Chikungunya, in two Brazilian cities. Even without any data on the two target diseases, their model achieves high prediction accuracy four weeks ahead. Gautam (2021) uses COVID-19 data from Italy and the USA to build an LSTM transfer model that predicts COVID-19 cases in countries that experienced a later pandemic onset.

These studies provide empirical evidence that transfer learning may be a valuable tool for epidemic forecasting in low-data situations, though research is still limited. In this study, we aim to contribute to this empirical literature not only by comparing different types of knowledge transfer and forecasting algorithms, but also by considering two different pairs of endemic and novel diseases observed in Brazilian cities, specifically (i) dengue and Zika, and (ii) influenza and COVID-19. With an additional analysis on simulated time series, we hope to provide theoretical guidance on the selection of appropriate disease pairs, by better understanding how different characteristics of the source and target diseases affect the viability of transfer learning.

Zika and COVID-19 are two recent examples of novel emerging diseases. Brazil experienced a Zika epidemic in 2015–16 and the WHO declared a public health emergency of global concern in February 2016. Zika is caused by an arbovirus spread primarily by mosquitoes, though other transmission methods, including congenital and sexual have also been observed. Zika belongs to the family of viral hemorrhagic fevers and symptoms of infection share some commonalities with other mosquito-borne arboviruses, such as yellow fever, dengue fever, or chikungunya. Illness tends to be asymptomatic or mild but can lead to complications, including microcephaly and other brain defects in the case of congenital transmission.

Given the similarity of the pathogen and primary transmission route, dengue fever is an appropriate choice of source disease for Zika forecasting. Not only does the shared mosquito vector result in similar seasonal patterns of annual outbreaks, but consistent, geographically and temporally granular data on dengue cases is available publicly via the open data initiative of the Brazilian government.

COVID-19 is an acute respiratory infection caused by the novel coronavirus SARS-CoV-2, which was first detected in Wuhan, China, in 2019. It is transmitted directly between humans via airborne respiratory droplets and particles. Symptoms range from mild to severe and may affect the respiratory tract and central nervous system. Several variants of the virus have emerged, which differ in their severity, transmissibility, and level of immune evasion.

Influenza is also a contagious respiratory disease that is spread primarily via respiratory droplets. Infection with the influenza virus also follows patterns of human contact and seasonality. There are two types of influenza (A and B) and new strains of each type emerge regularly. Given the similarity in transmission routes and to a lesser extent in clinical manifestations, influenza is chosen as the source disease for knowledge transfer to model COVID-19.

For each of these disease pairs, we collect time series data from Brazilian cities. Data on the target disease from half the cities is retained for testing. To ensure comparability, the test set is the same for all models. Using this empirical data, as well as the simulated time series, we implement the following transfer models to make predictions.

  • Random forest: First, we implement a random forest model which was recently found to capture well the time series characteristics of dengue in Brazil. We use this model to make predictions for Zika without re-training. We also train a random forest model on influenza data to make predictions for COVID-19. This is a direct transfer method, where models are trained only on data from the source disease.
  • Random forest with TrAdaBoost: We then incorporate data from the target disease (i.e., Zika and COVID-19) using the TrAdaBoost algorithm together with the random forest model. This is an instance-based transfer learning method, which selects relevant examples from the source disease to improve predictions on the target disease.
  • Neural network: The second machine learning algorithm we deploy is a feed-forward neural network, which is first trained on data of the endemic disease (dengue/influenza) and applied directly to forecast the new disease.
  • Neural network with re-training and fine-tuning: We then retrain only the last layer of the neural network using data from the new disease and make predictions on the test set. Finally, we fine-tune all the layers’ parameters using a small learning rate and low number of epochs. These models are examples of parameter-based transfer methods, since they leverage the weights generated by the source disease model to accelerate and improve learning in the target disease model.
  • Aspirational baseline: We compare these transfer methods to a model trained only on the target disease (Zika/COVID-19) without any data on the source disease. Specifically, we use half the cities in the target dataset for training and the other half for testing. This gives a benchmark of the performance in a large-data scenario, which would occur after a longer period of disease surveillance.

The remainder of this paper is organized as follows. The models are described in more technical detail in Section 2. Section 3 shows the results of the synthetic and empirical predictions. Finally, Section 4 discusses practical implications of the analyses.

Access the full paper [via institutional access or paid download].