With John Hopfield recently receiving the Nobel Prize in Physics, it’s the perfect time to dive into his groundbreaking work on neural networks. In this series, I’ll explore the fascinating world of Hopfield networks—an elegant and biologically inspired model for associative memory. These networks not only provide insights into brain function but also have broad applications in computational neuroscience and machine learning. In this first post, we’ll break down the core principles of Hopfield networks, their characteristics, and I’ll walk you through Python implementations to get you started with this powerful concept.

When dealing with dynamical systems, one usually faces the problem of partially-observed data. The engineering literature deals a lot with partially-observed linear dynamical systems, but do we have any hope to make predictions when our system is nonlinear? To answer this question, we will have to discuss one of the most amazing results in this context: Takens’s theorem, which actually allows us to give a positive answer to this question.

When starting this series of posts, I claimed that their intent was to argue that Einstein’s description of Brownian motion was his most groundbreaking contribution among the ones he presented in his annus mirabilis. In the first blog-post I presented Einstein’s approach to the problem, and in the second one, I showed how to solve the Heat equation (i.e., the diffusion equation). Now we have all the tools that we need to discuss how Einstein’s approach to Brownian motion led to the proof of the existence of Atoms.

In this second post about Brownian Motion I will discuss how to solve the Heat Equation. This will lead to a brief introduction to the Fourier Transform and what it is, in my opinion, a nice way of thinking about its meaning. Once we have the explicit form of the solution, we can derive the distinctive feature of all diffusion phenomena: the linear dependence of their variance with respect to time.

The study of Brownian motion laid the ground for developing the field of stochastic processes. Einstein was the first to shed light on this phenomenon, but I think this work’s importance is usually underestimated. In this series of 3 blog posts, I want to argue that this work is conceptually and practically even more groundbreaking than special relativity. In this first part, we will show how Einstein drew the analogy between the Brownian motion and the (seemingly unrelated) heat diffusion.

Predicting the future has always been a crucial goal for humans. In a certain sense, it can be said that it is the fundamental task of science: while it is basically always possible to explain past observations, well-grounded scientific theories must be able to predict future outcomes of experiments. Otherwise, they are rejected (or, at least, questioned).

Before digging into some of the saucy details of dynamical systems theory, I think it is time to introduce at least a bit of machine learning in this blog. The chance came naturally yesterday, as I wanted to learn how to use Jax, which is a sort of improved numpy, with a “functional soul”. I’m not digging into details about the codind part since I’m far for beeing confident with the library and this post will mainly revolve around the conceptual part of the topic. Yet, I will show all the code I used, so that you can follow through and appreciate the functionalities offered by Jax. If you are not confident with the machine learning ideas, you will maybe find some parts of this blogpost a bit mysterious, but you should still be able to follow through as I’ve tried to explain each step (at least at high-level, which is all we need for now).

The butterfly effect is the popular concept that “the flap of a butterfly’s wings in Brazil can set off a tornado in Texas”. This idea, which is in fact misunderstood and misrepresented in many occasions, came from the seminal work of Edward Lorenz, which was conducting methereological simulations. In this brief post, I will reproduce its experiment using a basic numpy simulation to show the origin of its incredible discovery.

In this first post I’ll simply be discussing some basic aspects of my field of study, which lies at the intersection of Dynamical Systems (DS) and Machine Learning. The scope of this post (and presumably, the following ones) is two-fold: on one hand, it will be used to develop the notation that I’m going to use through the blog posts. On the other hand, it’ll be nice way to familiarize with markdown and the blog format, learning the technical aspects and improving scientific writing.