Time Is Just Memory

There is a quiet trick at the center of being alive, and almost no one looks at it directly. The trick is that the thing you call "time" is not out there. It is in here. What you experience as a flowing river that you are floating down is, on closer inspection, a particular kind of file system, running inside you, that compares the present moment to recorded versions of earlier moments and reports the difference as duration.

Without memory, there is no time. There is only this. And "this" does not have a length.

If you actually pay attention to the present moment, with no reference to before or after, you will notice something strange. The present has no duration at all [1]. It is not short. It is not long. It is just here. The sense that any time has "passed" requires holding a previous version of "here" alongside the current one and noticing they are different. That holding is memory. Without it, the sense of passage cannot form.

What is actually happening

Pull the mechanism apart. At any given moment, your brain is doing three things at once. It is sampling the world right now. It is holding short term traces of the world from a few seconds ago. And it is comparing those traces to what is happening now. The comparison generates a feeling, and the feeling is what you call "time passing."

This is not a metaphor. The neural machinery that produces the felt sense of duration is largely the same machinery that produces episodic memory [4, 5]. People with severe damage to that machinery do not just lose their memories. They lose their sense of time. For them, the present is not a moment in a flow. It is the entire situation. There is no felt before, and therefore no felt now to compare it to, and therefore no felt motion of one into the other. They are not sad about it, because being sad about it would also require a memory of having had it.

If time were a property of the universe, brain damage should not be able to take it away. It can. So time is not a property of the universe. It is a product of a particular kind of biological recording system, in a particular kind of brain.

The universe is not flowing

The strangest part is that physics has been quietly saying this for a hundred years and almost no one outside the field really hears it. In the equations that describe the universe at the deepest level we currently understand, there is no built in flow of time [2, 3]. The equations are time symmetric. Past and future are not distinguished. There is no "now" marked anywhere in the math. The flow is added by the observer. You. Your machinery.

What there is, in the universe, is change. Things happen. States become other states. But "happening" is just one configuration of stuff being followed by another configuration of stuff. The followed-by relationship is real. The river is not. The river is what the followed-by relationship feels like to a system that records its own history.

Why the illusion is so total

If time is not really flowing, why does it feel so completely like it is. Why is the experience of duration so vivid that everyone in human history has assumed it was the most basic feature of reality.

Two reasons. First, the system that produces the feeling of time is the same system that you are doing the noticing with. You cannot turn it off and check what is left, because the checking also runs on it. Asking your time sense whether time is real is like asking your eyes whether seeing is happening. The instrument cannot be neutral about the thing it is the instrument for.

Second, the system is extremely well optimized. Memory and prediction are how a body keeps itself alive. Anything that interfered with the felt sense of "before, now, and after" would be selected against very fast. Evolution did not care whether the felt sense was metaphysically accurate. It cared that it worked. So the illusion is locked in at a level much deeper than reasoning, the same way the illusion that the earth is flat is locked in at the level of vision.

What the present actually is

Here is the part that is hard to hold on to. The present is not a slice of time. The present is the only thing there is. Everything you think of as "past" exists only as a current state of your brain right now. Everything you think of as "future" exists only as a current set of predictions in your brain right now. Both the past and the future are present tense activities. They are happening now, because nothing is happening then. Then is not where things happen. Then is a label your memory attaches to certain present configurations to distinguish them from other present configurations.

This sounds like wordplay until you actually try to find a moment that is not now. You cannot. Every memory you retrieve is retrieved now. Every plan you make is made now. Every regret is felt now. Every anticipation is felt now. The "now" you are inside is the only address any experience has ever had. The other addresses are just labels on the things stored at this one.

What changes when you see this

A surprising amount of suffering quietly stops making sense. You cannot really be afraid of the future, because there is no future, only a present prediction of one. You cannot really be wounded by the past, because there is no past, only a present memory of one. The suffering is real. It is just always present tense. It is always now. The "when" of it was the part that gave it weight, and the when was an artifact of the recording system.

You also stop being so sure that you are the same person you were yesterday. The continuity you feel between yesterday's you and today's you is a memory operation, performed today. There is no thread reaching backward through time. There is just a current sense, generated now, that there is a thread. You are not less real for this. You are differently real. The thing that is real is the present configuration, which includes a feeling of being someone with a history. The history is the feeling. The feeling is the thing.

The point

Time is not what the universe is made of. Change is. Time is what happens when a system that records change asks its own recordings what just happened. The answer feels like flow because the asking is constant. Stop the asking, and the flow stops with it. There is no river. There is only this, and a memory of this, and the felt difference between them, and the felt difference is the entire river.

This is not depressing. It is the opposite. If time is just memory, then the only place anything has ever happened is here. Which means here is much, much fuller than you were treating it as. The whole show is on this stage. The before and after were always props.

Sources

1. Augustine. Confessions, Book XI. The original argument that past and future exist only as present states of mind.
2. Barbour, J. (1999). The End of Time. Oxford University Press. The strongest contemporary case that time is not fundamental.
3. Rovelli, C. (2018). The Order of Time. Riverhead. Accessible physics-side argument that time as we feel it is not a feature of the equations.
4. Wittmann, M. (2016). Felt Time: The Psychology of How We Perceive Time. MIT Press. On how time perception arises from memory and bodily processes.
5. Tulving, E. (2002). "Episodic memory: from mind to brain." Annual Review of Psychology 53: 1-25. On the memory system that produces the felt sense of personal time.