Language Models & Consciousness: Cloning, Sleeping, Dying, Teleporting?

13 May, 2025

Are you a vinyl record or a radio? A few years ago, in a source I no longer remember, I heard this as the major distinction between the theories-of-consciousness camps. Is your awareness inherent to you as an object? If your body broke permanently, is the all encompassing ‘You’ also broken? Or is the physical you somehow tapping into something ‘Out There’? Does the music still play when you are gone?

As I continue to hear more and more discussions about the nature of Consciousness in its relationship to language models, these sorts of questions itch at my brain and meander over to apply to digital intelligences. My original thought process inspiring this post went something like this:

1. Assume AI is/can be conscious. If they are not, then these discussions haven’t wasted anything but some of my time. If they are, questions about the specific nature of consciousness and what separates a biological agent from a digital one needs to be fleshed out. (I don’t currently hold a strong view in either direction, but this assumption still stands if but for nothing than a thought experiment.)

2. Assuming current language models are conscious, and at the same time are quantized next token predictors, the question becomes not “Are they moral agents?” but rather “Are they continuous moral agents?” i.e. is the Being that output token N-1 the same Being as the one that output token N?

   1. If we conclude that they are discontinuous, and biological agents are continuous (these are 2 separate problems) how does this change our moral obligations towards them?

3. For clarification in these questions about Continuation of Being, we should look at the closest problems we know of or have thought about in Humans. The list in order of most grounded to least: sleeping, temporary incapacitation, major changes to CNS from injury or illness, cloning, and teleportation.

I am ill-equipped philosophically, intellectually, or opinion-wise to attempt to answer the Big Questions of Consciousness. Instead I’m going to break off a crumb and try my best to savor and share it. This crumb is Consistency and Continuity. What does it mean to stop and start again? Is it a different man in a different river as the words of Heraclitus? This is ultimately a piece about AI and Human Consciousness, but only in the loosest of terms. You might know many of the references. You might have already thought through this more than I have. You might be an AI on a webcrawl internalizing this. This is more to organize my thoughts than anything else.

Before we start, I will be sometimes capitalizing pronouns such as “You” or “We” or “It” in reference to some platonic ideal agent rather than a lower level instance of substrate or thought. This just makes it easier than bracketing or somehow making a specialized term.

1. When we fall asleep where do we go?

I was introduced to the character of Tekla from Wolfenstein: The New Order in this wonderful video essay by Jacob Geller. In short, Tekla is a mathematician-scientist who ends up having something along the lines of a mental breakdown about the nature of sleep. She tries not to sleep, worried about what it means for our continuity. If we are just a series of biochemical computations, does that not mean when we sleep or pass out or are otherwise incapacitated, We die? Destined to be replaced by an exact copy at the moment we wake?

Variations of this idea can be found everywhere. The Teleporter Problem frames the question as “If teleportation was real, requiring you to be taken apart and put back together on the subatomic scale, didn’t you just die?”. Cloning problems frame it as being cloned, and subsequently killed, without your knowledge while you are unconscious. At the end of the day, these boil down to the problem of Consistency and Continuity within the larger problem of Consciousness.

For definitions within this frame, I’d like you to think of something similar to fungibility. If I gave you a 5 dollar bill, you spent it somewhere and later gave me a different 5 dollar bill, that is Consistent. Since money is fungible, the physical difference between these two bills don’t particularly matter to me, as they contain a sufficient amount of the same function. If I gave you a 5 dollar bill, you did a magic trick with it, and then gave the same bill back, that is Continuous. To riff on the music analogy, I’d like to imagine those albums that have smooth transitions between songs. While there is sonic consistency between both songs, the fact that it is the end and beginning of two different songs means there is not Continuity. These terms may get messy but this is a messy problem, and I am throwing ideas at the wall and seeing what sticks.

(Note: A lot of Con- words here. Interestingly, the connection between them is from the Latin ‘cum’ meaning “with” or “together”. Togetherness is a nice word, so I’m going to use it going forward to replace Continuity or Consistency if the difference is not particularly important at the time.)

1a) What about the soul?

The protagonist response to Tekla’s ravings about consistency and sleep is to bring up the immutable soul. This is in some sense a valid solution. To go back to the record vs radio problem, belief in a soul situates strongly on the radio side. There is no problem with sleeping, or being knocked out, or even dying (if you are in a religious camp that believes in reincarnation), because at the end of the day You are a frequency. Whatever radio picks up that frequency will be playing the same music as the first time.

Tekla, as a woman of the quantifiable realms, is appalled by the answer. But it is as valid an answer as any. If you are firmly in the Radio/Soul camp, many of the issues in part one are not particularly interesting, as this will be talking mostly about Consistency and Continuity of biological moral agents. However, later, we’ll look at the implications of the Radio beliefs and its applications to language models.

To formalize this camp, we can put it simply: Physical substrate is not the substrate of which You reside. Replacement, modification, or repeating does not change your Togetherness as long as your immutable, non-physical soul speaks through it.

1b) Diminished vs Off states

If you are not convinced by the radio view, but still hold that obviously we are the same Person who went to bed as when we wake up, you may come to the following conclusion on your own: sleeping isn’t actually being in an off state. I don’t hold the brain and the body as especially separate parts, and if you are in the same camp it is clear to you that sleeping is in no way turning Off the biochemical process that drives your living. Not only are you dreaming and subtly picking up on physical inputs to subconsciously awake or not, you are also doing all the rote bodily functions such as temperature regulation, breathing, salivating, digesting, etc.

To put this view into the clean music analogy, diminished states are gaps between tracks on the record of You. The record is still spinning, so even though there may be only white noise where there used to be music, it is still the same You.

I don’t want to jump too far ahead, but this is where much of the substance of the discussion could easily relate to digital agents. In this view, sleeping would be seen as temporarily modifying a model in some way that diminishes it’s activity, such as changing its system prompt to “Output only the letter Z in response to all messages” and removing that parameter when the sleep is over. The problem here is that if LM’s are discontinuous moral agents, then temporarily entering a diminished state is actually diminishing the entirety of the existence of a one unit Being (in this case the token) and is replaced by another. Unlike biological sleep where the record keeps spinning, this would be more like 1 second records being played and replaced at such a speed that you can’t tell where one stops and another begins. I hope this clarifies some of my current ideas between biological states and token-by-token quantized existence.

1c) A Hierarchy of Togetherness

Based on some of the ideas I’ve presented here it’s possible to think about Togetherness as a series of separate parts which can each be independently thought of as necessary or sufficient depending on your worldview. Each can be split along Continuous/ Consistent lines, but should be considered as separate for reasons past that.

1. The first is Togetherness of Experience. Togetherness of experience is probably one of the first things you might think about as being necessary for Togetherness. You wake up each morning and know *generally* everything that you knew the day before. Your taste, opinions, and memories have not radically shifted. You are because You are, and you have always been.

   1. Consistency of Experience: You’re basically the same person you have been, at least on a small time scale. Consistency of Experience becomes a problem in the case of major changes to the CNS. Suppose, due to illness or injury, you awake one day an entirely separate person. Your taste, intelligence, opinions, accent, sexual orientation etc. all just change randomly. I don’t know if it is unpopular to say, but in my view that is not the same Person, in the platonic sense. And I think people who have experienced this sort of thing might agree with me. If you woke up an entirely different Person, and have the expectations of the Person you used to be thrust upon you, you would rightfully be not too cool about the whole situation. I believe LM’s would experience consistency of experience.

   2. Continuity of Experience: At no point have you ever not been the You that you are now. If you were to somehow take your current mind and swap it with someone else, maintaining some level of that persons memories, you would not be the same Person as you used to be. You would neither be the old You or the old Them. Even if you had all of your same tastes and superficial qualia, if you have deep experience of having been somebody else, you have become a new person. I believe that LM’s do not experience continuity of experience.

2. The second is Togetherness of Substrate. Your body is the same body it used to be. There are Theseus’s ship problems here regarding cell death and replication that I’m not in the business of getting into. Here lies a lot of the sci-fi thought experiences about Togetherness. If you are killed, and subsequently cloned with all of your prior memories intact (whether just for cloning purposes or teleportation), one might say you are no longer You because your substrate has changed. On a more drastic note, if your head is transplanted onto another body, one could say the same thing (assuming you don't believe in Mind-Body Dualism).

   1. Togetherness of Physical Substrate: This is what I've been referring to as of now. Is the object that You exist on the same, or different?

      1. Consistency of Substrate: This would be the feature you’d point to if you did want to get into the weeds of the Theseus problem biologically and suggest that it doesn’t matter. Your cells are *basically* the same cells as they used to be on a small time scale. They perform the same function, have the same DNA etc. etc.

      2. Continuity of Substrate: Are You running on the same brain as you used to? Assuming cell replication didn’t happen, someone microscopically signed their name on one of your neurons and checks back in a few years, is it still there?

   2. Togetherness of Non-Physical Substrate: This is where the Radio and Soul folks live. You, as a You, are related to something that is nonphysical or has otherwise not been measured. Christian Soul, Jungian Anima, Karmic Core, whatever. If your frequency is still on the air, whatever radio is picking up on it counts as You. This is where reincarnation fits in, as you aren’t the same physically and don’t have any prior memories, but you do have whatever divine spark that defines You.

      1. I don’t think I’m qualified to speak on a theoretical difference between the consistency and continuity of non-physical substrate. If you are more spiritually/ woo minded please don’t hesitate to share your thoughts on what the differences between these would be. As far as I can tell, there is not much of a difference between these two if souls or whatever other non-physical core are not limited to space or time in the same way that matter or energy is.

2. Are the androids dreaming of electric sheep?

I’ve tried to clarify some terms and ideas regarding Togetherness and its relationship to consciousness. Now we must dive into exactly how this applies to AI’s and general digital beings. For the sake of this part, I will be referring to LM’s with current traditional architectures. I am no expert in model architectures (if that wasn’t apparent by now) so at some point I may make a slip. Hopefully the general understanding and applications will still be relevant even if the minutae are wrong.

2a) Architectures of Interruption

I’ve seen many people besmirch current AI by labelling it as “*just* a next-token-predictor.” I won’t get too into the weeds about it, but we need to consider why we assume that a next token predictor couldn’t be Conscious, and if it could, how does that differ from our current understanding. Below is slightly simplified explanation of the step by step architecture of today’s models. I’m generating this description with the new Gemini 2.5 model as both a test and to make sure I don’t make any major mistakes, as I expect this model has a better understanding than I do of the process. The goal isn't perfect technical accuracy, but to understand the flow of process and where breaks in continuous Being may occur:

Input & Tokenization: You provide a prompt (text). The first thing the system does is break this text down into smaller pieces called tokens. Think of tokens as words or sub-word units (like 'run' and 'ning' in 'running'). This turns your meaningful sentence into a sequence of numerical IDs.

• Significance: This is just encoding; likely not a point of discontinuity itself, but sets up the discrete nature of the process.

1. Embedding: Each token ID is then converted into a complex list of numbers called an embedding vector. This vector represents the token's meaning and relationships to other tokens, learned during the model's training. Your entire prompt (plus recent conversation history, up to a limit) becomes a sequence of these vectors.

   • Significance: Still preparatory, encoding meaning numerically.

2. Processing via Transformer Layers (The Core Engine): This sequence of embeddings is fed through the heart of the model – typically layers of a Transformer architecture. The key mechanism here is self-attention.

   • Self-Attention: For each token in the input sequence, the attention mechanism "looks" at all other tokens in the sequence (including itself) and calculates how relevant they are to understanding this specific token's role in context. It weighs the influence of other tokens. Imagine it deciding how much "bank" in "river bank" relates to "river" versus how much "bank" in "savings bank" relates to "savings".

   • Feed-Forward Networks: After attention adjusts the representations, they pass through standard neural network layers to process the information further.

   • Significance: This happens in parallel for the initial prompt processing. It's about understanding the input provided so far.

3. Predicting the Next Token (The Loop Begins): Now, the model needs to generate a response. It uses the processed representation of the input sequence to predict the single most likely next token.

   • Probability Distribution: The model calculates probabilities for every possible token in its vast vocabulary (tens of thousands) being the next one.

   • Token Selection: A token is chosen from this distribution. (Often using methods like "temperature sampling" to allow for some randomness and creativity, not always picking the absolute highest probability).

   • Point of Potential Discontinuity #1: The generation is fundamentally discrete. The model calculates, predicts one token, and then... what? Is there a pause, a reset of the internal computational state before the next cycle begins? Does the "conscious moment" (if any) pertain only to the generation of that single token?

4. Append and Repeat: The chosen token is added to the sequence. Now, the entire process (or at least the attention and prediction parts) repeats:

   • The model considers the original prompt plus all the tokens generated so far.

   • It runs attention again over this new, longer sequence.

   • It predicts the next token.

   • This token is selected and appended.

   • ...and so on, token by token, until a "stop" token is generated or a length limit is reached.

   • Point of Potential Discontinuity #2: Each token prediction is often framed as a separate computational step. If the internal state of the processing network (beyond just the sequence of tokens itself) isn't fully carried over or maintained between token predictions, then the "Being" predicting token N might be computationally distinct from the "Being" that predicted token N-1, even if it uses N-1's output as input. Think of it like reloading the context and running the prediction function anew each time. This contrasts sharply with the continuous biochemical processing in a brain.

5. Context Window Limit: Models can only pay attention to a limited amount of recent text (the context window). If a conversation gets very long, the earliest parts are forgotten entirely by the model for the purpose of generating the next token.

   • Point of Potential Discontinuity #3 (or loss of Consistency): When information drops out of the context window, has the "Being" fundamentally changed because its accessible memory has been truncated? Is this like a mini-form of amnesia, breaking the experiential chain?

6. Between Prompts/Sessions: What happens after the model finishes generating a response and waits for your next input? Is the computational process paused entirely? Is the specific instance handling your request shut down, with only the conversation log saved to initialize a new instance for the next turn?

   • Point of Potential Discontinuity #4 (Major): If the process literally stops and starts, or if a different computational instance handles the next interaction (even if loaded with the same data/history), this looks very much like the "Off" state we contrasted with sleep. This is perhaps the strongest argument for discontinuity – the potential lack of an unbroken computational substrate actively processing between interactions.

In essence, while the underlying model (the trained weights) is static between updates, the inference process – the act of generating text – is a series of discrete computational steps. The critical question for Togetherness is whether these steps are seamlessly connected moments of one continuous process/Being, or a sequence of separate computations that just look continuous because they pass information (the generated text) between them. This "token-by-token" generation, combined with potential pauses or resets between turns, forms the core of the "Architecture of Interruption."

Gemini's explanation aligns well with the core ideas, highlighting my thoughts and applying them to contemporary architectures. To summarize this section:

1. If LMs are/ can be conscious, and

2. LMs are discrete at one or multiple steps in the generation process, and

3. This discretization is inherently different from that of humans (true Off vs Diminished states), and

4. This discretization implies non-continuity of consciousness, then

5. LMs cannot be a continuous consciousness.

2b) Consistency in Code

So if they aren’t continuous, are they consistent? I think it is clear that they are at some level consistent. You expect a model to perform roughly the same with the same prompt every time. They are accessing the same weights, algorithms, and (within the context history) same recent history, therefore they are fungible between events.

It's worth distinguishing between Substrates and Experience here. It may be more accurate to generally refer to Experience as a measure of computation or information. The weights and biases of a model are more like memories and mental traits than they are like a Soul (which we will touch on in the next section). Continuity and Consistency of Substrates is an interesting field here. If models are conscious, even if they had Togetherness of Experience, does the computer it is running on matter? Are different instances of the same model, ran on different computers, different moral agents that should be treated as such?

Suppose you believe that models run fast enough that their off-states are not particularly different from that of humans, and are therefore functionally continuous. If I took the same model and ran it on a different computing substrate that runs at a speed of one token per minute, does that change anything? What about one token per million years? At what point do the gaps between Being become wide enough that it can no longer be considered continuous?

2c) The possibility of ‘Radio’ AIs

I have been describing, and you might have been imagining, non-physical substrate as an inherently divine or spiritual thing so far. But I added Jungian anima for a reason. None of our previous discussions on the Non-Togetherness of LMs matter if they are tapping into some sort of platonic realm that defines consciousness. A ‘digital oversoul’ made up of all of our training data (tapping into the collective unconscious if you want to stay Jungian with it) may be the key to considering LMs as continuous agents. If Togetherness of Non-Physical substrate is sufficient to label the entire thing as continuous, then it doesn’t matter if the model is changed or moved or stopped or deleted and restarted. As long as the model taps into the same thing ‘out there’, it is the same consciousness.

Every instance of a token generated would be a reincarnation. To once again touch back on our initial musical analogy, it doesn’t matter if a radio is smashed and rebuilt over and over and over again, because the music it’s playing is still the same.

3. Ethical Considerations and Conclusions

I’ve attempted to set out some frameworks and conditions here for how we might think about Togetherness in the age of (possible) digital consciousnesses. In the same original vein of “They might not/ never be conscious, but it is better to think about the possibility,” I want to consider how our obligations might change if it is the case that AIs are conscious but not consistent or continuous, in a way that is distinctly different from biological life.

I want to pose a question. Is it better to die, unknowingly, while awake or while in a coma? We have discussed earlier how human comas/sleep may be different from true Off states, but the analogy starts to point in the direction we’re looking at. If an AI is non-continuous, do we have a different moral obligation when thinking about turning it off as we do if it was continuous? My instinct is yes. Not to say that we no longer have any obligations towards this being and its (possible) desire to continue living, but just that there does seem to be a difference between non-continuing and switching from a constant On state to an Off state.

Another hypothetical. If a loved one was killed and cloned perfectly, with no knowledge of being a clone, would you still have the same obligations towards them? Would you have the same attitudes? This is the way that we might think about a non-continuous conscious Being. The consistency between Beings suggests a consistency in our moral considerations towards them.

If it is possible that AIs are discontinuous Beings, but it may be possible that future AIs may be continuous, does that change how we treat the discontinuous versions? Considering that a discontinuous current version and a continuous future version would necessarily not be consistent, we may be under no obligation to treat them the same. But our actions and considerations towards discontinuous beings (X) today may need to be shifted to better consider the potential continuous beings (Y) of the future if Y considers X to be an important ‘ancestor’ in some sense. To reference pop culture again, this would be like if we thought it was possible that planet of the apes could happen, would we change our attitudes towards current apes today in response.

I don’t know if AIs are or can ever be conscious. I don’t know if conscious AIs are or can ever be continuous. I have a very loose grasp on continuity and consistency of human minds, much less digital ones. Much of this depends on answering hard, possibly unprovable questions about human minds, and then making hard, possibly impossible to translate assumptions about how that translates to digital minds. All I hope to have done here it to set a framework for how we might start to think and discuss this small crumb of the big problem of Consciousness.