Which most of us neuroscientists hated because a neural network is a biological network. […] Conflating the term with research on actual neural networks.
Yeah that’s fair, co-opting the term in computing was bound to overtake its original definition, but it doesn’t feel fair to blame that on the computer scientists that were trying to strengthen the nodes of the model to mimic how neural connections can be strengthened and weakened. (I’m a software engineer, not a neuroscientist, so I am not trying to explain neuroscience to a neuroscientist.)
mostly because they called it “neural networks” which sounded super cool.
To be fair… it does sound super cool.
It boogled the mind how anyone could believe a prediction model could have consciousness.
I promise you the computer scientists studying it never thought it could have consciousness. Lay-people, and a capitalist society trying to turn every technology into profit thought it could have consciousness. That doesn’t take AI, though. See, for example, the Chinese Room. From Wikipedia, emphasis mine, “[…] The machine does this so perfectly that no one can tell that they are communicating with a machine and not a hidden Chinese speaker.”
Also, though it is from a science fiction author Arthur C. Clarke’s third law, “Any sufficiently advanced technology is indistinguishable from magic.” applies here as well. Outside of proper science perception is everything.
To a lay-person an AI Chatbot feels as though it has consciousness, the very difficulty with which online forums have in telling AI slop comments from real people is evidence to how well an LLM has modeled language such that it can be so easily mistaken for intelligence.
There is no understanding. No thinking. No ability to understand context.
We start to diverge into the philosophical here, but these can be argued. I won’t try to have the argument here, because god knows the Internet has seen enough of that philosophical banter already. I would just like to point out that the problem of context specifically was one that artificial neural networks with convolutional filters sought to address. Image recognition originally lacked the ability to process images in a way that took the whole image into account. Convolutions broke up windows of pixels into discreet parameters, and multiple layers in “deep” (absurdly-numbered layer-count) neural networks could do heuristics on the windows, then repeat the process to get heuristics on larger and larger convolutions until the whole network accurately predicted an image of a particular size. It’s not hard to see how this could be called “understanding context” in the case of pixels. If, then, it can be done with pixels why not other concepts?
We use heuristics
Heuristics are about a “close enough” approximation for a solution. Artificial neural networks are exactly this. It is a long-running problem with artificial neural networks that overfitting the model leads to bad predictions because being more loose about training the network results in better heuristics.
Which further feed emotional salience (attention). A cycling. That does not occur in computers.
The loop you’re talking about sounds awfully similar to the way artificial neural networks are trained in a loop. Not exactly the same because it is artificial, but I can’t in good conscious not draw that parallel.
You use the word “emotion” a lot. I would think that a neuroscientist would be first in line to point out how poorly understood emotions are in the human brain.
A lot of the tail end there is about the complexity of human emotion, but a great deal was about the feedback loop of emotion.
I think something you might be missing about the core difference between artificial and biological neural networks is that one is analogue and the other is digital. Digital systems must by their nature be discreet things. CPUs process instructions one at a time. Modern computers are so fast we of course feel like they multitask but they don’t. Not in the way an analogue system does like in biology. You can’t both make predictions off of an artificial neural network, and simultaneously calculate the backpropogation of that same network. One of them has to happen first, and the other has to happen second, at the very least. You’re right that it’ll never be exactly like a biological system because of this. An analogue computer with bi-directional impulses that more closely matched biology might, though. Analogue computers aren’t really a thing anymore, they have a whole ecosystem of issues themselves.
The human nervous system is fast. Blindingly fast. However computers are faster. For example videos can be displayed faster than neurons can even process a video frame. We’ve literally hit the limit of human frames-per-second fidelity.
So if you will, computers don’t need to be analogue. They can just be so overwhelmingly fast at their own imitation loop of input and output that biological analogue systems can’t notice a difference.
Like I said though the subject in any direction quickly devolves into philosophy, which I’m not going to touch.
I’m familiar with Chinese room and yes that’s exactly what I was trying to infer with my example of how a video looks like a person. Acts like a person. But is not a person. I didn’t want to go into the Chinese room experiment but that was what I was thinking of.
The heuristics that humans use are not really like the probability statistics that learning models use. The models use probability cut offs. We use incredibly error prone shortcuts. They aren’t really “estimates” in the statistical way. They are biases in attention and reasoning.
I enjoyed your speculating about the use of analog for processing closer to real humans vs virtual.
I think you are partially correct because it’s closer to biology. As you said. But it also can’t change. Which is not like biology. 🤷
Humans don’t actually compute real probability.
In fact humans are so poor at true statistical probability, due to our biases and heuristics, it’s actually amazing that any human was able to break free from that hard wired method and discover the true mathematical way of calculating probability.
It quite literally goes against human nature. By which I mean brains are not designed to deal with probability that way.
We actually have trouble with truly understanding anything besides. “Very likely and basically assured” and “very unlikely and basically like no chance”.
We round percentages to one of those two categories when we think about them. (I’m simplifying but you get what I’m saying,)
This is why people constantly complain that weather predictions are wrong. 70% chance of rain means it certainly will rain. And when it doesn’t. We feel lied to.
I mentioned emotion and you are 100% correct that it’s a tricky concept in neuroscience (you actually seem pretty educated about this topic).
It is ill defined. However. The more specific emotions I refer to are approach/avoidance. And their ability to attract attention or discourage it.
To clarify, Both approach and avoidance emotions can attract attention.
Emotional salience : definition. = Grabs attention at an emotional level , becomes interesting. Either because you like it or you don’t like it (I’m simplifying)
Stimuli with neutral emotional salience will not grab attention and be ignored and will not affect learning to the same degree as something that is emotionally salient.
Your personal priorities will feed into this as well. Dependent on mood and whatever else you have going on in your life. Plus personality.
It’s always changing.
LLMs have set directions that don’t fluctuate.
The loop I describe is not the same as an algorithm loop.
An algorithm loop feeds data and cycles through to get to the desired outcome.
Sort of like those algorithms for rubric cube solutions (idk if you know what I’m talking about).
You do the steps enough reiterations and you will solve the puzzle.
That’s not the same as altering and evolving the entire system constantly. It never goes back to how it was before. Its never stable.
Every new cognitive event starts differently than the last because it is influenced by the preceding events. In neuroscience we call this priming.
It literally changes the chances of a neuron firing again. And so the system is not running the same program over and over. It’s running an updated program on updated hardware. Every single iteration.
That’s the process of learning that is not able to be separated from the process of experience nor decision making; At any level.
Within or beyond awareness.
May I ask what your expertise area is in? Are you a computer scientist?
You do seem to know a bit more about neuroscience than the average person. I also rarely meet anyone who has heard of the Chinese room thought experiment.
Also I agree we are getting into philosophical areas.
But the Chinese room argument is very flawed, at least if we assume that consciousness does in fact arise in the brain and not through some supernatural phenomenon.
Suppose we know the exact algorithm that gives rise to consciousness. The Chinese room argument states that if a person carries out the algorithm by hand, the person does not become consciousness. Checkmate atheists.
This is flawed because it is not the axons, synapses, neurotransmitters or voltage potentials within the brain that are conscious. Instead, it appears that consciousness arises when these computations are carried out in concert. Thus consciousness is not a physical object itself, it is an evolving pattern resulting from the continuous looping of the algorithm.
Furthermore, consciousness and intelligence are not the same thing. Intelligence is the ability to make predictions, even if it’s just a single-neuron on/off gate connected to a single sensory cell. Consciousness is likely the experience of being able to make predictions about our own behavior, a meta-intelligence resulting from an abundance of neurons and interconnections. There is likely no clear cutoff boundary of neural complexity where consciousness arises, below which no consciousness can exist. But it’s probably useful to imagine such a boundary.
Basically, what if thinking creatures are simply auto-correct on steroids (as Linus Tordvals put it). What’s unreasonable about treating intelligence as a matter of statistics, especially given that it’s such a powerful tool to model every other aspect of our universe?
Well that’s not my interpretation. Consciousness arises from understanding. True understanding. Not stimulus in- behavior out.
Consciousness is not a simple exchange or matching task. Which is what the Chinese room illustrates.
There is more to it.
The Chinese room is modern LLMs.
Human brains are altered by every stimulus. Physically they are constantly changing at the neuron level.
The way inhibitory neurons work … It does not work in a way that (at present) can be predicted very accurately beyond a single or small number of neurons.
As I like to say. Every moment the brain is updated biologically. Biological changes. Connections weakened, strengthened, created, destroyed.
This happens constantly.
You can’t use statistics to predict these kind of events.
Although the neuro definition of “consciousness” is debated. It is generally considered “awareness”.
It’s something that is a product of many processes in the brain.
And we haven’t even touched on brain occillations and how they impact cognitive functions. Brain occillations are heavily tied to consciousness/awareness. They synch up processes and regulate frequency of neuron firing.
They gatekeep stimuli effects as well.
The brain is so unbelievably complicated. Research on ERPs are the best we have for predicting some specific brain spikes of cognitive activities.
You may find the research on it to be less than where you think it is.
Neuroscience knowledge is far below what most people think it is at (I blame click bait articles).
However it’s still an interesting area so here is the wiki.
The thing is that we do not really know what consciousness is or how it arises. So I think we need to be careful when we decide how it does not arise, or what “true” consciousness is.
I think it is likely that consciousness emerges on the aggregate macro-level from processes that are simple on the micro-level. Such phenomena do lend themselves to be described or indeed understood best with statistics.
In particular, I think it’s a mistake to assume that consciousness can only arise by mimicking the exact functioning of a human brain. (Noting here that there is debate on whether other animals can be considered conscious with no clear cutoff or criteria). However I think that the criteria that you mentioned (continuous rewiring of neurons, oscillations of activity et.c.) could easily be added to an ML model, and I think those exact things will be added to ML models down the line.
-I think it is likely that consciousness emerges on the aggregate macro-level from processes that are simple on the micro-level. Such phenomena do lend themselves to be described or indeed understood best with statistics.
How’s that?
If it’s not defined and we have no way to measure the activity that would allow us to even create a model, how could statistics be used?
I don’t forsee the tech needed to record an entire brain’s electrical information including it’s constantly changing state, every neuron, and every connection, (all architecture), including it’s myelination, while the human is alive - happening any time in the foreseeable future.
Not only that. Then interpreting any statistical prediction in a real output of behavior or “experience”.
Not everything can be measured. An inability to measure something does not mean it’s not real and rooted in the material world.
We know enough to know that consciousness/awareness or any other definition you want to give it, and mental processing occurs in the brain.
We can’t measure it at a whole brain level . But we know that’s where it’s happening and that it’s a product of biological phenomena.
I’m sorry if this sounds rude but LLMs are not self updating systems. They are nothing like human processing. They don’t loop in the ways biology does. They don’t constantly change. They have fixed algorithms.
They can’t be. When parameters are too open with prediction models they just become nonsense.
This has been demonstrated many times.
Also consciousness in animals is supported by lots of research in neuroscience.
Even in the tiny flatworm.
It’s not unique in humans. And it appears to exist in animals.
The complexity would obviously vary based on biological limitations. But it appears to be, at least in part, a product of sensory processing.
Which is often why you find it being referred to as ,“awareness” or “self awareness”.
Awareness of self is necessary for any organism to distinguish itself from its environment and act accordingly. To be able to predict outcomes and act accordingly. To even know what can be consumed or mated with. Awareness of environment creates approach and avoidance emotions/behaviors.
Of course this is one of many theories regarding what consciousness is. But this one seems like a pretty solid description and explains why it would exist in animals.
Yeah that’s fair, co-opting the term in computing was bound to overtake its original definition, but it doesn’t feel fair to blame that on the computer scientists that were trying to strengthen the nodes of the model to mimic how neural connections can be strengthened and weakened. (I’m a software engineer, not a neuroscientist, so I am not trying to explain neuroscience to a neuroscientist.)
To be fair… it does sound super cool.
I promise you the computer scientists studying it never thought it could have consciousness. Lay-people, and a capitalist society trying to turn every technology into profit thought it could have consciousness. That doesn’t take AI, though. See, for example, the Chinese Room. From Wikipedia, emphasis mine, “[…] The machine does this so perfectly that no one can tell that they are communicating with a machine and not a hidden Chinese speaker.” Also, though it is from a science fiction author Arthur C. Clarke’s third law, “Any sufficiently advanced technology is indistinguishable from magic.” applies here as well. Outside of proper science perception is everything.
To a lay-person an AI Chatbot feels as though it has consciousness, the very difficulty with which online forums have in telling AI slop comments from real people is evidence to how well an LLM has modeled language such that it can be so easily mistaken for intelligence.
We start to diverge into the philosophical here, but these can be argued. I won’t try to have the argument here, because god knows the Internet has seen enough of that philosophical banter already. I would just like to point out that the problem of context specifically was one that artificial neural networks with convolutional filters sought to address. Image recognition originally lacked the ability to process images in a way that took the whole image into account. Convolutions broke up windows of pixels into discreet parameters, and multiple layers in “deep” (absurdly-numbered layer-count) neural networks could do heuristics on the windows, then repeat the process to get heuristics on larger and larger convolutions until the whole network accurately predicted an image of a particular size. It’s not hard to see how this could be called “understanding context” in the case of pixels. If, then, it can be done with pixels why not other concepts?
Heuristics are about a “close enough” approximation for a solution. Artificial neural networks are exactly this. It is a long-running problem with artificial neural networks that overfitting the model leads to bad predictions because being more loose about training the network results in better heuristics.
The loop you’re talking about sounds awfully similar to the way artificial neural networks are trained in a loop. Not exactly the same because it is artificial, but I can’t in good conscious not draw that parallel.
You use the word “emotion” a lot. I would think that a neuroscientist would be first in line to point out how poorly understood emotions are in the human brain.
A lot of the tail end there is about the complexity of human emotion, but a great deal was about the feedback loop of emotion.
I think something you might be missing about the core difference between artificial and biological neural networks is that one is analogue and the other is digital. Digital systems must by their nature be discreet things. CPUs process instructions one at a time. Modern computers are so fast we of course feel like they multitask but they don’t. Not in the way an analogue system does like in biology. You can’t both make predictions off of an artificial neural network, and simultaneously calculate the backpropogation of that same network. One of them has to happen first, and the other has to happen second, at the very least. You’re right that it’ll never be exactly like a biological system because of this. An analogue computer with bi-directional impulses that more closely matched biology might, though. Analogue computers aren’t really a thing anymore, they have a whole ecosystem of issues themselves.
The human nervous system is fast. Blindingly fast. However computers are faster. For example videos can be displayed faster than neurons can even process a video frame. We’ve literally hit the limit of human frames-per-second fidelity.
So if you will, computers don’t need to be analogue. They can just be so overwhelmingly fast at their own imitation loop of input and output that biological analogue systems can’t notice a difference.
Like I said though the subject in any direction quickly devolves into philosophy, which I’m not going to touch.
I’m familiar with Chinese room and yes that’s exactly what I was trying to infer with my example of how a video looks like a person. Acts like a person. But is not a person. I didn’t want to go into the Chinese room experiment but that was what I was thinking of.
The heuristics that humans use are not really like the probability statistics that learning models use. The models use probability cut offs. We use incredibly error prone shortcuts. They aren’t really “estimates” in the statistical way. They are biases in attention and reasoning.
I enjoyed your speculating about the use of analog for processing closer to real humans vs virtual.
I think you are partially correct because it’s closer to biology. As you said. But it also can’t change. Which is not like biology. 🤷
Humans don’t actually compute real probability. In fact humans are so poor at true statistical probability, due to our biases and heuristics, it’s actually amazing that any human was able to break free from that hard wired method and discover the true mathematical way of calculating probability.
It quite literally goes against human nature. By which I mean brains are not designed to deal with probability that way.
We actually have trouble with truly understanding anything besides. “Very likely and basically assured” and “very unlikely and basically like no chance”.
We round percentages to one of those two categories when we think about them. (I’m simplifying but you get what I’m saying,)
This is why people constantly complain that weather predictions are wrong. 70% chance of rain means it certainly will rain. And when it doesn’t. We feel lied to.
I mentioned emotion and you are 100% correct that it’s a tricky concept in neuroscience (you actually seem pretty educated about this topic).
It is ill defined. However. The more specific emotions I refer to are approach/avoidance. And their ability to attract attention or discourage it.
To clarify, Both approach and avoidance emotions can attract attention.
Emotional salience : definition. = Grabs attention at an emotional level , becomes interesting. Either because you like it or you don’t like it (I’m simplifying)
Stimuli with neutral emotional salience will not grab attention and be ignored and will not affect learning to the same degree as something that is emotionally salient.
Your personal priorities will feed into this as well. Dependent on mood and whatever else you have going on in your life. Plus personality.
It’s always changing.
LLMs have set directions that don’t fluctuate.
The loop I describe is not the same as an algorithm loop.
An algorithm loop feeds data and cycles through to get to the desired outcome.
Sort of like those algorithms for rubric cube solutions (idk if you know what I’m talking about).
You do the steps enough reiterations and you will solve the puzzle.
That’s not the same as altering and evolving the entire system constantly. It never goes back to how it was before. Its never stable.
Every new cognitive event starts differently than the last because it is influenced by the preceding events. In neuroscience we call this priming.
It literally changes the chances of a neuron firing again. And so the system is not running the same program over and over. It’s running an updated program on updated hardware. Every single iteration.
That’s the process of learning that is not able to be separated from the process of experience nor decision making; At any level. Within or beyond awareness.
May I ask what your expertise area is in? Are you a computer scientist?
You do seem to know a bit more about neuroscience than the average person. I also rarely meet anyone who has heard of the Chinese room thought experiment.
Also I agree we are getting into philosophical areas.
But the Chinese room argument is very flawed, at least if we assume that consciousness does in fact arise in the brain and not through some supernatural phenomenon.
Suppose we know the exact algorithm that gives rise to consciousness. The Chinese room argument states that if a person carries out the algorithm by hand, the person does not become consciousness. Checkmate atheists.
This is flawed because it is not the axons, synapses, neurotransmitters or voltage potentials within the brain that are conscious. Instead, it appears that consciousness arises when these computations are carried out in concert. Thus consciousness is not a physical object itself, it is an evolving pattern resulting from the continuous looping of the algorithm.
Furthermore, consciousness and intelligence are not the same thing. Intelligence is the ability to make predictions, even if it’s just a single-neuron on/off gate connected to a single sensory cell. Consciousness is likely the experience of being able to make predictions about our own behavior, a meta-intelligence resulting from an abundance of neurons and interconnections. There is likely no clear cutoff boundary of neural complexity where consciousness arises, below which no consciousness can exist. But it’s probably useful to imagine such a boundary.
Basically, what if thinking creatures are simply auto-correct on steroids (as Linus Tordvals put it). What’s unreasonable about treating intelligence as a matter of statistics, especially given that it’s such a powerful tool to model every other aspect of our universe?
Well that’s not my interpretation. Consciousness arises from understanding. True understanding. Not stimulus in- behavior out.
Consciousness is not a simple exchange or matching task. Which is what the Chinese room illustrates.
There is more to it.
The Chinese room is modern LLMs.
Human brains are altered by every stimulus. Physically they are constantly changing at the neuron level.
The way inhibitory neurons work … It does not work in a way that (at present) can be predicted very accurately beyond a single or small number of neurons.
As I like to say. Every moment the brain is updated biologically. Biological changes. Connections weakened, strengthened, created, destroyed.
This happens constantly.
You can’t use statistics to predict these kind of events.
Although the neuro definition of “consciousness” is debated. It is generally considered “awareness”.
It’s something that is a product of many processes in the brain.
And we haven’t even touched on brain occillations and how they impact cognitive functions. Brain occillations are heavily tied to consciousness/awareness. They synch up processes and regulate frequency of neuron firing.
They gatekeep stimuli effects as well.
The brain is so unbelievably complicated. Research on ERPs are the best we have for predicting some specific brain spikes of cognitive activities.
You may find the research on it to be less than where you think it is.
Neuroscience knowledge is far below what most people think it is at (I blame click bait articles).
However it’s still an interesting area so here is the wiki.
https://en.wikipedia.org/wiki/Event-related_potential
The thing is that we do not really know what consciousness is or how it arises. So I think we need to be careful when we decide how it does not arise, or what “true” consciousness is.
I think it is likely that consciousness emerges on the aggregate macro-level from processes that are simple on the micro-level. Such phenomena do lend themselves to be described or indeed understood best with statistics.
In particular, I think it’s a mistake to assume that consciousness can only arise by mimicking the exact functioning of a human brain. (Noting here that there is debate on whether other animals can be considered conscious with no clear cutoff or criteria). However I think that the criteria that you mentioned (continuous rewiring of neurons, oscillations of activity et.c.) could easily be added to an ML model, and I think those exact things will be added to ML models down the line.
-I think it is likely that consciousness emerges on the aggregate macro-level from processes that are simple on the micro-level. Such phenomena do lend themselves to be described or indeed understood best with statistics.
How’s that?
If it’s not defined and we have no way to measure the activity that would allow us to even create a model, how could statistics be used?
I don’t forsee the tech needed to record an entire brain’s electrical information including it’s constantly changing state, every neuron, and every connection, (all architecture), including it’s myelination, while the human is alive - happening any time in the foreseeable future.
Not only that. Then interpreting any statistical prediction in a real output of behavior or “experience”.
Not everything can be measured. An inability to measure something does not mean it’s not real and rooted in the material world.
We know enough to know that consciousness/awareness or any other definition you want to give it, and mental processing occurs in the brain.
We can’t measure it at a whole brain level . But we know that’s where it’s happening and that it’s a product of biological phenomena.
I’m sorry if this sounds rude but LLMs are not self updating systems. They are nothing like human processing. They don’t loop in the ways biology does. They don’t constantly change. They have fixed algorithms.
They can’t be. When parameters are too open with prediction models they just become nonsense.
This has been demonstrated many times.
Also consciousness in animals is supported by lots of research in neuroscience.
Even in the tiny flatworm.
It’s not unique in humans. And it appears to exist in animals.
The complexity would obviously vary based on biological limitations. But it appears to be, at least in part, a product of sensory processing.
Which is often why you find it being referred to as ,“awareness” or “self awareness”.
Awareness of self is necessary for any organism to distinguish itself from its environment and act accordingly. To be able to predict outcomes and act accordingly. To even know what can be consumed or mated with. Awareness of environment creates approach and avoidance emotions/behaviors.
Of course this is one of many theories regarding what consciousness is. But this one seems like a pretty solid description and explains why it would exist in animals.