This line highlights the core difference between panpsychists and their critics.
Personally, I think panpsychism is an elegant, simple, and natural solution to the Hard Problem. It strikes me as having a strong prior probability, i.e. if you were describing a hypothetical universe to me, I'd put the odds of "everything in the universe is capable of perception" at something like 50/50.
Critics seem to think this is bizarre, and put it in the bucket of "extraordinary claims require extraordinary evidence". I understand why, but I disagree. I think this is yet another example of anthropocentric thought, which has continually fallen to more inclusive models (e.g. heliocentrism and evolution)
It does work though, which makes it clear that consciousness is a property produced by a mechanism in the brain.
Pansychism says that everything manifests qualia, not that you can’t change what kind of qualia with drugs — obviously you can, as anyone who has taken recreational ones knows. You can also change it with a bullet, though that doesn’t revert after.
The idea is that there's always "something that it's like to be X", where X can equally be a conscious human, an anesthetized human, a dead human, or an electron.
On the other hand, brains certainly do seem to function like qualia detectors (as evidenced by everyone talking about qualia). So, who knows, maybe there's something further to be explored there; maybe it's related to the combination problem (https://en.wikipedia.org/wiki/Panpsychism#Combination_proble...).
The idea that this makes it unviable is called Logical Positivism. Personally, I think Logical Positivism is short-sighted and uncomfortable with ambiguity. But that's more of an aesthetic preference than a scientific assertion.
However, I was not convinced by the material argument, which seems just as magical as any other explanation, as there comes a moment when we have to believe “then something magic happens and the creatures suddenly have real awareness of their surroundings.” Having read the book, which denounced pan-psychism, I actually found myself more favorably inclined to pan-psychism , since it clearly is no more magical then any other explanation.