This comes down to action economy shenanigans. It requires an AoO (for spell casters, this is practically never going to happen as there's no chance one would trigger for Viserys like it might for someone like Richard) or a readied action. It's almost never worth waiting for an enemy to cast a spell to begin with, since you're just... waiting for it to happen, doing nothing else, and you can cast tons of spells to shut down an enemy trying to do various things via Celerity, or counter spelling, or focus on taking down spell effects already running with a Chain Dispel, and so on.