I mean, I honestly can't see the Grey College losing here. Their base quite literally doesn't exist, can open new entrances in weird places to resupply, and is ridiculously hard to even detect (let alone access).
You try besieging a group that you can't even find, and which can literally Teleport supplies in if you somehow counter their whole "invisible flying agents" thing.
I also can't see the Grey College actually winning a fight against the entire Empire, but that's not really the point 😓
Other colleges seem to be less well-defended than the GC (if only because they have a physical location that can be shelled back to the stone age) but they often have whole traditions focused around hiding out in the wilderness + secondary campuses. They'd suffer, but I'd expect several of them to survive in some form.
And of course right now they could probably emigrate to K8P, because the Greys have been doing some sword-Diplomacy