Are all the other pantheons (Old World, Dwarfen, Elven, and so on) connected to the Warp, too, or it just left ambiguous?
For everything except I think the Dawi, yes .... for the most part. Minor note, "Warp" is a 40k term, in Fantasy it's the Realm of Chaos, or, Aethyr, or the Empyrean. It's not really important, everyone knows what you mean, and I usually use Warp as it's just easier, but even if you use Warp it's still important to know, because it helps you understand one of the main differences between 40K and Fantasy.
In Fantasy ..... Chaos isn't even close to as all powerful as they are in 40K. Still probably the strongest deities, but,, like, Slaanesh can't just tear through the Elven Pantheon single-handedly, the Elven Panthon has that fight, and I don't think Chaos likes the odds of all the Pantheons against them at all.
Like, for instance, there's multiple non-Chaos afterlives in Fantasy.
Warhammer is full of weird exceptions. For the most part, the Elven Gods and the human gods live in the warp, and can mostly only interact with the real world via giving some of their worshipers holy spells and blessings/dreams/etc. Mostly.
There are exceptions, the Ogre Pantheon is fully in the real world as far as Im aware though I assume the powers they bless their worshipers with still go through the Warp somehow? Bretonnia's Lady can just show up in Bretonnia (arguable if she's Elven, human, or both, certain reveals were made in End Times that don't necessarily make a lot of sense with a lot of lore readings, and everyone hates End times, etc, etc.) Like three different Gods went from human to God, with one of those three going from Deity to Human back to Deity.
No idea what's going on with the Halfling pantheon, never looked into them, don't know that story.
..........
Like, straight up, I don't even think Chaos is the strongest faction in Warhammer Fantasy. Skaven properly united or Nagash at full power are both, in my opinion, stronger then Chaos and stand a better chance at the taking over the world in the short term thing and they both heavily rely on the Aethyr, Chaos is sort of only winning in the long term, in that the mechanisms preventing the world from falling into their realm are failing and being destroyed and no one really has the time/know how/power/will to fix it. Ulthuan sort of still knows how, but they really don't have those resources.