True! But that's sort of my point. Charlemagne may have Krell's armor and axe, plus his own tainted presence/power from the Dark Gods, but he's one guy. But he's capable of walking for, like, weeks without anyone even seeing him, especially if he doesn't take the main roads/paths. After the Vaults, his warband was entirely dead, his main spellcaster/magus was dead, and he was alone.
He was not, necessarily, like some Chaos Lords who are just walking wastelands, who leave burnt and hellish tainted earth behind them in vast gulfs like Archaeon or Crom might.
Charlemagne was running the most open band of Chaos Worshippers in Tilea/Estalia, and it still wasn't big enough of a deal to make the international Rumor Mill. It might not have seemed like it from Roland's/the Interlude's perspective, but Roland had/has a literal homing system thanks to his Grail Visions that are telling him precisely what direction to go in to get after his son, which pushed them to the desperation moves of sacking/razing more openly to get more resources to try and keep ahead of him and the band of Furies and their Furiosa.
All I was saying is that the wilderness is big enough that if you aren't actively sacking/raiding/razing/invading/attacking/etc. various settlements, it is possible to just sorta move through 'em without people noticing as much. And those that do, like isolated trappers/hunters/etc. are just as likely to have their disappearances blamed on it being the Great Forests of the Empire rather than specifically some raving Chaos Champion who cut them down and ate them.