To get on one of my hobby horses, the "masquerade" subgenre of urban fantasy is really, really, REALLY hard to justify with purely in-character explanations. That doesn't mean there aren't good masquerade stories. Just, you kinda have to go into them with a tacit agreement to not ask certain questions, or the whole thing falls apart.
Yeah, but like. I'm not convinced that these are actual problems with a genre that need to be solved.
First reason is that, well, with a good enough writing a stock genre convention ceases to be a genre convention and just becomes a part of the world. It doesn't even take
that much good writing, just writing that's just a little bit clever about it.
Like in World of Darkness where the name Masquerade comes from the fact that the Masquerade is implausible is part of the setting. Because the Masquerade is an actual social order among Vampires, that they're really obsessed with upholding, because they know they'd get exterminated in like five seconds if it wasn't. Upholding it requires constant clean up and mass societal manipulation and things are often teetering on the brink. Hell, the entire premise of the new game is that the Masquerade actually broke, and it's going to deal with the consequences of that.
If someone is still complaining about it at this point, then to me that says they're just complaining that the convention is being used at all, not that it doesn't feel plausible as written.
The second is that the mentality behind rational fiction (because it sure ain't unique to them) tends to treat genre conventions as Lego blocks that make up a genre and are inherent to it, instead of just creative trends that tend to be tied to a genre. And they put way, way too much stock in these conventions as cornerstones of genre.
Fantasy gets this a lot, where you got a lot of writers with a relatively limited reference pool for the genre trying to subvert tropes or really high falutin attempts at making them justified in the world. They'll think they're super clever for having an Adventurer's Guild or Thieves Guild exist in the most literal fashion possible, and then going into excrutiating detail why it makes sense, without ever stopping to think whether they even
need that shit. You don't. Fantasy doesn't need adventurer's guilds, it doesn't need dungeons, it doesn't need dragons, it doesn't need an arbitrary reason why guns don't exist. You can create any kind of damn world you want, and it would probably be more interesting than trying to one up the most basic bitch genre conventions while constantly nudging me going "Eh? Eh?" to remind me that this world I'm supposed to get immersed in is in fact a work of fiction.
Practical Guide to Evil is a big one in these category. It's probably one of the "best" examples of rational fiction, and comes one of the closest to being it's own thing separate from that. But it constantly shoots itself in the foot because it's preoccupied with meta-commentary and trying to
'play' with stock fantasy tropes. It bogs down the story with obtuse bullshit, ties all the tension and conflict in abstractions instead of the actual events of the plot, undermines the actually effective attempts to make the setting feel like an organic world. And I don't get why it would be impressive to you unless most of your ideas about the genre came from RPG sourcebooks.
Also, you're really not all that clever for pointing out that destiny and Black and White morality enforced by greater powers is dumb. Especially not if you're the Sword of Good, a Yud short story that a lot of people are impressed by but I think is piss that makes it seem like Yud's barely even read/watched Lord of the Rings, let alone examples of the genre that are actually interesting.
Also I've seen Prachett mentioned once or twice, and he's like a model for a lot of these writers. And the thing that you have to understand about Pratchett is that not only does he have a way better grasp of the fictional concepts he's playing with. But that his world is this big, real feeling world where fantasy tropes are real because it's
parody. Not parody as in your story constantly pointing out that tropes exist and that's super clever, but actual jokes and humour based around it. And his books broke out into being more dramatic works within the parody as a result of the series getting more sophisticated over time. You can't just brute force that end state by having fantasy tropes be self-consciously real in a setting.
Pratchett also doesn't stan for Dark Wizard Stalin. That's a common theme in both above mentioned works.