*flashback to Fate Stay/Night*
Not necessarily, and not immediately.
If they don't have Independent Action, or some way of draining prana from a leyline or something, it happens pretty quickly. For Carmilla to stay alive after her master died, she would need to almost non-stop drain blood basically 24/7, and even with that her already relatively poor fighting skills would tank and even a master would have a decent chance of beating her.
Once you start throwing around 'oh, if X is alive they'll keep on killing people so the only possible way is murder', then you've given yourself a very self-serving hammer. Olga isn't the Joker and this isn't Warhammer.
We don't need to kill her to knock her out of the war. Carmella doesn't really have any skills or Phantasms that'd keep her around once she's axed for good, not outside some contrivance with the blood-sucking skill letting her take in a ton of energy at once. But remember: Olga's wanted by the Church for killing civilians. If she goes full Zuko after being handed over to Jeanne, then the conversation can be revisited. Yet I doubt after being in the Saint's care there will be any issue.
Okay, let me try to be objective here.
Fact: Olga has no problem killing civilians in large numbers if she thinks it'll give her an advantage.
Fact: Olga seriously wants revenge against us.
Knowing these two things... do you really think that Carmilla dying would stop her from lashing out at us?
And even if the answer is, "yes," trying to subdue is harder than trying to kill, and Olga seems very confident that Assassin can kill Rider this time. Killing Olga kills two birds with one stone; she's no longer a threat, either current or future, and Assassin is basically dead.
Moreover, as I have said multiple times, Olga isn't a person deserving of sympathy. She is easily an example of how immoral Magus' can get. And while we aren't going on a crusade against all evil Magus' (a large percentage of them), she's our enemy, she's in front of us, and killing her is easier than trying to keep her alive.
I'll drop this at this point unless there's a response because it seems people either disagree with or otherwise aren't receptive to my opinion, but I really think going easy on her (as in, trying not to kill her) is a mistake.