The point isn't that people would speculate; the point is that, because the project is a PI project, it will draw spies like flies to rotten meat, and for that matter probably raiders. If the Batarians still exist we can be certain they'd be sending a fleet.
And it
would negatively impact our PR. The Quarians are still rather universally seen as the species who created an AI race, then panicked and tried to destroy them in secret, then
failed to do so, for all intents and purposes unleashing a likely-hostile, galaxy-destroying von Neumann plague on the universe. Openly supporting the Migrant Fleet is basically political suicide.
And it's not like the Migrant Fleet will be especially grateful, either: they'll gladly take whatever charity we give them, and will turn around and throw it into their conflict with the Geth, because that's where their political leaders are pointed right now. Without years of careful political maneuvering, including installing Tali as one of the five members of the Quarian Admiralty Board and personally saving the life of one of the other members, the Migrant Fleet is still inclined to take any advantage they get and immediately throw it into trying to piss off the Geth. And no, a strict agreement that they won't piss off the Geth won't work, as they have
already signed such a document with the Citadel, and are obviously ignoring it: "After being refused aid by the Citadel Council, the quarians fled the system in what remained of their fleet. Shortly thereafter, the Council stripped the quarians of their embassy as punishment for their carelessness, though a treaty was agreed upon forbidding an attack on the geth in order to avoid provoking them."