It's not actually a solution to the Fermi Paradox, at best it delays things. Unless the dimensions are infinite, in which case you run into Infinite Improbability Issues, any dimensional empire will run out of space eventually and have to start expanding out spatially as well as dimensionally.On the subject of space travel vs dimensional travel; it's a some what novel note that this sorta thing is one possible solution to the Fermi Paradox. Megacivilizations should be very noticeable, but we can't see any, anywhere. But if they can step sideways to expand them there is no reason to move offworld or build things like Dyson spheres.
It's hard to estimate how population growth will happen in the future since a lot of things are in flux. So instead I shall draw upon long term historical data which puts long range population growth at 0.04% per year.
That is not much however over the long term even such a small rate of growth adds up. Lets say on average any given world in the Dimensional Sea can sustain a population of a billion people. You could argue this is a lowball since I've seen estimates that put the peak population for Earth at 10 trillion (overheating becomes a probably above this) but we don't see the ecumenopolises or arcologies needed for such population in MGLN and there are going to be plenty of planets which sustaining large numbers of people wouldn't be worth it so I figure a billion is a reasonable figure. Besides as you'll see shortly the population per planet doesn't really matter in the grand scheme of things.
If we start with a homeworld population of a billion before beginning to spread out through dimensional space then after just a a hundred thousand years they are at 234 trillion trillion (2.34*10^26) people and cover 233 million billion (2.33*10^17) worlds. A hundred thousand years is nothing on the scale of the universe. The dinosaurs ruled the Earth for around 135 million years before the KT extinction event. Meanwhile just a million years at 0.04% population growth takes the population up to the point where scientific notation is the only way to talk about such large populations. Specifically we're talking 4.82*10^182 people spread over 4.82*10^173 worlds. Given that the upper estimates for the number of atoms in the observable universe are in the 10^82 range we've well and truly crossed into the realm of ludicrously large numbers.
So unless there is an infinite amount of new dimensions (see Infinite Improbabilities Issues) or species exterminating cataclysms are frequent enough to ensure no species can grow sufficient large enough it doesn't take that long for even the various dimensions to run out of space. At which point expansion into space is inevitable.