You are correct but it is easier to repel 10M of imperial army then 100K of Luna Wolves.
And we will be repelling all 20 legions at one point or another, after all we will be existential threat to Imperium.
Doubt it, they can't just focus on us. They couldn't do that for any of the imperiums threats, for a multitude of reasons but mostly:
1: Distance and regions: Each primarch was assigned a region and to expand it to other planets to reunite humanity. The primarchs on the other side of the region closest to us can't get to us just due to sheer distance without abandoning their region for multiple years.
2: We are not alone: We are not the only threat to the imperium, not even the only AI intelligence either. If the imperium even dedicated 10% of their armies to us, the rest of the imperium will start failing due the numerous other threats that accost it. Yes they can spend a lot of resources to defeat us, but they cannot at the snap of their fingers.
3: Big, powerful, and slow: The imperium at this time is really powerful, however it reacts relatively slow to threats outside of those closest to those threats. Even if the imperium decided we need to die now, it would take decades to redirect supply lines and factories to taking us down without disrupting the rest of the imperium. They could do it, but it won't be quick.
4: Home is scary: Our region of space sucks, that mutated ship thing? That's the intro to this place basically. There's lots of scary things here of many types, which is why the imperium isn't gonna find us soon. Because trying to colonize the ghoul stars is a horrid task that takes a lot of resources and dedicated effort. We will probably find the imperium before they find us.
5: The emperor of man sucks but not really: He could come to us, he could come here personally sure. Do you think for an instant that the chaos gods won't interfere? Coming into this hellhole part of space, something that even if he could crush us fast, would still mean 2+ years away from the imperium? They will jump on the opportunity fast and he knows this. The emperor is one of the biggest chess pieces the imperium has, he can't be thrown around willy nilly which he knows. He can't afford to go to us personally, not without a good plan for the rest of the imperium first.
There's more reasons, but those are more minor in nature. These are the big reasons we won't be the giant focus of the imperium, at least not that easily. Yes we will fight, but it's not the entire imperium, not at the start at least.