First of all, we've only attacked the Nomads when they attacked us first. We were never the ones to deal the first blow. But sure, let's take a look at our notable enemies, see what they're really made of.
First we have the nomads. You say they are weaker than us, but there's one thing we can say about them: They might be uncultured barbarians, but they sure know how to fight. Heck, one of the arguments for making with the Khemri was that a Hero Duo from up north would cause us no end of trouble, and we pretty much had to make the Stallion March to hold them off.
Then we have the Xohyr. Our ancestral enemies, monsters who captured and sacrificed so many civilians they were able to build a wall with their skulls. A foe so terrible we needed to borrow the strength of the aforementioned Nomads before we even wanted to fight them. Not only that, they had the world's first True City. We might not want to admit it, but Xohyr was probably the birth of Culture in our region. They were without a doubt our peers.
They might be a bit of a footnote, but I want to bring up the Spirit Talkers as well. We didn't fight much against them directly and neither of us were particularly martially inclined, but they had a profound effect on our military. It was them who taught us the power of mysticism and elitism in warfare, and each of their Berserkers could take on an entire unit of regular troops on their own. You can draw a direct line between the International Games we are building today and the legacy of the Spirit Talkers.
There's also the Thunder Horses/Speakers. Again, not the most frightening foes we've faced, but they are descended from Wendiktos himself. A guy so badass the Ymaryn themselves gave him their blessings and who wielded a fallen star in battle. He led an army so mighty that when they eventually lost, the Ymaryn as a people learned that even the mightiest of heroes can fall.
And finally, we have Khemri. Oh by Gwy, we have the Khemri. Now this was a worthy opponent. A young girl blessed with an unthinkable talent for violence; a God given flesh descending on the battlefield to match her; Armies so large and battles so bloody the battlefield had enough cadavers to fill an enitre city! It was a glorious battle, so terrible and awe-inspiring we immediately began working on ways to fight them again without killing them!
Compared to that, what does the Hill Kingdom have? What exactly makes them worth the effort to take out? They are cowards who are fundamentally unable to face their foes head on, who only attack them once their back are turned.
An insect like that isn't worth the underside of our boots.