[X] Protecting the villages. The problem is that the lowlanders will be too numerous and too varied for the People to stop them all. The solution is to protect the villages by constructing a physical barrier to keep out the raiders.
The known threats to us right now are primarily raiders, and the places where they come from are not centralised as much as us so we don't have to worry about well-organised attacks on are territories that would require a trained military to repel, not to mention we are too far from each other to make conquest or even large-scale raids feasible (not that there is any present demand for us to conquer them anyway). Fortifications can even lower the impact of surprise raids like what Curan did in what kicked off our first war with the Merntir, so we don't have to lose a couple hundred people. So as much as I like the effectiveness and flexibility of trained militaries, the current situation and that of the forseeable future favours fortifications.
When known threats start getting more organised, have advancements in siege warfare and the willingness/resources to commit to it, as well as being close enough to make sieges happen, we are going to need a trained military to relieve besieged territories as they will otherwise fall to a determined attacker. Just not now.
Our warriors are lesser, but our logistics are better than our foes.
I agree with all your other points, but this particular one can also be addressed by training. It would result in our warriors being on par if not better than known threats while also having superior logistics.
Im all for wanting walls, but we need warriors to man them, if we just have walls and no one manning them then they are useless!
Poorly trained militaries/militia (ie. us) actually derive more benefit from fortifications than trained armies. The former are unused to marching and thus travel less distances and have increased straggling (people falling behind and thus unavailable for fighting), are unused to formation fighting and are unresponsive to orders, lack proficiency in using weapons, and will crack under pressure faster. Thus the latter will, most of the time, soundly trash the former in the field of battle.
Fortifications alleviate much of these, as the garrison doesn't have to march, the defenses break up enemy formations, they present enough difficulties to the attacker in just getting through them to somewhat even the odds in personal combat, and the psychological safety provided by them means that the garrison's morale does not crack as fast as if they were in the field. These also benefit trained armies too, its just that poorly trained armies do not stand a chance without them, while a trained army does.