I mean, you say that, but like, the concept of distance is currently under fire, because any attempt to quantize gravity as a field theory does weeeeeiiird things to spacetime and there's a decent contingent of physicists who are seriously considering the idea that spacetime is just an emergent result of quantum correlations.I'm not sure how much more fundamental you can get than "You cannot formulate a consistent set of physical laws which allow FTL while also having both cause and effect and the concept of distance"
Also, FTL doesn't really do anything to causality? Local causality, anyway, remains just fine. It's true that you get some, again, weird nonclassical correlations when you zoom out, but ... we already have to live with that, it's called quantum mechanics

(Now. It's true that, if a CTC exists, then you do have to give up on locality; not everything is predictable just from things in your environment. But again, locality is also under fire, as part of the same giant question mark called 'quantum gravity.')