My point is not that there would be not impact (albieth indirect), it's that other things on your list ALSO have an impact, potentially even greater.
I mean, luxury goods COULD end up being taxed, but straight up gold could also be taxed. Taxes COULD end up being used to military ends against the empire or our allies, but so could the military forces saved by our research on skaven and greenskin. Mundane books COULD be used to refine their understanding of the Old World in order to better plan their raids.
There is no reason that the slight improvement in their economy due to a minor trade in luxury would be worst than all of those outcomes IMO.