I know no analogy is ever perfect, but there is a big difference: Joffrey was the expected heir. Renly's and Stannis' claim about his parentage were right, as things were, but really, that was some outrageous claim probably not believed by all too many. So, really, Joffrey being crowned was just going down the expected line of succession, while Renly and Stannis established rival claims (based on a justification that turned out to be actually true, but still, rival claims).
Meanwhile, in our case, we are the expected heir and the line of succession as it is expected to go down. On account of our sex, people could justify alternate claims - but it would indeed be those with the alternate claims then. Basically, in a way, any rivals making a claim on behalf of a child half-brother would be more akin to Renly's Stormlands/Reach alliance.