Manderly's mentality here is "You miss 100% of the shots you don't take". And Show wise basically every house in the Seven Kingdoms threw offers at her. The Manderlys figure they're big enough that's not completely impossible and if they ask Rhaenyra directly they might have a better chance. Still a long shot and they know it.
Yeah I figure it was this, but asking for it immediately after asking for a big favor came across at least to me as not a great look. Like, it'd be one thing if they gave us some big gift or something to turn up the charm, show what they can bring to the table. Or at least discuss what they could bring to the table. Jason Lannister's an ass but he was making his pitch to canon!Rhaenyra with gifts and an offer to build her a dragonpit (which yes he obviously has an ulterior motive for, but it'd be pure upside for canon!Rhaenyra), and he didn't even think he'd get to be king.
In contrast, the Manderlys here are opening by asking for a massive favor, and their offer is just "I promise not to usurp you"... which, thanks, I know it's better than some would, but is really below the bare minimum. Add in how they're leading into it with bringing up this old marriage agreement which was so obviously ridiculous and unfavorable, and how uncomfortable Rhaenyra is with the whole thing and it doesn't come across well.
IMO we should just refuse this and not bring him along on the progress. If other voters are worried that refusing out of hand will seem harsh, and we should bring him along for the gentler let down - I think the fact that they know this is a long shot will prevent them from taking any offense, especially if we do push for granting them the contested castle. Bringing him along and then never talking to him on the other hand seems much more likely to lead to
greater offense, since we're stringing them along and then not seriously considering it. Talking to him is going to cost us actions.
Additionally, if we bring him along we're going to encourage other people to shoot their shot as well, and end up in this situation more frequently. Other people will assume that we
are seriously considering him, or at least might be, and feel like the need to get their foot in the door and pressure us for this now, before they could lose their chance.