It's not nearly as insane as you believe. Walder Frey doesn't want Robb dead, he wants prestige and he wants one of his daughters married to the King in the North, and he wants anything else he can weasel out of the negotiations. He can't do that unless he offers us bread and salt to start negotiations in the first place.
As for taking the castle, yes, it's not a simple matter at all, but we know and Frey knows that if Robb doesn't return safely from the Twins the entire northern army will go berserk. They won't stop until they see every Frey dead.
What would happen if we go in person is Walder probably either calls us stupid for coming in person or arrogant, doesn't matter which since he insults us either way, and then we remind him that negotiations won't take place without bread and salt which he'll give us because he wants us to give him things too.
EDIT: Although if you see Roslin Frey as an unacceptable compromise I understand why you want Roose and Greatjon to go in our place.
It's exactly as insane as I believe it to be. Going
will show him that we're stupid and arrogant, and one way or another he
will make us pay for our stupidity and arrogance. Here's how...
1. Due to his greater leverage from having us in his power, he will not only demand the marriage but he'll demand something else on top of it. So if you're willing to accept the marriage for a crossing, just send Catelyn and be done with it.
2. He'll decide that we're too stupid and arrogant to risk backing, because we're clearly an idiot for putting ourself in his power. At best he decides to just kick us out and wash his hands of it. Alternatively he takes us hostage, guest right or no - he can just deny ever having given us bread and salt, and nobody will be able to prove otherwise.
And yeah, our army will be pissed if he takes us hostage, but even if they go berserk that's not a guarantee that he'll lose. Far from it. The Twins are a rather formidable fortress, or more accurately two and a half formidable fortresses. Taking it by storm will not be easy, even with a five to one numerical advantage. First, we'd have to take the eastern Twin, which is a good castle with high curtain walls, a deep moat, a barbican and a portcullis, so the castle is basically an island castle. But let's suppose our army, in a berserker rage (which, by the way, is a terrible condition to be in for a castle offensive, due to the disruption of discipline), manages to somehow breach it and start flooding in after taking the grievous casualties one would expect when storming a well defended castle. Do the Freys admit their defeat and throw down their arms? No. They withdraw their forces across the bridge. The Northern forces then have to march themselves across the bridge, but they'll be opposed. Half way across the bridge is the Water Tower, which has arrow slits to kill people coming down the nice bridge killzone, portcullises that would need to be broken through to keep them there, and murder holes to start dropping boiling oil on them when they manage to start getting through them. But ok, after taking even more grievous casualties getting through that miniature fortress guess what? They get to go through the whole thing again, since they've got another nice, narrow killzone to go through in order to get to the western Twin, which they they then have to somehow storm, and at that point they have to do it tired and battered while the Freys will now fight completely to the death since they don't have a good retreat available without abandoning their home. And oh, if they succeed they'll need to refortify everything before the Lannisters arrive, because if you've wrecked the defenses then their army will have an easy time getting through to catch up to slaughter whatever remains of the Northern army.
There's a reason the Lannister army didn't try to take the Twins in canon, even with part of the Frey army having left to join up with Robb.