(Barely) got through the flagship by using a fully upgraded Kestrel. The four lasers were key to victory, but hacking the engine was the decisive factor. Even so, it got me to like 20% health. There is no way I should be having this much trouble.
If you're running on a lower difficulty setting (Normal or
especially Easy), get yourself a cloak and practice the timing on it. On the first stage, run it at full power and cloak when the missiles launch. It will nearly guarantee misses (in fact, it
will if you have 40% evade or better to start with, and since you're cloaked you can reallocate power to your engines to ensure that, then back out to medbay/oxygen/whatever to fix yourself up in the interval) and give you a full fifteen seconds to get your weapons charged up for a major volley-fire.
The layout of the boss changes with the difficulty. On Normal and Easy, I believe, it still has separated rooms for all its guns; teleport boarders into them in the order of missile (by far the biggest threat), ion (opens you up for the lasers and the beam), then beam (because you want to keep at least one crew alive, the beam will disappear after stage 2, and the triple laser is no threat by itself as long as you have at least three ranks of shield). Because the subsystems are separated the flagship will be unable to repair them; you can then work on eliminating the crew in the main body of the ship in perfect safety.
How you do that depends on the size (and upgrades) of your teleporter room, what kind of boarders you have, and your weapon layout. In general, a good approach is to hammer the shield room; the crew will swarm in to repair it, which will make your shots weaken them for your boarders, and damaging the room will weaken the flag's defences. Then, teleport boarders in and pop the medbay. Keep an eye on the flag's health while you do this, you don't want to kill it and lose your boarding team. I'd usually put the boarders in the pilot room with a two-man team or the shield room with a four-man.
If you have at least one crystal you can completely forgo shooting at the flag and just beam into the pilot room, locking it down when the pilot gets to low health. You can guarantee at least one kill per boarding cycle like that, which means its only a matter of time.
For the second and third stages, keep your cloak at just one power, and turn it on a couple of seconds after you get the "Superweapon Charging" notification. At just one power, it will still last through the drone spam/death blossom, and there'll be enough time for it to come off cooldown before the superweapon comes back online.
In the second stage, again, teleport aboard and smash the missile launcher; then move on to the beam cannon, then the drone control system. The drones are the bigger threat, but if you leave teleporting your guys in there until last, they then have free run of the ship and can crush the shield generator and pilot station to leave the flag completely defenceless against your guns.
The third stage's mind control can be very irritating, especially since with the Zoltan shield it can be difficult to kill it right away. If you've murdered the flag's crew, though, you don't have to deal with being boarded at the same time, which means you can cycle in crew to take the mind-controlled person's hits, then run them back to the medbay. Be careful doing this, because a rank 3 mind control system will let an Engi kill a Rock, but it'll keep it from wrecking its system until you can take down the Zoltan shield. Then it's a matter of teleporting aboard to roast the missile bay and then mind control unit. As with the second stage, once you've blasted the mind control station, your guys have free run of the ship and can smash the shield and pilot stations to leave the flag defenceless.
A defence drone I is very useful in all three stages to reduce incoming missile damage, but it is
especially for the second stage for stuffing the huge amount of boarding drone spam the flagship sends at you. Your life is much easier if your guys aren't distracted by enemy boarding drones and hull breaches, which remove the manned-system bonuses.