True, but a direct hit would still likely set the warhead off, which while it's no Type-93 it'll certainly do major damage.
TNT is so insensitive you can
blow it up and it won't explode. Tests were done.
That wasn't good enough for the USN, so they replaced it with Torpex in 1942, which was just as insenstive but half again as powerful. Then they replaced
that with HBX, which is as powerful as TPX, but much less sensitive. Unlike Type 98 filler (I know, shit ton of "Types"), which is extremely sensitive. Though not as bad as Type 94, which was rather powerful but far too sensitive even for Japan. And by "too sensitive for the IJN", I mean "prone to spontaneously exploding".
No, the concern is the fuel (methanol), which is a wonderful way to get a big ol' fire going. Or, if you're unlucky, a fuel-air explosion.
Not as dangerous, but they'll still go boom if they get hit and given how they're pretty much useless anyways... what's the point in having them?
Well, no, they're not "useless". First, the IJN thinks they work, so they're going to maneuver to evade them. The biggest problem with the Mark 15 (and most common problem) is failure to explode on impact. Until it smacks into the ship and doesn't explode, it will have every appearance of running straight, hot, and true, which means that IJN ships will be frantically maneuvering to dodge them. This is because when they work they have a
five hundred pound warhead (or, later, a 800 pound HBX one that's basically a 1200 pound TNT warhead). It's not the thousand pound warhead of the Long Lance, true, but that's more than enough to ruin anyone's day, and, critically,
the IJN does not know that American torpedoes are flawed, and even if they did they do not know which specific torpedoes are going to fail. That means they have to treat every fish in the water as if it's going to ruin their day, because if that's the one torpedo that everything goes right on and it hits because they didn't maneuver to evade it, their ship is in for a really bad time.
Now, this doesn't mean the mounts on Atlanta should stay - they shouldn't. They don't contribute at all to her role of bullying destroyers and enemy aircraft, she's extremely unlikely to have opportunities to employ them, and her torpedo mounts take valuable deck space that could instead have more guns, not to mention adding topweight to a ship that already has stability issues.