I think mental instability from magic is based more on who (or what) taught you...
Cthulhu-esque magic springs from an alien understanding of the multiverse. To practice those rituals would take the mind where it is ill equipped to handle the alien thought processes involved.
Native magic which are not so outside human experience shouldn't be as hazardous to practice, they merely tap forces mankind lost access to. (Whether the force became inaccessible or the nonawakened waged war on the awakened, via witch hunts, inquisitions, etc.)
Eh. Given the belief component, there's a lot of reasons for it to be dangerous or harmful to sanity, given the tendency for deities to be at least a little incomprehensible to mortals.
I guess I'm not familiar enough with Cthulu mythos to know what "awakened" is supposed to mean beyond a buzzword.
I'm not even going on Cthulhu mythos, just the idea that mages can see things non-mages can't. Which ends up being pretty hard to distinguish from schizophrenia, other hallucinatory mental issues, or being on hallucinogens, from the outside perspective, unless you can somehow prove it to them.
Yeah well I've lost 2 cousins to depression and/or other mental issues pushing them to the point they took their own lives, so forgive me if I'm a bit freaking sensitive about this.
A large chunk of my immediate family has bipolar (or just depression, with my dad), as do I. My experience has been atypically easy, my symptoms well-controlled. My brother's has not. My parents could quite easily have died from self-medicating, though they kicked the illegal drugs/drinking before I was born, and I'm pretty sure self-medication overdoses are why I never got to meet either of my genetic grandfathers. I don't exactly treat mental health as a joke.
That said, a gift for magic in this setting seems likely to be very easy to be confused with various mental health issues, particularly schizophrenia, and for good reason. Generally, seeing things that others can't, or hearing a voice, for example, telling you how to raise a golem to defend a city, is going to raise red flags with people. Even the slightly more acceptable New Age-y stuff about seeing auras is going to get you funny looks and a reputation.
People that see things that others cannot perceive, are at best labeled 'odd' and at worst might be committed to psychiatric facility for being delusional. I believe BadKatt was only inferring that having such a sense might get an otherwise rational and sane person wrongly labelled as 'crazy'. And a family that possesses the ability; might appear to ignorant outsiders as a family history of mental instability.
A not uncommon conclusion in any sort of urban fantasy novel with the "most people can't see magic" aspect, which itself has ancient roots. I've read a few stories where the main character was labeled as mentally ill, when they were actually magically gifted in some way, and they didn't necessarily glorify it. In many ways,
not being crazy was a significantly worse scenario for the characters, because that meant the monsters were real. Nor was this said to be true of everyone.
Really, I do think it usually results from a self-aware look at how people on the outside would perceive an ability to see a world others can't, or hearing voices from a deity instructing you in magic, or various other things. Until and unless you can back it up, like, say, raising a golem, it's inevitably going to make you look just a bit nuts. I know there are a (hopefully) very small number of people who deliberately avoid treatment for things like bipolar or schizophrenia because they think it's a sign they're magical or the like, but I really doubt those are the most common reasons for it. Usually, bad experiences with medication or doctors, fear of being labeled mentally unwell, general distrust for authorities, and/or the effects the condition has on logic, emotion and cognition are more than enough, unfortunately.
