Wouldn't you rather spam invis and other spells while hidden..? Silent casting is my favorite perk in this game. Some spells are extremely useful when sneak-casting, and you wouldn't even think it. Also the throw voice shout works wonders with this build. Lots of high level enemies tank destruction spells but if you're hidden you can SUPER SPAAAAAAAM.