AoE damage vs armor is the trade off that should be made. The manner in which the tradeoff occurs well...
In my opinion just restricting armored classes from just the highest tier isn't enough. The difference between using t4 of a spell and t5 of a spell is noticeable, but it isn't big.
Although you could scale it.
Cloth cast up to t6
Leather up to t5
Chain up to t4
Plate up to t3
As for CoT. What I'd do is make the damage CoT does to the mob, proportional to the damage you take from the mob.
Atm the only restriction is you have to get damaged, if you're getting hit for 0 Cot does 0 damage. But if you get hit even for single digits, the damage from CoT is insane. If you're getting wailed on as a caster, you'll dish out decent damage. But if you're barely getting scratched it won't be nearly so high.