I get it too, just not as bad as you do, it's most noticable if I'm leaving a city area or have zoomed somewhere else to look at something then recenter on my spot on the map. I try to make sure I close my map before leaving a city. Try turning some of the options off and see if that helps. Then all you need to do is turn the ones you need/want back on when you need your map.