You have several issues here:
1) ATC sending you the wrong way. I have experienced this in both VFR and IFR flight. I do not know what causes it, but it happens. But, there will be times ATC will take you a little off the GPS track because ATC is guiding you basically by waypoints that GPS might not use. But, there are glitches that send you off to the wrong spot. ATC recently sent me 180-degrees when I had filed a IFR flight plan from KMEM to KPIE (Memphis to St. Petersburg, FL) It never routed me according to the flight plan. In fact, it never routed me other than 180-degrees. Gotta be a glitch because we would have kept going south forever. I ran the route just to be sure it was a glitch and that somwhere along the route it would send me to KPIE. It didn't. Glitch.
2) Being handed off to several controllers within the same control area is very normal and realistic. Each controller supervises a specific geographical area. So as you travel you will encounter different controllers controlling different and distinct airspace within the same center.