Trevor: If the heading remains constant in a given direction I would say you come a long way, so don’t give up! For that it sounds more like you have an offset that you need to take in to the calculation (for some reason) as you point out.
On my Sparki I get the same readings wherever I turn my Sparki and the magnometer on my phone shows a pretty decent earth magnetic force of 40-50 mikro Tesla. However from the Sparki I only get a maximum of -15 to +10 milli Gauss on any given axis. Thats with the wheels not turning, when I turn the wheels however the readings goes all over the place and far more than a couple of milli gauss, so the only conclusion I have drawn from my testing is that the motors should not turn when you do the readings from the magnometer since they will be faulty. Apart from that my readings are so small I can only assume my magnometer is not working properly, but I am hoping Joe can weave some magic with the driver code!