The grouping up and down, side to side with different loads is a very common thing. It has to do with rifling twist, bullet bearing surface, velocity, phase of the moon and alignment of the planets. Don't let it bugger you - find the grouping you like best and adjust the sights accordingly.