Something is incorrect about the way the swager is working. Sounds like it isn't going in all the way. I own the Dillon and its ram has a slight radius at the bottom that widens the mouth of the primer pocket just a little extra as it bottoms out. Assuming the RCBS has that, you want to be sure it went all the way home.
If you have some Lee sizing lube, use a Q-tip to put some inside each primer pocket or on the ram before swaging. That can help. The Lee product dries, so it won't bother the primers later, and it may help them slip in. If you use any other case lube for this, you will need to remove it later to prevent primer contamination.
If you have one of the hand primer pocket reamers, apply that to one of the swaged cases and see what it does? If it removes much material, the swage is inadequate. Also note that you sometimes need to run the swager a second time after the first reloading is fired because the brass a swager raises around the perimeter of the primer pocket partially flows back in when pressure slams the case head into the bolt face.
P.S., Mr. Broom, if you haven't guessed by now, the primer pocket swager uses a primer pocket-shaped ram to force the vestiges of a military crimp aside after decapping. The alternative is to cut them away with a primer pocket reamer or a primer pocket profile cutter, such as the accessory one for the Wilson trimmer. If you don't do this, there are varying degrees of difficultly seating a replacement primer.