There have been several comparisons I've read over the years on primer size. The CCI's have ALWAYS been a little larger diameter than the others.
If your .204 brass is on the slightly "tight" side of the spec, there you have it.
Several reloaders I know reload brass so much, that towards the end of it's life when pockets get expanded slightly; they switch to CCI primers.