Yeah, it's tough, but why apply any more force than is required to anything ? I cycle my Mark X .30-06 just hard enuf to eject the shell onto the bench. If you're hunting, and need a quick followup shot, then don't hesitate to eject vigorously to be sure the shell ejects.... My local gunsmith told me not to baby this style action. He said this action was used to fight in WWII and is tough. If I smoothly cycle the action, shells don't eject sometimes. If I go ahead and cycle the action with authority the shell sails about 10 feet through the air...
You didn't say whether they were reloads - if you had a high primer, some of the firing pin force could have been taken up in seating the primer, leaving a reduced force to ignite the primer.
Do an experiment: with a double-checked empty chamber close the bolt partially, and pull the trigger. See how far up the bolt handle can go, and still release. If open too far, the trigger should not be able to release the firing pin. See whether firing it does actually close the bolt if the handle is up a little.
I suspect you had a bad or contaminated primer, if it's really just the one.