I typically carry 4 rounds with me when I go whitetail hunting here in the South East (I have been known to carry one magazine full only - which is four for all my bolt/pump/semi rifles I've ever owned, for a long time now). I now use a single shot rifle but I still carry four. I'd probably do just fine with two.
I normally only have to use one round per deer, but I will use more, if necessary, to make sure the deer doesn't suffer long. So far, I'm 5 for 5 this year (all deer expired within a minute of being shot the first time). I don't think I've needed more than one shot per deer in the last 15 years. If I'm ever able to go after elk, I'll probably take 8, just in case (there are wolves/bear/etc. up that way and I might need to have more with me to feel safe).