As a qualifier, I hunted for 10 years in north-central Wisconsin...if it was 30 degrees out, we were sweating and ticked off because it was way too warm for the deer to be moving much. 15 or less was what we preferred.
Walmart sells a Rocky-brand line of thermal base layer that is a poly fabric that is light-weight, loose-fitting and very warm...it is less than $40/set and extremely effective.
I base what I will wear on a given day on the predicted high temp. My rule of thumb is pretty simple: If it's going to be 30 or above, I wear 3 layers, with the middle layer being fairly light, possibly even a cotton sweatshirt and pants. If it's going to be LESS than 30, my middle layer will be a performance fabric, like Thinsulate/Goretex pants and shirt. At 30 or less, my outer layer is ALWAYS a wind-proof, bulky (trapped air IS the key!) combination of a thigh-length parka and bib overalls. The bibs are crucial because of how they help keep the cold off your kidneys...I wouldn't go hunt in cold weather with any kind of regular pants unless I had bibs over them.
If the high temp will be 40 or more, I will still wear three layers, but the top layer will not be as heavy. For temps that high, I usually go with a matched set of ScentLock gear as my third layer.
For my feet, I've tried many different combinations and finally settled on a very thin moisture-wicking poly sock on cool days and the same, with a 2nd wool pair, on COLD days. The key to keeping my feet from getting painfully cold (they don't stay warm, really) is to wear a good quality boot that is about 1.5 sizes too big. I wear a 9.5, but buy size 11 boots. This allows my feet to not be cramped, restricting blood flow, and allows for enough air space in the boot to act as an insulator. In my opinion, a boot that fits well in plain cotton socks will be too tight with a second layer on and is the biggest culprit for cold feet. Please note that all of this only applies if you are hunting from a tree stand or ground blind. If you will be walking a lot during your hunts, none of the above will work, for you feet. If you're hunting from a ground blind and have a great deal of trouble keeping your feet from aching from being too cold, invest in a pair of "boot blankets". They aren't worth the trouble from a tree stand and can't be walked in, but if you carry them in to a blind, your feet won't be walking back to camp at 9:30.
Finally, I wear a fleece, full-face mask and one of those tube-shaped muffs to put my hands in. I stick 2 hand-warmer packs in the muff and wear a thin, lightweight pair of gloves while hunting. I keep a bulkier pair of gloves in my pack, but only wear them while walking or as a 2nd layer, if my hands get very cold, for some reason. Another step I take, when hunting from a tree stand, is to use the safety harness that my wife sewed a 1/2" foam rubber pad into. It comes up to about the middle of my back and was originally meant to add comfort, but it had the surprising bonus of adding incredible warmth! (We made ours from those cheap, 2-piece folding seat pads you find on the shelf at every sporting goods store, during season. Our updated design allows it to be removed, the pad taken out and the whole thing washed...because it gets sweaty!)
For truly staying warm, focus on your head, hands and lower back (kidneys). Let these areas get chilled and you won't feel warm, no matter what else you have on. If on stand, wear boots your feet swim in. Move very slowly when getting to your stand: You will avoid getting sweaty, which is the real point, but you'll also spook less game. Most of all, learn to get a feel for when you are not warm and when you are truly "cold". For many people, especially from warm climates, "cold" is whenever it's less than 74 degrees.
To me, I have learned to differentiate between the sensation of cold air in my lungs (which I really enjoy, frankly) and the sensation of a shiver running from head to toe, because my body is starting to struggle at keeping my core temperature up. When this happens, I begin doing isometric exercises (basically, just tightening muscle groups, w/o moving) for 20 counts, rotating between arms, pecs, quads and glutes. If you do three sets of each it will take about 4-5 minutes and help fight off the chill by increasing blood flow and generating heat by burning calories.
Nothing improves your odds of success more than sitting all day, no matter how cold it is, so it really takes a system to keep warm. My experience at doing exactly that, on many occasions, has led to the system above.