I had a 25-'06 and found it burned more powder than I liked for varmint and didn't have the bullet weight I prefer for big game. With that being said, it is really an excellent choice for both kinds of hunting, as long as you're willing to compromise just a tad on both of those counts. I sold the 25-'06 and later regretted it, even though it provided nothing of value over the 243 and 270 I already owned. If I were restricted to just one rifle for varmints and big game, that M70 FTWT in 25-'06 was just about ideal.
Given that I can have more than one rifle, I prefer a hot 22 for any varmint smaller than a coyote...which is just about all of them. The 25-'06 is in a kind of no-man's land, from my perspective.