This.You are asking whether a varmint load will work on deer, but I would just use a suitable deer load to shoot varmints. Way too much is made of an inch of trajectory. If you know what it is, you can compensate for the drop at range. You can't compensate for bullet construction. Varmints just need to be dead, and any deer bullet for the .25-06 (even a lighter one given the smaller deer) will do just that.
I’ve used bullets meant for deer on countless prairie dogs and coyotes.
I use my .308’s loaded with hornady interlocks, Remington corelokts, Hornady SST, Interbonds, or accubonds for EVERYTHING! All 150gr-165gr loads. 100gr in my .243 for everything as well.