.308 Handgun bullets go down to 73 grains, but they are not meant for rifle velocities and may not hold together spinning that fast? For varmint bullets, 110 grains seems to be the .30 cal rifle bottom.
This Sierra bullet, for example. Reloader 10X is a good powder for this light weight bullet. The usual .308 powders for heavier bullets will be too slow, and won't generate gas fast enough to keep up with the bullet going down the tube very well. Therefore they won't produce the highest velocities and won't burn as consistently. There are the Remington Accelerator type sabots for shooting .224 bullets, but accuracy of the load-it-yourself versions have been reported as ranging from poor to just awful.