Sounds like you have a pretty neat rifle!
Here's what I would suggest:
1.) Slug the barrel to see what the groove diameter is
For best results the bullet should be at least .001" larger
An undersized bullet will sometimes keyhole especially if it is too hard.
Important! -Use a b.p. lube.
What bullet are you using?
2.) Use a .06" thick poly wad between the powder & the bullet
3.) Up the b.p. charge to 70-75 grs. compressing it with the bullet.
Unless you are using SWISS brand BP you will likely have to clean the barrel every 7-10 rounds to maintain accuracy.
Hopefully that will give you better results.