Broken Bow's Beavers Bend recreation area has cabins to rent- I've seen bikes parked outside of them but never stayed there myself. Might be a good base camp.
$115 a night, looks like a few are available.
If you do Broken Bow, you could ride north on 259 & pick up the Talimena Parkway and it's a pretty ride. Once you get on the Talimena & to the Queen Wilhelmina Lodge, you can get a $5 camp site at the little country store at the bottom of Rich Mtn (272) & they serve breakfast too. Been there a bunch & it's fun for an overnight... keep your ear plugs handy cause the train comes through all night long! That's the only downside but it's not that bad.