It depends on the adventure riding.
I have done some that is nothing more than dirt roads and it really isn't any more demanding than street riding. Go somewhere like North Carolina and do a 250 mile day of pavement on the right roads and you will be WHOOPED by the time the day is over! Then there are some dirt "roads" that might be only a few miles long and by the time you reach the other end, you can barely stand up, can count your pulse by the pulsing of the edges of your field of view, and you might be seeing dancing spots in your vision
Skill level has a lot to do with it as well. If you watch enduro racing, the really good guys make it look easy and they aren't expending anywhere near the amount of energy as the not so good guys... That is part of what makes them so good, the ability to get through stuff in a way that is the smoothest and least difficult, letting the bike do the work, so they don't wear themselves out. Fatigued riders make mistakes. Mistakes lead to injury to bike and rider. This is why good training and practice of correct techniques can make all the difference in the world.