I did that once, albeit walking, not driving, and the drone had no auto flight programming. It was really hard to do anything more than walking on flat ground. Bumpy ground, difficult. Try to grab something on a shelf, difficult. This was a few years back, so tech is better, having auto follow would free up some brain power, but it’d still be real rough.

There’s a video of a bunch of soccer players trying to play using VR and a camera above the field, it’s got some funny moments.