Huh, okay, after some very cursory googling it looks like you're right. I was misunderstanding how VR focal distance worked.

Then your point definitely stands: taking off a headset in order to mix up your focal distance does sound like a pain.