Yep - that's why I bought an auto for the yacht. Normal procedure is mudweight up, hoist sails, then back up front to pull the mudweight in before sailing away. One day ( maybe ) the wind will change as I walk back and the boom collect me on the way across.
On the other hand, when sailing the performance dinghy, with a fair chance of going swimming, I wear a buoyancy aid.
Then there are times when I'm just motoring - and MY decision I tend not to wear one