I have a similar background, elec-mech undergrad, masters in robotics doing SLAM, spent time doing electrical design for industrial usage, then software doing computer vision, now working on drone autopilots.

I 100% agree on needing to design and regulate the design of safety critical CV (and particularly ML based CV) algorithms so that, if nothing else, the failure mechanisms are quantifiable and limited in their impact.