That's why, at least if you're not a master, you don't try to use both actively and keep one in an idle position. When you block, you defend and attack with the other sword in one motion, removing the need to coordinate both arms at once.
I can assure you there are plenty of cases of anyone...