My aunt used to drive barefooted when she had a car.
But I always put my footwear on whenever I drive. I could still
remember my early driving days when I used to keep using this
particular pair of shoes whenever I drive simply because I feel
more comfortable in it (used this same pair during my practical
driving lessons).
As time goes on (and more confident), tried other shoes, sandals as
well as slippers. But from my experience, I would strongly advise
fellow drivers NOT to wear slippers as this could be quite
dangerous(quite loose grip). Choose those sandals with support at
the back of the heel as this would give your feet a better grip on
your footwear. Consequently, you would have better control of the
clutch and brake pedals.
But most importantly, one should feel comfortable with the
footwear(or without it) before proceeding to drive.