For a casually smart outfit, consider wearing a dress shirt and black shorts — these items fit nicely together. Dial up the fashion factor of this ensemble by wearing a pair of white leather derby shoes. A no-brainer combo appropriate for middle-aged men.