A varsity jacket and navy chinos are totally worth adding to your list of must-have casual styles. Inject a sense of sophistication into this outfit with black and white loafers. No doubt, a fail-safe option if we're talking outfit ideas for middle-aged gentlemen.