For a casually smart outfit, try teaming a pea coat with blue jeans — these pieces go really well together. Why not add athletic shoes to the mix for a more casual twist? And if we're talking casual fashion for middle-aged men, this combo looks good on most gentlemen.