For something on the casual end, try pairing a navy crew-neck t-shirt with blue jeans. To give your outfit a more casual twist, complement your ensemble with navy athletic shoes. This ensemble should encourage anyone who wasn't sure about wearing casual outfits as a 40-year-old gentleman.