Why not go for a short sleeve shirt and dark purple chinos? These pieces are very functional and will look good paired together. You know how to dress down this ensemble: grey athletic shoes. This combo should prompt anyone who was doubtful about wearing casual outfits as a middle-aged man.