As you can see here, it doesn't require that much effort for a man to look sharp. Consider teaming a blazer with a tan turtleneck and you'll look incredibly stylish. You could go down the casual route on the shoe front by finishing with tobacco leather snow boots. This pairing should erase any doubts you had about dressing casually in your 40s.