Go for a straightforward yet casually dapper option pairing a dark brown cardigan and cargo pants. Add a pair of tobacco leather brogues to this outfit to make the outfit a bit sleeker. Casual combinations for middle-aged guys aren't actually that hard, as you see.