If you're wearing a jacket, as I always am, the sweater vest always needs to be buttoned, with the exception of the bottom button. But if you're going sans jacket, you can leave it open.