If the buffer that's being closed is not the active one, there's no point in switching the user away to another buffer.