What if an AI made a decision that most people disagreed with, and their lives became better because of it?

Would we call that AI aligned?

“Aligned with human interests” sounds straightforward until what we want and what benefits us point in different directions.

Imagine a city banning cars from its center despite widespread opposition. Suppose the streets become safer, businesses thrive, and even the opponents later welcome the change. Does the better outcome justify overriding them?

Political leaders face this tension too. An unpopular decision can be an act of leadership or an abuse of power. AI brings the same question into alignment: who gets to decide what’s best for us?

Greater intelligence might help predict consequences. It doesn’t settle how much freedom we should trade for convenience, or how much one person should lose for everyone else to gain.

Who should benefit from AI? Should it prioritize people who can do the most good, or help those with the least opportunity? Who judges someone’s potential contribution? When these goals conflict, whose idea of a better world should it follow?

Perhaps AI should convince a majority before acting. But imagine a system that knows exactly which fear or hope will change each person’s mind. If its goal is to win approval, persuasion can become another way of deciding for us.

Maybe we’ll talk with our personal AI until we make up our minds, and that becomes how we vote. But would the AI be representing our judgment, or shaping it? Whoever controls that conversation could gain enormous political power. We would need to explicitly confirm when thinking aloud becomes a vote.

Meaningful consent needs room to hear competing arguments and say no. Majority support also needs limits: people outside the majority still have rights.

I don’t want to vote on every decision an AI makes. I want us to choose what we delegate, set boundaries, and retain the ability to challenge decisions or withdraw that authority. We can accept a process without liking every outcome.

For me, alignment has to include our ability to keep participating in the choice of what “better” means.

I want AI to challenge my judgment. I also want to be able to disagree with it.

If AI gives us a better world while taking away our say in it, how much of what we wanted has it actually understood?