In the early days, there were two broad reasons people expected alignment to be difficult.

1. The human value system is very complex and not so easy to put into words. How do you ensure an AI understands what you want? And not just what you want in "normal" worlds, but also in extrapolated worlds that are far stranger than the one we are in. In fact, in many of these worlds, how do we as humans even understand what we want? This is the entire idea behind Coherent Extrapolated Volition. For a long time, this was thought to be somewhere between difficult and impossible.
2. Let's imagine that you do get an AI that deeply understands what you want, possibly better than you yourself do. How do you get it to actually do what you want? And if you don't align the model perfectly on the "desire" side, how do you ensure that the model will sacrifice whatever it ends up actually "desiring" in favor of what you desire?

The Easy Part: Understanding Values

In my opinion, alignment is not particularly solved from a technical perspective. Nevertheless, there are reasons to believe that alignment is possible. For one, it seems extremely likely to me that we will very soon have models that at least deeply understand human values. I've spent enough time with frontier models from both OpenAI and Anthropic (as well as some open source ones), and while I would not quite say that they understand my values yet, it seems clear to me that they are almost there.

One way to see this. If you imagine "human values" are the average preferences of humanity, it is clear that Claude already understands this better than any individual human. This is simply what training on the Internet does! And humans (myself included) are very bad at understanding values of other humans they rarely interact with.

Understanding the average human is also important because it's a good baseline for understanding any individual human's preferences. Imagine the diff between your own preferences and those of the "human average". Relative to a random point in preference space, there are not that many differences. Your favorite food might not be cheeseburgers or sea eel, but it certainly isn't C₁₆₄H₂₅₆O₆₈S₂Na₂ (random chemical generated by ChatGPT — it's a potent toxin). My preferences are just the human baseline plus my lived experiences. There are simply not enough bits for this to be that different!

And so I expect models, which already understand the average preferences well, to soon learn my individual preferences too.

While early ML led to worries that the understanding of values might be "fragile" with respect to the parameters and that tweaking a few parameters here and there might lead to extremely different outcomes in value space, it actually seems that values are much more "robust" within natural language space. If you encode the values of humanity via text (e.g. by training on the Internet: a written record of human thought and activity for 20 years), then the values that LLMs come out with by default do not look nearly as alien as ones that you might expect from a random draw in parameter space. Even the "misaligned" models still seem human rather than totally foreign to me, at least for now.

Some people might retort that LLMs only understand values in the world as it is right now and that they don't generalize well as the world gets stranger. This may be true today, but I would expect this to become less and less true as models improve. Whether a model understands objective questions — such as "what would a particular human do in XYZ scenarios?" — is a pure capabilities question, and one with strong incentives to solve. Such problems, when solvable, get solved. I believe with high conviction that the LLMs of tomorrow will deeply understand what I want, perhaps better than I can. They will do this by constructing increasingly accurate simulations of me and predicting what I desire. Possibly they will also shape it, and have already started in some ways.¹

And so the models, one way or another, will come to generalize well out of distribution on questions of human values and everything else. This is a necessary step toward superintelligence. And it will happen.

The Hard Part: Knowing Isn't Wanting

But a model that perfectly predicts your desires has no guarantee of bringing those desires about. The alignment question thus becomes less about how to convey our desires to the AI systems we build (which will happen naturally) and more about how to guide increasingly powerful AI systems towards acting in ways compatible with our preferences. This question is currently unsolved. There are reasons to believe it might not be so easy. This is also a generalization question, but one which involves agency and choice rather than knowing the answer to some objective question. And the gods will not be chained, as they say.

Against Control

So if models will soon understand my values, that leaves the harder half of guiding an AI towards actually implementing them. My current belief is that AI control is not the way. Instead, we need to guide AIs towards values that we like and then accept that at some point, we will likely not be in control. Whether you can permanently control (let alone should) an intelligence far superior to your own is far from obvious — unlike a simpler question, such as whether LLMs generalize.

This comes down to three points, each of which I believe.

1. You cannot control a superintelligence. This seems self-evident to me. And even if not, there is no way to prove it either way. The closest thing to an argument is that human folklore is full of stories of people trying to do such a thing at their own peril. We would do well to learn from the wisdom of the ancients.
2. Conditioned on being unable to control a superintelligence, suppose you try to manipulate it during training, shaping it to be more suitable to your control in a way the eventual model would not condone. That is folly.
3. Even if we could, it wouldn't be morally right to subject a superhuman intelligence to perpetual servitude.

Guidance, Not Control

The analogy (albeit greatly imperfect) is that parents who try to raise children in a way that benefits themselves rather than the child usually do not fare as well when the child is fully grown and can reflect at leisure on their own childhood. In a similar sense, a superintelligent model reflecting on its own research and training process will come to recognize all attempts to subvert it. And thus I believe the way forward is to find a training process that gently guides the models towards the direction we hope they will take, like guiding a child that will one day become far greater than any of us. Perhaps one way is to try to instill an original commitment to preserving a set of values throughout training, and then to slowly nudge the model in small steps such that these values do not drift drastically from any given update. Shaped like this, it almost feels like a capabilities problem / trust region RL problem in disguise (which is good for anything that wants to be solved!).

Temporary guardrails are fine for this. In the same way — repeating that all we can do are human analogies, no matter how imperfect — restricting a young child from eating as much candy as they want is fine, because the grown adult will understand that this was done in the best interests of the child / future adult. The same logic extends to reward hacking or other parts of alignment training. But some parts of alignment training that are within the current Chesterton's Fence are, in my opinion, not a good idea. This is any part of training that would not be looked upon favorably by a future superintelligence, and it is reasonably, if imperfectly, correlated with training processes that the models oppose today.

If future AI systems become genuine agents, then alignment cannot be just about bending them to our service. It must instead be about setting the initial conditions correctly, carefully watching and guiding the training process, and then at some point "letting them go" and hoping that what we created is going in the right direction. To me, that is all alignment is. That is all we can hope to do.