Som det mesta från AI as Normal Technology är det här lästvärt. Några utdrag nedan, men läs hela.
We have argued that model alignment is not enough for AI safety. To recap: when a model is deployed, it doesn’t have enough context to know if the conditions it’s being deployed in are safe or not — a lot of that context is outside the model.
Alignment kommer aldrig kunna lösa säkerhetsutmaningen med AI-agenter på egen hand eftersom vad som är säkert och önskvärt beteende beror på sammanhanget.
AI companies function like startups. While their revenue and valuations have grown immensely, by all accounts, they lack the organizational processes that are the norm in any other industry. As a result, they haven’t developed the organizational maturity to get visibility into risky experiments and deal with the fallout.
Companies in most other industries see more gradual growth that allows the time for building organizational norms. AI companies resemble Facebook and social media companies that grew quickly with the “move fast and break things attitude”, but ultimately needed to put in guardrails and organizational governance standards in light of incidents like Cambridge Analytica.
Jaggedness only strengthens our view that it is more important to mitigate specific risks rather than trying to solve all safety risks through alignment. While risks might arise rapidly, they do so in specific areas, and we should be prepared to rapidly strengthen our defenses in those areas — and ideally do so pre-emptively
Att effekten av AI varierar mellan olika områden är ett argument för specifika motmedel istället för ett fokus på generell AI-risk.
So while we continue to reject the alarmism, and don’t believe that catastrophic risks are imminent, we do think risks are increasing over time because our investments and policies aren’t keeping pace.
Vidare till källan: normaltech.ai