Review: Human Compatible: Artificial Intelligence and the Problem of Control by Stuart Russell

Professor Russell’s book starts out with an entertaining journey through the history of AI and automation, as well as cautionary thinking about them. This discussion is well informed – he is a renown AI academic and co-author of a comprehensive and widely used AI textbook.

Having provided historical background, the remainder of the book argues two main points: (1) the current approach to AI development is having dangerous side-effects, and it could get much worse; and (2) what we need to do is build AIs that can learn to satisfy human preferences.

Concerning the dangers of AI, the author first addresses current perils: misuse of surveillance, persuasion, and control; lethal autonomous weapons; eliminating work as we know it; and usurping other human roles. I found this part of the book an informative and well-reasoned analysis.

Beyond AI’s current perils, the author next addresses the possibility of AIs acquiring superhuman intelligence and eventually ruling and perhaps exterminating humankind. The author believes this is a definite possibility, placing him in basic agreement with works such as Bostrom’s Superintelligence and Tegmark’s Life 3.0. AI’s existential threat is the subject of continuing debate in the AI community, and Russell attempts to refute the arguments made against his position.

Russell bases his case for AI’s existential threat on two basic premises. The first is that in spite of all the scientific breakthroughs required to initiate superintelligence (well documented by Russell), you cannot rule out humans achieving these breakthroughs. While I appreciate this respect for science and engineering, clearly some human achievements are more within reach than others. Humans understanding human intelligence, let alone creating human-level machine intelligence, seems to me too distant to speculate about except in science fiction.

Russell’s second premise is that unless we change course, superintelligence will be achieved using what he calls the standard model, which creates AIs by optimizing them to meet explicit objectives. This would pose a threat to humanity, because a powerful intellect pursuing explicitly defined objectives can easily spell trouble, for example if an AI decides to fix global warming by killing all the people.

I don’t follow this reasoning. I find it contradictory that an AI would somehow be both super intelligent and bound by fixed concrete objectives. In fact in the last part of the book, Russell goes to great pains to illustrate how human behavior, and presumably human-level intelligence, is far more complicated than sequences of explicit objectives.

In the last part of the book Russell advocates developing provably beneficial AI, a new approach that would build AIs that learn to satisfy human preferences instead of optimizing explicit objectives. While I can see how this would be an improvement over homicidal overlords, I don’t think Russell makes the case that this approach would be even remotely feasible.

To point out how we might grapple with provably beneficial AI he spends a good deal of time reviewing mathematical frameworks that address human behavior, such as utility theory and game theory, giving very elementary examples of their application. I believe these examples are intended to make this math accessible to a general audience, which I applaud. However what they mainly illustrate is how much more complicated real life is, compared to these trivial examples. Perhaps this is another illustration of Russell’s faith that human ingenuity can reach almost any goal, as long as it knows where to start. Like scaling up a two-person game to billions of interacting people.

I was very pleased to read Russell’s perspective on the future of AI. He is immersed in the game, and he is definitely worth listening to. However, I have real difficulty following his extrapolations from where we are today to either superintelligence or provably beneficial AI.

How Will AI Impact Organizations?

We can think of AI as software that helps organizations more effectively reach their goals, e.g., by reducing costs and increasing revenues.

Gaining benefits from AI, or any other innovative technology, requires organizational change. New strategies. New job descriptions. New workflows. New org charts. Training.

What makes AI different? Are the challenges faced by organizations adopting AI different from those encountered in adopting other software innovations?

After all, using computers to revolutionize organizations is nothing new. IBM developed the SABRE reservation database for American Airlines back in 1964, replacing manual file cards with a system that could handle 83,000 reservation requests. Pretty disruptive!

So how is AI changing organizations? Let’s take an example – the financial industry. AI’s ability to find patterns in mountains of data can help financial organizations:

  • Make more accurate and objective credit decisions by accounting for more complex relationships among a wider variety of factors
  • Improve risk management using forecasts based on learning patterns in high volumes of historical and real-time data
  • Quickly identify fraud by matching continuously monitored data to learned behavioral patterns
  • Improve investment performance by rapidly digesting current events, monitoring and learning market patterns, and making fast investment decisions
  • Personalize banking with smart chatbots and customized financial advice
  • Automate processes to read documents, verify data, and generate reports.

To make these improvements using AI, a financial organization needs to undertake the sort of activities needed to introduce any new software into their operations and products, such as:

  • Establish strategic priorities and budgets
  • Clarify and communicate objectives and plans with stakeholders
  • Work with software developers/vendors/users to establish and carry out software/system development projects
  • Create/modify procedures and organizations to take advantage of the new software
  • Hire, train, retrain the workforce as needed
  • Monitor results and adapt as required.

What are the special challenges AI brings to these activities?

The first challenge is AI’s high profile. Managers feel compelled to catch the wave of the future, and workers fear they will lose their jobs. As a consequence:

  • Managers may undertake AI projects with unrealistic expectations. AI can be extremely effective, however only when there is access to large volumes of data relevant to an operational role that truly benefits the organization
  • Employees essential to successful adoption of the new systems may stand in the way or quit if they see AI as a threat.

Clearly due diligence is required in the first case, and effective employee engagement in the second.

A second challenge is an “all or nothing” aspect of AI. To reap the benefits of AI, the core AI technology must be fully integrated with an organization’s IT infrastructure and business operations. Notice in the financial organization example above, how many aspects of the organization could be affected by AI. To successfully integrate AI, an organization must be “all in”. To do this requires particularly high levels of communication, investment, and cross-organizational participation.

A third challenge is that with successful adoption of AI, the requirement for personal growth and change is pervasive, up and down the organization. Leaders, engineers, and operators all need to learn and embrace the changes brought about by AI. For many, this can be an exciting opportunity for career growth and more fulfilling jobs. Others will mourn the lost relevance of hard-won experience. The organization must be prepared to invest in training, re-training, and professional development. The more AI takes over routine data gathering and analysis, the more important ‘soft’ skills will be to every worker.

Finally, a fourth challenge is that even a very capable AI can produce unintended results. For example, although AI-based analysis can lend objectivity to credit decisions, training AIs using historical data can promulgate past biases. Also, when highly-trained AIs encounter situations they have never seen before, the results can be unpredictable. This means AIs need human supervisors, and these supervisors are dealing with a whole new kind of employee!

Next: What kinds of jobs will AI impact?