11-695: ML in Production. Christian Kaestner & Eunsuk Kang
Human-AI Interaction
Eunsuk Kang
Required reading:
Building Intelligent Systems by Geoff Hulten (2018), Chapter 8.
Guidelines for Human-AI Interaction. Saleema Amershi, et al., in CHI 2019.
Optional reading:
Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-user Expectations of AI Systems. Kocielnik, et al., in CHI 2019
Learning Goals
- Understand the risks of poor interaction design
- Understand the challenges behind designing human-AI interactions
- Understand the basic elements of user interaction design
- Consider design considerations for AI-based systems
- Modes of interaction: Automate or augment?
- Mental model: User understanding of what AI is doing
- Dealing with errors: Guide user towards prevention & recovery
- Feedback and control: Align user feedback with AI improvement
What's coming next
Risks of Poor Interaction Design
Poor Interaction Design Confuses Users
Poor Interaction Design Annoys Users
Poor Interaction Design Causes Harm
Poor Interaction Design Causes Harm
Poor Interaction Design Causes Harm
- Radiation therapy system at Panama City public hospital (2001)
- Therapist draws block shapes to determine treatment area
- Software computes final radiation settings
Poor Interaction Design Causes Harm
- Same shape drawn in different order, double the radiation dose
- 28 patients overdosed; 8 dead
- Therapists charged with 2nd degree murder (but are they really to blame?)
Risks of Poor Interaction Design
- Interaction design is not just about visual presentation!
- Poor interaction design can:
- Cause confusion or misunderstanding
- Prevent the user from effectively performing their task
- Increase mental and physical burden
- Drive users away from the product
- Contribute to security or privacy issues
- Cause physical (injuries, deaths) and societal harms (bias, misrepresentation)
Usability Concepts
(This will be a brief tour to a complex subject. If you are interested, consider taking 05-318/618: Human-AI Interaction)
Dimensions of Usability
- Learnability: How easy is it for users to accomplish tasks the first time?
- Efficiency: After learning, how efficiently can users perform the tasks?
- Memorability: Can users remember to perform the tasks after a period of not using the system?
- Errors: How often do users make errors, how severe are these errors, and how easily can they recover from the errors?
- Satisfaction: How pleasant is it to use the design?
Interaction Cost
- Mental and physical effort needed to perform a desired task
- Task memorization & recall, context switch, track system state
- Reading, scrolling, clicking, typing, waiting for UI changes
- Goal of usable design: Minimize interaction cost while allowing users to perform their tasks
Usability & AI
- AI has potential to greatly reduce interaction costs
- Automate tasks through personalization & predictions
- But also introduces new usability challenges
- Q. What's new or hard about AI-based systems?
Usability & AI
- AI has potential to greatly reduce interaction costs
- Automate tasks through personalization & predictions
- But also introduces new usability challenges
- Unpredictability: AI makes mistakes, sometimes unexpectedly
- Opaqueness: User has difficulty understanding how system works
- Evolution: AI behavior changes over time, surprising users
Design Considerations for AI
- Modes of interaction: Automate or augment?
- Mental model: User understanding of what AI is doing
- Dealing with errors: Guide user towards prevention & recovery
- Feedback and control: Align user feedback with AI improvement
Modes of Interaction
Modes of Interaction
- Automate: Take action on user's behalf
- Augment: Provide options or additional information
- Prompt: Ask the user if an action should be taken
- Organize: Display a set of items in an order
- Annotate: Add information to a display
- Hybrid of above
Selecting Appropriate Mode of Interaction
- Identify the tasks that the user wants to achieve
- For each task, decide between automate vs. augment
- Automate when:
- User lacks knowledge/ability to perform the task (e.g., prediction)
- Boring, repetitive, dangerous tasks
- The effect of action can be reversed
- Augment when:
- High stakes & accountability is needed
- Difficult to communicate the user's need to AI
- User enjoys performing the task (e.g., driving)
Other Factors to Consider
- Forcefulness: How strongly to encourage taking an action?
- Active: Automate action or interrupt user and ask for confirmation
- Passive: Suggest action, but do not require immediate answer
- Frequency: How often does interaction occur?
- When a new prediction is available or model changes
- Periodically (e.g., suggest action every hour)
- Only when explicitly initiated by user
- Cost: What is the effect of a wrong prediction?
- If cost is too high, consider augmenting rather than automating
- If possible, provide a way to undo the action of AI
Example: Design suggestions in PowerPoint
- Automate or Augment? Why?
- Forcefulness? (active vs. passive)
- Frequency?
Example: Fall Detection
- Automate or Augment? Why?
- Forcefulness? (active vs. passive)
- Frequency?
Mental Model
Mental Model
- What the user believes about the system
- "How does the system work? How does it respond to my actions?"
- User plans actions and reacts to system based on this mental model
- Challenge: Aligning system with the user's mental model
- Inherent mismatch between user's & designer's models
- User's model may be preconceived based on prior experience
- User's model and/or system evolves over time
Example: Shopping Cart Checkout
Mental model for shopping cart = A linear sequence of familiar steps
- Browse for items
- Add items to cart
- Choose checkout
- Enter shipping & billing data
- Press Order
- Get confirmation
Breaking Mental Model
- Anti-pattern: Interrupt linear flow & bring user back to a
previous step
- Create an account, open a new dialog to enter preferred address...
- Breaks user's mental model => failure to convert into sales
- ~60% of customers abandon their shopping cart
Mental Model for AI-Based Systems
- User: "What is AI doing, and how do I use it?"
- Opaqueness: Typically less transparent than traditional apps
- AI will make mistakes, often unpredictably
- Unclear inputs: What are possible actions? Which of these actions matter? When does my action take effect?
- Lack of control over output: Why am I being given these recommendations? Why is the output displayed in this order?
- Lack of trust over output: How do I know the output is correct?
Mental Model for Voice Assistants?
Q. Can you describe what it does? What it cannot do?
Mental Model for Voice Assistants?
- Unclear, inconsistent mental model
- An interface for other services?
- "Handy helper"?
- Knowledge repository? Fact-finding tool?
Misalignment in Voice Assistants
- AI often fails to meet user expectations
- (1) User doesn't know how to get AI to do X
- (2) User says X, but AI can't do X well
- Users settle on simple tasks over time; small but limited improvements
Misalignment in Mental Models
“So, this week, I realized that I don't use my IA nearly as much as I thought I did. I do use it often. However it's very much normally the same like five things over and over again."
- User settles on a suboptimal mental model & fails to benefit from the full capabilities of AI
Principles for Aligning Mental Model
- Identify user's existing mental models
- Find similar apps & identify common patterns
- User interviews, walkthroughs, prototype testing
- Design & evolve the system to conform to the user's model
- Collect & analyze errors made by user
- Identify potential mismatch vs. user's mental model
- Improve/adjust the user's mental model
- Set the user's expectations through onboarding
- Increase transparency and explain decisions made by AI
- Allow user to adjust system behavior to match their expectations
Onboarding: Set User's Mental Model
- Provide examples of how the system works
Onboarding: Set User's Mental Model
- Be explicit about what system can and cannot do
Transparency: Explain How Decisions Are Made
- Explain how the user's input actions influence output
Dealing with Errors
Dealing with Errors
- User errors: Mistakes made by users (e.g., click on a wrong button)
- Lots of work in cognitive science & human factors
- Error taxonomies, human performance modeling, task analysis, ergonomic analysis, etc.,
- Often due to misalignment of mental models
- System errors: Failure to provide an outcome expected by the user
- Due to mistakes made by an ML model
- Our focus in this lecture
Example: Scheduling Assistant
- Analyze e-mail content for possible meeting scheduling
- Suggest creating a new meeting based on inferred information
Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-user Expectations of AI Systems. Kocielnik, et al. (CHI 2019)
Dealing with Errors in ML
- Define types of errors & their costs
- False positives vs. false negatives
- Optimize for one with lower costs
- Q. For meeting scheduling, which are more acceptable?
- Detect & record occurrences of errors
- Collect telemetry or user feedback
- Q. Telemetry to collect for meeting scheduler?
- Identify sources of errors
- Poor/bias training data, noise in data, data drifts
- Provide meaningful error messages to the user
- Provide an explanation for the error
- Suggest actions to fix the error (e.g., "Edit details" option)
- Give user controls to recover from and mitigate the effect of an error
- e.g., delete or modify incorrect meeting schedule
Setting User Expectations for ML Errors
- Be upfront about how well the system performs (e.g., model accuracy)
- Temper the user's expectations and avoid surprises
Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-user Expectations of AI Systems. Kocielnik, et. al. (CHI 2019)
Error Messages: Suggest user actions
- Tell the user what the AI needs in order to behave as intended
- Guide the user towards ways to recover from/prevent further errors
Breakout: Dealing with Errors
Design suggestions/fall detection
- In #lecture, type:
- Possible error(s):
- How to detect the error:
- How to allow the user to recover from error:
- What additional data to collect (from user) to reduce future errors:
Feedback and Control
Types of Feedback
- Implicit feedback: Data about user behaviors collected by system
- e.g., times of day, duration of usage, recommendations accepted/rejected, click patterns, etc.,
- Explicit feedback: Prompted or deliberately provided by user
- Surveys, ratings, thumbs up, feedback forms, etc.,
- Design considerations for feedback
- Align feedback with improving interactions (and AI)
- Acknowledge user feedback & respond immediately
- In addition to feedback, provide a way for user to adjust AI behavior
Responding to Feedback
- When possible, respond to feedback with an adjustment to AI behavior
Giving User Control
- Provide a mechanism for user to adjust system behavior
Giving User Control over ML Behavior
- Provide a mechanism for the user to control the types of ML errors
- Scheduling assistant: Adjust thresholds to achieve trade-offs between precision vs recall
Guidelines for Human-AI Interactions
Human-AI Interactions
Human-AI interactions must be considered throughout the entire ML lifecycle!
- Requirements & design
- Understand user needs & their mental models
- Explicitly design system to match the mental model
- During interaction
- Consider factors for interaction (automate vs augment, forcefulness, frequency)
- When errors occur
- Provide an explanation & actionable information
- Provide ways for user to adjust AI behavior
- Maintenance and evolution
- Collect user feedback and improve model
- Adjust system design to reduce mental model mismatch
Summary
- Goal of usable design: Minimize interaction cost
- Automation does not necessarily imply reduced cost!
- Interaction design considerations for AI
- Modes of interaction: Automate or augment?
- Mental model: User understanding of what AI is doing
- Dealing with errors: Guide user towards prevention & recovery
- Feedback and control: Align user feedback with AI improvement
Human-AI Interaction
Eunsuk Kang
Required reading:
Building Intelligent Systems by Geoff Hulten (2018),
Chapter 8.
Guidelines for Human-AI Interaction. Saleema Amershi, et al., in CHI 2019.
Optional reading:
Will You Accept an Imperfect AI? Exploring Designs for Adjusting
End-user Expectations of AI Systems. Kocielnik, et al., in CHI 2019
