Dr. Lia DiBello is the Chief Science Officer of ACSILabs, Inc, which makes the FutureView Platform — a virtual reality training platform used by the US military and by certain large businesses to accelerate expertise. I last talked to Lia about her groundbreaking work explicating the mental model of business expertise, and in fact created this podcast to interview her. She’s back today to talk about her use of AI to accelerate expertise.

Neil Sahota is the CEO of ACSILabs. Neil was part of the original IBM Watson team that won the Jeopardy challenge. After that challenge, he was, amongst things, the person responsible for IBM’s Watson ecosystem strategy, and an IBM Master Inventor (which is a designation for IBM employees who have made outstanding contributions to the IP creation process at IBM). Since leaving Big Blue, Neil has been Artificial Intelligence Advisor to the United Nations, published a book about AI in 2019, and started the UN’s AI For Good Initiative, which is currently hosted under the International Telecommunications Union.

In this episode, we take a human-first perspective on using and deploying AI in real world business and military contexts. We discuss Lia and Neil’s history with AI, talk about good and bad implementations of AI they’ve seen in real-world environments, and discuss what it means to get more folks to an expert-level at human-AI symbiosis.

Shownotes

ASCI Labs — https://acsilabs.org/

ASCI Labs in the Navy STP marketplace — https://acsilabs.org/acsi-labs-has-met-the-requirements-to-be-listed-in-the-navy-stp-virtual-transition-marketplace/

AIQ (Artificial Intelligence Quotient): Helping People Get Smart about the Smart Machines They Are Using — https://medium.com/about-work/helping-people-get-smart-about-smart-machines-they-are-using-f9e0095846fe

AIQ: Artificial Intelligence Quotient — https://www.psychologytoday.com/us/blog/seeing-what-others-dont/202007/aiq-artificial-intelligence-quotient

Hoffman, R.R., Mueller, S.T., Klein, G., & Litman, J. (2018). "Metrics for Explainable AI: Challenges and Prospects." Technical Report, Explainable AI Program, DARPA, Washington, DC. — https://arxiv.org/abs/1812.04608

Timestamps

• (00:00) - Lia DeBello and Neil Sahota on Human-AI Symbiosis
• (00:07) - Introduction
• (02:08) - Using AI for Expertise Acceleration
• (17:42) - Good Applications of AI Historically
• (23:20) - Mistakes With Applying AI
• (28:09) - Watching Experts Use AI
• (30:45) - AI Systems as Extensions of Human Expertise
• (35:04) - Why Won't AI Replace Humans?
• (41:55) - How To Train Junior Folks For AI Symbiosis
• (45:42) - Teaching Creative Thinking With AI
• (50:52) - Moving Beyond Chatbots
• (57:55) - Adapting Company Processes to New Tech
• (01:00:00) - How Should AI Systems Designers Think About Affordances
• (01:03:51) - Possible to Design Normal Business Processes for Expertise Acceleration?
• (01:08:09) - What Should Expertise Researchers Learn from AI Researchers?
• (01:09:49) - What Should AI Researchers Learn From Expertise Researchers
• (01:14:43) - ACSI Labs in the Navy STP Marketplace
• (01:20:00) - Where Folks Can Find Neil and Lia

Stan Slap is the founder and president of SLAP, an international consulting firm that helps organisations get maximum commitment from their manager, employee, and customer cultures. SLAP has worked with the who's-who of the Fortune 500, companies that you've heard of like Costco, Oracle, HP, Caterpillar, Microsoft, Amazon, General Electric, Intel, Viacom, Tesla, Glassdoor, Warner Music Group — and some companies that you haven't; all told, SLAP has worked with companies doing business in 44 countries. Stan is also the author of two books: Under The Hood and Bury My Heart in Conference Room B.

This podcast dives into Slap's ideas about understanding, managing and shaping effective employee cultures. His model for culture is the first coherent model that I've found that is useful and tested at scale; it is therefore more powerful than most other formulations of employee culture that you'll find.

SLAP Company — https://slapcompany.com/
Under The Hood — https://www.goodreads.com/book/show/22571574-under-the-hood
Commoncog summary of Under The Hood — https://commoncog.com/under-the-hood/
Bury My Heart at Conference Room B — https://www.goodreads.com/book/show/7810760-bury-my-heart-at-conference-room-b

• (00:00) - Introduction
• (02:32) - Employee Culture is About Business Performance
• (06:56) - What a Culture Is
• (10:21) - Thinking About Culture as an Organism
• (19:40) - What Management Gets Wrong
• (22:56) - What To Do When You've Lost The Culture's Trust
• (26:36) - Tension Between Managing Individuals vs Managing Culture
• (30:09) - Giving a Culture Energy
• (35:16) - Introducing Change to a Culture
• (42:28) - Doing Cultural Understanding at Scale
• (50:48) - Four Step Process for Cultural Commitment
• (01:00:04) - Danger of Corporate Values
• (01:04:29) - Managing Culture in a Rapidly Scaling Company
• (01:10:51) - Dealing with Bad Subcultures
• (01:18:14) - Stan's Closing Message
• (01:25:49) - Where Folks Can Find Stan

Colin Bryar joined Amazon really early in its life and spent twelve years as part of Amazon's senior leadership team.

For two of those years he was 'Technical Assistant' to Jeff Bezos, as known as 'Jeff's shadow', during which he spent each day attending meetings, traveling with, and discussing business and life with Jeff. After Amazon, he and his family relocated to Singapore for two years where Colin served as Chief Operating Officer of e-commerce company RedMart, which was subsequently sold to Alibaba. Along with his ex-Amazonian colleague Bill Carr, Colin is co-author of Working Backwards, a book on an insider's look at how Amazon works. Bill and Colin are co-founders of Working Backwards LLC, where they coach executives at both large and early-stage companies on how to implement the management practices developed at Amazon.

This podcast is a really deep dive into the practice of the Amazon Weekly Business Review, which remains to this day one of Amazon's secret operating weapons, and a big part of what makes for a great operator.

Working Backwards, book: https://www.goodreads.com/en/book/show/53138083
Working Backwards https://www.workingbackwards.com/
Commoncog summary of Working Backwards: https://commoncog.com/working-backwards/

• (00:00) - Introduction
• (01:24) - Colin's Background
• (04:00) - Joining Amazon
• (06:31) - The Data Situation in Early Amazon
• (08:22) - Being Jeff Bezos's Shadow
• (10:37) - Living in Singapore
• (12:07) - Writing Working Backwards with Bill Carr
• (15:17) - The History of the Weekly Business Review
• (17:50) - How the Amazon WBR is different
• (20:45) - Customer Experience Metrics vs Business Metrics
• (22:33) - Controllable Input Metrics vs Output Metrics
• (30:39) - What a Typical WBR Looks Like
• (34:22) - Why Glancing at Metrics is Important
• (35:43) - What kinds of discussions should you have in the WBR?
• (37:30) - Understanding Variation
• (41:28) - Stories About Figuring Out Controllable Input Metrics
• (48:19) - Applying the WBR to internal business functions
• (49:49) - Introducing the WBR to a New Company
• (55:55) - Applying the WBR to New Products
• (01:01:04) - Not Using Surveys as Primary Research on Customers
• (01:04:17) - What Makes for a Good Operator?
• (01:05:23) - Operating Cadence
• (01:07:05) - What Colin Wishes All Operators Knew Tomorrow
• (01:07:57) - What You'd Wish You'd Known
• (01:09:50) - Would Many of These Lessons Apply to Early Stage Startups?