AI in the Real-World: For Better and Worse

logo

AI in the Real-World: For Better and Worse

Use spacebar / arrows to advance; Esc for overview; s for notes.

A survey of the AI landscape in user assistance content in 2025-ish

Presented by

Lief Erickson

lief.erickson@intuitivestack.io

www.intuitivestack.io

2 Feb. 2026

Can a submarine swim?

Can a submarine swim?

Worse

When did it last snow in Minneapolis?

Sometimes it snows in April

 

Egg on face

Alt text for images

Agent

  • Microsoft Style Guide

  • WCAG guidelines

  • Company-specific rules

Benefits

  • Improved accessibility for all users

  • Consistent, literal descriptions across images

  • Faster publishing without adding review burden

  • Scales with content growth and reuse

Photo of a dog wearing sunglasses

Why AI Projects Fail

Goal is too broad.

Let’s build an AI that answers all customer questions

Assuming AI can fix bad content and bad data

The AI will clean it up as it learns

Full-text search + embeddings = “semantic”

We don’t need taxonomies or ontologies. The LLM will find the connections automatically

Vector databases = understanding

Embedding the content captures its meaning, so we don’t need metadata or structure.

Ignoring governance (until something goes wrong)

We don’t want to slow things down with policies or review boards.

Trying to corral chaos

You're making me deal with questions I'd rather avoid.

Better

How Alkemi Technology uses GPT Agents

AI doesn’t replace the work of a technical writer. It extends it. The real value comes from how we guide it and how rigorously we validate outcomes at every step.

-Tara English-Sweeney

User Story Architect GPT: Reverse-engineered product documentation into user stories we didn’t have.

Document Breakdown Assistant: Drafted overview topics in a specified format.

Short Description Creator: Created concise overviews.

Style Guide Assistant: Checked tone and consistency.

Transcript > SME review > refine into article

And many more automate-the-boring-stuff agents

How Mintlify Uses Claude

Looked at personal workflow and decided what to offload to Claude.

LMMs help non-writers to write better.

Motto to team: You are smarter than Claude.

Detailed prompt instructions in Claude.md.

Breaking any rule is failure.

Document new features: Claude can examine a Git diff and describe what is being changed.

Claude summarizes changes >> Claude Verify >> Propose changes to docs. Tip: Keep context window small. Not wholesale changes, but focused updates.

Claude is good at the mechanics but not at seeing the strategy behind the content.

CT Smith at Payabli: 10x Efficient

SEO audit to find existing/missing frontmatter and metadata (3 hours, $15, Claude Code to update)

For making voice notes outlines, proofreading, and brainstorming

Vibe code custom React components for internal processes. “A livesaver.

Generate monthly reports for management

AI chatbots aren't a magic spell that compensates for crappy content.

-CT Smith

Best Available Human

Study of 5,172 support agents at a Fortune 500 firm

+14% productivity overall (issues resolved/hour)

Biggest gains for newer or lower-performing agents: +34%

Strongest impact in moderately rare situations where agents lack experience but data exists

If top performers stop contributing new approaches

Brynjolfsson, E., The Quarterly Journal of Economics (May 2025)

Not as good as a true expert

Better than trainees or novices

GenAI often performs as well as or better than the best available human, even if it falls far short of elite expertise

GenAI is a proxy for a tutor, mentor, or laziness

Not automation, but access to top-performer judgment

How I use AI today

Personas for “interviews”

Notetaker

Code mentor

Voice and tone checker

Image drafter

Disc golf coach

Connect with me

End

logo

www.intuitivestack.io