Hey, Megan, let's do a podcast. Great idea. What

should we talk about?

One of the really, really fantastic things about

my job in the universe is that I get to meet all sorts

of really fantastic people and have really interesting

conversations. And I've decided to share those

conversations. So, Sam Rogers, welcome. It is

so exciting to finally get to have this

conversation — actually, I've been thinking about this conversation for the last few

months — and to dig into your

work and where you are and where you see things right now.

So can you start off by introducing yourself? Sure.

Yes. Hi, Megan. Thanks for having me. I'm Sam Rogers,

founder and CEO of PAICE dot Work. I also run Snap Synapse,

where I've spent over two decades helping organizations with their

learning management systems, creating learning content, more generally

making change stick. For years, I was the guy

that companies called when their learning ecosystem wasn't working right. And

have helped with migrations, integrations, workflow fixes, all that unglamorous

stuff that makes learning manageable at work.

Well, and I love the different directions at which

you come at this work, and a lot of those nuts and bolts,

Sam, make it actually happen in the world,

right? Like, it's one thing to design a program, but if you can't make it

happen in the world, it's like bears in the woods

and whatever bears are supposed to be doing in woods. So I,

you know, I also appreciate — the boring, nerdy

person in me sees the boring, nerdy person in you and actually making

these out. You and I met at Learning Dev camp years

and years ago, back

when we were just children, I'm sure. You were doing

a lot of video and learning design for video, and

you also share

this ecosystem framework, or

sense about you. And you were also one of the first LXD

people that I knew that went really deep into AI. Lots and lots of

people did, but you're one of the

ones who was very early in helping

people get comfortable with it, helping people

do smart things with it, right? And

although I will say that I remembered you made a playlist for

Learning Dev Camp that was so catchy — an

AI-generated playlist — it was so catchy that I caught myself two weeks later,

you know, just kind of humming something in the shower, and I'm like, where did

this song come from? Where did this song come from? Where did this song...? And

all of a sudden I started singing and realized it was the Learning Dev Camp

song. It came from AI.

Wow, that is creepy. So, but,

when you and I chatted at Dev Learn, right, you're

getting really strategic about this and looking

forward to the kinds of things that we will need

as an industry, as a society, as an ecosystem, not just

as learning designers, right? And it's around the assessment of an

individual's and an organization's actual capabilities

with AI — not just usage. So

how did you get to this point? How did you realize we needed this?

Well, for better or for worse, I never really think in terms

of like what L&D needs. I think more generally about,

you know, business needs. And every business

needs to use AI well, and of course, most

don't yet, right? Because it's new to the business workflow.

But they don't even really have the best way to

talk about, let alone measure, what good looks like

in this new context. So these are

problems that I'm solving with PAICE. I built

a measurement and workflow system to

make AI adoption teachable, governable,

real. Not just that people tried the tool, but that it's actually

producing outputs that people value. I

love it. I love it. Right, so it's not just, "did you use it? Did

you try it?" But "are you using it well?" I think that's fantastic.

Okay, so stop for a minute. Let's look

at PAICE dot work, and I want to start with the super basics. What does

PAICE dot work stand for? It's an acronym, right?

It is. It's P A I C E

and it stands for People plus AI Collaboration

Effectiveness. It's a framework basically for measuring

if people — how well people can work with AI safely,

repeatably, in a way that can be taught, managed, and governed. Here

AI usage. And that's a start. But really

I'm more interested in capability. That is the shift where

L&D becomes truly essential. I

actually use the acronym P A I C E in a couple

places. It's also the capability

measures of Performance, Accountability,

Integrity, Collaboration and Evolution that spells out

PAICE. I actually — I think I came up with that one first, but

there's also the scoring tiers and the products

and all that kind of thing. I maybe went a little too hard on the

branding, but having a common mnemonic is definitely helpful.

Absolutely. And I really like

— common to both of those is the word "collaboration," right?

And so I think that that's the key with seeing AI tools

as being a collaboration. It's a new kind of partner. I

hesitate to anthropomorphize it, but it is a

different kind of work. And so I'm really excited

about the collaboration, but at the same time, in order to implement tools, in order

to use tools well, the people need to collaborate and be able to

have different kinds of conversations as well. So I'm really digging the

collaboration space on this. Yeah, great, me too.

So typically, right when, when I talk to organizations and

we say, what are you measuring with

your AI tools? They'll say, "wait, wait, wait, wait, wait, wait!" It's too early to

be measuring results and business and all this stuff and everybody tries to, but there's

this conversation around, should we be measuring dollars? Which

is one piece of things, but we also look at, right, organizations

are asking their people, "are you using tools?" So self reporting tools usage.

Depending on your scale

in the organization, your tools actually keep track of who

uses them and for how long a session and how many minutes and how

many sessions a week and whatnot. That is only though

— it's notably — that's only on the approved and paid

for tools by the company. Not all the things that you might bring along

for the ride as well. People are tracking

completion of required training, right? "When we pushed out

the compliance training about AI, did you, did you take it?" We all know

what an effective learning metric that is. And then

several of my clients

are, you know, they're — how many of their people, how many of their client-facing

people have certified, you know, completed an AI

certification program or something like that, right? So still a measure

of activity, but kind of a

measure of "I have completed a

thing." What does PAICE dot work measure,

actually? Yeah, I'm glad

you brought up the training itself is a perfect

analogy here. So the old butts-in-seats metric, you know,

it doesn't tell us a whole lot. We know that.

We don't just want to show the activity-based checkboxes of attended,

and completed, and passed, and — all those things are needed, certainly. They're

just not enough. They never really were enough.

But they're just the first hurdle that we had to clear in any

kind of learning intervention. But very soon after we clear that bar,

the next one comes into focus, which is what difference does it make? So

just like with any learning intervention, AI can

make the kind of difference that the business is hungry for or it

can do something else that isn't that. What PACE

aims to do is quantify the risks of AI,

not on a technological basis, but on the human level.

So the shorthand I often give is, it's like FICO for

AI risk. That's a very US-centric term, I know, but

PAICE is global in scope. It's basically

helping the organization assess what are the risks

of giving these high-powered tools to our workforce.

I think that's really, really

so important. I remember an early conversation I had with Josh Cavalier,

and there's conversation about why are some

organizations adopting AI and why are so few doing it.

This was several years ago and we had this conversation around

organizational risk. These tools do things, these

tools have access to a lot of things. And the, the gateway

to that, by the way, are humans. But the,

because of the — the possibility of risk is so

great, both frequency and impact, that...

this is, this is different. This is different than rolling

out calculators. This is different than rolling out the internet. This is different.

So I think this is such important work.

You've written a white paper, and I've had a

chance to dig in and give a

round or two of thoughts on that, which was so much

fun. Thank you so much for that opportunity. There's a couple

of terms now we've got PAICE dot work covered. There's a couple terms that

I'd like for you, in this context, to

discuss and define a little bit more. One is, what do you mean by "AI

collaboration"? Yeah, well, well, first I just want to say again,

thank you, Megan, for your time and generosity on that version of the white

paper. I've since made some pretty substantial revisions that incorporate

the feedback that you and others have provided. The new version is much

shorter and introduces fewer terms that need definition. But, but

the "AI collaboration" one that you kind of foreshadowed there,

yeah. What do we mean when we talk about AI collaboration?

Because it's not people using AI, it's

the ability of a person or a team to reliably

produce good work with AI in the loop,

with its known limitations, with review checkpoints

with the required organizational

accountability. So in L&D terms, it's

a performance system. This isn't like skill badging or certifications.

It's a tool-agnostic means of aggregating

human intelligence and AI intelligence

to output work.

Love it. Love it. Okay, so thinking about

that then, what do we mean by "governance readiness"?

Yeah, we've seen a lot about AI

readiness in general and well, "ready or not, here it comes." At

PAICE, we refer more to governance readiness, which

is the organization's ability to scale AI without all

the confetti. I'm not opposed to confetti and celebrating things; I've

been known to throw some confetti on occasion. But this

version of readiness includes

guardrails, it includes review cycles, it includes escalation

paths, ownership; it's hard to take readiness

as a term seriously with those elements

missing. So that's what we mean when we talk about

governance readiness. I kind of like that.

So a lot of times "readiness" is in a conversation around literacy and

individuals' skill sets, or

change capacity, or interest and

willingness, right? "Readiness," "willingness," right? But governance is

a word nobody really, or very few

people, really get excited about. But governance is

what we need to make sure that we're doing this in a sane and

orderly way that is safe and effective for the

organization overall. The readiness to have that

governance infrastructure, I think is just at a level above both of

those. So I kind of love that a lot.

Great. Okay, so behavioral

observation is the next thing I want you to dig in

because PAICE is not a multiple choice test.

That's right, yeah. So, we can

— we're very familiar in L&D with testing what people know about

something. That doesn't mean that they're going to do anything

or that their knowledge would be predictive of what they would do

or how people feel about something, you know, with a sentiment survey or something like

that. The reason I made PAICE is

because those things don't "matter harder"

when it comes to AI. Like there's the

conversation we have in L&D all the time about trying to get closer

to the things that actually matter and what is actually predictive

of behavior. But the best thing to observe is behavior

itself. So that's what PAICE does.

That's what it's built to do. And it's not,

it's not asking you what you know about AI. Which is great because

you don't actually have to know a lot about AI in order

to use AI well, just like you don't have to be a great

mechanic to be a great driver, right? Like I

want to get this thing where I want to go. The ability

to do that is really what PAICE is focused on, of

watching that. In the same way that you take a driver's

test; there's the written part of the test. Yes, you have to know

stuff. But then someone sits in a car with you and they take you on

a route that is unpredictable, that maybe you've never been on

before. And they're watching how you perform

in that circumstance and giving you your

license based on how you perform against that

rubric. That's the best analogy for what PAICE is

doing by watching how people use AI in

this kind of simulation circumstance and

then being able to provide very specific

feedback and ratings based on that assessment.

I love this. And actually the way in which

you're doing this with PAICE dot work has become my new example

of when I'm, when I'm describing Will

Talheimer's Learning Transfer Evaluation model. And people are like, "well, what's the difference between

a tier 6 decision competence and a

tier 7 — sorry, tier 5 decision competence, 6 task competence" — like

task competetence. We actually watch somebody do it.

And this is a great example by the way.

So super, super cool. Now, but here's the thing; as I

think back to my own driver's test and what a nerve-wracking experience that was,

but that gave me the ticket to do something, right? It literally gave me the

keys, right? Or license to drive.

This kind of measurement is a

bold and maybe challenging move from a

measurement perspective, right? I can imagine it makes some people

uncomfortable to be assessed this way, particularly

if they like to think of themselves as having great skills. Everybody is an above

average driver, right? By some miracle of metrics,

everyone's above average. Yeah, everybody is above — like, "I'm a good driver, of course!"

And what are

some of the change components that an organization should

be thinking about if they are looking

at something like PAICE dot work? Well, the

first thing to say is that I built

PAICE dot work on Privacy by Design principles. So

as far as like the person who sat in the car with you and

your nerve-wracking experience of your driver's test, PAICE

is built to scale

that kind of individual assessment and

observation, but it's scaling it using AI, so

there's not actually a person watching. So when you go through the

assessment, no one knows that you did. No one

sees your score unless you choose to show it off. Teams

and organizations can still see that their people did something in

aggregate. They can still see any gaps and dangers that exist

at a higher-level view, which is really what they want anyway.

Well, most of the time, I should say. Anyone who's been responsible

for compliance reporting knows that there's

inevitably a persecution motive that develops within the

organization of, you know, "let's fire all the low

scores" or the, you know, all that kind of thing that happens. I've designed

PAICE to be immune to that. So not even I

know how specific individuals are scoring. I just

see that anonymized individuals are getting their scores.

That way I can't be coerced into helping fire people. You know, the

platform can't be used that way. But

eventually PAICE dot work will probably have some competition.

I'm happy to let the competition take that

legally-riddled and misaligned work. Our mission is

to enable safer and more effective

people-plus-AI collaboration by providing

independent capability measurement.

We're also a public benefit corporation, so we're fully

committed to scaling a profitable company, but we're structured so that we

never have to choose between profit and principle.

So that's important for people to know: You're not being

observed in that traditional sense. This is a new capability that AI

opens up that has never really existed before. And it's also important for

organizations to know that what you're getting is a view

of your people, not a view of individuals,

like in a learning management system or something. Okay. And

that's, that's perhaps different than what people are expecting from

L&D or from their organizations. So probably an important message.

I will find the link and add it to

this podcast when we publish it, but there was an interesting piece

of research that was done with one of the large recruiting

firms and they conducted a trial in which

they allowed applicants to either interview with a

human interviewer, an AI interviewer — actually,

they didn't — group one had to interview with a human.

Group two had to interview with an AI. And group three

got the choice. And there was some interesting,

interesting findings that came out. One was that people who

interviewed with the AI tools actually scored higher.

Humans did the scoring. It was just the interview was

with the AI humans did the scoring. And so some of

their thinking was that it freed people up. They didn't feel quite so, so

judged. And so that's interesting.

What they also found in that third group where they allowed people to choose

was that women and minorities

tended to choose the AI more frequently. Yeah. And

their thinking behind this and some of their follow up

was that they were feeling like they would be more assessed as an

individual and not based on their social identity

characteristics. I thought, "mind blowing." Yeah. This is a tool for

equity and participation.

Yes. And they're probably right, is my

instinct. The studies have yet to be done,

but AI can amplify whatever it is that we

want to focus on and use it for. So one of the big worries is

certainly security and privacy

and all those kinds of things. I've directed PACE to use

AI to make it impossible to deconstruct

an individual's identity from what it is that's being gathered. Like if we

aim it at Privacy by Design, it stays private. If we

aim it towards equity, it works great for that. When we aim

towards accessibility, it works great for that too.

It's a matter of how we drive that AI output to

where it is that we choose to go, where we want to go. So

I get a little meta about that and I'm doing my best to

practice what we're preaching with PAICE.

I love it. I absolutely love it. Okay,

this has been fascinating, but I want to take a

slight shift here. I am always

curious about people's work quirks.

What are productivity habits — things that make you either

super effective or super annoying to the people around? The

variety is always interesting and surprising. I always learn something more about people that

I didn't know before. Do you have any favorite habits

or tips? Your favorite fidget, your favorite distraction?

So I'm probably a pretty quirky worker. I get along well in

anybody's office. But 25 years ago I was the only person with three

computers on my desk. One development box, one kind of

handicapped baseline computer for testing, my own personal laptop.

Today I'm still doing something like that. Three computers on

my desk now, but I'm also working across

ChatGPT, Claude, Gemini, Perplexity, usually a local model like

Deepseek, all at once. And I find myself

serving as the human transport layer between them all very

often. I have my own Obsidian vault, which I

am kind of obsessive over. That's where text and

all the other files land. I'm like Markdown native,

so anything that's coming to me, I'm

often like porting in and out of Markdown,

which is just a basic text format. And I

do all of my, all of my jotting still in a paper

notebook, I think, which is... which is probably a

quirk at this point. I use mine constantly because just of the

quality of thinking that comes out is just better for certain things.

What I don't use unless I have to

is mobile devices. I was one of the first independent

iOS developers long ago. I taught my smartphone cinema workshops

for years. I still believe in the promise, but I can't ignore the history

since then. So, yeah, I would

say don't text me stuff so much. I'm

not the guy that wants to do all of that on my phone.

And I would encourage anybody who wants to create things that matter to stop

consuming things that you don't... don't like through your phone.

The costs outweigh the benefits, I think. So

I'm always the one trying to find a better way to like

not go through the phone for everything. Does that count?

It totally counts. Maybe not what one would have expected from somebody

who's got three computers running all the time. So I love it. I love it,

love it, love it. Sam, thank you. Thank you so

much for hanging out with me for a few minutes. Thank you so much for

the work that you have done and are doing

for the industry. And I will, I am

sure I will bump into you at some interesting conference or some interesting

conversation again in the future. So thank you so much. Yeah, always

a pleasure, Megan. Thanks so much.

So how'd that go? Megan? That was so much fun.

I've known Sam for years and just really

appreciate — he's always surprising, right?

So he's this kind of easygoing, unassuming guy

and then he starts talking and you're thinking, oh my gosh, he

is thinking about things at a very deep level. And I appreciate

that so much about him. And he's so willing to share and make it

practical with other people. And that's something really

awesome about him. He came up to me at DevLearn and literally

gave me a paper print out of his white paper

and asked for feedback. And I thought, well, that's

cool. That gives me something to do on the plane. And so old school.

Like he was talking about on paper and pen. Old school. I go out and

I'm like scribbling in my thoughts and I'm drawing pictures

and, "what do you mean by this?" And "you've got two of those." And "I'm

kind of confused here," but "this is really cool. You got to amplify this."

And then I had these, like, 18 pages

of handwritten notes, scribbles all over, like, how do I

get... do I put this in the mail? Do I like, what

do I do? So no kidding. I filmed it with my

phone. I like, took my phone,

and of course after Sam said he doesn't like to use phones. But anyways,

the the video camera in my phone and I talked

through all of my notes, and it was probably the weirdest way to give

feedback, but worked,

so. And it was a fun paper to dig into. So

it was great to have this conversation. It's cool to see the iteration and the

evolution of the work that he's doing. I know you've got

one more thing to share, Megan. I myself am really

eager to get in and try the tool out,

both as an individual and maybe as an organization.

And I think that this kind of thing

is something we'll see, in the coming years, ore and more of. Sam

mentioned he'll have competitors. He absolutely will have competitors

for this. But what I love is that this first one

is like a research project project. It is

an amplifier for us. So super cool stuff.

I love my job. This is Meg Fairchild and Megan

Torrance and this has been a podcast from

TorranceLearning. Tangents is the official podcast of

Torrance Learning (as though we have an unofficial one). Tangents

is hosted by Meg Fairchild and Megan Torrance. It's

produced by Dean Castle Castile and Meg Fairchild, engineered and

edited by Dean Castile, with original music also

by Dean Castile. This episode was fact checked

by Meg Fairchild.