WEBVTT

1
00:00:00.000 --> 00:00:04.000
Okay. Portugal. Brazil. Hey.

2
00:00:04.000 --> 00:00:10.000
I'm from India. Iowa, Poland.

3
00:00:10.000 --> 00:00:14.000
Love it. Very cool.

4
00:00:14.000 --> 00:00:20.000
We just went live on LinkedIn and Twitter. So, I want to welcome everyone.

5
00:00:20.000 --> 00:00:24.000
This is going to be really exciting.

6
00:00:24.000 --> 00:00:30.000
So, we have already a bunch of activity in the chat.

7
00:00:30.000 --> 00:00:36.000
Hey, Kenan, from Istanbul. Yeah. So, this is really cool.

8
00:00:36.000 --> 00:00:46.000
We'll give it maybe another minute or so as people find this event,

9
00:00:46.000 --> 00:00:49.000
potentially, like through Luma.

10
00:00:49.000 --> 00:00:57.000
But, yeah, a few hundred people, actually over a few hundred people registered.

11
00:00:57.000 --> 00:01:04.000
So, we're deeply, deeply appreciative and super excited.

12
00:01:04.000 --> 00:01:09.000
Hey, Eduardo from Mexico.

13
00:01:09.000 --> 00:01:13.000
Ruchik C.A. is at California.

14
00:01:13.000 --> 00:01:17.000
Asha is in New York. Excellent.

15
00:01:17.000 --> 00:01:23.000
That's so cool. If you just joined, we're going to get started.

16
00:01:23.000 --> 00:01:29.000
But, please continue to say hi. Gabriel is in Portugal.

17
00:01:29.000 --> 00:01:33.000
And, we'll just kick it off.

18
00:01:33.000 --> 00:01:41.000
So, what I want to do, actually, that we've never done before.

19
00:01:41.000 --> 00:01:45.000
So, this is also pretty exciting.

20
00:01:45.000 --> 00:01:52.000
Is, I'm curious, what is your role at your company?

21
00:01:52.000 --> 00:01:56.000
So, I'm a developer relations engineer.

22
00:01:56.000 --> 00:02:02.000
Jerel is staff engineer. Yep. An engineer.

23
00:02:02.000 --> 00:02:06.000
We'll get to intros here in a second. Yeah.

24
00:02:06.000 --> 00:02:11.000
But, it helps us to know who's in the room with us.

25
00:02:11.000 --> 00:02:16.000
So, yeah. Very cool. Yeah. Very cool.

26
00:02:16.000 --> 00:02:20.000
And then, I'll just leave this poll up as we kick it off.

27
00:02:20.000 --> 00:02:25.000
So, we can start off with a quick agenda.

28
00:02:25.000 --> 00:02:31.000
What we're going to cover today is functional copilots.

29
00:02:31.000 --> 00:02:33.000
Really, how to make them work.

30
00:02:33.000 --> 00:02:49.000
And so, you may have already been tasked with, you know, you've got basically the mandate from above.

31
00:02:49.000 --> 00:02:53.000
Or, maybe you are in the, I see some people are in leadership.

32
00:02:53.000 --> 00:03:06.000
So, maybe it was your mandate to, let's add copilots and workflows into our production applications.

33
00:03:06.000 --> 00:03:21.000
And so, it's not a new story, but we're going to talk about how to build them, how to make them reliable, and then some really key insights.

34
00:03:21.000 --> 00:03:32.000
We'll save a few key insights till the end, where we'll talk about how to get your agent to learn.

35
00:03:32.000 --> 00:03:37.000
And so, I, next slide please, Jerel.

36
00:03:37.000 --> 00:03:41.000
So, some quick intros. I'm Nathan Tarbert.

37
00:03:41.000 --> 00:03:46.000
I'm the Senior Developer Relations Engineer here at CopilotKit.

38
00:03:46.000 --> 00:03:56.000
And I spend most of my time meeting with our users, teams, and customers.

39
00:03:56.000 --> 00:04:10.000
My goal is to be the person right in the middle between you building with CopilotKit and our engineering and product department.

40
00:04:10.000 --> 00:04:13.000
If you run into blockers, I'm happy to help.

41
00:04:13.000 --> 00:04:17.000
And then, I love bringing solutions back.

42
00:04:17.000 --> 00:04:20.000
So, my goal is to make you successful.

43
00:04:20.000 --> 00:04:26.000
I am joined by Jerel, who I will pass it off to.

44
00:04:26.000 --> 00:04:30.000
And I'll let you give your own intro, Jerel. Let people know what you've been working on.

45
00:04:30.000 --> 00:04:35.000
Sure. Hi, everyone. I'm Jerel. I'm a Staff Applied AI Engineer at CopilotKit.

46
00:04:35.000 --> 00:04:38.000
And I'm the guy responsible for OpenMuse, OpenDots, and OpenDuet.

47
00:04:38.000 --> 00:04:43.000
We've just seen some of our open source applications out there.

48
00:04:43.000 --> 00:04:47.000
I'm primarily the guy behind the community-maintained projects.

49
00:04:47.000 --> 00:04:52.000
At the same time, we also have OpenBot, which is a key part of life.

50
00:04:52.000 --> 00:04:54.000
We're going to talk about that in a bit later.

51
00:04:54.000 --> 00:04:59.000
Basically, my job is to help show and apply AI in different use cases.

52
00:04:59.000 --> 00:05:04.000
That's why I'm super passionate about what we're going to talk about today, which is functional agents.

53
00:05:04.000 --> 00:05:10.000
And I'll get back to that again later on. Back to you, Nathan. Thanks, Jerel.

54
00:05:10.000 --> 00:05:12.000
Very cool.

55
00:05:12.000 --> 00:05:18.000
So, for people who are new here, maybe you aren't aware.

56
00:05:18.000 --> 00:05:24.000
We are CopilotKit. Can you go to the next slide, please, Jerel? Thank you.

57
00:05:24.000 --> 00:05:27.000
We're CopilotKit, and we are open source.

58
00:05:27.000 --> 00:05:33.000
We have two massive projects.

59
00:05:33.000 --> 00:05:36.000
Well, we have actually a lot of massive projects.

60
00:05:36.000 --> 00:05:39.000
But CopilotKit and AG-UI, I'll get to AG-UI in a second.

61
00:05:39.000 --> 00:05:43.000
But combined, we have 55,000 stars.

62
00:05:43.000 --> 00:05:49.000
And we are very passionate about open source. So, some quick context on CopilotKit.

63
00:05:49.000 --> 00:05:53.000
We are the framework for building agentic applications.

64
00:05:53.000 --> 00:05:59.000
So, a typical app has a front end, but your agent lives on the back end.

65
00:05:59.000 --> 00:06:04.000
We're the layer that connects the two and gives your agent a UI.

66
00:06:04.000 --> 00:06:15.000
We also are a key, and we'll talk about this later, in taking that further to help your agent learn.

67
00:06:15.000 --> 00:06:32.000
So, the frameworks that we support are React, React Native, Angular, View for the web, but it goes beyond the web into Slack, into MS Teams, Discord, and WhatsApp.

68
00:06:32.000 --> 00:06:48.000
And you get a full suite of building blocks when you build with CopilotKit, where you get generative UI, human in the loop, front end tools.

69
00:06:48.000 --> 00:06:53.000
And we support all of the services.

70
00:06:53.000 --> 00:07:02.000
So, with CopilotKit, we'll talk about this later.

71
00:07:02.000 --> 00:07:16.000
But when it comes to business use cases, we are, you know, Fortune 500's use CopilotKit all the way down to hobbyists.

72
00:07:16.000 --> 00:07:21.000
And we're also the company behind AG-UI.

73
00:07:21.000 --> 00:07:29.000
So, AG-UI is an open protocol underneath it all that CopilotKit is built on top of.

74
00:07:29.000 --> 00:07:34.000
And it's an open standard for how agents and front ends talk to each other.

75
00:07:34.000 --> 00:07:38.000
Essentially, the bridge between the agent and the user.

76
00:07:38.000 --> 00:07:41.000
And we're really that connective tissue.

77
00:07:41.000 --> 00:07:58.000
So, what gives the same agent, you know, work in a browser can work in Slack or React Native or OpenBot without rewriting your entire application or even rewriting your agent.

78
00:07:58.000 --> 00:08:03.000
So, that leads me to why we're all here today.

79
00:08:03.000 --> 00:08:09.000
Building functional Co-Pilots. And Jerel, I will pass it over to you.

80
00:08:09.000 --> 00:08:15.000
All right. Let's get this party started. Awesome. All right.

81
00:08:15.000 --> 00:08:21.000
Thanks, Nathan. So, by the end of this talk, we have a bunch of ground to cover.

82
00:08:21.000 --> 00:08:23.000
So, I'm going to like brisk walk through it.

83
00:08:23.000 --> 00:08:30.000
But essentially, I want everyone here to end with this after the next 50 minutes.

84
00:08:30.000 --> 00:08:38.000
Which is basically, you'd be able to design and build functional agents with CopilotKit and learn how to improve their performance in production.

85
00:08:38.000 --> 00:08:47.000
Also, having like a mental framework of what are the things you need to care about and like consider when building functional agents.

86
00:08:47.000 --> 00:08:52.000
But before we dive into it, what are functional agents in the first place?

87
00:08:52.000 --> 00:09:01.000
Essentially, I define these as like copilots that are like specific to a particular job and industry.

88
00:09:01.000 --> 00:09:07.000
And they've gone through a recent uptake up and again.

89
00:09:07.000 --> 00:09:10.000
It started with coding with Cursor.

90
00:09:10.000 --> 00:09:19.000
You know, the whole idea with Cursor was, let's bring this LLM, this agent, inside of where the developers are working, inside of the IDE.

91
00:09:19.000 --> 00:09:20.000
It started as a port.

92
00:09:20.000 --> 00:09:28.000
Then they later on, they built their entire product from scratch, agent first, as like more users gathered into it.

93
00:09:28.000 --> 00:09:35.000
What we're seeing now in 2026, and this is why we're discussing this and we have like a lot of people like join in for us today,

94
00:09:35.000 --> 00:09:40.000
is that this is now bleeding across different functions or different verticals.

95
00:09:40.000 --> 00:09:46.000
Harvey and Legora for legal support share healthcare for a bridge.

96
00:09:46.000 --> 00:09:51.000
And a quick shout out for like the folks at Kino AI, they're also like super interesting.

97
00:09:51.000 --> 00:09:54.000
They're bringing this into the side of like video editing.

98
00:09:54.000 --> 00:10:03.000
Essentially, it's a good example or like it's a good indicator of how like AI has progressed in the last couple of years.

99
00:10:03.000 --> 00:10:08.000
Like now, we've started with coding because it's like the easiest and we have like the frameworks ready.

100
00:10:08.000 --> 00:10:21.000
But once we start building for different functions, we see that there's a lot of things we need to bridge and like guide that weren't like necessarily hurdles in coding and in Cursors up and rise.

101
00:10:21.000 --> 00:10:31.000
So we're going to be talking about these and considerations of what are the factors you need to consider to like take it into production.

102
00:10:31.000 --> 00:10:36.000
And how do you actually ensure that you're setting yourself up for success in the long run?

103
00:10:36.000 --> 00:10:42.000
And by allowing your agent to learn like from user behavior and like improving that piece.

104
00:10:42.000 --> 00:10:51.000
Okay. So while we're all here, I saw like Nepal, around 60% of the people here are engineers and around 20% are leaders.

105
00:10:51.000 --> 00:10:57.000
And like we have like a diverse like set of people here.

106
00:10:57.000 --> 00:11:06.000
Most of us here have been tasked or took the initiative of like, how do we bring agents into this particular role?

107
00:11:06.000 --> 00:11:08.000
Like agents for X.

108
00:11:08.000 --> 00:11:14.000
And I'm sure many of us here have been asking, hey, can we build an agent for our supply chain department?

109
00:11:14.000 --> 00:11:16.000
Or can we build an agent for the HR department?

110
00:11:16.000 --> 00:11:20.000
So essentially, this is going to be like a core crux of the conversation.

111
00:11:20.000 --> 00:11:26.000
And just a quick recap from how this has progressed in the last four years.

112
00:11:26.000 --> 00:11:29.000
And it's just four years, it seems like an eternity ago.

113
00:11:29.000 --> 00:11:40.000
We started from chat assistance where we chat GPT and we go with like, hey, can you write me a function to do this particular task?

114
00:11:40.000 --> 00:11:49.000
And then copy paste that into your IDE or like, or can you write me a contract clause for this particular piece that I'm trying to negotiate?

115
00:11:49.000 --> 00:11:51.000
Copy paste that into your doc.

116
00:11:51.000 --> 00:11:58.000
Now we're moving into functional copilots where we're bringing that into many different verticals.

117
00:11:58.000 --> 00:12:02.000
And it's just like a huge improvement.

118
00:12:02.000 --> 00:12:05.000
Like you see the huge improvement in that across industries.

119
00:12:05.000 --> 00:12:15.000
So we see here, like coding cursor, where we did a quick MVP for Open Harvey, which is just essentially an open source legal agent.

120
00:12:15.000 --> 00:12:18.000
As well as like, you know, AI.

121
00:12:18.000 --> 00:12:29.000
Recently, it's like, instead of like using Premiere and integrating via a plugin, how about we rebuild the entire like video editing software and start AI first.

122
00:12:29.000 --> 00:12:32.000
So essentially, these are the two schools of thought.

123
00:12:32.000 --> 00:12:36.000
The legal space are more integration heavy.

124
00:12:36.000 --> 00:12:46.000
But other verticals like coding and video AI and I'm seeing more and more getting into this are more like, let's rebuild and make this AI first.

125
00:12:46.000 --> 00:12:50.000
So yeah, like plugin and rebuild.

126
00:12:50.000 --> 00:12:55.000
What's the right answer? I don't think there is one. You need to do both.

127
00:12:55.000 --> 00:13:05.000
Again, you need to meet, you need to meet and build for where your users are and also anticipate like where they will be.

128
00:13:05.000 --> 00:13:12.000
And we see a strong push and pull with this from like, okay, we need to integrate, we need to build a plugin for this particular app.

129
00:13:12.000 --> 00:13:17.000
And then later on, we find that more and more, we start building it inside of the app.

130
00:13:17.000 --> 00:13:27.000
You can see that in Figma, you can see that in Claude, where there was initially a couple years ago, like they had strong integration and they were actively working together to like make those strong integrations.

131
00:13:27.000 --> 00:13:33.000
But now we see them like actually like build similar features.

132
00:13:33.000 --> 00:13:38.000
Figma now has more agentic features and Claude is now more design related features.

133
00:13:38.000 --> 00:13:46.000
So essentially when you're building like functional copilots, these are the things you need to like consider and build for.

134
00:13:46.000 --> 00:13:52.000
Like if you're building for a particular job, what are the tools that they are using?

135
00:13:52.000 --> 00:13:58.000
What are the patterns, UX patterns, UI patterns that they're used to in their day-to-day jobs?

136
00:13:58.000 --> 00:14:09.000
And then how do we make the trade-off decision of whether to build or integrate or integrate then build and then like the trade-offs across that.

137
00:14:09.000 --> 00:14:18.000
So before we get into this, I'm going to do a quick lay of the landscape.

138
00:14:18.000 --> 00:14:28.000
To build a functional agent, there's a lot of layers to it from choosing the right model, agent harness, data integration, context and knowledge, domain capabilities, products, and security and governance.

139
00:14:28.000 --> 00:14:31.000
It's a lot to cover for a 15-minute call.

140
00:14:31.000 --> 00:14:34.000
And I don't think we'll be able to like scratch the surface of all of this.

141
00:14:34.000 --> 00:14:43.000
So a couple of things that are out of scope for this conversation, we won't be getting a deep dive into agent harness and models.

142
00:14:43.000 --> 00:14:53.000
But what I've done instead is I've created the open repository to help you facilitate and guide through this activity on your own in your day job.

143
00:14:53.000 --> 00:14:56.000
Like, hey, I want to build this functional copilot.

144
00:14:56.000 --> 00:15:02.000
And Nathan has shared the repository in chat. So you can just like work through with this.

145
00:15:02.000 --> 00:15:09.000
And it's essentially like a set of skills that will help you like think through this stack.

146
00:15:09.000 --> 00:15:12.000
And it's better to expect some changes around that.

147
00:15:12.000 --> 00:15:17.000
But I hope that will help you get through this in your actual day job.

148
00:15:17.000 --> 00:15:21.000
One thing to talk about while we're here is security and governance.

149
00:15:21.000 --> 00:15:32.000
We're going to split like the two, I'd say, key milestones when we're building functional agents, which is MVP and prototype and production grade and ready.

150
00:15:32.000 --> 00:15:35.000
I'm going to talk about the MVP and prototype.

151
00:15:35.000 --> 00:15:40.000
It's very important that you still do the MVP and prototype so that you know if it's actually worth it.

152
00:15:40.000 --> 00:15:54.000
But for as you get into building for functional use cases and where business and actual data is involved, you realize that some of these are actually very important to secure.

153
00:15:54.000 --> 00:16:02.000
So there's like security and governance is becoming like more and more important as this goes across.

154
00:16:02.000 --> 00:16:09.000
And like there's going to be some trade-offs in the skill that in the repository created.

155
00:16:09.000 --> 00:16:12.000
There's also this thing you can talk about it like, hey, we're on MVP mode.

156
00:16:12.000 --> 00:16:14.000
And hey, we're on like production mode.

157
00:16:14.000 --> 00:16:19.000
And what are the things we need to consider when we're going to like deploy this now?

158
00:16:19.000 --> 00:16:28.000
Okay, so let's say you're tasked to build a legal tool for your legal team, like a legal copilot.

159
00:16:28.000 --> 00:16:33.000
You first start with who are the key players inside of the story?

160
00:16:33.000 --> 00:16:37.000
What are their responsibilities? What do they care about?

161
00:16:37.000 --> 00:16:40.000
And essentially do a map of like, who do we need to build for?

162
00:16:40.000 --> 00:16:53.000
And what are the layers of depth and care do they have for the workflow or for the function you're building for?

163
00:16:53.000 --> 00:16:57.000
And we're going to go through this. Again, this is just an example.

164
00:16:57.000 --> 00:17:01.000
There are many of these, but you typically get this from talking to your stakeholders.

165
00:17:01.000 --> 00:17:06.000
Like, hey, who are the people we're interacting with?

166
00:17:06.000 --> 00:17:15.000
And one of the things that you need to do is to actually talk to them and map out what they care about.

167
00:17:15.000 --> 00:17:23.000
For those who have been in the space of like building startups and engineer for the past, for more than four years, like before AI,

168
00:17:23.000 --> 00:17:28.000
you'd know there are different modes of thinking for this that we still use in Apply today.

169
00:17:28.000 --> 00:17:33.000
Like product thinking, systems thinking when you're starting to building, and then design thinking,

170
00:17:33.000 --> 00:17:36.000
which is what we're actually going to be talking about across this.

171
00:17:36.000 --> 00:17:39.000
It's like principles borrowed from all three.

172
00:17:39.000 --> 00:17:45.000
And as we've seen how AI changed our roles, like it's up to us to actually do bits and pieces of all three.

173
00:17:45.000 --> 00:17:51.000
It used to be like a team of specialists, but now you have to like master all three modes of thinking

174
00:17:51.000 --> 00:17:53.000
when you're building functional copilots.

175
00:17:53.000 --> 00:17:59.000
Especially if you're like a one-man team or like when you're tasked to like build a lot,

176
00:17:59.000 --> 00:18:04.000
like you're tasked to build a lot and you don't really have the time and there's potential mounting up.

177
00:18:04.000 --> 00:18:08.000
You need to do like all three roles for that.

178
00:18:08.000 --> 00:18:15.000
So just laying like the ground for this.

179
00:18:15.000 --> 00:18:19.000
Again, there are many ways and milestones to set this up.

180
00:18:19.000 --> 00:18:24.000
But essentially for the repository, for the sake of the stock, I've just like created like,

181
00:18:24.000 --> 00:18:28.000
Hey, first things first, we need to gather requirements and do research.

182
00:18:28.000 --> 00:18:35.000
And I've essentially asked AI to like, Hey, help me like codify how I'm doing things when I'm building applications.

183
00:18:35.000 --> 00:18:40.000
And I'm going to walk you through like how I've been building them in the stock as well.

184
00:18:40.000 --> 00:18:45.000
But essentially there are five steps, which is one research and user map.

185
00:18:45.000 --> 00:18:54.000
In my personal workflow, I do a lot of like mood boarding and research before we actually get into like building.

186
00:18:54.000 --> 00:19:02.000
So one of the things that have changed in the last like couple of years is in the past, there was this thing called like the prompt engineer.

187
00:19:02.000 --> 00:19:11.000
Like we have to be detailed with the prompts, you have to put the right context there, put like instruct the agent the right way.

188
00:19:11.000 --> 00:19:20.000
But what I found to building across building with AI almost every day for the last four years is that we're doing less and less of that.

189
00:19:20.000 --> 00:19:25.000
And more and more of like working with your agent to build the context.

190
00:19:25.000 --> 00:19:29.000
So some people call this context engineering. I just call this research.

191
00:19:29.000 --> 00:19:31.000
Essentially what you're doing is research.

192
00:19:31.000 --> 00:19:37.000
You talk to people, you gather data, you map out existing workflows.

193
00:19:37.000 --> 00:19:50.000
You take UI, UX maps from existing tools and then analyze it and then figure out what to do with it for the task at hand.

194
00:19:50.000 --> 00:19:55.000
For the designers in the room, I saw like around 15% answered in the poll.

195
00:19:55.000 --> 00:20:00.000
Essentially, this is like the mood boarding and UX mapping and codified into a skill.

196
00:20:00.000 --> 00:20:06.000
I used to do this by hand back when I was a designer, like almost a couple of lifetimes ago.

197
00:20:06.000 --> 00:20:16.000
You take a lot of screenshots, map them in Figma, like give it a lot of comments and then write it into a PRD and then give it to the engineer to ship.

198
00:20:16.000 --> 00:20:19.000
But now it's like, okay, we need to get this done within the day.

199
00:20:19.000 --> 00:20:23.000
So what we're going to do instead is like, okay, what is the name of the tool that they're using?

200
00:20:23.000 --> 00:20:26.000
Is it up live? Can I download it?

201
00:20:26.000 --> 00:20:29.000
And then can I point my agent to it and actually map the workflow?

202
00:20:29.000 --> 00:20:38.000
And I find that instead of doing this for a day to a week in a team context,

203
00:20:38.000 --> 00:20:43.000
like we're able to shrink the design sprint to a couple of hours with your agent.

204
00:20:43.000 --> 00:20:47.000
And that's the goal with the skill user map. So it's like roles, software jobs and flows.

205
00:20:47.000 --> 00:20:52.000
And I'm going to show like what that looks like in a bit. The next is context and access.

206
00:20:52.000 --> 00:21:01.000
When you're building functional workflows and you're moving away from MVP and prototype plan, you actually need to connect this to real systems.

207
00:21:01.000 --> 00:21:08.000
So this stage is mapping out where do we need to connect this to?

208
00:21:08.000 --> 00:21:13.000
And what does that actually look like? Is that a direct integration? Is that an API key?

209
00:21:13.000 --> 00:21:16.000
Is that an MCP? Is this something we're going to build in the app?

210
00:21:16.000 --> 00:21:18.000
Is this something that we're going to connect in Slack?

211
00:21:18.000 --> 00:21:22.000
So we're going to have to like map out what that looks like. Then lastly, spec and build.

212
00:21:22.000 --> 00:21:29.000
Once you map the context, it's a lot easier to build the right thing once you do the first two things right.

213
00:21:29.000 --> 00:21:39.000
And I find that if you spend most of your time researching and context building, you can just leave your agent on autopilot and go to sleep.

214
00:21:39.000 --> 00:21:42.000
When you wake up, it's already done and it's the right thing for you.

215
00:21:42.000 --> 00:21:45.000
And you can give it a couple of feedback. But essentially, that's the key part.

216
00:21:45.000 --> 00:21:55.000
So these first three and a little bit of like the fourth piece, which is to deploy, is the MVP and like prototype plan, which is very important before you even get into production.

217
00:21:55.000 --> 00:22:01.000
Because you have to like understand and play with it and show it to stakeholders.

218
00:22:01.000 --> 00:22:04.000
Hey, is this what we want to like share?

219
00:22:04.000 --> 00:22:17.000
One key hack I'm doing like before securing, like secure and cover, because once you get into step five, it's going to be a huge, like the iteration cost is a lot slower.

220
00:22:17.000 --> 00:22:20.000
One thing you do there is like, hey, let's record all of the flows.

221
00:22:20.000 --> 00:22:24.000
And your A can do this, by the way, like record GIFs and flows for the user map.

222
00:22:24.000 --> 00:22:29.000
So that when I wake up the next day, I have like videos I can play and I can just review.

223
00:22:29.000 --> 00:22:32.000
Did it build the right thing according to my expectations?

224
00:22:32.000 --> 00:22:35.000
Or what did it get wrong within the shortest amount of time?

225
00:22:35.000 --> 00:22:41.000
Next is like, okay, so what does this look like?

226
00:22:41.000 --> 00:22:53.000
So step one research is, in the past, this would have taken like me and a UX researcher, and then we're going to map out the entire flow.

227
00:22:53.000 --> 00:22:59.000
We're going to go through and watch them do the entire thing. So what emails do they send?

228
00:22:59.000 --> 00:23:04.000
Can we get a copy of those? Can we, what documents, tools do they use?

229
00:23:04.000 --> 00:23:09.000
And what, how do they use them?

230
00:23:09.000 --> 00:23:13.000
Do they do comments? And how do they manage versions?

231
00:23:13.000 --> 00:23:18.000
Like, what are the different things they're doing across a particular job to be done?

232
00:23:18.000 --> 00:23:22.000
So a key part there is, let's say you're building a legal copilot.

233
00:23:22.000 --> 00:23:31.000
For anyone to understand this a bit well, you just need to like refresh memory if you've ever had experience working with a lawyer.

234
00:23:31.000 --> 00:23:42.000
You'd find that they primarily interact with emails and documents and they do a lot of revisions, redlining revisions, especially when the other side has a lawyer as well.

235
00:23:42.000 --> 00:23:45.000
And then you have to like go into like back and forth with them.

236
00:23:46.000 --> 00:23:52.000
And then sometimes they maintain a spreadsheet to like keep the financial figures in mind.

237
00:23:52.000 --> 00:23:56.000
And then lastly, this is something that is less apparent, but it's more internalized.

238
00:23:56.000 --> 00:24:02.000
They went to school to understand the legal space, the rules, the policies, the things to do right.

239
00:24:02.000 --> 00:24:09.000
And the things that would be like a cluster fire of like, like things to do wrong.

240
00:24:09.000 --> 00:24:14.000
So you also need to like create those playbook. And like, I think that's the hardest part.

241
00:24:14.000 --> 00:24:21.000
And there's many, there's many ways of doing this from interviews to buying datasets.

242
00:24:21.000 --> 00:24:32.000
And then we also are sharing a new, we also are doing this thing that I'm going to share earlier, which is learning from user behavior, which will help you build this up.

243
00:24:32.000 --> 00:24:36.000
We have to be PG dated. So like cluster fire. Okay.

244
00:24:36.000 --> 00:24:41.000
And then next you have to map out the time spent.

245
00:24:41.000 --> 00:24:55.000
This really helps when you want to then like, so this part is also very important, especially when you want to then justify the time spent and token spent building this thing.

246
00:24:55.000 --> 00:24:59.000
You have to quantify like, hey, how much did it used to cost?

247
00:24:59.000 --> 00:25:03.000
And how much value are we actually doing?

248
00:25:03.000 --> 00:25:07.000
This part is, I'd say, one of the most important part that people miss.

249
00:25:07.000 --> 00:25:14.000
It's that, okay, this entire process is a 10 day process that we can bring down to a single day.

250
00:25:14.000 --> 00:25:25.000
Like that's a lot of value, especially once you consider in like many different, many different, like, like people's time spent on this thing.

251
00:25:25.000 --> 00:25:32.000
And you compute their salaries, it becomes like an exorbitant amount of time and across an organization.

252
00:25:32.000 --> 00:25:40.000
That's how you justify investments in functional copilots, which I'd say is a very important thing for your role as the majority of us here engineers.

253
00:25:40.000 --> 00:25:48.000
One of the things that I'd say my engineering friends struggle with, and I used to struggle with, is that how do we justify the value of what we're building?

254
00:25:48.000 --> 00:25:50.000
And this part is the thing you need to do well.

255
00:25:50.000 --> 00:25:54.000
It's like, okay, what did it used to cost in terms of time?

256
00:25:54.000 --> 00:26:05.000
Do some quick rubber hat, like, like, napkin calculation of like, what is that actually in financial value in terms of time, like exponential mandates, you know?

257
00:26:05.000 --> 00:26:09.000
And then that's when you figure out like, hey, this is something worth automating.

258
00:26:09.000 --> 00:26:12.000
Or, hey, this is just actually a 10 day thing.

259
00:26:12.000 --> 00:26:15.000
And the best we can do is bring it down to nine days.

260
00:26:15.000 --> 00:26:20.000
And it's like a one day difference that's not worth automating, because like the things are out of our control.

261
00:26:20.000 --> 00:26:30.000
So by mapping this out, you get a key, you're able to make informed decisions on what to automate and what to leave alone.

262
00:26:30.000 --> 00:26:42.000
Okay, so for our leadership people here, and I'd say the senior engineers and designers, you must have heard this across time, like the job to be done for your merit.

263
00:26:42.000 --> 00:26:48.000
I'd say it's very important to understand this and like internalize this in today's age.

264
00:26:48.000 --> 00:26:58.000
It's not commonly talked about anymore as it used to be in like pre-AI stage of like software development lifecycle.

265
00:26:58.000 --> 00:27:07.000
But like understanding this is what would separate an MVP that gets to production and an MVP that dies in prototype land.

266
00:27:07.000 --> 00:27:26.000
So, but essentially what this is, is to understand job to be done, you have to understand that people don't buy your software, your application, your AI agent, like for its features.

267
00:27:26.000 --> 00:27:33.000
People hire your solution to solve a problem that they have. And that's the job to be done.

268
00:27:33.000 --> 00:27:46.000
There are some key frameworks here, but essentially like what I want to keep in mind is when like this particular thing, when a situation happens, I want to be able to do this so that I can get this particular value.

269
00:27:46.000 --> 00:27:53.000
And again, since this is a mock situation or in the legal use case, like the value differs company to company.

270
00:27:53.000 --> 00:27:58.000
Okay. So the next step is to context and access.

271
00:27:58.000 --> 00:28:07.000
We, this is an increasingly more important part of building agents in 2026.

272
00:28:07.000 --> 00:28:19.000
Um, and I recently wrote a book about this in the last month or so, but essentially there are many layers of AI agents starting with like, hey, here's a chat bot inside of an app.

273
00:28:19.000 --> 00:28:33.000
Second is like, hey, let's bring that agent inside of our machine, like laptops, like Claude, like Hermes and OpenClaw and give it access to essentially that the entire laptops applications and whatnot and let it do stuff.

274
00:28:33.000 --> 00:28:42.000
And that like grow a lot of value and locked a lot of value for people in, um, their day-to-day like use cases.

275
00:28:42.000 --> 00:28:52.000
Um, but recently we've seen the third, the third and fourth layers mature in the last couple of months where we now have brought that into the cloud.

276
00:28:52.000 --> 00:28:57.000
Now your agent is not just using your computer, which is also like a huge security concern, by the way.

277
00:28:57.000 --> 00:29:10.000
Um, they now have their own sandbox, um, like cloud computers that they manage and like allow them to be always on because they're now separated from your machine.

278
00:29:10.000 --> 00:29:18.000
No, no more walking around with your laptop half open and trying to connect with your hotspot running across like, uh, like place to place, you know.

279
00:29:18.000 --> 00:29:23.000
Um, now your agent actually has their own computer and that's, um, level three.

280
00:29:23.000 --> 00:29:31.000
Um, the fourth layer here, before we get into the fourth layer, we actually, a quick shout out to the OpenBot team.

281
00:29:31.000 --> 00:29:36.000
We, we have, this is essentially what we've been building with OpenBot, OpenMuse, Duet and DOTS.

282
00:29:36.000 --> 00:29:46.000
Um, essentially we're open sourcing this layer, like the third layer of the new agentic, um, like paradigms.

283
00:29:46.000 --> 00:29:54.000
And, um, we, we allow you to self deploy this, like, and essentially like make it more accessible for, for enterprises.

284
00:29:54.000 --> 00:30:04.000
Um, the fourth piece here is actually something that got overtaken quite recently, but it's, uh, still a huge piece of how people work.

285
00:30:04.000 --> 00:30:12.000
It's now bringing this agent, like the, the first three layers into organizations in a multiplayer setting.

286
00:30:12.000 --> 00:30:21.000
Like we've seen this early this year with Claude Tag, where you can just add Claude in your organization and give it a task, go back and forth.

287
00:30:21.000 --> 00:30:28.000
Um, and that's like, like now like, uh, a multiplayer setting for like agents with their own computers.

288
00:30:28.000 --> 00:30:31.000
And it's like driving a lot of value across organizations.

289
00:30:31.000 --> 00:30:35.000
So when you're building your own functional use case, you have functional copilots.

290
00:30:35.000 --> 00:30:39.000
You need to consider what context does it need to have?

291
00:30:39.000 --> 00:30:42.000
Is it relevant for this agent to be accessible in Slack?

292
00:30:42.000 --> 00:30:54.000
Or is it something that we need to lock in and keep, um, the access close to, um, Gmail and, um, like, like the drive that you're storing the information on?

293
00:30:54.000 --> 00:30:57.000
So that's a key thing that we are, uh, that you have to consider here.

294
00:30:57.000 --> 00:31:02.000
Oh, by the way, if you are building agents to connect to Slack, we've built Channels SDK.

295
00:31:02.000 --> 00:31:09.000
And it allows you to like help, it allows you to like really like, uh, it's an open source library that we have that allows you to connect agents to Slack.

296
00:31:09.000 --> 00:31:15.000
And it's like, uh, like really straightforward. Okay. So next is spec and build.

297
00:31:15.000 --> 00:31:24.000
If you do the first two stages right, this is the part that I'd say is the easiest now.

298
00:31:24.000 --> 00:31:34.000
And it used to be the hardest, like, like, uh, before AI, this one, this spec and build took weeks and months to actually nail.

299
00:31:34.000 --> 00:31:41.000
But now like the hardest thing to do is the first two pieces research and context access.

300
00:31:41.000 --> 00:31:47.000
Um, spec and building is something that you can do overnight and almost asynchronously.

301
00:31:47.000 --> 00:32:02.000
Like once you gathered the right research and the right context, um, and you put it in a place that's accessible to your agent and read me in a doc or in a HTML file or in transcripts, it's, it's almost instant.

302
00:32:02.000 --> 00:32:05.000
It's almost, um, autopilot at that point.

303
00:32:05.000 --> 00:32:20.000
But if you skip research and context and go straight to spec and build, you're going to pay the price of iterating more and more and spending a lot more tokens just because like, Hey, um, like it gets quite close to what you want, but it's not really there.

304
00:32:20.000 --> 00:32:26.000
And then you present it to your stakeholders and then your stakeholders say like, and that's kind of wrong.

305
00:32:26.000 --> 00:32:38.000
I mean, you're like, you're eroding the trust bank and that, that's something I don't suggest doing, especially in the corporate settings that, um, when you present something and they, it's not hitting, like, it's like, do that enough times.

306
00:32:38.000 --> 00:32:43.000
And everyone's trust and like your ability to build will also like, like, uh, diminish.

307
00:32:43.000 --> 00:32:44.000
So you have to be mindful.

308
00:32:44.000 --> 00:32:55.000
And the way to build this right is by doing the first two stages, because it's now like a lot easier to spec and build out agents, um, and energetic workflows.

309
00:32:55.000 --> 00:33:03.000
So in CopilotKit, one of the things that we do is, uh, we bridge your applications context to your agents.

310
00:33:03.000 --> 00:33:07.000
And this allows you to do a lot of like cool things.

311
00:33:07.000 --> 00:33:22.000
Like when you're writing, um, like let's say in the legal use case, when you're writing documents, when you're writing, uh, uh, the thing that we've mapped earlier from the research and build, uh, just going back a sec.

312
00:33:22.000 --> 00:33:27.000
Um, when you're, what we've identified here is one, they need access to email.

313
00:33:27.000 --> 00:33:33.000
Two, they need documents. Um, they need like a document reader and a document editor.

314
00:33:33.000 --> 00:33:37.000
Um, they need a way to manage revisions. They need a way to compute.

315
00:33:37.000 --> 00:33:44.000
And then there, there, there has to be a way to like set up rules that the agent might not get, um, in the first round.

316
00:33:44.000 --> 00:33:58.000
So when we interpret that in this, um, in the spec and build is, is we have to consider like, okay, um, are we going to build a plugin to Word, um, and Google Doc?

317
00:33:58.000 --> 00:34:03.000
Or are we going to rebuild Word and Google Doc from scratch within the open source library?

318
00:34:03.000 --> 00:34:05.000
That's a key decision you have to make.

319
00:34:05.000 --> 00:34:13.000
Um, in the earlier years of AI, I'd say they were just, just a couple of years ago, honestly, but it feels so long.

320
00:34:13.000 --> 00:34:20.000
Um, you, people really lean in towards building a plugin, like, okay, let's integrate existing tools.

321
00:34:20.000 --> 00:34:35.000
Um, but in 2026, like, I'd say that's, and it's a hot take, that's the wrong thing to do, especially as organizations and these giant corporations are waking up and locking in their application.

322
00:34:35.000 --> 00:34:45.000
You know, like one thing that separates Metamuse from the other applications is it has access to Instagram and Facebook that Grokbot just doesn't have.

323
00:34:45.000 --> 00:34:56.000
Grokbot has access to X and then, um, all of these different, like differentiators, um, that they lock their agents in and out of is what's going to make or break like a functional use case.

324
00:34:56.000 --> 00:35:01.000
Yes, you still need to like figure out how to connect your agent to an MCP.

325
00:35:01.000 --> 00:35:15.000
It's still an important thing, but if that's your entire strategy that puts you to risk, um, especially when you are, um, like, especially when these giant, like, applications start to lock down and build their own agents.

326
00:35:15.000 --> 00:35:19.000
It's a hot take, but I see it coming. So what, what to do, what to do now?

327
00:35:19.000 --> 00:35:35.000
Yeah. So, um, at, let's say you are like Harvey or like Kino AI, which is like the video editing software, instead of like integrating with Premiere Pro or like Microsoft Word, build the set of features that you release your seed.

328
00:35:35.000 --> 00:35:39.000
Um, uh, and you scope it, you map it out with your agent so that it's easier.

329
00:35:39.000 --> 00:35:44.000
There are a bunch of open source libraries that you can bet to like, like bridge the gap.

330
00:35:44.000 --> 00:35:47.000
And if you build that inside your application, that gives you a lot more context.

331
00:35:47.000 --> 00:35:56.000
And, um, CopilotKit allows you to connect your app state or app features to your agent, um, and manipulate it directly.

332
00:35:56.000 --> 00:36:06.000
So instead of like integrating to a software built for humans, you're building a software ground up that is AI first, that your agent natively understands and interacts with.

333
00:36:07.000 --> 00:36:11.000
Um, like another like function that we have here is like, like use for the tool.

334
00:36:11.000 --> 00:36:17.000
So it allows you to like manipulate like the state of the application as well, run a task, graph a red light, fill the cell.

335
00:36:17.000 --> 00:36:28.000
Um, now when we are like on the agent experience, one thing that, uh, we are going to briefly discuss is like generative UI and components like use components.

336
00:36:28.000 --> 00:36:39.000
Like how do we make the experience not just blocky words of text with, um, and there's some generative UI components out there that in and of itself disturbs a talk.

337
00:36:39.000 --> 00:36:41.000
And if you're interested, shoot us a message.

338
00:36:41.000 --> 00:36:43.000
Like that's, that might be something we're going to talk about.

339
00:36:43.000 --> 00:36:49.000
Um, but generative UI is an entire spectrum of things that would allow you to improve your agent experience.

340
00:36:49.000 --> 00:36:56.000
Um, and then lastly, um, human in the loop, which is a key part in designing functional copilots.

341
00:36:56.000 --> 00:37:05.000
It's understanding when to bring something up to the human and require the plan.

342
00:37:05.000 --> 00:37:10.000
Um, require, uh, require human intervention for the plan.

343
00:37:10.000 --> 00:37:13.000
So that's a key part of this as well. Um, okay.

344
00:37:13.000 --> 00:37:21.000
So, um, from the task to the approval gate, uh, like essentially here, this is like the human in the loop component.

345
00:37:21.000 --> 00:37:25.000
Uh, let me see if I can screen share here.

346
00:37:25.000 --> 00:37:34.000
Uh, let me just, oh, I accidentally turned off the agent that's running.

347
00:37:34.000 --> 00:37:39.000
Let me just run that up and I will circle back to this. Uh, okay.

348
00:37:39.000 --> 00:38:00.000
Um, I'm going to circle back to the demo, but, uh, one thing here is like when, one key part when building functional agents is the trust between the, the professional and your agent.

349
00:38:00.000 --> 00:38:14.000
It's not often talked about, but one key part in building a good agent experience is like having the right level of show and hide and inspect.

350
00:38:14.000 --> 00:38:15.000
Uh, what do you mean by that?

351
00:38:15.000 --> 00:38:18.000
It's like, essentially it's just an exercise of trust building.

352
00:38:18.000 --> 00:38:27.000
If you were a professional lawyer and you abstract everything away into this dropdown section, like, like done, um, and recorded reason.

353
00:38:27.000 --> 00:38:37.000
I like, and you're bombarded with like, uh, like a done, like contract that's already done as your first time interacting with AI.

354
00:38:37.000 --> 00:38:56.000
It's really hard to trust that because you can't see the process, um, of like making it like, like it's, it's the old, like, show your solution, um, paradigm in, in school where we, where we, uh, like to understand how to arrive at a certain point.

355
00:38:56.000 --> 00:39:06.000
And it's like, like to trust the end goal, you want to be able to like monitor how it was made and at the same time, like track that.

356
00:39:06.000 --> 00:39:10.000
So this is also an important part in the secure and governance part later on.

357
00:39:10.000 --> 00:39:15.000
It's like, okay, logs and audits, um, and rollbacks is a key part.

358
00:39:15.000 --> 00:39:25.000
Um, and then, uh, let me see if we got it up now.

359
00:39:25.000 --> 00:39:30.000
Okay. I got it up and running. Okay.

360
00:39:30.000 --> 00:39:42.000
Let's see if I can quickly swap my screen here just for a sec.

361
00:39:42.000 --> 00:39:44.000
Okay. Okay.

362
00:39:44.000 --> 00:39:49.000
So just to showcase what that looks like, uh, let's say, so this is Open Harvey.

363
00:39:49.000 --> 00:39:51.000
This is part of the repository we shared.

364
00:39:51.000 --> 00:39:57.000
Um, and currently it's using, um, a mock so that it's like a lot easier to like, like show and render.

365
00:39:57.000 --> 00:39:58.000
We don't have to wait.

366
00:39:58.000 --> 00:40:04.000
Um, but a mock is actually like mocking like the response of an LLM that, uh, it would work in reality.

367
00:40:04.000 --> 00:40:11.000
So let's say let's review, um, that like an MSA, which is something we've identified that the users need.

368
00:40:11.000 --> 00:40:17.000
Um, showing the review plan here, um, for most users is like, uh, you can just like go ahead and build that.

369
00:40:17.000 --> 00:40:28.000
But for the professionals you're building for, for the legal professionals you're building for, it's an important thing to show like, okay, um, are we aligned with how the agent is going to tackle this task?

370
00:40:28.000 --> 00:40:37.000
And even if it's just a brief run through over what's happening, that allows, um, that builds trust between the professional and the agent.

371
00:40:37.000 --> 00:40:40.000
Um, and let's say we approve the plan here.

372
00:40:40.000 --> 00:40:49.000
Uh, we see, again, this is fast because we are, uh, running a recorded demo on this, on, on the back of the a mock.

373
00:40:49.000 --> 00:41:03.000
But essentially that would take a longer time and then, um, it would show that, okay, what, what's processing and how it's, um, like, like what's processing and how it's manipulating like the data here.

374
00:41:03.000 --> 00:41:16.000
And then one key part, uh, cause again, this is a quick build for this demo is if you're building to replace, like, it's a lot, it's super useful to pattern match what they're already used to.

375
00:41:16.000 --> 00:41:19.000
Like, and it's easier to do with AI.

376
00:41:19.000 --> 00:41:23.000
Let's say you, like, agents work a lot in Microsoft Word or in Google Docs.

377
00:41:23.000 --> 00:41:28.000
Um, ask your AI to do the slash user map and point it to those applications.

378
00:41:28.000 --> 00:41:37.000
Let's do a user map of Google Docs. Let's do a user map of Microsoft Word so that you can, you can, uh, yes, uh, so this demo, you can run this locally.

379
00:41:37.000 --> 00:41:44.000
It's in the repository, um, under slash starters, um, and, uh, just to expect some updates on it.

380
00:41:44.000 --> 00:41:50.000
But at the same time, this is, uh, this is something, this is a starter you can actively, like, like, modify and build with.

381
00:41:50.000 --> 00:41:57.000
Um, and once you do the user map, your agent will now understand what you mean by, like, click copy.

382
00:41:57.000 --> 00:42:02.000
Microsoft Word. And before you get into that, you need to see it to actually map out.

383
00:42:02.000 --> 00:42:08.000
I like looking and asking my agent to actually do a mood board of the entire application.

384
00:42:08.000 --> 00:42:17.000
So that I know that it understands and it gathers it into context of the different screenshots or, um, like, the particular news flow.

385
00:42:17.000 --> 00:42:20.000
And then it makes it easier to build once you have that.

386
00:42:20.000 --> 00:42:26.000
Because in the past, you'd have to describe that in words and in PRDs and in user stories.

387
00:42:26.000 --> 00:42:29.000
I'd say it's super hard to even do that, even for an agent.

388
00:42:29.000 --> 00:42:34.000
It's a lot easier to show and ask your agent to like, hey, let's build context together.

389
00:42:34.000 --> 00:42:40.000
Download it inside of your computer and then, like, go through the entire flow, map out.

390
00:42:40.000 --> 00:42:44.000
It's a lot better if you can do this with your, um, with your professional.

391
00:42:44.000 --> 00:42:46.000
But at the same time, that's a, that's a key thing.

392
00:42:46.000 --> 00:42:49.000
Or just, like, like, do it yourself and, like, giving context to your agent.

393
00:42:49.000 --> 00:42:55.000
Anyway, that's, that's an example of, like, building for functional agents with, um, CopilotKit.

394
00:42:55.000 --> 00:43:02.000
And, um, a quick behind the scenes here is CopilotKit runtime connects you to threads and memory.

395
00:43:02.000 --> 00:43:09.000
Uh, we still need to find a way to, like, deploy this in the cloud, which is, you can do that in Daytona, E2B, AWS, or QuickCloud.

396
00:43:09.000 --> 00:43:12.000
Or do that in self-hosted local way. That's something we also support.

397
00:43:12.000 --> 00:43:14.000
Connect that with Slack and Teams.

398
00:43:14.000 --> 00:43:20.000
And then your application is, like, essentially, um, a web app that, like, runs with these.

399
00:43:20.000 --> 00:43:29.000
Um, I'm risking to swear that's short on time, but there's a key part I want to show on, like, how do you actually do secure and govern.

400
00:43:29.000 --> 00:43:33.000
Um, there's a lot of things to deploy. There's many different flavors in doing this.

401
00:43:33.000 --> 00:43:38.000
I did my best of, like, trying to consolidate how I think about deployment inside of a skill.

402
00:43:38.000 --> 00:43:40.000
So it's inside of the repository as well.

403
00:43:40.000 --> 00:43:51.000
So you can, uh, like, set this up in, set this up in, uh, uh, like, work with your agent to deploy this that meets your needs.

404
00:43:51.000 --> 00:43:54.000
You can just, like, work with it in the deploy skill.

405
00:43:54.000 --> 00:44:02.000
Um, lastly is one key part that I'd say is also something you need to consider in the MVP.

406
00:44:02.000 --> 00:44:09.000
Especially when you're building for business use case is the, the guardrails, um, building the audit rail.

407
00:44:09.000 --> 00:44:11.000
It's a lot easier to build now.

408
00:44:11.000 --> 00:44:18.000
Um, I've also, like, consolidated a checklist in, inside of, like, the secure and govern, um, like, part of the repository.

409
00:44:18.000 --> 00:44:22.000
So that you can just ask your agent to, like, hey, um, are we, like, secured and governed?

410
00:44:22.000 --> 00:44:24.000
What are the things we need to care about?

411
00:44:24.000 --> 00:44:34.000
Um, because often it's easy to miss these things, but this is what makes or breaks the production application, um, makes or breaks a prototype breaking into a production application.

412
00:44:34.000 --> 00:44:36.000
You have to, like, consider these things.

413
00:44:36.000 --> 00:44:39.000
Named approvers is very important, especially for business use cases.

414
00:44:39.000 --> 00:44:42.000
Like, who approved this thing?

415
00:44:42.000 --> 00:44:50.000
Um, like, that's important for many different reasons, but it's also just, like, to ask questions about, like, hey, um, what happened here whenever something went wrong?

416
00:44:50.000 --> 00:45:02.000
Or, like, hey, like, like, uh, like, understanding who's responsible for the thing, especially as you build, like, human to agent, like, like, essentially bridges with, um, these functional copilots.

417
00:45:02.000 --> 00:45:07.000
Um, and then lastly, again, this is, like, a preview of what it looks like, like, the scale.

418
00:45:07.000 --> 00:45:12.000
It will give you, like, an HTML file with things to check for your application.

419
00:45:12.000 --> 00:45:21.000
And it gives you, like, a prototype or a production, like, like, barrier, like, this is safe for our laptop demo, but you know what?

420
00:45:21.000 --> 00:45:28.000
You need to check these, these boxes first before you get into, like, production, or else you might be putting your company at risk.

421
00:45:28.000 --> 00:45:32.000
Okay. So, one last thing. Okay.

422
00:45:32.000 --> 00:45:42.000
Once, let's say you've gotten first a one to five done and deployed, and your stakeholders love it, you deployed it internally, people are using it.

423
00:45:42.000 --> 00:45:46.000
Like, but after a while, you see your agent breaks down.

424
00:45:46.000 --> 00:45:57.000
And the reason for, like, your agent fails and breaks down for a couple of, like, key moments inside of the workflow.

425
00:45:57.000 --> 00:46:05.000
And it becomes complex to build data sets or complex to identify and improve.

426
00:46:05.000 --> 00:46:13.000
This is why we actually spent a lot of time, like, thinking about this and, like, helping our, like, customers, like, like, build on this.

427
00:46:13.000 --> 00:46:23.000
It's, like, because one of the key problems here is, like, it's really hard to build a data set for fine tuning or describing the problem in the first place.

428
00:46:24.000 --> 00:46:26.000
Because it's really hard to build context around it.

429
00:46:26.000 --> 00:46:32.000
Like, it's not a linear trajectory, a yes or no you can build a neural network with.

430
00:46:32.000 --> 00:46:36.000
Like, it's a lot, a series of steps that are unreliable.

431
00:46:36.000 --> 00:46:39.000
Like, the steps changes each time to accomplish the task.

432
00:46:39.000 --> 00:46:45.000
Even if the task is the same, like, the steps to get to the task is very different across time.

433
00:46:45.000 --> 00:46:51.000
So, it's really hard to, like, build and improve on.

434
00:46:51.000 --> 00:46:55.000
Like, to build a data set that you can, like, use your agent to improve on.

435
00:46:55.000 --> 00:47:02.000
So, this is why, let's say, you've gotten live and your agent isn't working as well as you are.

436
00:47:02.000 --> 00:47:04.000
How do you improve it? This is a problem we care about.

437
00:47:04.000 --> 00:47:08.000
And this is why we've also launched CLiF. This is still in early access.

438
00:47:08.000 --> 00:47:13.000
We're working with design partners to get this, like, to a broader audience.

439
00:47:13.000 --> 00:47:15.000
So, essentially, if you're interested, like, shoot us a message.

440
00:47:15.000 --> 00:47:17.000
Like, later on, we have, like, a slide for that.

441
00:47:17.000 --> 00:47:26.000
But to explain what this is, Nathan talked about AG-UI, which is essentially connecting your agent and your user.

442
00:47:26.000 --> 00:47:32.000
Like, your agent and your users inside of the application.

443
00:47:32.000 --> 00:47:33.000
We're essentially bridging that gap.

444
00:47:33.000 --> 00:47:40.000
And by doing so, we have contacts on the agent trajectory and the user trajectory.

445
00:47:40.000 --> 00:47:45.000
So, by doing this, we get contacts for both.

446
00:47:45.000 --> 00:47:49.000
And it's not going to our, like, service. By the way, we don't have that.

447
00:47:49.000 --> 00:47:52.000
It's going directly to your service or your cloud.

448
00:47:52.000 --> 00:47:56.000
And you can, like, self-host this so that everything stays local in your business sense.

449
00:47:56.000 --> 00:48:01.000
But by doing this, you get what we call a full stack trajectory.

450
00:48:01.000 --> 00:48:05.000
So, agent trajectory, which is the series of steps to do a particular task.

451
00:48:05.000 --> 00:48:08.000
And the user trajectory is, like, what does that look like?

452
00:48:08.000 --> 00:48:13.000
Like, the series of steps to do a particular task for the full stack trajectory.

453
00:48:13.000 --> 00:48:25.000
And that is a, like, this dataset is becoming more and more important to own for your organization.

454
00:48:25.000 --> 00:48:28.000
Again, this is something that you're going to self-host and deploy.

455
00:48:28.000 --> 00:48:40.000
Like, it's more and more important because, like, your professionals and your agents now start to work together and mesh between tasks and their day-to-day jobs.

456
00:48:40.000 --> 00:48:45.000
And if you, like, have a better understanding of this, it's a lot easier to improve on.

457
00:48:45.000 --> 00:48:59.000
So, the key example here is that, let's say, an agent trajectory tried, and that this is just, like, the agent trajectory, like, the evals you get from LangSmith and, like, building with LangSmith and, like, the other eval providers there.

458
00:48:59.000 --> 00:49:12.000
But they don't really have context on the user actions, which is, like, the highlighted part here is, like, here's, like, the agent actions, but what did the user actually click and work about?

459
00:49:12.000 --> 00:49:19.000
So, what this looks like is, let's say, in a financial use case, your user clicks across, like, the different sections of the app.

460
00:49:19.000 --> 00:49:25.000
Essentially, that is what we keep track of in the full stack trajectory, user, agent, user, you can see this here.

461
00:49:25.000 --> 00:49:32.000
And that's something you can, like, gather in and build, like, a dataset on.

462
00:49:32.000 --> 00:49:41.000
So, AG-UI is connected across majority of the frameworks out there, so you don't need to migrate whatever, like, agent framework you're using.

463
00:49:41.000 --> 00:49:47.000
And it's also connected to majority of the channels from web apps, Slack, mobile, and Teams.

464
00:49:47.000 --> 00:49:58.000
And then, by doing this, you get to label what's learning from, like, there's also, like, a way to thumbs up and thumbs down these, like, tune their repository, clean it up.

465
00:49:58.000 --> 00:50:09.000
Then, once you get to that, you get, once you've done that, you have many different ways to improve your agent that's just not accessible for you because, like, before this, because it's really hard to get these contacts.

466
00:50:09.000 --> 00:50:17.000
In the past, when I was still a designer, we'd record screens from people using our applications.

467
00:50:17.000 --> 00:50:27.000
We still do this, like, in a production application with, like, different, like, telemetry trackers.

468
00:50:27.000 --> 00:50:30.000
But it's really hard to bring that together with your agent.

469
00:50:30.000 --> 00:50:35.000
And that's something uniquely possible with, like, what we've done with, like, CopilotKit, and AG-UI.

470
00:50:35.000 --> 00:50:45.000
So you can, like, just start building these product insights, like, and can turn them into skills, eval candidates, and can use this dataset as you scale into the organization to fine-tuning URL.

471
00:50:45.000 --> 00:50:47.000
So, okay, so that's the learning piece.

472
00:50:47.000 --> 00:50:53.000
But what are the, like, now, let's say, it's gotten approved from prototype and internal use case.

473
00:50:53.000 --> 00:50:56.000
Now we're going to productize this across the entire company.

474
00:50:56.000 --> 00:51:02.000
Now you're opening up a whole can of worms that, like, look daunting to tackle.

475
00:51:02.000 --> 00:51:08.000
But once you know what these are, it's a lot easier to, like, work around them.

476
00:51:08.000 --> 00:51:11.000
And there's a lot of, like, providers you can use or custom build.

477
00:51:11.000 --> 00:51:22.000
One thing that we can, like, learn from is, like, OpenBot, which is something that we've built for enterprise in mind.

478
00:51:22.000 --> 00:51:39.000
So we have key guardrails there, like, and audit logs for the genetic applications, which is an important piece, especially that you need to keep track of all of these user and agent actions across the use case.

479
00:51:39.000 --> 00:51:43.000
So it's not just for improving the application.

480
00:51:43.000 --> 00:51:51.000
Some use cases and some functions require this for compliance and regulatory purposes.

481
00:51:51.000 --> 00:51:55.000
So you need to, like, set these up in a way that supports that.

482
00:51:55.000 --> 00:51:59.000
Again, all of our, like, software here is mostly open source.

483
00:51:59.000 --> 00:52:01.000
So you can, like, self-host them.

484
00:52:01.000 --> 00:52:07.000
If you have any, like, if you need any help, like, deploying that, you can talk to our team in setting that up.

485
00:52:07.000 --> 00:52:12.000
So everything you saw today is ready to deploy in the repository that Nathan has shared.

486
00:52:12.000 --> 00:52:17.000
If you weren't able to gather the link, it's just on the, this functional copilot's app here.

487
00:52:17.000 --> 00:52:21.000
And then one, let's see a Paul here.

488
00:52:21.000 --> 00:52:24.000
It's like, have you ever built a functional copilot?

489
00:52:24.000 --> 00:52:28.000
And, like, love to see if, like, where people are out of that.

490
00:52:28.000 --> 00:52:37.000
And then last thing, if you are going to take anything away from this song, it's that when you're building a functional copilot,

491
00:52:37.000 --> 00:52:46.000
this means that you have to build an environment where your users and agents can work on tasks together.

492
00:52:46.000 --> 00:52:51.000
And across all different use cases, across all different functions, this remains to be true.

493
00:52:51.000 --> 00:53:00.000
Because, like, our workflows have now just merged and some industries have adapted faster than others.

494
00:53:00.000 --> 00:53:04.000
So you have to figure out, like, what are the functions that we get into this.

495
00:53:04.000 --> 00:53:11.000
And then lastly, if you build it with CopilotKit, you get to learn from both your user actions and your agent traces.

496
00:53:11.000 --> 00:53:20.000
And that just allows you to, like, get a lot of, tap into a data source that is mostly unavailable otherwise.

497
00:53:20.000 --> 00:53:24.000
So that's a key important piece here. Okay.

498
00:53:24.000 --> 00:53:28.000
That's pretty much it. Nathan?

499
00:53:28.000 --> 00:53:31.000
Awesome.

500
00:53:31.000 --> 00:53:36.000
Super educational, Jerel.

501
00:53:36.000 --> 00:53:42.000
A lot of people, I'm sure, are also interested in the slides. Will those be available?

502
00:53:42.000 --> 00:53:45.000
Yes. Very nice. Yeah.

503
00:53:45.000 --> 00:53:49.000
Yeah, we can definitely share.

504
00:53:49.000 --> 00:53:53.000
We will be sharing the recording and we'll also share the slides.

505
00:53:53.000 --> 00:53:58.000
And also the QR code is for CLiF.

506
00:53:58.000 --> 00:54:05.000
So it's everything that Jerel talked about in getting your agent to learn.

507
00:54:05.000 --> 00:54:14.000
And it's using AG-UI to do that under the hood between every user interaction in your application.

508
00:54:14.000 --> 00:54:22.000
And then teaching your agent or guiding your agent or modifying your agent.

509
00:54:22.000 --> 00:54:27.000
The goal is to improve your product.

510
00:54:27.000 --> 00:54:31.000
So everything is self-hostable and it's still in.

511
00:54:31.000 --> 00:54:35.000
We launched it a few weeks ago, but it's still in early access.

512
00:54:35.000 --> 00:54:43.000
So we would love to hear from people that use it and get your feedback on it.

513
00:54:43.000 --> 00:54:48.000
You can talk to our team about it and they will get you all set up.

514
00:54:48.000 --> 00:54:50.000
So, amazing.

515
00:54:50.000 --> 00:54:55.000
We are, I know we had questions.

516
00:54:55.000 --> 00:54:58.000
So we have a couple minutes left.

517
00:54:58.000 --> 00:55:04.000
There was, let's see, let me scroll up.

518
00:55:04.000 --> 00:55:19.000
One question we had Jerel from Faramars asked, can I push a signal fan out into Slack channel

519
00:55:19.000 --> 00:55:23.000
or DM and take a reply back into the same agent thread?

520
00:55:23.000 --> 00:55:29.000
And when does agent tag leave preview? Yes.

521
00:55:29.000 --> 00:55:33.000
So that's something you can do right now with the open source software.

522
00:55:33.000 --> 00:55:38.000
And if you need help setting that up, you can always talk to our engineers

523
00:55:38.000 --> 00:55:42.000
and we can walk you through how to self-host this.

524
00:55:42.000 --> 00:55:44.000
But I personally have done that with OpenBot.

525
00:55:44.000 --> 00:55:49.000
Like, hey, here's our open source version of Grockbot.

526
00:55:49.000 --> 00:55:55.000
And then if I want to carry the conversation in Slack and bring that back into OpenBot

527
00:55:55.000 --> 00:55:58.000
and then have a way to connect the two together,

528
00:55:58.000 --> 00:56:01.000
it's something you can do with channels, SDK, Slack integration.

529
00:56:01.000 --> 00:56:08.000
And then there's still some pieces in our manual that you have to figure out in terms of deployment.

530
00:56:08.000 --> 00:56:15.000
But having the SDK makes it a lot easier to match and integrate across what you're building.

531
00:56:15.000 --> 00:56:17.000
It doesn't have to be OpenBot.

532
00:56:17.000 --> 00:56:22.000
Like any application you build towards, you can just integrate with our channels, SDK.

533
00:56:22.000 --> 00:56:34.000
And it will give you access to not just Slack, but you can do that in iMessage teams and whatnot.

534
00:56:34.000 --> 00:56:41.000
And in terms of bidirectionalness, you can have that conversation in Slack

535
00:56:41.000 --> 00:56:44.000
and then continue that in OpenBot and then request for contacts in Slack.

536
00:56:44.000 --> 00:56:48.000
And that's something that you can do or ask your agent in OpenBot.

537
00:56:48.000 --> 00:56:54.000
Like, hey, send this message out in Slack and it's going to show that. Yeah.

538
00:56:54.000 --> 00:57:00.000
Yeah, totally. And then so other people were asking throughout the presentation.

539
00:57:00.000 --> 00:57:06.000
By the way, Jerel, you got a lot of claps. Oh, thank you. Mega props. Yeah.

540
00:57:06.000 --> 00:57:10.000
Yeah, it was really good.

541
00:57:10.000 --> 00:57:13.000
And we are at time.

542
00:57:13.000 --> 00:57:22.000
But I just shared our Discord link because if there's any questions that didn't get covered,

543
00:57:22.000 --> 00:57:26.000
please join our Discord. We're happy to answer. I'm hanging out there.

544
00:57:26.000 --> 00:57:31.000
Jerel is hanging out there. Most of our engineers are there as well.

545
00:57:31.000 --> 00:57:34.000
Well, how many times can I say as well?

546
00:57:34.000 --> 00:57:44.000
There's thousands of AI builders or people building agent applications that are great

547
00:57:44.000 --> 00:57:50.000
to network with. And so the community as a whole is amazing.

548
00:57:50.000 --> 00:57:55.000
And so you're welcome, Will. We appreciate it.

549
00:57:55.000 --> 00:57:57.000
So thank you, everyone.

550
00:57:57.000 --> 00:58:04.000
We're sorry we didn't get to every single question, but please, yeah, just join our Discord

551
00:58:04.000 --> 00:58:07.000
and we can continue the conversation there.

552
00:58:07.000 --> 00:58:14.000
Jerel, I want to thank you again for, it's a lot of work putting, we've been planning this

553
00:58:14.000 --> 00:58:19.000
for a while, so I really appreciate it. And thanks everyone for coming.

554
00:58:19.000 --> 00:58:20.000
Thanks for having me.

555
00:58:20.000 --> 00:58:28.000
We'll see you all in Discord or on X or on LinkedIn.

556
00:58:28.000 --> 00:58:32.000
Reach out to us. Thank you. Cheers. Bye-bye.

