WEBVTT

00:00:17.846 --> 00:00:24.233
<v Matt>Hello nina oh matt i have a guest today you do this

00:00:25.008 --> 00:00:31.813
<v Matt>is ben ben yeah ben's a buddy of mine out in berlin with me he's been saving

00:00:31.813 --> 00:00:35.333
<v Matt>the world with various startups he's been working now let you introduce yourself

00:00:35.333 --> 00:00:38.253
<v Ben>some more about uh i'm ben

00:00:39.094 --> 00:00:45.663
<v Ben>the software engineer i studied compute engineering and then became a software

00:00:45.663 --> 00:00:48.878
<v Ben>engineer that was about 20 years ago.

00:00:49.475 --> 00:00:52.573
<v Ben>I've been with four different startups

00:00:52.573 --> 00:00:56.908
<v Ben>and in between I worked for larger companies and quickly fled those.

00:00:57.296 --> 00:01:01.913
<v Ben>The CTO that I originally worked for, my first job out of college,

00:01:01.913 --> 00:01:07.193
<v Ben>she compared it to being a pirate at a startup or working for the Navy.

00:01:07.533 --> 00:01:13.047
<v Ben>So the pirate lifestyle really appealed to me. So that's why I keep joining startups.

00:01:15.493 --> 00:01:16.597
<v Matt>Yes. Go ahead.

00:01:17.107 --> 00:01:21.108
<v Nina>I was going to ask, is it you prefer the startups because there's less red tape

00:01:21.230 --> 00:01:25.423
<v Nina>and more freedom and of technologies you can choose and stuff like that?

00:01:25.423 --> 00:01:28.089
<v Nina>Or I guess, why do you prefer startups?

00:01:29.212 --> 00:01:35.158
<v Ben>I think it's a personal preference, and neither of them, I would say,

00:01:35.158 --> 00:01:38.228
<v Ben>is wrong, and everybody has their own preference.

00:01:38.228 --> 00:01:46.966
<v Ben>But I like startups for the ability to bring on larger changes in the product and in the code base.

00:01:47.213 --> 00:01:50.958
<v Ben>There's much more space for experimentation and exploration.

00:01:51.717 --> 00:01:58.344
<v Ben>At a startup that I worked at, we really leaned hard into microservices and

00:01:58.965 --> 00:02:03.348
<v Ben>almost every two weeks we would tear down a microservice and rebuild it,

00:02:03.348 --> 00:02:04.778
<v Ben>sometimes in a different language.

00:02:05.078 --> 00:02:11.248
<v Ben>And that is obviously harder in a larger infrastructure, in a larger company

00:02:11.248 --> 00:02:18.184
<v Ben>where there's significantly more stakeholders, where the product is much more mature and where

00:02:19.113 --> 00:02:23.509
<v Ben>I would say product fit has already been established. Some of the startups that

00:02:23.509 --> 00:02:29.279
<v Ben>I've worked for had to pivot, and that I think is also a very interesting process

00:02:29.279 --> 00:02:32.289
<v Ben>that I was really happy to be part of.

00:02:32.289 --> 00:02:37.859
<v Ben>Obviously, it's a bit of a downer to figure out that your business model doesn't

00:02:37.859 --> 00:02:43.405
<v Ben>quite work, but being able to create something new is very exciting to me and it's very appealing.

00:02:44.273 --> 00:02:46.635
<v Matt>What size are you guys at now, developer-wise?

00:02:47.511 --> 00:02:52.161
<v Ben>I think, don't quote me on it, but I think we're 12.

00:02:54.085 --> 00:03:02.032
<v Ben>Yes, we have some integration engineers and I would like to have them be included as engineers.

00:03:02.842 --> 00:03:08.049
<v Ben>And I think the threshold for what is an engineer now with AI has definitely

00:03:08.049 --> 00:03:10.612
<v Ben>changed. But yeah, about a dozen.

00:03:11.413 --> 00:03:18.149
<v Matt>And how many products would you say? Like just one product line or multiple? We have three products.

00:03:18.778 --> 00:03:22.943
<v Ben>The first one was a wedge product. The company is called Meadow.

00:03:23.510 --> 00:03:28.499
<v Ben>It's in a higher ed space. So we take tuition payments and process them for

00:03:28.499 --> 00:03:33.243
<v Ben>students and really provide more transparency in that process.

00:03:33.928 --> 00:03:39.089
<v Ben>This is the second higher ed startup that I've joined. The first one is called Simplicity.

00:03:39.362 --> 00:03:42.638
<v Ben>And that was also my first job straight out of the college.

00:03:43.145 --> 00:03:47.999
<v Ben>It's definitely a very interesting space. Higher ed, a lot of very smart people

00:03:47.999 --> 00:03:50.032
<v Ben>with really bad software.

00:03:50.195 --> 00:03:55.539
<v Ben>I don't know if you are familiar with that space at all, but they're very reluctant

00:03:55.539 --> 00:04:00.542
<v Ben>to change. Some of these institutions are hundreds of years old,

00:04:01.248 --> 00:04:03.289
<v Ben>and a lot of them are also public.

00:04:03.589 --> 00:04:09.249
<v Ben>So that brings its own kind of hurdles in terms of regulations and rules that need to be followed.

00:04:09.765 --> 00:04:13.909
<v Ben>The release cadence kind of follows the academic year. So sometimes it's very

00:04:13.909 --> 00:04:16.207
<v Ben>hard to release something mid-year

00:04:17.162 --> 00:04:19.285
<v Ben>But yeah, it's been very interesting.

00:04:20.124 --> 00:04:24.321
<v Nina>So with your experience then in startups where, you know, you kind of have to

00:04:24.321 --> 00:04:28.778
<v Nina>refactor something or completely redo an entire microservice,

00:04:30.392 --> 00:04:36.835
<v Nina>how has AI then kind of changed like your day-to-day workflow in a startup?

00:04:37.810 --> 00:04:42.501
<v Ben>A specific example where we started to cut you off, but that specific example

00:04:42.501 --> 00:04:46.402
<v Ben>back then we still used to handwrite code. But the olden days,

00:04:46.721 --> 00:04:49.651
<v Ben>we still had IDEs and all that good stuff.

00:04:51.215 --> 00:04:56.836
<v Ben>I definitely, I mean, development as a whole has gotten so much faster, right?

00:04:56.836 --> 00:05:02.966
<v Ben>And rewriting something with well-defined contracts, which is obviously what

00:05:02.966 --> 00:05:07.918
<v Ben>enabled us to swap these microservices in and out, having well-defined contracts.

00:05:08.304 --> 00:05:14.736
<v Ben>I would imagine it would be significantly faster now. And if it is Go or JavaScript

00:05:14.736 --> 00:05:21.576
<v Ben>or TypeScript or whatever language, I think it has become less relevant now. with AI.

00:05:21.896 --> 00:05:27.366
<v Ben>I don't I've coded in all of these languages I've also coded in Go and I've

00:05:27.366 --> 00:05:30.716
<v Ben>done Ruby on Rails and I think

00:05:31.874 --> 00:05:37.006
<v Ben>Yeah I wish we had AI back then to kind of coach us and kind of create these

00:05:37.006 --> 00:05:42.712
<v Ben>things faster, allow us to do much more fast prototyping to experiment more.

00:05:43.663 --> 00:05:51.355
<v Matt>Would you say like in, where's the bottleneck now? Like for the larger corporations, it's the people.

00:05:51.680 --> 00:05:54.838
<v Matt>They can't keep up at the pace of the developers anymore.

00:05:55.512 --> 00:05:58.496
<v Matt>They're just cranking out code too fast.

00:05:59.575 --> 00:06:04.086
<v Ben>It's funny that you say I'm sitting on six PRs right now waiting for reviews

00:06:04.086 --> 00:06:09.632
<v Ben>and it's a real problem that we've been discussing a lot at Neto.

00:06:10.383 --> 00:06:18.196
<v Ben>We've enabled product now to submit PRs but they don't they don't review them,

00:06:18.196 --> 00:06:20.496
<v Ben>so they're just contributing further to the problem.

00:06:23.424 --> 00:06:29.356
<v Ben>We obviously have Claude look over and provide code reviews and score them and

00:06:29.756 --> 00:06:32.376
<v Ben>a lot of our reviewers know based on that and

00:06:33.747 --> 00:06:38.541
<v Ben>I still find myself looking at the file changes to kind of double-check that,

00:06:38.541 --> 00:06:44.791
<v Ben>but I feel like that's maybe a habit that I have to get rid of, right?

00:06:44.791 --> 00:06:48.871
<v Ben>I still want to understand how the code works and to truly fully understand

00:06:48.871 --> 00:06:52.259
<v Ben>how something is implemented rather than just reading the blurb.

00:06:53.030 --> 00:06:58.481
<v Ben>That Claude or Codex pops out with the warnings and whatnot.

00:06:59.142 --> 00:07:05.301
<v Ben>But yeah, it's code reviews right now to have a better understanding of what

00:07:05.301 --> 00:07:09.901
<v Ben>we're actually creating, which I think we're somewhat losing touch.

00:07:09.901 --> 00:07:13.081
<v Ben>This has been a comment that I've provided in the past.

00:07:14.111 --> 00:07:19.218
<v Ben>We're so fast to create things, we're losing touch a little bit with what exactly we're creating.

00:07:19.923 --> 00:07:25.211
<v Nina>Yeah this is that is a huge issue that i mean i experienced it at my last gig

00:07:25.211 --> 00:07:26.837
<v Nina>where things were moving too fast,

00:07:27.464 --> 00:07:34.681
<v Nina>and just using using code like automation or pr reviews pr pr review automation

00:07:34.681 --> 00:07:37.015
<v Nina>with Claude or whatever tool,

00:07:37.723 --> 00:07:39.546
<v Nina>i i don't want to lose touch with,

00:07:40.261 --> 00:07:42.761
<v Nina>like what we're actually merging myself.

00:07:42.761 --> 00:07:51.526
<v Nina>So I know that a company's project manager would potentially get upset if you're,

00:07:52.107 --> 00:07:55.061
<v Nina>spending extra time looking at differences in files.

00:07:55.061 --> 00:08:02.001
<v Nina>But I think every dev I've spoken to has felt like we're losing touch with what

00:08:02.001 --> 00:08:03.127
<v Nina>is actually being written.

00:08:03.806 --> 00:08:06.691
<v Nina>And I feel like that's down the line.

00:08:08.335 --> 00:08:13.500
<v Nina>We'll always have AI to kind of correct things now, but I don't want to have

00:08:13.500 --> 00:08:20.000
<v Nina>like a fire drill randomly in my career because I didn't use my own eyes to

00:08:20.000 --> 00:08:21.752
<v Nina>look at code, if that makes sense.

00:08:22.703 --> 00:08:28.020
<v Nina>And although like fire drills are like far few between because of AI,

00:08:28.020 --> 00:08:33.026
<v Nina>it's still something that I miss doing my code reviews a little bit.

00:08:33.592 --> 00:08:37.979
<v Nina>Matt, you said you're code reviewing six PRs right now at once. Yep.

00:08:38.171 --> 00:08:43.610
<v Matt>So has Ben. Ben's got nine, maybe? I submitted them. I'm just waiting for somebody

00:08:43.610 --> 00:08:47.605
<v Matt>to tickle I only got three of my own out there.

00:08:48.287 --> 00:08:50.600
<v Nina>Yeah. And actually, there was

00:08:50.600 --> 00:08:57.381
<v Nina>a video that was released this week by the creator of Rails, I believe.

00:08:57.960 --> 00:09:04.090
<v Nina>And he was going into a whole bunch of topics about just the speed of things

00:09:04.090 --> 00:09:09.563
<v Nina>and the ability to program in different languages now, but Everybody's coding less.

00:09:09.719 --> 00:09:14.790
<v Nina>And I always think, how is that going to affect, how is that going to even affect

00:09:14.790 --> 00:09:18.586
<v Nina>our ability to review code down the line if we're not coding as much?

00:09:19.752 --> 00:09:23.381
<v Nina>So I always think, like, where's that balance now?

00:09:23.970 --> 00:09:31.095
<v Nina>He has a line in this video where he says, he asks the audience at the summit,

00:09:31.675 --> 00:09:34.735
<v Nina>how many of you in the audience actually write a line of code now?

00:09:35.390 --> 00:09:41.380
<v Nina>And five people raised their hand in a room of like hundreds and just with that

00:09:41.380 --> 00:09:44.762
<v Nina>in mind i mean is there anything that you guys do,

00:09:45.593 --> 00:09:49.970
<v Nina>do you guys actually take the time every once in a while to code something by

00:09:49.970 --> 00:09:54.620
<v Nina>hand and maybe just use Claude as like an assistant in a new language because,

00:09:55.750 --> 00:09:58.570
<v Nina>i do that occasionally because i'm always worried about skill regression.

00:09:59.771 --> 00:10:05.215
<v Nina>When it comes to actually architecting and coding because i learned how to architect,

00:10:05.958 --> 00:10:12.460
<v Nina>applications through coding and making mistakes along the way so this is a question

00:10:12.843 --> 00:10:16.447
<v Nina>do any of you code for yourself by hand anymore,

00:10:17.439 --> 00:10:19.122
<v Nina>no no okay.

00:10:19.342 --> 00:10:22.030
<v Matt>Even for like a one-line change i'll have claude do it

00:10:22.828 --> 00:10:24.214
<v Nina>Really all okay.

00:10:25.183 --> 00:10:30.168
<v Ben>I think the best analogy that i've heard is i know i know how to drive a stick

00:10:30.168 --> 00:10:32.018
<v Ben>shift but in an automatic car

00:10:32.888 --> 00:10:40.078
<v Ben>why i think the knowledge is good to to to understand how gears work and how

00:10:40.078 --> 00:10:44.168
<v Ben>a clutch works and and whatnot and gives you a better appreciation i think it

00:10:44.168 --> 00:10:45.719
<v Ben>makes you a better driver but

00:10:46.596 --> 00:10:50.978
<v Ben>like same as in today's cars they have even automatic cars allow you to shift

00:10:50.978 --> 00:10:53.298
<v Ben>up and down but why would I want to do that?

00:10:53.758 --> 00:10:56.785
<v Ben>And even worse so, I don't have the time to do that.

00:10:57.469 --> 00:10:57.731
<v Nina>Right.

00:10:58.614 --> 00:10:58.918
<v Matt>I think

00:11:01.313 --> 00:11:08.274
<v Matt>we're hitting a wall with AI and Agile development is not working anymore.

00:11:09.158 --> 00:11:15.358
<v Matt>We had the same thing happen with Waterfall and Agile when the convention over

00:11:15.358 --> 00:11:18.223
<v Matt>config started happening and you were able to crank out some more code.

00:11:18.554 --> 00:11:21.243
<v Matt>And that started to push more into Agile.

00:11:21.684 --> 00:11:23.948
<v Matt>Now we have AI, we're going faster.

00:11:24.465 --> 00:11:28.948
<v Matt>And I think really where it's hitting and seems Ben's also hitting it,

00:11:28.948 --> 00:11:36.328
<v Matt>like PR is where is like the tip of the spear of the breakdown of agile.

00:11:36.328 --> 00:11:38.084
<v Matt>And I think PR is going to have to go away.

00:11:38.742 --> 00:11:40.652
<v Matt>I don't think we can rely on them.

00:11:41.528 --> 00:11:45.957
<v Matt>Human PRs, it's going to have to be agentic reviews.

00:11:46.995 --> 00:11:51.571
<v Ben>I mean, it's checkpoints, right? We need more checkpoints.

00:11:51.931 --> 00:11:57.201
<v Ben>And over the last year, more so than ever, my work style has changed.

00:11:57.201 --> 00:12:02.110
<v Ben>At first, AI would generate the code and I would still write the unit tests, right?

00:12:02.434 --> 00:12:06.159
<v Ben>To at least know and guard against,

00:12:06.426 --> 00:12:11.140
<v Ben>to be able to write down the scenarios that I want to guard against.

00:12:11.882 --> 00:12:15.901
<v Ben>Then that went away and I was never really a huge fan of writing tests to begin

00:12:15.901 --> 00:12:22.503
<v Ben>with. but then we relied on end-to-end tests and now there is no time for that.

00:12:23.518 --> 00:12:26.920
<v Matt>There's no time to write them or there's no time to run the suites?

00:12:27.735 --> 00:12:31.371
<v Ben>The suites are definitely being run but I'm not generating them anymore.

00:12:31.371 --> 00:12:35.641
<v Ben>I'm not handwriting them anymore. I'm not instructing Cypress to say,

00:12:35.641 --> 00:12:41.820
<v Ben>okay, now you click on this button and that button now. It's all done via code, via Claude.

00:12:42.040 --> 00:12:53.137
<v Nina>Yeah, now that Agile methodology is kind of not as fruitful as it once was, and we require, I guess,

00:12:54.194 --> 00:12:57.258
<v Nina>I mean, do you guys think we need less checkpoints, right?

00:12:59.136 --> 00:13:05.970
<v Nina>I just wonder how it's going to change if we're not doing PR review anymore.

00:13:06.242 --> 00:13:12.220
<v Nina>Let's say PRs hypothetically fall to the side because we put so much trust in agents.

00:13:12.948 --> 00:13:17.858
<v Nina>Do you think companies are going to adopt maybe having an end of sprint kind

00:13:17.858 --> 00:13:22.713
<v Nina>of review where you kind of go over everything that's been merged in? And I guess,

00:13:24.798 --> 00:13:28.578
<v Nina>where do you guys see the next kind of, what do you guys see as like the next

00:13:28.578 --> 00:13:36.064
<v Nina>steps? Like what would be a good plan other than agile for modern AI development?

00:13:36.825 --> 00:13:41.638
<v Nina>Is there anything like in your day-to-day that you wish people implemented or

00:13:41.638 --> 00:13:47.468
<v Nina>your project manager implemented that kind of accompanied this new fast-paced

00:13:47.468 --> 00:13:48.648
<v Nina>kind of work environment?

00:13:49.755 --> 00:13:53.668
<v Matt>It's rough. i mean trying to invent a whole new system off the top of my head

00:13:53.807 --> 00:13:57.388
<v Nina>Right right but you know while you're working do you ever think man i wish we

00:13:57.388 --> 00:14:01.680
<v Nina>had a system for this or that or this or that i mean like,

00:14:03.058 --> 00:14:08.378
<v Nina>i i i think like i've thought with everything moving fast like everybody we

00:14:08.378 --> 00:14:10.197
<v Nina>were working together matt i thought that,

00:14:11.053 --> 00:14:14.928
<v Nina>i had the idea like oh i wish we had like yeah an end of sprint meeting where

00:14:14.928 --> 00:14:17.380
<v Nina>we go over absolutely everything that's been

00:14:17.653 --> 00:14:21.548
<v Nina>that's been merged or just have an overview view of every PR and everybody look

00:14:21.548 --> 00:14:27.208
<v Nina>at it together and just have like a big overarching like kind of architecture discussion.

00:14:27.671 --> 00:14:30.697
<v Matt>You heard of shape-up or Linear cycles

00:14:31.424 --> 00:14:32.667
<v Nina>No I haven't.

00:14:33.293 --> 00:14:39.278
<v Matt>So basically the shape-up version shape-up and Linear cycles are basically the

00:14:39.278 --> 00:14:41.390
<v Matt>same thing where you have like six week sprints

00:14:42.066 --> 00:14:47.988
<v Matt>and then you have like a three-week cooldown afterwards and that would be like

00:14:47.988 --> 00:14:53.149
<v Matt>kind of where you fix bugs you do kind of architecture reviews of what's happened

00:14:54.192 --> 00:14:58.099
<v Matt>That could possibly be a new system that pops out of this.

00:14:58.673 --> 00:15:04.457
<v Matt>But that's also written up by DHH as well, ShapeUp is, and then Linear kind

00:15:04.457 --> 00:15:05.586
<v Matt>of took it and ran with it.

00:15:05.935 --> 00:15:10.204
<v Matt>So Linear is based off of longer cycles, not just two-week sprints.

00:15:11.847 --> 00:15:16.546
<v Matt>The idea kind of being that two weeks is time too small to do anything.

00:15:17.547 --> 00:15:24.885
<v Ben>These days, two weeks is too long. so you get a lot done in two weeks compared to before

00:15:25.694 --> 00:15:31.306
<v Nina>Yeah yeah it seems like this type of methodology shape up it,

00:15:32.130 --> 00:15:39.197
<v Nina>six weeks is a long long span is that like six weeks before release like i've

00:15:39.197 --> 00:15:42.711
<v Nina>never heard of companies i guess using this before.

00:15:43.373 --> 00:15:48.297
<v Matt>Yeah i've only heard of Basecamp using it. I've never heard of anyone like

00:15:48.297 --> 00:15:50.125
<v Matt>in a larger company using it.

00:15:51.039 --> 00:15:55.228
<v Matt>It's just too loose for most larger companies.

00:15:55.349 --> 00:16:01.852
<v Matt>They want the kind of two-week control over what gets done and not a divergence

00:16:02.488 --> 00:16:06.043
<v Matt>between like an infrastructure team and an application team.

00:16:07.111 --> 00:16:12.227
<v Ben>We used it for a second at the startup that I'm currently at and we abandoned

00:16:12.227 --> 00:16:13.604
<v Matt>it. What happened? What happened?

00:16:14.676 --> 00:16:16.312
<v Ben>It cycles for too long.

00:16:19.408 --> 00:16:25.741
<v Ben>I also, by the way, I hate this intense focus, and I think it's American,

00:16:26.101 --> 00:16:28.222
<v Ben>on finishing something at the end of the quarter.

00:16:29.235 --> 00:16:33.032
<v Ben>I don't know if you guys ever experienced it, but for a lot of startups,

00:16:34.472 --> 00:16:38.812
<v Ben>I did a stint at PayPal, and at the end of the quarter, everything had to be done.

00:16:39.312 --> 00:16:43.322
<v Ben>Something had to be delivered, and if a project didn't fit, it either didn't

00:16:43.322 --> 00:16:47.060
<v Ben>get tackled or it got broken up. But yeah, we're

00:16:49.551 --> 00:16:53.512
<v Ben>I mean, we've also been changing things fairly rapidly.

00:16:53.892 --> 00:17:00.632
<v Ben>So we used to be two weeks friends, then six weeks cycles, and we're trying

00:17:00.632 --> 00:17:05.377
<v Ben>new things, which is also something that gets afforded to us by being a startup.

00:17:06.438 --> 00:17:08.892
<v Matt>Yeah, like the quarterly stuff's coming from the top usually,

00:17:08.892 --> 00:17:13.072
<v Matt>because you have your quarterly deadlines and that gets pushed down to managers'

00:17:13.072 --> 00:17:16.075
<v Matt>reviews and the managers have to then hit things on the quarter.

00:17:16.822 --> 00:17:21.401
<v Ben>The quarterly reports also for public companies are also very intense.

00:17:22.568 --> 00:17:28.564
<v Ben>I've heard DHH and conventional configuration. Am I amongst Rails people?

00:17:28.929 --> 00:17:34.455
<v Matt>No, I've done Rails stuff, but also Spring picked it up. So Java does it now, too.

00:17:35.996 --> 00:17:41.637
<v Matt>Java prefers to keep it to annotations instead, but a lot of the stuff is spilled over.

00:17:42.425 --> 00:17:46.815
<v Matt>And I've done a lot of Python, like FastAPI and stuff like that,

00:17:47.099 --> 00:17:48.649
<v Matt>which is just Python Rails.

00:17:49.727 --> 00:17:56.416
<v Ben>I've done Ruby on Rails for about a year for a first startup out of DC.

00:17:56.836 --> 00:18:02.352
<v Ben>And we went to a Ruby on Rails conference in Portland, out of all places.

00:18:03.050 --> 00:18:04.761
<v Ben>And it felt a little culty.

00:18:05.460 --> 00:18:09.016
<v Matt>It is culty. Have you watched the video? It's great. He basically tells them

00:18:09.016 --> 00:18:10.824
<v Matt>Rails is stupid and stopped doing Rails.

00:18:11.607 --> 00:18:13.609
<v Nina>And he made Rails. Yeah.

00:18:14.271 --> 00:18:17.854
<v Matt>He's just like, we're done. We're done. What are we doing here, guys?

00:18:18.242 --> 00:18:21.296
<v Nina>Yeah he became a big rust guy yeah because of,

00:18:22.462 --> 00:18:27.436
<v Nina>like he said i hate he said i used to hate rust when i had to look at it now

00:18:27.436 --> 00:18:30.386
<v Nina>because i don't have to look at it and an agent can write it i love it it's

00:18:30.386 --> 00:18:36.171
<v Nina>the best thing ever i hate rails it's really funny yeah i think we linked it in the document.

00:18:36.746 --> 00:18:41.326
<v Matt>Yeah we'll put in the show notes too yes you should definitely watch it yeah

00:18:41.326 --> 00:18:49.663
<v Matt>he just like he just rails no pun intended into other developers and it's basically

00:18:50.636 --> 00:18:57.236
<v Matt>he says 37Signals doesn't write code anymore all the AI writes the code he reviews

00:18:57.236 --> 00:19:04.145
<v Matt>some of it and so do other people but mostly it's getting like checked automatically with AI and merged in

00:19:05.205 --> 00:19:10.066
<v Ben>there's still a sunload progression in my YouTube consumption I used to listen

00:19:10.066 --> 00:19:14.816
<v Ben>to things at 1.25 and then I cranked it out to 1.5.

00:19:14.816 --> 00:19:18.127
<v Ben>Now I just put it into Gemini and tell it to summarize it for me.

00:19:19.456 --> 00:19:21.653
<v Matt>I think that's exactly what Nina did for the video.

00:19:21.993 --> 00:19:26.351
<v Nina>Yeah, that is exactly what I did. I have it open right now on my computer.

00:19:27.408 --> 00:19:32.253
<v Nina>It transcribed it with Whisper, and then I pulled out keynotes because I just

00:19:32.253 --> 00:19:35.454
<v Nina>didn't want to listen to him talk, and I could just read it faster.

00:19:38.522 --> 00:19:44.853
<v Nina>And that's actually one of the projects. I don't know if you showed Ben ETA brain. No, I haven't.

00:19:45.659 --> 00:19:50.163
<v Matt>We were talking about it earlier. here but yeah yeah but he also goes into that

00:19:50.163 --> 00:19:54.163
<v Matt>too he's into our idea of agents as search

00:19:55.061 --> 00:20:02.735
<v Matt>basically the idea is like Elasticsearch is garbage and so unless you're like google able to like do

00:20:03.813 --> 00:20:08.223
<v Matt>the customization needed to get what you need out of it it's a lot better just

00:20:08.223 --> 00:20:13.189
<v Matt>to have with an agent that could go through it and do the search for you.

00:20:14.281 --> 00:20:18.713
<v Matt>If, especially for us, like we're just transcribing podcasts,

00:20:18.713 --> 00:20:22.789
<v Matt>so the data size is not large.

00:20:22.975 --> 00:20:27.549
<v Matt>So you can grep through the whole thing fast enough with enough agents just ripping through it.

00:20:28.031 --> 00:20:33.101
<v Matt>As long as you can spin up those reps fast enough, it works well.

00:20:35.131 --> 00:20:36.292
<v Ben>I just have done.

00:20:36.930 --> 00:20:37.940
<v Nina>Oh, sorry.

00:20:38.248 --> 00:20:38.939
<v Ben>Sorry, go ahead.

00:20:39.252 --> 00:20:44.767
<v Nina>I was going to say we're thinking about how to add Jev into it as well. That's like next task.

00:20:45.441 --> 00:20:49.829
<v Nina>If you think it's something that you want to do, Matt, I'm kind of curious for that.

00:20:51.104 --> 00:20:53.524
<v Matt>Yeah, I was wondering where you got to it. Jev is just a classifier.

00:20:53.524 --> 00:20:58.644
<v Matt>Basically, it would help you pick if results returned are good or not,

00:20:58.644 --> 00:21:05.026
<v Matt>and then it would be a filter step in the process of the agent's aggregating data.

00:21:06.098 --> 00:21:12.144
<v Ben>I did during the last episode you guys talked about Jev and I mostly agree

00:21:12.144 --> 00:21:14.124
<v Ben>with it, it's a classifier

00:21:15.509 --> 00:21:20.304
<v Ben>I do think the one legitimate use case that I've seen so far is picking a cheaper

00:21:20.304 --> 00:21:25.714
<v Ben>model or analyzing a prompt and then directing it towards a model to have a

00:21:25.714 --> 00:21:27.237
<v Ben>little bit more cost control

00:21:27.920 --> 00:21:33.714
<v Matt>I think since that episode an open source model has come out so Jev was was

00:21:33.714 --> 00:21:38.559
<v Matt>proprietary and the only thing out there for I think three or four days

00:21:39.584 --> 00:21:43.673
<v Matt>which is amazing by the way the open source one's called Jeff

00:21:44.471 --> 00:21:45.051
<v Nina>Oh my gosh,

00:21:47.700 --> 00:21:53.362
<v Nina>Wow, that's four dates. Things are moving so quickly.

00:21:53.965 --> 00:21:59.360
<v Nina>I don't even know how to keep up anymore. I need an agent to summarize all the

00:21:59.360 --> 00:22:02.325
<v Nina>new things coming out and determine the best one.

00:22:03.341 --> 00:22:08.080
<v Nina>Yeah, so Jeff is completely open source. So is it that you're saying that we

00:22:08.080 --> 00:22:13.715
<v Nina>don't, there's no, I mean, it was cheap before with Jev. Jev wasn't that costly.

00:22:13.870 --> 00:22:17.608
<v Matt>Yeah, and I don't know what the system requirements are for it.

00:22:18.422 --> 00:22:21.313
<v Matt>It may eat up our tiny Hetzner facts.

00:22:22.055 --> 00:22:22.415
<v Nina>Yeah.

00:22:23.019 --> 00:22:30.993
<v Ben>I also think cost controls are coming, right? The days of token matching are long gone.

00:22:31.349 --> 00:22:38.161
<v Ben>I haven't received instructions specifically, but I don't send everything to Fable anymore, right?

00:22:38.621 --> 00:22:42.520
<v Ben>Sonnet or Opus can do the same task.

00:22:42.900 --> 00:22:49.918
<v Ben>Hey, create a feature flag or do this one or two-line PR. I don't need Fable to do it.

00:22:51.220 --> 00:22:56.670
<v Ben>But prices are going to dramatically increase. And I think that is the interesting part about Jev, right?

00:22:56.670 --> 00:23:01.512
<v Ben>It's supposed to reduce costs. It's a cheap alternative, which I don't,

00:23:02.250 --> 00:23:08.610
<v Ben>at least from my point of view and from what I've read, cost controls haven't

00:23:08.610 --> 00:23:10.070
<v Ben>been in the super forefront.

00:23:10.070 --> 00:23:13.856
<v Ben>It's been more about creating better quality and better models.

00:23:14.651 --> 00:23:19.620
<v Matt>I mean, even on the user side, we're still in POC phase for almost everything,

00:23:19.620 --> 00:23:22.070
<v Matt>implementing AI agents.

00:23:22.569 --> 00:23:27.880
<v Matt>Like, you're not really going to care about that up front until you have something

00:23:27.880 --> 00:23:30.743
<v Matt>working and then you're going to fine-tune the costs.

00:23:32.198 --> 00:23:40.911
<v Nina>I personally have never hit my max token usage on like a $200 Claude subscription.

00:23:41.125 --> 00:23:45.358
<v Nina>But I know like if you work with Java or if you work with different types of

00:23:45.728 --> 00:23:49.018
<v Nina>languages that eat up tokens, I know you can hit it pretty quick.

00:23:49.018 --> 00:23:54.118
<v Nina>But even like with making my own harness, I have not hit the token usage.

00:23:54.118 --> 00:23:59.438
<v Nina>I see a lot of people complaining a lot of the time that they hit that token

00:23:59.438 --> 00:24:01.950
<v Nina>usage, but maybe that's where Jev fits in for them.

00:24:02.490 --> 00:24:05.996
<v Nina>Have you guys ever hit your token usage on like a $200 subscription?

00:24:06.850 --> 00:24:11.512
<v Matt>Yeah, only when I'm doing multiple agent work. And that's usually when I'm doing UI work.

00:24:12.105 --> 00:24:18.886
<v Matt>The backend rarely uses that many tokens. Backend works pretty straightforward.

00:24:19.522 --> 00:24:25.058
<v Matt>Where UI work will require images and Cypress tests going back and forth over

00:24:25.058 --> 00:24:27.259
<v Matt>the agent, which will eat up most of the tokens.

00:24:28.370 --> 00:24:35.138
<v Ben>I think something that I've done that definitely I mostly use the plan provided

00:24:35.138 --> 00:24:37.844
<v Ben>to work but if I try to one-shot something

00:24:38.913 --> 00:24:42.878
<v Ben>I've definitely asked it, spin up five agents come up with a plan,

00:24:43.298 --> 00:24:47.598
<v Ben>align them all against each other or sometimes I even tell them to build the

00:24:47.598 --> 00:24:50.382
<v Ben>entire feature and then compare them against each other

00:24:51.718 --> 00:24:56.285
<v Ben>that is expensive that is expensive and we're churning through a ton of tokens

00:24:57.014 --> 00:25:03.438
<v Matt>are you talking about like two different like layouts of uis and show them to you or or differently

00:25:03.937 --> 00:25:10.438
<v Nina>Be building it this way versus that way and doing direct tests performance with

00:25:10.438 --> 00:25:13.741
<v Nina>two different versions i've done that and that's definitely eight up tokens,

00:25:14.491 --> 00:25:18.513
<v Nina>but i don't do it often but there's value to it for sure,

00:25:19.658 --> 00:25:20.974
<v Nina>Is that what you mean?

00:25:21.636 --> 00:25:25.138
<v Ben>Yeah, so in Cursor, there's a multitask mode.

00:25:25.138 --> 00:25:31.152
<v Ben>So spin up five different models, use five different models, use Grok, Fable,

00:25:31.493 --> 00:25:35.118
<v Ben>and whatever else is out there, and then just perform that same task and then

00:25:35.118 --> 00:25:40.402
<v Ben>come to a consensus decision, measure the performance of each solution, essentially.

00:25:41.368 --> 00:25:43.307
<v Nina>That is really, that's, yeah.

00:25:43.882 --> 00:25:50.421
<v Matt>How do you digest it when it comes back? Do you have it make a separate rapport?

00:25:51.107 --> 00:25:53.522
<v Matt>How do you judge what comes out of it?

00:25:54.384 --> 00:25:57.098
<v Ben>I will ask you to summarize the findings.

00:25:58.820 --> 00:26:06.428
<v Ben>I'm having a really hard time reading AI Slop, and I've been trying to add things

00:26:06.428 --> 00:26:09.494
<v Ben>to the agent to make it easily digestible.

00:26:10.414 --> 00:26:16.088
<v Ben>Definitely one late night, I got a response from an agent, and I was just like,

00:26:16.088 --> 00:26:22.368
<v Ben>explain it like I'm five. And then I still didn't get it. And I jumped to explain

00:26:22.368 --> 00:26:24.438
<v Ben>it to me like I'm a golden retriever.

00:26:24.798 --> 00:26:26.492
<v Ben>I just kept on going down.

00:26:28.380 --> 00:26:34.633
<v Ben>But yeah, it's difficult to search through that. But yeah, for that specific

00:26:34.750 --> 00:26:39.478
<v Ben>process where I use multitask mode, I have it, summarize it in the end and bring

00:26:39.478 --> 00:26:40.710
<v Ben>me a consensus decision.

00:26:42.182 --> 00:26:45.962
<v Nina>I did that with ETA Brain. And that was the first time I've done that.

00:26:45.962 --> 00:26:50.427
<v Nina>I asked it to look at the past two branches that had been merged where,

00:26:52.950 --> 00:26:59.847
<v Nina>we updated the Elasticsearch. We migrated from having our server in front-end-based

00:27:00.108 --> 00:27:03.322
<v Nina>migrated into the Java code.

00:27:03.322 --> 00:27:09.252
<v Nina>And then I said, okay, let's do a comparison in speed when asking questions.

00:27:09.252 --> 00:27:14.602
<v Nina>And so it checked out master and, you know, it rebuilt everything.

00:27:15.258 --> 00:27:16.899
<v Nina>It did Claude and Chrome.

00:27:17.912 --> 00:27:21.749
<v Nina>And yeah, it took like an hour and a half.

00:27:21.946 --> 00:27:27.212
<v Nina>And I did eat up a bunch of tokens, but it came back with a nice table where

00:27:27.212 --> 00:27:32.337
<v Nina>the speed improvements were in terms of how it went through every single pipeline.

00:27:33.153 --> 00:27:38.151
<v Nina>And that was my first time using AI to do an extreme test like that.

00:27:38.778 --> 00:27:41.622
<v Nina>And it was something that didn't even dawn on me. Like, oh yeah,

00:27:41.622 --> 00:27:45.512
<v Nina>I can do these huge performance tests really quick now with AI,

00:27:45.512 --> 00:27:47.033
<v Nina>whereas that would take...

00:27:48.106 --> 00:27:52.386
<v Nina>Long time to the point where i wouldn't even cross my mind to something feasible

00:27:52.386 --> 00:27:56.901
<v Nina>to do because it would just be such a time time suck but yeah,

00:27:57.589 --> 00:28:03.398
<v Nina>i uh but i still didn't hit my max token so i still didn't i still didn't hit the end which was weird,

00:28:04.204 --> 00:28:08.346
<v Nina>but yeah do you come up with um i'm curious in your workflows ben if you do

00:28:08.346 --> 00:28:13.676
<v Nina>you come up with i guess like consistent performance performance testing like

00:28:13.676 --> 00:28:16.162
<v Nina>in md files like are there i,

00:28:17.009 --> 00:28:21.676
<v Nina>guess how do you how do you log your your your different prompts for doing robust

00:28:21.676 --> 00:28:25.898
<v Nina>testing i guess i'm asking broadly like what,

00:28:26.879 --> 00:28:33.132
<v Nina>how you automate things like is it is it that you do you use like a harness for your ai,

00:28:33.782 --> 00:28:38.986
<v Nina>workflow automation or is it more like you you have your own that you made or

00:28:38.986 --> 00:28:43.290
<v Nina>is it that you just you just rely on like a hierarchy of Claude files and,

00:28:44.062 --> 00:28:48.806
<v Nina>go-tos like what's your setup like when you work day to day it's.

00:28:49.397 --> 00:28:55.841
<v Ben>The last one so we have a number of md files in in our monorepo so

00:28:56.486 --> 00:29:01.046
<v Ben>we switch everything to a monorepo like when we started really started using

00:29:01.046 --> 00:29:06.006
<v Ben>ai to to provide better context and these md files

00:29:06.650 --> 00:29:12.757
<v Ben>then become PR requests that we discuss at a bi-weekly engineering meeting to

00:29:15.121 --> 00:29:19.991
<v Ben>Create better outputs and set certain standards. Not necessarily coding standards,

00:29:19.991 --> 00:29:23.781
<v Ben>but I think lately we had some problems

00:29:23.781 --> 00:29:27.621
<v Ben>with the data model and AI pushing really hard for us to change it.

00:29:28.260 --> 00:29:33.741
<v Ben>But we were quite confident in it. So this is something that we discussed and

00:29:34.181 --> 00:29:39.028
<v Ben>persist and agents on defile and then tell it that this is the way we want it.

00:29:39.740 --> 00:29:44.651
<v Ben>So it's something that kind of grows. It's a, for lack of a better phrase,

00:29:44.651 --> 00:29:50.160
<v Ben>a living document, and we discuss these things as a group, and it goes through the PR process.

00:29:51.251 --> 00:29:55.531
<v Matt>How big are your tickets, you would say, these days? Are they like full features

00:29:55.531 --> 00:29:59.284
<v Matt>now, or are they still broken up into individual pieces?

00:30:00.390 --> 00:30:03.964
<v Ben>I was thankful that recently, that they were too granular.

00:30:04.661 --> 00:30:10.761
<v Ben>I've thought about this, and I like granular tickets it's because I have done some self-reflection.

00:30:11.161 --> 00:30:16.391
<v Ben>Smaller PR reviews gain more feedback. And that is not true anymore,

00:30:16.391 --> 00:30:19.881
<v Ben>right? This just creates more PRs that nobody wants to look at.

00:30:21.198 --> 00:30:26.436
<v Ben>So they should be getting bigger. And that was my aha moment.

00:30:26.825 --> 00:30:32.878
<v Ben>So I would get handed a project and I would instruct to AI, hey,

00:30:33.151 --> 00:30:36.208
<v Ben>come up with a plan, break this into Linear tickets.

00:30:36.208 --> 00:30:40.517
<v Ben>We use Linear. We used to use JIRA. I'm not a huge JIRA fan,

00:30:41.038 --> 00:30:45.924
<v Ben>maybe bordering analogy, but yeah, they're supposed to be getting bigger.

00:30:46.619 --> 00:30:51.528
<v Ben>And as part of that, it's going to be less PRs, late, which PR reviews are hard

00:30:51.528 --> 00:30:56.298
<v Ben>to combine. And if it's a thousand lines of code, I used to shy away from like

00:30:56.298 --> 00:31:00.820
<v Ben>changes, be like, whoa, this is like a thousand line change. Can you break that down?

00:31:01.428 --> 00:31:05.568
<v Ben>Nowadays, it's the same effort. If I don't look at the files,

00:31:05.568 --> 00:31:12.566
<v Ben>I just need to read to what the AI prompted, what the rating of it is. And yeah.

00:31:13.422 --> 00:31:19.338
<v Matt>What about team size? Do you think, because now you've got bigger PRs,

00:31:19.898 --> 00:31:27.828
<v Matt>people working on one monorepo, you're going to end up with an unmanageable amount of conflicts?

00:31:29.093 --> 00:31:33.833
<v Ben>Yes. I mean, from the time that I publish a PR, it will get rebased at least

00:31:33.833 --> 00:31:37.140
<v Ben>five times because people are just pushing so much more code, right?

00:31:37.655 --> 00:31:43.253
<v Ben>Team size, I'm a big supporter of having at least two people on a project,

00:31:43.593 --> 00:31:48.353
<v Ben>even if your secondary is just a PR reviewer or a sounding board or

00:31:49.562 --> 00:31:53.593
<v Ben>maybe even a silent partner or whatever you may call it because people get hit

00:31:53.593 --> 00:31:55.059
<v Ben>by the bus. It does happen.

00:31:55.388 --> 00:31:59.713
<v Ben>And you want to break that silo as fast as possible Because with us creating

00:31:59.713 --> 00:32:04.993
<v Ben>so much code and having things be in silos and nobody understanding what is

00:32:04.993 --> 00:32:07.933
<v Ben>actually going on is so much more commonplace now.

00:32:08.313 --> 00:32:15.310
<v Ben>But we also want to create a bunch of features. So everybody's kind of like lone wolfing it with AI.

00:32:15.677 --> 00:32:17.893
<v Ben>AI as their co-pilot and then just

00:32:20.273 --> 00:32:25.823
<v Ben>punching out code. Which, again, I have absolutely no context.

00:32:25.823 --> 00:32:31.133
<v Ben>If there's no more PRD, no more engineering document, because there's just a

00:32:31.133 --> 00:32:34.371
<v Ben>plan, I don't know what you're trying to do.

00:32:35.141 --> 00:32:40.190
<v Ben>I'm just looking at the code and looking at the review and trying to decide if it's safe or not.

00:32:41.742 --> 00:32:46.552
<v Nina>Should have something that i would imagine or like to have in the future is

00:32:46.552 --> 00:32:49.423
<v Nina>like when you're in jira or you're in you're in,

00:32:50.582 --> 00:32:55.472
<v Nina>you're looking at a diff if there was an ai summary of the diff in the context

00:32:55.472 --> 00:33:02.216
<v Nina>of the code base just at the top just off to the side like yeah you know up at the top,

00:33:02.981 --> 00:33:07.512
<v Nina>just you know for thousands of lines just have an ai summary for PRs rather

00:33:07.512 --> 00:33:11.577
<v Nina>than the actual summary that's, you know, at the top.

00:33:12.094 --> 00:33:16.582
<v Nina>Like, where you could click and it would automatically scroll down to each line

00:33:16.582 --> 00:33:19.690
<v Nina>that's relevant to that key point. I don't know.

00:33:20.520 --> 00:33:23.932
<v Matt>I'm getting closer to that with my chief of staff, Bob.

00:33:23.962 --> 00:33:29.042
<v Nina>Oh, that's right. I didn't think, yeah. Yeah. You technically have something.

00:33:29.042 --> 00:33:34.254
<v Matt>I've kind of gotten fed up with GitHub and Bitbucket and slowly making my own.

00:33:36.193 --> 00:33:41.052
<v Matt>It's a lot more like this should do i even need to look at this is kind of what

00:33:41.052 --> 00:33:48.302
<v Matt>the purpose of it is now and it and then it has a button for me to just approve or it has

00:33:49.220 --> 00:33:53.842
<v Matt>in the or i have to go into the box and then look at the actual code and it's

00:33:53.842 --> 00:33:58.332
<v Matt>getting it's getting better i got it just gives like me a score out of 10 and

00:33:58.332 --> 00:33:59.872
<v Matt>then now it gives me which i

00:33:59.872 --> 00:34:04.100
<v Matt>added is helpful a table with like the top three reason why it's not a 10.

00:34:04.904 --> 00:34:09.431
<v Matt>And usually, maybe not usually, at least a third of the time,

00:34:09.995 --> 00:34:13.362
<v Matt>I'm able to see why a bad score really isn't a bad score.

00:34:13.664 --> 00:34:20.015
<v Matt>And then I still have to fail out and go to the actual PR for the other two thirds.

00:34:21.067 --> 00:34:23.902
<v Matt>I really think that's like kind of, we're trying to get closer to like,

00:34:23.902 --> 00:34:27.411
<v Matt>what do you actually need from a PR? Give it a thumbs up.

00:34:29.337 --> 00:34:32.211
<v Nina>Wish there was a way we could showcase it somehow.

00:34:32.588 --> 00:34:36.373
<v Matt>I've got way too much client. I'd have to make a fake one. Maybe next time.

00:34:37.677 --> 00:34:39.920
<v Nina>Make one with fake. It would take a second to make one with...

00:34:40.222 --> 00:34:45.746
<v Matt>Yeah, I have it in the blog that I wrote about it, but an actual working one, I can make fake soon.

00:34:46.811 --> 00:34:53.517
<v Ben>I've tried to do something similar once to get more of a feel for what the code

00:34:53.517 --> 00:35:01.557
<v Ben>actually does to give me each file and pseudocode out very concisely what a function does and

00:35:02.537 --> 00:35:05.562
<v Ben>what the change actually encapsulates.

00:35:05.823 --> 00:35:10.817
<v Ben>I don't need to, if it is a new endpoint, I don't need to look at the controller

00:35:10.817 --> 00:35:16.527
<v Ben>or I really want to just see what the service does, like strip all the boilerplate

00:35:16.527 --> 00:35:18.457
<v Ben>out and just give me the essence of it,

00:35:18.917 --> 00:35:22.797
<v Ben>but relate it to a specific line of code so I can actually take a look at it.

00:35:22.797 --> 00:35:26.834
<v Ben>It felt very satisfying to do it, but again, I think it takes too long.

00:35:28.041 --> 00:35:34.271
<v Matt>Reviewed takes too long? Looking at the actual code to actually trace that huge

00:35:34.271 --> 00:35:40.791
<v Ben>change to like, hey, AI says the service now does something slightly different,

00:35:40.791 --> 00:35:42.441
<v Ben>and this is how it was implemented.

00:35:42.821 --> 00:35:46.641
<v Ben>This is the service file. Here are the three lines of code. There's an if statement

00:35:46.981 --> 00:35:49.860
<v Ben>there and whatnot. It just takes too long.

00:35:50.535 --> 00:35:58.158
<v Matt>Yeah, like is there a shift to be like, now you need to kind of prove not to greenlight the PR?

00:35:58.744 --> 00:36:03.951
<v Matt>Like I felt before the idea was you need to see if this is good enough to merge

00:36:04.582 --> 00:36:10.017
<v Matt>and now I feel you need to see if this is bad enough to actually warrant comments

00:36:12.159 --> 00:36:15.738
<v Ben>I still call grades it out of the five if it is a

00:36:16.479 --> 00:36:21.581
<v Ben>if it is a three or below I've got more work to do right and if I when this

00:36:21.581 --> 00:36:27.341
<v Ben>happened before and I and I argued against it and I got the the score up to a four.

00:36:27.341 --> 00:36:31.141
<v Ben>And a five out of five is generally a single line change or whatnot.

00:36:31.141 --> 00:36:34.531
<v Ben>I barely, I don't even worry about the files anymore.

00:36:34.771 --> 00:36:39.375
<v Ben>At a four out of five, I do want to see the comments. I want to see how they were resolved.

00:36:40.145 --> 00:36:43.971
<v Ben>At a three out of five, those comments don't only need to be resolved.

00:36:43.971 --> 00:36:47.521
<v Ben>They need to be argued against. And there needs to be some argumentation on

00:36:47.521 --> 00:36:52.643
<v Ben>the developers to have on why AI is wrong and how it was resolved.

00:36:53.823 --> 00:36:57.493
<v Matt>I've gotten my reviews to now when I actually do dive in. I mean,

00:36:57.493 --> 00:36:58.721
<v Matt>I'm still diving in with AI.

00:36:59.220 --> 00:37:06.511
<v Matt>It's going to give me failure cases that it thought of, but either they're not covered in tests.

00:37:07.226 --> 00:37:12.350
<v Matt>And so it has to create that test first before it's a valid comment to add to the PR.

00:37:12.641 --> 00:37:15.543
<v Matt>And then that code needs to go into the comment as well.

00:37:16.206 --> 00:37:21.495
<v Matt>And then if it's an architectural one, it needs to diagram the previous,

00:37:21.622 --> 00:37:23.915
<v Matt>the current architecture within the PR.

00:37:24.286 --> 00:37:29.483
<v Matt>And why thinks it's bad and then a correct one so that like there's something

00:37:29.483 --> 00:37:36.628
<v Matt>for i mean really the other person's agent to come in and and pick up and see and like actually

00:37:37.333 --> 00:37:39.353
<v Matt>bite into and try to fix with it

00:37:40.408 --> 00:37:46.463
<v Ben>we have a skill it's called real or paranoid and i think with certain models

00:37:46.463 --> 00:37:48.547
<v Ben>there's definitely, I mean,

00:37:49.843 --> 00:37:52.563
<v Ben>Some of the warnings are really paranoid.

00:37:53.086 --> 00:37:56.502
<v Ben>Time zones, like what it worries about.

00:37:57.603 --> 00:38:01.293
<v Ben>I shouldn't talk about time zones. Time zone bugs still do exist,

00:38:01.293 --> 00:38:03.711
<v Ben>and I shouldn't anger the time zone gods.

00:38:04.151 --> 00:38:07.753
<v Ben>But there's definitely some things where it is over-worried,

00:38:07.753 --> 00:38:11.413
<v Ben>and so we developed kind of a skill to kind of ground it a little bit.

00:38:12.468 --> 00:38:17.118
<v Matt>Yeah, it's definitely paranoid. It depends Depends on the model you're using, though.

00:38:17.842 --> 00:38:23.537
<v Matt>I feel Opus 5 now is a little more loose and just tries to get the job done than Fable was.

00:38:25.043 --> 00:38:28.893
<v Matt>I've been using Opus 5 exclusively now. I don't go back to Fable.

00:38:30.143 --> 00:38:33.418
<v Matt>I go to Codex if I need a second opinion now.

00:38:34.458 --> 00:38:41.423
<v Ben>We found the same thing. Opus 5 is dumb enough to accept some of these things.

00:38:41.423 --> 00:38:44.406
<v Ben>Opus, Fable gets a little paranoid.

00:38:46.244 --> 00:38:50.119
<v Matt>Do you want to change topics to Nina's fictional writing app?

00:38:50.270 --> 00:38:53.362
<v Matt>I want to know if you did, added the payment stuff.

00:38:54.304 --> 00:38:58.345
<v Nina>No, I did not. No. Come on, Nina!

00:38:58.816 --> 00:39:04.168
<v Nina>Because the people who write the fiction, Matt, they're writing fiction for free.

00:39:04.632 --> 00:39:09.903
<v Nina>Therefore, my demographic would not want to pay to write.

00:39:10.570 --> 00:39:11.504
<v Matt>Couldn't add ads?

00:39:12.786 --> 00:39:16.534
<v Nina>I was thinking about ads, but it would ruin it.

00:39:16.534 --> 00:39:23.164
<v Nina>The beautiful ui it will it would i have like an animated ui such an artist

00:39:23.164 --> 00:39:25.669
<v Nina>different theme well okay okay,

00:39:26.344 --> 00:39:31.214
<v Nina>i'm gonna there are there will be plenty of apps to make money with plenty like

00:39:31.520 --> 00:39:35.677
<v Nina>the forge forge video forge audio those can work,

00:39:36.498 --> 00:39:41.644
<v Nina>like i know this is an app where i actually have users who use it and they they

00:39:41.644 --> 00:39:44.638
<v Nina>literally keep spamming hey can you add this hey can you add that?

00:39:45.084 --> 00:39:46.442
<v Nina>And yes, I'm doing it for free.

00:39:47.410 --> 00:39:50.126
<v Nina>I'll figure it out. Give me some time.

00:39:50.774 --> 00:39:52.225
<v Ben>What about a donation button?

00:39:52.875 --> 00:39:57.171
<v Nina>There is a donation button. There's a Buy Me a Coffee button and that's good enough for me.

00:39:57.310 --> 00:39:59.052
<v Ben>Maybe you can have a pop a little bit more.

00:39:59.710 --> 00:40:07.082
<v Nina>I'm going to make it bigger. I'm going to make it fly across the screen and right in their face.

00:40:07.964 --> 00:40:12.754
<v Nina>It actually is pretty inconspicuous right now. I should probably make it bigger. It's kind of tiny.

00:40:14.223 --> 00:40:17.273
<v Matt>You probably have AI figured out. Tell it to be a marketer and figure out how

00:40:17.273 --> 00:40:19.096
<v Matt>to get people to click on this donate button.

00:40:19.796 --> 00:40:22.163
<v Nina>How do we psychologically convince the user to...

00:40:22.603 --> 00:40:24.108
<v Matt>I mean, it's what everyone else is doing.

00:40:24.961 --> 00:40:27.683
<v Nina>Yeah, I didn't even think to ask a lot of questions like that.

00:40:28.918 --> 00:40:34.433
<v Nina>Pretend you are a person who is skilled in marketing and branding and want to

00:40:34.433 --> 00:40:36.770
<v Nina>trick the user into clicking this. Yeah.

00:40:38.283 --> 00:40:42.540
<v Nina>I know you think it's foolish, but they won't use it if I start charging.

00:40:43.192 --> 00:40:47.227
<v Nina>I mean... Have you guys added, when you guys add AdSense to a website,

00:40:48.066 --> 00:40:54.353
<v Nina>is there a way to add AdSense in a way that you could make it like a little

00:40:54.353 --> 00:40:57.925
<v Nina>transparent or were really inconspicuous?

00:40:59.200 --> 00:41:02.453
<v Nina>Or are there rules and regulations against doing that? Like,

00:41:02.453 --> 00:41:07.443
<v Nina>can I have it behind a div that, and so they only see it like sometimes.

00:41:07.943 --> 00:41:11.873
<v Matt>I mean. Yeah, Google will know that it's like Google's.

00:41:11.873 --> 00:41:13.023
<v Nina>Okay, see, that's what I thought.

00:41:13.023 --> 00:41:16.237
<v Matt>You have like a million people focused on that specific problem.

00:41:16.889 --> 00:41:20.933
<v Matt>But like you could just have it on the side or something or you're just like,

00:41:20.933 --> 00:41:25.124
<v Matt>well, it's a discord, right? So like it could just be like a message that's no, I don't know.

00:41:25.363 --> 00:41:27.430
<v Matt>I don't know how you would do it in your discord app.

00:41:28.803 --> 00:41:30.797
<v Nina>I'll think about it. I'll think about it, do it.

00:41:32.600 --> 00:41:36.983
<v Ben>Doesn't that defeat the purpose? Aren't you being paid by impression and by

00:41:36.983 --> 00:41:43.054
<v Ben>click-through? So if you hide it well enough so nobody clicks through it, it

00:41:44.156 --> 00:41:49.758
<v Nina>Just going to stick with the donation button. I want to stick with that. It's, yeah.

00:41:50.490 --> 00:41:56.120
<v Ben>I mean, ramp it up a little bit. Don't go full on Wikipedia. Yeah.

00:41:57.354 --> 00:42:06.649
<v Ben>But if Wikipedia is a 10, maybe try a 4 or a 3. Maybe a B tested.

00:42:07.578 --> 00:42:08.426
<v Nina>Maybe what tested?

00:42:09.043 --> 00:42:10.122
<v Ben>Do some A B testing.

00:42:10.809 --> 00:42:15.936
<v Nina>Okay. Let me make a note. I'm just going to make a note to remember that.

00:42:16.753 --> 00:42:20.536
<v Ben>The only problem is you need to come up

00:42:21.636 --> 00:42:26.126
<v Ben>you need to have enough traffic to have a key value rate but with the donation

00:42:26.126 --> 00:42:30.946
<v Ben>button how many people click through how many people how many people stay on

00:42:30.946 --> 00:42:34.906
<v Ben>the site and just click it away there's a bunch of metrics that you can pull

00:42:35.515 --> 00:42:40.206
<v Nina>There's an active now so all the users on the website they have like an old

00:42:40.472 --> 00:42:44.506
<v Nina>like you know the old myspace where it would say like online now they have that

00:42:44.506 --> 00:42:48.256
<v Nina>on there so i can see there's maybe like four or five people on at once it's

00:42:48.256 --> 00:42:50.412
<v Nina>it's a baby app it's in its infancy.

00:42:51.227 --> 00:42:55.977
<v Matt>But users so your other ones don't have users

00:42:56.652 --> 00:43:03.626
<v Nina>That's because i haven't shared them well i could yeah right right i'll start

00:43:03.626 --> 00:43:08.502
<v Nina>sharing i'll start sharing apps but i need to redo them in rust all of them.

00:43:08.821 --> 00:43:12.107
<v Matt>Yeah and then redo them again and again and again

00:43:12.873 --> 00:43:18.563
<v Nina>Just rust I'm pretty sure rust's the end all for desktop applications yeah.

00:43:22.219 --> 00:43:26.969
<v Ben>I think in the first episode you were talking about electron is that what it is

00:43:27.184 --> 00:43:33.452
<v Nina>Yeah and now i've realized that electron is way too slow it for because it was

00:43:33.452 --> 00:43:40.282
<v Nina>for a media app it was for an app in which you're you're editing audio files it was like a daw so,

00:43:40.762 --> 00:43:45.130
<v Nina>that was horrible on the cpu because everything with electron is through chrome,

00:43:45.826 --> 00:43:51.162
<v Nina>it's it wasn't the best choice electron on it's very very slow in comparison

00:43:51.162 --> 00:43:54.325
<v Nina>they're like there's so much lag in it so i need to redo it,

00:43:55.281 --> 00:44:01.825
<v Nina>it was called forge audio and now i made something called forge video to help

00:44:01.941 --> 00:44:04.030
<v Nina>matt and i out with our podcast editing,

00:44:04.900 --> 00:44:13.232
<v Nina>and i did it in rust with Tauri and metal and it's so much faster there's there's

00:44:13.232 --> 00:44:15.841
<v Nina>no hammering up the cpu at all so,

00:44:16.712 --> 00:44:22.662
<v Nina>yeah it's yeah it was it's better for like real-time performance so i'm probably

00:44:22.662 --> 00:44:27.488
<v Nina>going to redo all my applications that's all my applications that are media.

00:44:28.578 --> 00:44:34.772
<v Nina>Media editing apps like at the time i was just very stubborn about using electron

00:44:34.772 --> 00:44:39.302
<v Nina>because it was what i knew and it being all javascript and that being what i

00:44:39.302 --> 00:44:42.625
<v Nina>was most comfortable coding in like maybe five years ago,

00:44:43.463 --> 00:44:49.142
<v Nina>yeah ai has like in terms of like helping me with research and figuring out

00:44:49.142 --> 00:44:51.720
<v Nina>like what's the best best way to build something,

00:44:52.638 --> 00:44:57.792
<v Nina>yeah i would not have had the ability to get out of my comfort zone if i if

00:44:57.792 --> 00:45:04.388
<v Nina>i didn't have ai to help like you know Claude to help research and code it for me,

00:45:05.009 --> 00:45:08.202
<v Nina>because the the learning what is it called the uh.

00:45:10.469 --> 00:45:16.189
<v Nina>Rust is very difficult to pick up quickly and I would never have experimented

00:45:16.189 --> 00:45:19.707
<v Nina>with it if I didn't have an agent coding in it for me.

00:45:20.744 --> 00:45:28.195
<v Nina>Especially coming from a background of preferring dynamic languages. So, yeah.

00:45:29.556 --> 00:45:33.849
<v Ben>I have a question. Both you guys brought up a specific technology and this is

00:45:33.849 --> 00:45:41.649
<v Ben>for my own selfish reasons. I recently, for work, we stood up at we set up OpenSearch,

00:45:41.649 --> 00:45:44.537
<v Ben>which is AWS's version of Elastic.

00:45:45.448 --> 00:45:49.938
<v Ben>Are you guys familiar with the offering? And how does it assist?

00:45:50.643 --> 00:45:54.569
<v Ben>Are the performance implications noticeable?

00:45:54.889 --> 00:46:01.449
<v Ben>And if you guys are familiar with how they fort, do you guys foresee OpenSearch

00:46:01.769 --> 00:46:05.577
<v Ben>deviating much further from Elastic?

00:46:06.508 --> 00:46:12.389
<v Matt>Or if it follows DocumentDB compared to MongoDB yeah it'll immediately slow

00:46:12.389 --> 00:46:15.097
<v Matt>down and not keep up performance wise

00:46:15.823 --> 00:46:21.059
<v Matt>unless you have super fine-tuned your mongos or your Elasticsearch i'm just

00:46:21.059 --> 00:46:26.057
<v Matt>guessing i don't know for real but unless you have something special in Elasticsearch

00:46:26.757 --> 00:46:31.864
<v Matt>you're not going to notice that much of a difference it'll just going to be easier to maintain,

00:46:33.180 --> 00:46:38.999
<v Matt>if that's become a problem, that infrastructure, to move it off to something

00:46:38.999 --> 00:46:40.741
<v Matt>that Amazon worries about is worth it.

00:46:42.229 --> 00:46:47.579
<v Matt>We did, I've done the OpenSearch migration, as well as a DocDB migration.

00:46:47.579 --> 00:46:53.569
<v Matt>And both the reasons is specifically to not have an IT person worry about the box.

00:46:55.319 --> 00:47:00.429
<v Ben>Hosted in AWS, so that was essentially the winning argument.

00:47:01.216 --> 00:47:06.811
<v Matt>I mean, it's a real argument. You save 150k on a person, at the very least.

00:47:07.832 --> 00:47:11.569
<v Matt>There's no way OpenSearch can cost that much more over Elasticsearch.

00:47:12.573 --> 00:47:16.195
<v Ben>It is expensive. It is surprisingly expensive.

00:47:17.868 --> 00:47:21.848
<v Matt>I've also been playing with Instinct, completely unrelated. Have you guys heard of Instinct?

00:47:22.549 --> 00:47:23.253
<v Nina>No, it's not.

00:47:23.926 --> 00:47:25.459
<v Matt>It's basically what OpenClaw wanted

00:47:25.459 --> 00:47:30.030
<v Matt>to be and so it's also the same as facebook's muse that just came out

00:47:31.103 --> 00:47:35.576
<v Nina>Oh it's uh it's oh it's it's like a it's like dot dots.

00:47:36.349 --> 00:47:40.079
<v Matt>I don't know i don't know if it is like dots or not it's basically the interface

00:47:40.079 --> 00:47:43.785
<v Matt>is whatsapp at least for me you can do it so that's like your messenger app

00:47:44.443 --> 00:47:50.962
<v Matt>and it's really easy to connect it to things unlike OpenClaw and you don't have to host it yourself

00:47:51.626 --> 00:47:57.009
<v Matt>so i'm giving it way too much data but i've been enjoying what i've been doing

00:47:57.009 --> 00:47:59.399
<v Matt>with it been doing it to like generate leads

00:47:59.945 --> 00:48:03.649
<v Matt>and kind of go out and research like other companies to like offer services

00:48:03.649 --> 00:48:08.015
<v Matt>to as well as find conferences that have cfps open

00:48:08.624 --> 00:48:12.349
<v Matt>and then write up a cfp that fits the conference and drafts and things like

00:48:12.349 --> 00:48:14.535
<v Matt>that it's been good i like it

00:48:15.698 --> 00:48:19.429
<v Nina>It's it's not like dots where it works like a wrong around the clock right like.

00:48:19.429 --> 00:48:24.189
<v Matt>You can make it work around the clock it is very opaque what is happening behind

00:48:24.189 --> 00:48:30.853
<v Matt>the app where i feel if dots is anything like amp you're kind of like seeing the sausage get made

00:48:31.868 --> 00:48:36.939
<v Matt>and for some reason like watching a thing in the terminal blink i have a lot

00:48:36.939 --> 00:48:41.656
<v Matt>less patience for than watching my whatsapp someone's typing go forward

00:48:43.445 --> 00:48:46.665
<v Nina>Many different so i know we're towards the end of time but how many different

00:48:46.665 --> 00:48:54.106
<v Nina>ai tools do each of you use because i'm curious because you have instinct you have grok ai.

00:48:54.385 --> 00:48:55.742
<v Matt>I don't have great i have am

00:48:56.845 --> 00:48:57.875
<v Nina>Uh okay and.

00:48:57.875 --> 00:49:03.388
<v Matt>Then i have uh i do codex and Claude such between the two codex

00:49:03.904 --> 00:49:08.305
<v Nina>Okay yeah i Just have Claude and ChatGPT.

00:49:09.278 --> 00:49:16.459
<v Ben>Predominantly used person, although I've recently been using Claude Code exclusively.

00:49:16.883 --> 00:49:22.435
<v Ben>And then for whatever reason, I have a subscription to all three of the biggest ones.

00:49:22.435 --> 00:49:29.775
<v Ben>I have a ChatGPT personal one, Gemini and Claude Code just for ChatGPT,

00:49:29.785 --> 00:49:34.075
<v Ben>mostly for writing, Gemini for general research, and Claude Code for random site

00:49:34.075 --> 00:49:35.075
<v Ben>projects that I want to do.

00:49:35.808 --> 00:49:41.735
<v Nina>Yeah. Yeah, same. I have, I recently canceled Gemini, but yeah,

00:49:41.735 --> 00:49:47.865
<v Nina>I just have ChatGPT for, for like writing or sometimes research outside of coding.

00:49:47.865 --> 00:49:52.090
<v Nina>And then yeah, Claude, Claude code for all of my side projects.

00:49:53.069 --> 00:49:56.673
<v Nina>But I am very curious about Dots that just came out.

00:49:57.468 --> 00:49:58.763
<v Ben>What's that supposed to do?

00:49:59.500 --> 00:50:02.315
<v Nina>It's an assistant that kind of works like around the clock.

00:50:02.472 --> 00:50:04.357
<v Matt>Okay. So it is the same thing. It's the same thing.

00:50:04.491 --> 00:50:13.654
<v Nina>Yeah, it's a 24-7 AI agent, and it uses the latest ChatGPT model,

00:50:14.437 --> 00:50:21.485
<v Nina>and supposedly is extremely fast, and the highest subscription tier is $500.

00:50:21.485 --> 00:50:26.725
<v Nina>So I don't know what the lowest subscription tier is, but I'm curious to try

00:50:26.725 --> 00:50:29.461
<v Nina>it, at least. Not the $500 one, though.

00:50:30.341 --> 00:50:30.785
<v Matt>It should be good.

00:50:30.825 --> 00:50:32.228
<v Nina>That's, yeah.

00:50:32.894 --> 00:50:37.556
<v Matt>Right. I think we got to end it there. So that's that sweet.

00:50:38.538 --> 00:50:39.431
<v Ben>Thank you so much for having.

