Justin Skycak (00:00) don’t take notes when you’re learning. no point, just work exercises. the nuance here is that when there is information that’s not part of reference material, you can’t go look it up after, you’re brainstorming something with someone or a thought occurs to you, and you don’t want to miss it, write it down.
Write it down, because otherwise you don’t have the reference material. There’s a big difference between these two things.
Jason Roberts (00:00) Paul Graham wrote great essay that the maker, has to just get in the zone and think really, really hard about something where it could take hours sometimes to really get into the zone. Whereas,
the manager is like the email and the phone call, the meeting, and let me look this up and check this and just lots and lots of little tasks, both of these things are important, right? In life
and in business, you don’t get to just like, I’m just going to build stuff and then I’m not going to do these other things. Cause you, you get nothing and you can’t, if you spend all your time, just set up meetings and coordinating and stuff like nothing serious gets done. it’s like, gotta have both.
Jason Roberts (00:00) It’s really important you find someone who’s like a complement so like you make up for each other’s weaknesses and shortcomings as opposed to you’re both like the same person. You have the same strength. That tends to not work. Right.
And I think that’s what makes really good relationships of any kind and marriage business partnerships, is that you have this complementary skill set and you appreciate what the other person can do. You’re like, thank God you’re doing this. Thank God. you know, and then you respect it and then you appreciate it and then you just like, I got this, you got that.
Jason Roberts (00:00) Once I’m like in the deep work, I resent.
being pulled out of it I don’t want to do the meeting. I don’t want to do the paperwork. I don’t want to return these emails. And even other people I like, it’s interesting stuff. It’s like, I’m like, damn it, I’m in the zone, you know? And then it’s like, I can’t.
get myself to focus, I have like this ADHD and I can’t lock in on something. And it’s sort of like for me, it’s almost like a day thing. Once you get in a certain mode for a day, it can be really hard to switch. Now, sometimes maybe it’s like, well, you go to lunch and you have this like break in the day and maybe you can reset. And some people are probably better at this than others. I struggle with it.
Jason Roberts (00:00) any successful endeavor requires a great team of people. And getting the right
group of people who not only are individually highly capable, that you all kind of like each other, work together, and trust each other, and all that kind of stuff. Sometimes it just comes together and you’re like, damn, we just did it. You what I mean? You run into people who, their roommates from freshman year in college are their best friends for life. You’re like, that is just luck.
Sometimes the girl you date when you’re in eighth or ninth grade and you marry, she’s like the most amazing girl. was like, that is just luck, man.
Jason Roberts (00:00) You can’t give somebody responsibility for something without giving them control. Right? That’s not fair. And it’s just, it’s not reasonable.
Justin Skycak (00:09) you want somebody who you can hold responsible for the output of the team, right? And the only way to have that person is to give them control over the team that they build. There’s no way otherwise. That’s the only way.
Jason Roberts (00:00) You cannot solve tomorrow’s problems today because you don’t really understand what those problems are, and you don’t even know if they’re real.
It’s mostly a waste of time. I’m not gonna say you can’t think at all about what the future might look like, but you got to be careful not to get caught going down that rabbit hole and spending too much time thinking that you can make all these predictions because you’re probably going to be wrong.
Or at least you’re going to be wrong about the relative importance. You kind of like, okay, here are all these things.
that were interesting or could be important. Let’s figure out what’s really the most important one, and let’s just kind of do that, and then we’ll pick our head up and go, okay, based on what we’ve learned from doing that, now we can talk about the next thing.
We all want to avoid problems and catastrophe. You want to try and get ahead of some of these things, but you just got to be careful about overestimating your ability to really pinpoint what they are. Cause what’ll happen is you’ll over-engineer things.
You came up with this vastly complex solution to a non-problem, to an imaginary problem.
Jason Roberts (00:00) I can pretty good at compartmentalizing and what I mean is living in a state of denial about things that have to get done. And you know that fire is burning. And you’re just like, I’m just going to pretend it’s not burning.
You know, and you’re kind of thinking, you’re wondering to yourself, like, I wonder how long I can let this burn. it’s not great, right? I’m lucky in that I have Sandy, my wife and co-founder, because she is very operational.
just get shit done. like, all right, make a list, collect the data, execute good enough, done, bang, bang, bang.
She treats me a little bit like the mad scientist, like, OK, you go to lab. I’ll take care. I’ll run interference. But I will call you every once in while. And you’re going to have to do certain things.
she’s so good at that And I think that’s what makes really good relationships is that you have this complementary skill set and you appreciate what the other person can do. You’re like, thank God you’re doing this. Thank God.
Jason Roberts (00:00) what are you doing? meta work like on a corporate environment where you cover your ass to show you’re doing stuff. Like, well, I’m writing these briefs I feel like there’s just a lot of brief writing.
Like, why do you spend half a day writing a fricking document?
I’ve always to bureaucracy and bullshit and meta work.
And it doesn’t impress me. I don’t give a shit. In fact, I’m annoyed.
The meta work just overwhelmed the process. And I think you see it a lot of companies.
The focus becomes that because
the politics and bureaucracy and it’s like nobody’s doing like one guy who’s actually doing anything. Everybody else is just
talking about who’s doing what and the plan and when this is going to do. meetings and approval And I can only talk about something for so long before we actually have to do it.
Justin Skycak (00:00) Everyone’s excited
but their excitement is leading them to go off in all these different directions. there’s the compounding of misalignment. And you gotta be constantly keeping people focused on the thing they’re supposed to be doing. And if you don’t do that, then somebody is one degree off course, two degrees off course, and you didn’t correct them
for the few days. And then you look up and you’re like, wait, where, where are you? Like, why are you all the way over there? It’s like, well, you let them go that far without pulling them back in.
Jason Roberts (00:00) you know, like when startups would go and try and raise venture capital and they have these pitch decks and they’d like in our growth chart and we’re going to do this. everybody knows this bullshit. The venture capital ists know it’s bullshit. You know, it’s bullshit. They kind of have to go through it just to, it’s just, that’s just part of the.
Justin Skycak (00:12) Yeah, the projecting revenue
like couple of years out from zero. We have zero right now. We’re going to pick up this many customers. Like, really? You can predict.
Jason Roberts (00:16) And we’re going to make a billion dollars.
Everybody’s like, great, okay, sounds
great. It’s like, is a big enough market, and then the profit margin and the cost of the, There’s big opportunity here. That’s all you need to say. But it’s nonsense because there’s just so much between that has to be done that’s just gonna dictate what’s gonna happen. You can’t make these predictions.
Justin Skycak (00:00) You do need to think hard about the stuff that you’re doing, but you don’t want to make it more complex. You want to think hard about how do I make it simple so that I’m not putting a straight jacket on
Jason Roberts (00:12) Buddy of mine in my first startup, used to say like, you better be careful what code you write because going to be supporting it for the rest of your life. That is going to be a thing that you have to deal with.
And if you’re solving problems that aren’t real problems, now you have all this extra code that you’ve got to lug around that are putting constraints on these other solutions. And now it takes five times as long to build. So there’s just so many reasons.
to try to avoid caught up building stuff that doesn’t really need to be built and just focus on the really critical stuff that you know how to solve now.
Jason Roberts (00:00) my inability to really imagine all the conditional levels of complexity that are involved in this thing that has to get done.
And I know this about myself and I’m still always underestimating how long it takes to do stuff.
Justin Skycak (00:09) Yeah.
is kind of funny because,
on one hand, you wanna just focus your field of vision on what you’re doing right now. But then at the same time, when it comes to like, actually like, okay, realistically, how much work is this? How much is it gonna take?
The way I kind of think of it is like, you don’t always know exactly what are the things that you’re going to do after what you see in front of you, but Like historically, maybe what you see in front of you is maybe the first 30%, 40%, 50 % maybe. like, just put a, put an extra two X or.
2.5x factor on there, even though you don’t know exactly what it’s representing, it’s gonna be there in some way. You’ll figure out what it is later, but it is there.
Jason Roberts (00:45) In my case, buyback.
Jason Roberts (00:00) Entrepreneurs have to be sort of self-deluding. It’s the only way that you can get started on an actual project because if you knew the actual, was really gonna take, you wouldn’t do it. It’s just, oh my God. You’re like, it’s gonna take me two months, three months, max,
and then you get going. And then, of course, it takes 18 months. In for a dime, in for a dollar, you made enough progress, you’re excited, okay. You need delusional people sometimes at the helm just to get everybody.
You distort reality around you because you believe so strongly that something is possible. And it’s not only possible, it’s possible in a relatively short period of time. And then you get everybody moving in the right direction. If you’re going to finish it, you’re going to have to do that. And then a lot of times, once you start making enough progress and learning enough, you get enough momentum and everybody’s, OK, yeah, yeah, we got this. It’s going to take more work, but we’re in it. So you’ve got to have
that level of self
Jason Roberts (00:00) Every time you build new features to allow more control, understand that you’re going to open up more
potential problems because you’ve made the system more complex because there’s a surface area of things that are happening to the system has increased, and they’re always going to be disappointed because they want to do more than what you’ve even added. You’re never gonna make everybody happy. But of course you add more control the more people get confused and then you get more email support and you’re just like, ⁓ man why are they even doing that, you know?
Jason Roberts (00:00) If I what are your life goals? You’re like, I got 100 life goals. Okay, so basically you’re gonna do nothing. Because you can’t do everything. So you say, no, my goal is this. Have one or two priorities, maybe three. You get beyond a few priorities and then you have to spread your time and effort among them all so much that nothing’s
Jason Roberts (00:00) It’s like looking at Photoshop. You’re like, pfft.
I gotta take like a semester long course to understand how to use this, even get started. I mean there’s tons of software like this AutoCAD and 3D modeling and rendering and they’re all like that and you’re just like oh my god.
Because every feature you add is putting walls down. It’s making stuff more complex. New users come in and are intimidated. They can’t even get started, and it’s just not a beginner product. So now you open up space for a competitor to come in and have a slimmed downed
version of what you do and say, well, we do the really important 80 %, it has 20 % of the features. It’s like something like we’re video editing. It’s like Cap Cut instead of Adobe Premiere. It’s super powerful, but like I need to just cut this video in like 15 minutes. Well yeah it’s probably not the thing. So it’s just it’s all trade offs.
Jason Roberts (00:00) you got to get people kind of.
working together rowing, you know, it’s like, how hard is it to keep these guys all pulling together? smaller person in the front is like, pull, pull, whatever. It’s like, do they really need that? I guess, yeah. Cause if they didn’t, they wouldn’t have that person, right? Cause it’s extra weight.
Or conductor in orchestra.
I don’t really know what they’re doing, but apparently they’re keeping everybody moving together and doing their part until everything syncs up and creates its effect. This seems to be required in human nature when you have multiple people working together type is critical.
Jason Roberts (00:00) This seems to be required in human nature when you have multiple people working together type is critical. when you’re working with people who are all working part-time or remote, it’s even harder than if they all show up to an office
and they kind of sort of can self coordinate. When they’re kind of doing this 10 hours a week and they’re working, they’re taking grad classes or working on a dissertation. I mean, it’s so easy for them to be like, well, what are we doing? You know, like do this, come
Jason Roberts (00:00) Buddy of mine in my first startup, used to say like, you better be careful what code you write because going to be supporting it for the rest of your life.
And if you’re solving problems that aren’t real problems, now you have all this extra code that you’ve got to lug around that are putting constraints on these other solutions. It’s like, oh, jeez, but there’s the thing, and it’s going to impact this thing. And now it takes five times as long to build. So there’s just so many reasons.
to try to avoid caught up building stuff that doesn’t really need to be built and just focus on the really critical stuff that you know how to solve now.
Jason Roberts (00:00) there’s just so many reasons.
to try to avoid caught up building stuff that doesn’t really need to be built and just focus on the really critical stuff that you know how to solve now.
smart people can invent an infinite number of problems themselves. They can really and they can make a great job convincing you and themselves that this absolutely has to be done. and I think sometimes smart people struggle
setting limitations, constraints on their own hubris and their ability to just think through anything. you just have to remember the times that you did that and you screwed it up
Jason Roberts (00:00) of the problems is is there’s a little bit of a fantasy that parents and some teachers play into. It’s like, just want them to fall in love with the subject.
Little kids or younger kids are enthusiastic about most things, especially if their parents are excited about it too and engaging with them. But you’re talking about middle school and up. It’s just not, it’s not a reliable thing.
Justin Skycak (00:18) Most of the time, that’s not going to work. If you are being held to account for kids learning the material, that’s not the strategy to lean
Like try to make that happen
for the kids who are receptive to it, but just understand that most kids are not going to be like that. So you have to actually focus on the mechanics of incentivizing people to do work. And a lot of kids maybe they go into software and later realize, Hey, math is actually pretty cool. They, might realize this later downstream. And they’re going to have to go through a little bit of, I don’t love it at the moment.
in order to get to a place where they do have a greater connection to it.
Jason Roberts (00:49) Yeah, it’s just
Jason Roberts (00:00) I would try and gamify everything that I did when I would teach and coach. So I remember when I would my son Colby when he was five, six, seven years old in soccer,
I want to take advantage of every minute because we only practice like once a week or maybe twice a week like for an hour. So what are you gonna get done? It’s hard to get anything done.
I want a ball at everyone’s foot the entire practice. We’re going to be doing stuff. I’m going explain as quickly as I can to get it across, and then I’m going to quickly turn it into a game to make it fun. And I would come up with a cool name like Zombie Attack.
And they would, I want to play Zombie Attack, right? You give it a cool name, and it worked. And it was a great way to get the most out of them. So it’s make things as fun as and as engaging as you can.
Jason Roberts (00:00) So I remember in second grade, we get a conversation or something from his teacher.
He walks up and he says, you know, Colby is really having a hard time and he’s really being disruptive or he’s not following directions. And so then he goes, so what I’m gonna do is I’m gonna send home like a
behavior card where it’s like seven categories and I’m gonna rank them a one through five and so the first few days it’s like threes and twos. And Sandy’s like, okay if it’s all fours and fives, I’ll give you a dollar.
Within the week,
He’s racking up a dollar almost every day. And then it’s like a month or five weeks go by and then we didn’t get the card I’m like, what’s going on with the card? And so with the teachers, they’re walking the kids. So he kind of came up to us and he’s like, no, it’s good. It’s fine.
No need anymore. That was end of it. But that incentive system made all the difference
Justin Skycak (00:00) It’s funny how minimal these incentives and gamifications have to be to work.
Like kids have imaginations, right? You can say something and they will run with it. And it’s just the idea of framing the thing that you’re doing. So when it comes to math,
Jason Roberts (00:08) It doesn’t take
Justin Skycak (00:14) when we talk about gamifying, we don’t mean you have to make everything into like an actual video game where everything is on the screen. Like you have a joystick and stuff doing math. You just need a little dose that makes it kind of interesting in that sense. Right.
Jason Roberts (00:30) You’re
framing it as a game. This is supposed to be fun. They’re like oh, so it isn’t work. It’s a game. Yeah, yeah, yeah, yeah, it’s a game. They’re like, oh, OK, game. And as long as it’s gamey enough, then it’s a game. It may not be the best game, but it’s a
Jason Roberts (00:00) When Colby and the original cohort of Math academy students
in seventh grade,
I was teaching calculus.
I would introduce a concept, do a couple on the board, and then everybody on the board. And everything was through a competition.
And it was about speed as well as getting it. And immediately they loved it.
They love being up at the board because they didn’t have to sit down. They’re all like move around and
you know, doing the thing. And of course, they enjoy competing.
We had a lot of people would come visit cause couldn’t believe it. I tell them and they’re like, you’re doing what?
I said, yeah, seventh graders are doing calculus. They’re like, And I remember this friend of mine, Gary, who was a mathematician at Caltech.
And he’s watching it and afterwards he’s just like, that was incredible.
But what’s really amazing is how fast they are. And I said, well, Gary, when you train for speed, you get fast.
The incentive structure was about winning the game, and they got really, really good and really, really fast.
Jason Roberts (00:00) middle school, kids can be really motivated, really excited to do things. There’s a lot to just do almost anything. Seventh grade, eighth grade, it starts to wane. You get to high school, they’re really…
And One of the problems that was really frustrating about high school is how the system is set up. They have a lot of pressure on them In the US, it’s about like,
you got to take advanced courses or AP courses and you got to get A’s because you got to have this great GPA. Because if you don’t have that, there’s no way you’re going to get into a decent college.
And then you gotta be in all of these extracurriculars and you can’t just be in them, no, you gotta be the student class president.
The thing is though, that creates a lot of stress and a lot of pressure. And then it becomes, I just got to get through this stuff. I don’t have time to love anything anymore.
Because any time there’s stress on you, or pressure on you, it’s hard to enjoy stuff.
It just takes the fun out of everything. Once you get to high school, it really gets to that.
Jason Roberts (00:00) For younger kids who are not
wanting to do Math Academy every day or you know as frequently,
you need to create incentive structures that they feel okay
Find what they’re excited about and set a nice carrot for them.
You find something that’s personally important to them, that they want, and you use that as an incentive to get them to doing it. Because they’re kids, they’re like, I wanna play band, and then by a month in, they don’t wanna practice anymore.
Or I want to be on soccer, but they don’t want to go to soccer practice because they want to play video games with their friends. Like, no, you’re on the soccer team, you got to do soccer practice. Even things that are fun, that they said they wanted to do, their motivation starts to flag because they’re kids.
As parents, you don’t always get to just be like Mr. Nice Guy. You have to say, like, no you’re doing this.
Now you can try and make it as painless as possible by setting up incentives. Like they have to understand, I pretty much have to do this anyway. But now I got this other incentive. You kind of got this sense of a stick
in the background. I mean, I’m going to have to do it, but I have this little carrot I’m excited about. So let’s focus more on the carrot.
Jason Roberts (00:00) of the problems that was really frustrating about high school is just how the system is set up. They have a lot of pressure on them
to get into a decent college.
Their stress is too high for them to be able to fall in love with some of this stuff.
I remember talking to some parents before Math Academy was in the high school, and I was
talking about, we could do a whole science thing and this and they’re like, listen, Jason, these kids have all these AP classes, they have all this stuff and they don’t have the time for all that.
And that started happening to me in the Math Academy program in the high school. We started to see kids who were good at it and liked it. It was just the pressure because their Math Academy classes were typically their hardest were like, what AP classes can I take to get an A? We had some kids who would switch out of Eurisko and just take AP Computer Science for the second semester because it was a joke, comparative they could just get an A.
They weren’t going to learn would get an easy A. And these are a couple of kids who went to MIT and stuff. They’re just like, I’m playing the game, man.
Justin Skycak (00:00) The most gifted kid that I’ve ever worked with, he was actually resistant to learning calculus around seventh or eighth grade.
had kind of learned a lot of arithmetic on his own and I had found some puzzles online that he wanted to work on.
And was a good use of time at the time. Eventually, you get to a point you’re not really progressing a whole lot in your mathematical development. I mean, you’re
way ahead compared to grade level, but like at some point you gotta make the leap.
Like there’s levels of math that you have to climb in order to just get further along in the talent domain in order to unlock new things for you to do.
Adults think like that. Kids don’t think like that cause kids don’t have the longer perspective. They don’t know what the long game is. They haven’t seen the long game play out.
And so what ended up pushing this kid over into okay, fine, I’ll learn calculus was that
he wanted to go take college level math courses, like in ninth grade. And so I was talking to him and his parents like, yeah, he can totally do that.
Problem is though, if he doesn’t know calculus, then not only is he going to struggle in these courses,
they’re not even going to let him into these courses if he doesn’t have the five on the AP Calc BC exam.
The big thing was getting his parents on board with it because if the parents on kid’s going to be on board one way or another.
The interesting part is once he learned all the calculus stuff, calculus became one of the things that he really enjoyed.
And now today, I still ⁓ work with him every other week.
He’s gotten through a lot of undergrad math, and so he’s actually sinking his teeth into research,
working university mathematician.
And there’s like bunch of derivatives being tossed around.
And it’s in this area that he was resisting back in seventh or eighth grade, and he’s having the time of his life right now. And if we had not pushed him through this segment of the journey that he was resistant to, he would not be doing what he’s doing right now.
Jason Roberts (00:00) A lot of public schools, if you get a D or an F in a school, they will still promote you to next course. I couldn’t believe when I first heard that when I was talking to a high school math teacher in the
Pasadena school district. It’s just like, what’s the point? If they don’t know Algebra 1, why are you sending them to Algebra 2? It’s just bureaucratic convenience. Move them along. Kid can’t read in first grade, move them to second grade. Still can’t read, the third grade. They just do it.
Bureaucracies really maximize convenience. And they’re just things that work out on their their plan, or whatever. And now if you come in and you say, my fifth grader should be doing algebra or whatever.
They’re going to be, uh no. What they won’t say is that’s a massive headache for us, and we don’t want to do it. So we’re going to come up with whatever reason we can to prevent that from happening because we don’t want to deal with the headache.
They do not want to do extra work. Nobody really wants to do extra work, but I tell you, the schools really don’t want to do it. And the teachers themselves, they got like my union, I don’t have to do this. And the principal’s like, I can’t make them do because the union and the bureaucratic and it’s a nightmare. So even if you could come to them and you say,
Well, can my daughter test into algebra? But if your daughter scored like a 97%, they’re like, oh, they did miss 3%. Don’t mind the fact that we promote people with an F to the next grade. They don’t know 3%. I’ve heard that. It’s like 93%, 95%, and they would not allow them.
Every once in a while you run into one teacher, one principal, school who’s like really open to it, but that’s super rare. It’s really frustrating.
Jason Roberts (00:00) The great thing about life is we’re all different, and we all have our things that come easy for us. We get some things for free and then we get stuff we have to work for. And part of life is figuring out what those things are.
It’s tough when kids are not really particularly stand out in anything, and they’re trying to figure out who they are. And it’s kind of tough because I don’t have anything to build an identity on that I can be really proud of. It doesn’t always have to be a competitive thing, but it’s just something.
The most important things that parents can do is encourage their kids to do stuff, to try things, especially things they think, I think you would like this. I think you might be good at It’s kind of scattershot when they’re really young. They just do everything up through 11 or so.
And they eventually kind of will find something.
and become skilled and leveling up in something and developing a sense of pride and a sense of accomplishment and success.
Jason Roberts (00:00) That’s why it’s so important for adults who can look ahead and say, look kid, I know you want to mess around this thing, but I know where you’re going, and I know what it takes to get there. And if you just guide the student to that point, they’ll be
vastly happier, vastly more successful, really realize their potential.
Justin Skycak (00:16) Let them find the things that they’re interested, don’t necessarily like push them like hardcore into things that they have no interest or no gift. Yeah, yeah, exactly.
Jason Roberts (00:24) that always backfires that always backfires
and the kids just give it up with piano or something your parents made him do it and then as soon as they’re old enough you’re like I am done I’m ever touching piano the rest of my life you know
Justin Skycak (00:33) Yeah.
But if your kid gravitates towards something that they really like doing net positive in their life.
and especially if they show a gift for it, they have some kind of like big advantage where this could really be a massive thing in the rest of their life.
Like you can’t let them quit at the slightest sign of difficulty. Kids sometimes will do that or like they love this thing until it gets a little hard and then they’re like, ah, I just want to stay in easy land. Part of supporting the kid means helping them get through those phases of the journey
that may not be the most enjoyable.
Justin Skycak (00:00) if your kid gravitates towards something that they really like doing net positive in their life.
and especially if they show a gift for it, they have some kind of like big advantage where this could really be a massive thing in the rest of their life.
Like you can’t let them quit at the slightest sign of difficulty. Kids sometimes will do that or like they love this thing until it gets a little hard and then they’re like, ah, I just want to stay in easy land. Part of supporting the kid means helping them get through those phases of the journey
that may not be the most enjoyable.
Speaker 2 (00:00) In software, just importing a solution from the library and saying I imported this model and I ran it. “now I’m a machine learning researcher engineer.” Like, no, no, no, you can’t just use the off the shelf. Like you can’t just use the, the theorem and wield the theorem and say, “now I am all powerful.” You actually have to go code from scratch, re-derive the result
from the bottom up to really understand the mechanics of what is it. It’s not enough to just take it off the shelf and use it. You need to know what went into building this thing.
Speaker 2 (00:00) You don’t want to let your natural gift or talent turn into a crutch. The moment you just rely on it and stop developing other parts of your game, that’s the moment when you start hitting like a really sharp asymptote, right? It reminds me of, in the original Pasadena program, oddly enough, sometimes it was the sharpest
sixth graders who could do the most work in their head, who would struggle the most a year later because they were so resistant to writing anything down. They were like, no, I don’t need pencil and paper. I just do all this in my head.
Their outsized working memory capacity became a crutch. They can only solve problems that they can immediately, like entirely fit in their working memory without usage of paper and pencil. That came back to bite them really fast.
Speaker 2 (00:00) Problem solving is like this, nebulous thing that everyone says it’s on the SAT. Like you got to get the kids to, think critically, to problem solve their way. how do we teach that? But once you, once you start drilling down into
what do the kids have to do? You realize that it all comes down to these skills that you can enumerate explicitly. And what’s more, you can arrange them in a knowledge graph. as a course. It’s a body of well-connected knowledge, hierarchical, just like anything else. so when people
refer to like, ⁓ the kid needs to learn problem solving, need to learn to think critically. just, there’s this whole body of knowledge that they just haven’t gotten, but it can be enumerated exhaustively.
Speaker 2 (00:00) The knowledge graph,
it’s this trick that allows you to trade off student work for instructor work. Instead of having the student go through this hodgepodge of problems and hopefully infer all the structure, and just have them grind through a large enough volume that they can infer. What you do is have the system,
figure this all out behind the scenes. It doesn’t mean that there’s less work that has to happen, but there’s less work for the student. We have to do more of the work, but that’s the trick. And we can do our work at scale.
Speaker 3 (00:00) I came in very, very little engineering background.
I had a strength in mathy coding, but until then I had just leaned into that so much that I never developed the rest of my coding abilities. And that was a severe weakness. But the silver lining of these severe weaknesses is once you shore them up, then you become so much more capable on so many more fronts.
It wasn’t fun realizing just how lacking I was on the software engineering side of things as opposed to the quant. But once I just dug in and leveled it up, suddenly there’s all sorts of things that became available to me that I was able to start working on that I wouldn’t have anticipated.
Justin Skycak (00:00) When you’re actually talking about solving concrete problems in the format of like a math test standardized test competition test,
justin 4 1 (00:00) When you’re actually talking about solving concrete problems in the format of like a math test standardized test competition test,
Justin Skycak (00:08) you have to be familiar with this class of problems that you’re trying to solve. Cause there’s just so much contextual information about that specific kind of problem and what type of methods will unlock it. you’re not going to invent them on the spot in a minute or two minutes or five minutes
justin 4 1 (00:08) you have to be familiar with this class of problems that you’re trying to solve. Cause there’s just so much contextual information about that specific kind of problem and what type of methods will unlock it. You’re not going to invent them on the spot in a minute or two minutes or five minutes
Justin Skycak (00:28) Yeah. You don’t have the time to approach it like a, like a research problem.
justin 4 1 (00:28) Yeah. You don’t have the time to approach it like a, like a research problem
Justin Skycak (00:32) or even like a take-home If you can’t figure it out in a couple of minutes, you’re
justin 4 1 (00:32) or even like a take-home If you can’t figure it out in a couple of minutes, you’re
Speaker 2 (00:00) You can explain anything if you break it a small step, to do too much. It’s usually they’re trying to take too big of a jump and then people just fall off. It’s like, you know, I used to rather than stairs, I used to think of it as like rocks placed across like a creek or something. And it’s like, okay, well, if they’re, three inches apart than grandma can do it. If it’s like eight inches, it’s like, well, you know, my mom and dad can do it. Okay, now it’s
a foot and a half, it’s like, well, I can do it and stuff. But now it’s good point where it’s like, well, my son can do it. Three feet, you know, it’s like, that’s what happens. It’s like, only super gifted, brilliant people can do because the pedagogy is so bad and the jumps are so big, you need to need someone who could do a standing broad jump of nine feet to make And it’s like, OK, why rather than writing textbooks that have these
six and eight and 10 feet jumps every two or three pages. And it takes them incredible mathematical aptitude to make those jumps. Otherwise they’re just like, man, I just, I don’t know. I’m like falling into the river and I’m floating down the creek and like, wait, this
Speaker 2 (00:00) One thing that I always, would say, this is what I think we need out of this course is
it needs to be at the level of what you would see at Harvard or MIT or Stanford or whatever. You don’t want someone to say, well, you know, that thing at Math Academy, I mean, you know, it’s just not bad. mean, it’s just kind of And so, you know, one thing I’d ask you to do when I we first created our
university level courses, linear algebra and multivariables, like go look, what can you find? What are they doing? If you can see any published final exams or syllabi from these institutions, what are they covering?
What’s the superset of the commonality, right?
So we always say like, we can reinvent a lot, but we do not exist in a vacuum. We exist world
where things are done a certain way and people have certain understandings and expectations. And if you drift too far off that, then people just do not know how to think about what your product is or what you’re doing. And it just becomes so much ⁓ friction in crossing that, it’s just, it can cause the product to fail.
Justin Skycak (00:00) What’s really funny, the learning curve happens, then you deploy something and it blows Yeah. All of sudden all these errors start happening down, emails are coming. And then that was a whole nother learning curve of how to write bulletproof code. Because stuff blowing up.
justin 4 2 (00:00) What’s really funny, the learning curve happens, then you deploy something and it blows Yeah. All of sudden all these errors start happening down, emails are coming. And then that was a whole nother learning curve of how to write bulletproof code. Because stuff blowing up.
Justin Skycak (00:15) It’s so stressful. It’s such a horrible experience. All these emails, nobody’s got tasks, this thing’s And so then you start going through the process of learning how to write code that won’t fall down, that’s logging everything. Cause then like what happened?
justin 4 2 (00:15) It’s so stressful. It’s such a horrible experience. All these emails, nobody’s got tasks, this thing’s wrong. And so then you start going through the process of learning how to write code that won’t fall down, that’s logging everything. Cause then like what happened?
Justin Skycak (00:28) are how to alert us.
justin 4 2 (00:28) are how to alert us.
Justin Skycak (00:30) you don’t realize things like cascading failures one thing fails, it causes another thing to fail. You don’t realize the magnitude of a screw up that can happen just from one small thing. And so you think like, well, I’ll just make things not break
justin 4 2 (00:30) You don’t realize that there’s things like cascading failures like one thing fails, it causes another thing to fail. You don’t realize the magnitude of a screw up that can happen just from one small thing. And so you think like, well, I’ll just make things not break.
Justin Skycak (00:45) And that’s the first step, but then there’s another step like, okay, what if something does break? we need to limit the breaking and its scope, cause just something’s going to break at some point, even if you’re as careful as possible.
justin 4 2 (00:45) And that’s the first step, but then there’s another step like, okay, what if something does break? we need to limit the breaking and its scope, cause just something’s going to break at some point, even if you’re as careful as possible.
Justin Skycak (00:58) or somebody is gonna run into some scenario that you just never thought would have ever happened, or somebody’s data is gonna be corrupted and then you gotta make sure that that doesn’t screw up with other people. so there’s just lots I guess more and more advanced sort of error handling and writing this kind of robust code.
justin 4 2 (00:58) or somebody is gonna run into some scenario that you just never thought would have ever happened, or somebody’s data is gonna be corrupted and then you gotta make sure that that doesn’t screw up with other people. there’s just lots I guess more and more advanced sort of error handling and writing this kind of robust code.
Justin Skycak (01:15) I mean, that’s why it’s good to start small and then like organically. Cause then the, these skills and tools and processes and everything can evolve over time so that
justin 4 2 (01:15) mean, that’s why it’s good to start small and then like organically. Cause then the, these skills and tools and processes and everything can evolve over time so that
Justin Skycak (01:28) when you do have a larger scale, it’s not like, no, what do we do now? It’s like, it’s just % bigger than it was.
justin 4 2 (01:28) when you do have a larger scale, it’s not like, no, what do we do now? It’s like, it’s just % bigger than it was.
Justin Skycak (00:00) the biggest growth happens when somebody says, here, you take care of it. And you’re like, oh, shit. yeah. was much right. I kind of felt like you just gave me code baby, this robot to take care of. And you’re like, don’t kill it. Like, trust you.
justin 4 2 (00:00) the biggest growth happens when somebody says, here, you take care of it. And you’re like, oh, shit. yeah. was much right. I kind of felt like you just gave me code baby, this robot to take care of. you’re like, don’t kill it. Like, trust you.
Justin Skycak (00:18) I mean, I did not study computer science in college. So I didn’t really have any sort of background in this. So was just learning the hard way, right? School of hard knocks. just, all right, crap, code blew up. Well, I guess they don’t really teach that a whole lot in computer science programs production code is where you learn.
justin 4 2 (00:18) I mean, I did not study computer science in college. So I didn’t really have any sort of background in this. So was just learning the hard way, right? School of hard knocks. just, all right, crap, code blew up. Well, I guess they don’t really teach that a whole lot in computer science programs production code is where you learn.
alex 4 2 (00:00) With this machine learning course, want people to do gradient descent by hand so it really gets under their skin.
Justin Skycak (00:00) with this machine learning course. want people to do gradient descent by hand. So it really gets onto their skin.
alex 4 2 (00:05) you actually come to code something like that, you know exactly what’s going on. It’s very easy to sort of look at a formula, create like a Python script, which is like 30 lines long. It kind of gives you an answer. It’s like, don’t really know what’s going on here.
Justin Skycak (00:05) you actually come to code something like that, you know exactly what’s going on. It’s very easy to sort of look at a formula, create like a Python script, which is like 30 lines long. It kind of gives you an answer. It’s like, don’t really know what’s going on here.
alex 4 2 (00:18) You know, people kind of the top down method of becoming machine learning engineers and never really did the bottom up part. So they kind of vaguely know what gradient descent is, but not really. And that’s not a good position to be in if you really want to make kind of like cutting edge or pushing the boundaries on things.
Justin Skycak (00:18) know, people that kind of went the top down method of becoming machine learning engineers and never really did the bottom up parts. They can’t vaguely know what gradient descent is, but not really. And that’s not a good position to be in. If you really want to make kind of like cutting edge technology or pushing the boundaries on things.
alex 4 2 (00:35) So yes, it is a necessary evil to kind of go through those painful calculations by
Justin Skycak (00:35) So yes, it is a necessary evil to kind of go through those painful calculations by
alex 4 2 (00:42) It’s like, dude, like now, you know, what I know you didn’t love it, but now you’ve mastered it. You know, intuition through repetition.
Justin Skycak (00:42) It’s like, dude, like now, you know, what I know you didn’t love it, but now you’ve mastered it. You know, intuition through repetition.
alex 4 2 (00:49) That’s where you learn. You develop that intuition and then you can get the abstractions stuff all fall into place. But if you try and skip the concrete examples, skip the repetitions, and go straight to the abstraction formulas, you’re like cargo cult
Justin Skycak (00:49) That’s where you learn. You develop that intuition and then you can get the abstractions stuff all fall into place. But if you try and skip the concrete examples, skip the repetitions, and go straight to the abstraction formulas, you’re just like cargo cult
math, you’re just parroting stuff. It’s like, they can’t really solve anything.
alex 4 2 (01:04) math, you’re just parroting stuff. It’s like, they can’t really solve anything.
alex 4 2 (00:00) With this machine learning course, want people to do gradient descent by hand so it really gets under their skin.
you actually come to code something like that, you know exactly what’s going on. It’s very easy to sort of look at a formula, create like a Python script, which is like 30 lines long. It kind of gives you an answer. It’s like, don’t really know what’s going on here.
You know, people kind of the top down method of becoming machine learning engineers and never really did the bottom up part. So they kind of vaguely know what gradient descent is, but not really. And that’s not a good position to be in if you really want to make kind of like cutting edge or pushing the boundaries on things.
So yes, it is a necessary evil to kind of go through those painful calculations by
Justin Skycak (00:00) The great thing about having a knowledge graph is that it’s a very strict implementation of mastery learning. We don’t let you do topic X unless you have mastered or at least proficient in all of the prerequisites for topic X. You have to have demonstrated that. Now, as curriculum designers, especially when we’re looking at some of these test prep topics.
justin 4 1 (00:00) great thing about having a knowledge graph is that it’s a very strict implementation of mastery learning. We don’t let you do topic X unless you have mastered or at least proficient in all of the prerequisites for topic X. you have to have demonstrated that. Now, as curriculum designers, especially when we’re looking at some of these test prep topics.
alex 4 1 (00:00) great thing about having a knowledge graph is that it’s a very strict implementation of mastery learning. We don’t let you do topic X unless you have mastered or at least proficient in all of the prerequisites for topic X. you have to have demonstrated that. Now, as curriculum designers, especially when we’re looking at some of these prep topics,
Justin Skycak (00:21) where I think is really powerful is it allows you to kind of include all those really subtle prerequisites that could easily slip under the radar. But if you’re just analyzing these problems really carefully, so that’s a really important prerequisite. I’m gonna make sure that that is a prerequisite of this topic, which means you know the student is prepared with all the necessary prerequisites before they even see the topic.
justin 4 1 (00:21) what I think is really powerful is it allows you to kind of include all of those really subtle prerequisites that could easily slip under the radar. But if you’re just analyzing these problems really carefully, so that’s a really I’m gonna make sure that that is a prerequisite of this topic, which means you know the student is prepared with all the necessary prerequisites before they even see the topic.
alex 4 1 (00:21) what I think is really powerful is it allows you to kind of include all of those really subtle prerequisites that could easily slip under the radar. But if you’re just analyzing these problems really carefully, so that’s a really I’m gonna make sure that that is a prerequisite of this topic, which means you know the student is prepared with all the necessary prerequisites before they even see the topic.
justin 4 1 (00:45) Another thing, of course, just by simply enumerating all the prerequisites, you can say to yourself, well,
alex 4 1 (00:45) thing, of course, just, just by simply enumerating all the prerequisites, you can say to yourself, well,
Justin Skycak (00:45) thing, of course, just, just by simply enumerating all the prerequisites, you can kind say to yourself, well,
alex 4 1 (00:50) hang on, this topic has got 15 prerequisites. That’s way too much. That’s cognitive overload. Huge issue right there. This needs to be split up, organized in a different way. So having that information just in front of you allows you to take all the things we know about the science of learning into account, incorporate all those sort of subtle prerequisites so that the student is not struggling at any point.
Justin Skycak (00:50) hang on, this topic has got 15 prerequisites. That’s way too much. That’s cognitive overload. Huge issue right there. This needs to be split up, organized in a different way. So having that information just in front of you allows you to take all the things we know about the science of learning into account, incorporate all those sort of subtle prerequisites so that the student is not struggling at any point.
justin 4 1 (00:50) Hang on, this topic’s got 15 prerequisites. That’s way too much. That’s cognitive overload, huge issue right there. This needs to be split up, organized in a different way. So having that information just in front of you allows to take the things we know about the science of learning into incorporate all those sort of subtle prerequisites so that the student is not struggling at any point.
Speaker 1 (00:00) One sort of temptation of the top down is, is it gives almost like a perception of expertise, a perception of skill, of familiarity with the problem that you can talk a good game.
I’ve been fooled a number of times by people because like, I want to believe that they’re good. Cause I don’t want to want to spend all this time digging in and testing them. And I was like, Oh yeah. So, you know, this and they, they use all the right buzzwords and they talk about the right methodology and the right libraries and the right frameworks. And that they talk knowingly and as if they have battle scars or whatever know, how you develop complex systems and quality software. and it’s, really like talking good game.
Because you You read some thought pieces by the area, right? So you get this sort of vibe or the sort of the language of the domain, who and what’s what and what’s cool and what’s not, what are people excited about? What are the important results? And, you could talk to somebody who’s an expert for 20, 30 minutes, like, oh, this guy knows his stuff.
Yeah, yeah, we should talk to him. It turns out it was an inch deep. But it gives you a quick payoff, It impresses people around you. might get you job. might get you grant maybe or research support or whatever environment you’re But then again, you run out gas. being able to make progress.
alex 4 2 (00:00) We have direct prerequisites, which are, in the knowledge graph, they’re sort of like, you know, like it’s like one edge connecting the topic to its its prerequisites.
Justin Skycak (00:00) we have direct prerequisites, which are in the knowledge graph. They’re sort of like, you know, like it’s like one edge connecting the topic to its its prerequisites.
justin 4 2 (00:00) we have direct prerequisites, which are in the knowledge graph. They’re sort of like, you know, like it’s like one edge connecting the topic to its its prerequisites.
alex 4 2 (00:07) in terms of direct prerequisites, I usually say that three or four is probably I start getting feeling uncomfortable when it’s anything more than that.
Justin Skycak (00:07) in terms of direct prerequisites, I usually say that three or four is probably I start getting feeling uncomfortable when it’s anything more than that.
justin 4 2 (00:07) in terms of direct prerequisites, I usually say that three or four is probably I start getting feeling uncomfortable when it’s anything more than that.
alex 4 2 (00:14) It’s like, this is, this is feeling like cognitive overload.
Justin Skycak (00:14) It’s like, this is, this is feeling like cognitive overload.
justin 4 2 (00:14) It’s like, this is, this is feeling like cognitive overload.
alex 4 2 (00:18) You know what’s interesting is that you said three to four prerequisites. I mean,
Justin Skycak (00:18) You know what’s interesting is that you said three to four prerequisites. mean,
justin 4 2 (00:18) You know what’s interesting is that you said three to four prerequisites. mean,
alex 4 2 (00:23) when you look working memory literature, somebody can hold about chunks of information in working memory. about your capacity.
Justin Skycak (00:23) when you look working memory literature, somebody can hold about chunks of information in working memory. about your capacity.
justin 4 2 (00:23) when you look working memory literature, somebody can hold about chunks of information in working memory. about your capacity.
Speaker 2 (00:00) Based on the SAT, based on all these foundational skills, they can be pulled together in so many different You can do the calculation. Let’s say you have like 100
subskills, 200 subskills, and you’re computing how many different combinations of three or four of those subskills, and you get an astronomically large number. Well, there’s no way that we’re gonna hit all of these combinations. But once you actually look at the exam and you see these combinations that show up over and over again, it’s a much, much smaller space.
It’s not this astronomically large number. It’s a large number, but like you can do it. If you’re willing to put in some elbow grease can take even a student who’s not particularly mathematically
gifted, can get them to fill in a lot of these gaps that a lot of genius students are just kind of inferring on the fly. You can make that explicit for them and scaffold them through that process and get them on that.
Jason Roberts (00:00) Anytime you wanna do something that’s outside the system, that doesn’t already have sort of a infrastructure and a thing in place, it really comes down to a force of will.
Because whatever idea you have, like, this is going to be this cool new thing, and here’s all the benefits. And that may be true, but there going to be so many obstacles. And frankly, people that are going to try and slow you down stop you. They’re going to try to derail you.
Jason Roberts (00:01) doesn’t mean you’re a blunt, instrument, Cause that doesn’t work. That just pisses people off. And then you get, instead of passive resistance, you get active resistance. You create direct enemies, people who hate you. You want to avoid creating enemies, cause they can do a lot of damage.
They may not come to your face and yell at you or something, but they will be sabotaging. They’ll be blowing up bridges and stuff in the middle of the night, right? You know, figured it out.
Justin Skycak (00:27) Yeah, because once somebody
hates you, then they just get pleasure out of opposing you no matter what it is. go from a state from like, okay, initially people are resistant to just like new stuff and stuff that makes more work for them. But once people hate you, like they might resist it just because they hate you, just to oppose you and get satisfaction out of it.
Jason Roberts (00:44) Right.
So you don’t want to offend people, demean people, make them feel bad. You want to try and make it as easy for them as and you want to help them get credit for it. Say, hey, let’s do this thing. This is going to be really cool. I’ll do the heavy lifting. And so you keep them in the loop. You keep them updated. Hey, just want you know, I’m this person. I think we can do this. What do you think? You kind of try and.
pull them along and make them feel as good about it as possible.
Jason Roberts (00:00) You want resisting you to be the hardest thing that they’re gonna have to do. Not because you’re their enemy but just because They’re just gonna hey, I like what he’s trying to do. It sounds kind of crazy it’s work, but Okay, fine,
and you just try and make them feel good about it. You’re pulling them along on this journey, Like Lord of the Rings, they’re a hobbit. You gotta come on this journey with me, right? They don’t wanna do it. They’re in their hobbit hole and they’re like, You kinda inspire them and they’re like, all right, it’s easy to go on this journey with Gandalf than it is to resist him. You kinda have to create that dynamic.
Jason Roberts (00:00) they did not quite realize the level of resistance that was going to come from the principals and superintendents and chief academic officers who just, they don’t believe in gifted education or advanced, they want everyone to just be at grade level, does everything at the same pace.
Also just makes our life easier too. Conveniently, right? You’ll notice that doing less, making everybody do the same thing, it not only makes them feel good in some ways, but just makes their life easier and it makes less work for them, right? Everyone wears a size medium shirt, now this is easy.
I don’t have to measure, oh, you’re extra small and you’re a large, extra large, you’re double X. Oh gee, we got all these different size shirts and how many of them we order of each and who’s wearing what size shirt? They don’t want to deal with that. Everyone wears a size medium shirt. You know, like, wait, you know, my daughter is really small. It’s kind of like a dress for her. You know, just cinch it. You know what? You know, it’s really a medium. And you’re just like, and my son, who’s actually big, it doesn’t even cover his stomach. They’re like, well, it’s like air conditioning. You know what mean? Just like, it’s bullshit, right?
Jason Roberts (00:00) what does willing something into existence mean? It means giving everything you got, marshaling all the resources refusing to give up and then just putting continual pressure on something until your goal is achieved. That’s what willing into existence is. It is not lying in bed at night trying to manifest it, thinking, I just really want this to happen for me.
It’s actions all the way down. Actions, actions, actions, actions. has to be something you really want because it’s not going to come easy. that’s what it takes.
Jason Roberts (00:00) communicate early and often. You almost cannot over communicate with parents. The error is, you talk to them and then two, three months go by and you don’t talk to them and they’re kind of out of the loop.
Justin Skycak (00:02) Mm-hmm.
Jason Roberts (00:11) kids falling off and then you’re like, this kid has spent the last six weeks doing nothing and it’s a whole problem.
Justin Skycak (00:17) Can’t let it like grow to be that big of a problem. Cause then if you do, then like everyone just digs in their heel. Like the problem is just too big to accept so everyone just wants to say like, no, you’re, you’re crazy. Your expectations are way too high. it makes it.
harder and harder to get things back on the rails because now there’s like some momentum in being off the rails. The kid is used to not doing a whole lot of work in class. Whereas if you catch that at the beginning, get it early, communicate with parent, the parent will catch the kid up that evening. If there’s only a deficit of 30 XP, 40 XP, guess what? The parent’s just going to make the kid do it that evening.
and then the kid’s going to come back, they’re fine. They’re on the rails the next day. We’re all good. Now we just maintain, maintain. But yeah, if the kid has racked up a deficit of thousands of minutes worth of learning that they didn’t do,
once things get to be that bad, it’s just like, it just becomes a bad explosive situation that you just, but the solution is just don’t let it get into that state. Cause that’s you never win.
Jason Roberts (01:27) Yeah,
Jason Roberts (00:00) When I was in high school, teachers didn’t meet with parents.
ever. but now because parents lobby through a barrage of emails to the teacher to the principal to the whatever and then just make everything really painful
The parents are acting like an agent. It’s like a sports agent. what they want, a lot of them want is their have like great grades so they can go to whatever college they want to go to, But they also want their kids to learn a lot, but they don’t want their kids to do a lot of work.
Right? And kids aren’t going to learn much unless they put in a fair amount of work, unfortunately, that’s how the world works.
no, you actually gotta do work. So, but parents, they, you know, they can, they wanna do less work. want my life, I want there to be low stress at home, but I want my kid, yeah, I my kid to be really educated, but I also want them to get great grades. Okay, well, the kids, there are some kids who like work really, really hard by default. Most of them don’t. there’s a whole spectrum.
But anyway, the parents at the end of the day, once they realize that, my God, my kid’s gonna get a C, or is he gonna get some Bs when I tell them they’re not gonna get A’s and they’re like, my God, this is really, they’re not gonna get to go to Harvard or whatever dream they have. And so then they serve as this, they wanna lobby the school. And so they do, and so that has helped lower the standards, I think, especially these private schools, right?
paying 30, 40, 50, $60,000 a year and, and, and, and, know, and my kid works really hard in this. these, these costs and the, and the superintendents and the head of school and stuff, they’re like trying to keep the parents at bay. Cause that’s what pain for the pain for the schools, for the administration and the teachers are parents, parents P for pain.
Jason Roberts (00:00) so I used to coach this men’s soccer team for years and all these guys who were these ex college players and I’m a bunch of them would coach these sort of elite academy level club soccer teams, you know, under 12, under 14, under 16. And I’d be like, I asked him like, how do you like that? Like, oh, it’s great. The kids are great. He’s like, but the parents are a nightmare. I’m like, really? He’s like, oh yeah. Oh yeah. It’s like.
Why didn’t my kid play left? They should play the right side. And why isn’t he starting? And they use it, you know, and it’s just a constant lobbying for the parents. It just ruins the experience for the coach. The coach is trying to, you know, not saying every coach makes good decisions or is fair, whatever, but it’s the parents. Parents equal pain for
coaches and teachers and administration. They’re trying to everything they can to limit said pain. And a lot of the changes you see in the curriculum, well, there’s no gifted class, accelerated classes. We don’t do a lot of homework. We have makeup tests and makeups on the makeups and we have all this stuff. It’s to the complaint, the agents, parents from…
lobbying and creating all this endless cycle of pain. so then you get this situation you have now. so the teachers are just like, whatever, man. I mean, just make it really easy.
Justin Skycak (00:00) if nobody’s checking the homework how’s the kid gonna learn? Because the kid doesn’t know how to do these problems correctly. The parents almost certainly don’t know how to do these problems correctly. the only kids that get through are the kids who have somebody in their life or whose parents can bring in somebody into their life who can effectively do what
the teacher is supposed to be doing. You’re right, yeah.
Jason Roberts (00:19) Or they’re at the 99th percentile. They’re just super
high aptitude and they’re really into it. And so they’re going to all these extra resources because they’re a physics nerd or whatever. Cause this is what they want to do. But that’s like that, that’s the one out of a hundred kid. everyone else is like, I don’t know what’s going on. Right. And you have a couple of kids who are drafting off that one kid. He’s helping his or she’s helping her little.
Justin Skycak (00:35) Yeah.
Yeah.
Jason Roberts (00:45) a few couple friends and they’re kind of figuring it out, but everybody else is just like, what’s happening?
Justin Skycak (00:47) Yeah.
Yeah, yeah,
yeah, exactly. And very quickly, it gets just so out of hand that like, even if you had a tutor sit with the kid and help them work through the homework problems, like they wouldn’t get a whole lot out of it because the problems are now so far ahead of their ability.
Jason Roberts (00:00) parents are not aware of actually how little their kids are learning. Whatever you think your kids are learning, in most cases, they’re learning a lot less. They know less, they’re doing less work, they have fewer skills, less understanding. You know, it’s bad. know, especially…
Things in math and physics and stuff like that that.
Justin Skycak (00:22) foundational for other skills that you might need to do. And also it’s a hierarchy
Jason Roberts (00:26) is it just takes a lot of hard, consistent work to keep making progress. It’s just that the standards have to be higher, they have to be more rigorous, they have to be quantitative. ⁓
That’s just is and if you’re not going to do it and going to make everything mushy and fluffy and we’re just going to have group discussions and projects then guess what? Everybody’s going to do less work. Kids, teachers, everybody. And then at end of the day, people come out and nobody knows anything other than like that one, two kids who were sort of were mostly self-taught and you know.
That’s it. That’s how it rolls.
Jason Roberts (00:00) most undergraduate math majors, We’re playing this little game of pushing symbols around, but nobody really has a great intuitive grasp of the subject itself because it’s not really taught. not really structured in that way.
pushing symbols around without concrete examples is just a really inefficient way to go about things. they just skip the concrete examples
And so you just do the theorem proof theorem and then so, so poorly done at the undergraduate level, almost without exception. There are probably a few, probably like 2 % of math professors in the undergraduate university level who take the pedagogy seriously and are really trying to teach them the rest just show up lecture, problem set, whatever.
Good luck.
Justin Skycak (00:00) discovery learning, like, save that until you’ve got your fundamental skills. it takes so much time
to go through and make a discovery. Like, how about let’s save that for when like making the discovery actually yields like serious results.
Jason Roberts (00:12) get more out of it. If we want to get more out,
we can get more from these reps if you have the component skills in place. we’re thinking of like, what’s the most efficient way from go to A to Z?
Justin Skycak (00:18) Yeah.
Jason Roberts (00:22) the total hours put in, if you try and do stuff too early, then there’s a great inefficiencies, you brought some stuff forward that you would have learned later, but in aggregate, spent more hours getting to Z because you’re not getting as much out of it. And you could have spent that time getting these really important requisite skills in place, you know.
Justin Skycak (00:43) Yeah,
exactly. And the end result of your discovery, even if you manage to make the discovery, like, guess what? Nobody gives a shit because like, it’s not real new, it’s new knowledge to you. You did research, relative to your own knowledge base, but humanity has more knowledge. It’s not like a publishable result mathematically. when you’re going to like invest a lot of time and effort into a project,
Jason Roberts (01:01) Stuff in there, yeah.
Justin Skycak (01:07) make sure it’s something that’s like actually like impressive.
Jason Roberts (00:00) rookies got to make rookie mistakes. And part of rookie mistakes is they don’t listen to senior people, right? No, was old people. They don’t know stuff, right? I just want to do it. Okay. Rookie. See you at the end. I’ll see you at the finish line. You know, how’d that go for you? You know what I mean?
Jason Roberts (00:00) The human body attempts to create homeostasis, like keeps everything the same. So whenever you want to change it, say get in better shape.
become more muscular or something. Well, guess what? Your body’s going to resist that. So you have to work hard to change your body. You have to go and lift heavy weights consistently, over a period of time and your body will change.
constantly have to be hammering away with intention.
The world is like that in a way too. There are all these sort of moving parts and these systems and these vested interests and people are just, they found a place in this world. This is my job or this is my department or this is my thing that I do and they don’t really want it to change typically.
and then somebody wants to come in and go we’re gonna change stuff they’re just like what
people don’t want to adjust. They don’t want to change. Right. Even if you can make a strong argument that this thing you want to do is going to be an improvement. It’s going to help kids or whatever. It’s easy for people to rationalize fighting against it or resisting it
Jason Roberts (00:00) If you just can every day, you just keep doing what you’re doing, you just keep executing and keep improving and you just never give up and you keep pushing… the universe will bend.
Okay? The universe will bend. It cannot sustain that kind of continual pressure, the continual effort. The world will just say, all right, fine. Fine!
Justin Skycak (00:00) calculus just felt like a video game
because climbing up this math ladder, there’s infinite progression. There’s no beating the game and having to go on to a new one. It’s an infinite game. And you know, like everything that you do in this game, you can take with you later in life. This is a game that actually matters. And it’s a game that your parents are actually like really, like they’re proud of you for spending like eight hours just like holed up in your room playing this game.
Jason Roberts (00:07) The infinite game of math, yeah.
You’re
playing eight hours playing Eve online or EverQuest or World of Warcraft. What are you doing? Like, I’m in my guild! And you’re like, what? Get outside. This is terrible.
Justin Skycak (00:28) Yeah, yeah, exactly.
Yeah, exactly.
And everybody’s like super impressed when you come out like, wow, my kid taught himself calculus over the summer. And it’s so funny how similar it is to just like, my kids spent the summer like playing eight hours a day of Call of Duty and ⁓
Jason Roberts (00:51) You’re not telling
people that unless you’re just complaining like my kid only plays Call of Duty. I don’t know. I’m worried they’re going to become deranged. You know, what do we do about this? It’s the opposite.
Justin Skycak (00:57) Yeah, exactly.
Yeah,
right. It’s it’s seen as a problem. But you just change it to like calculus or math. And suddenly, the perception just does a complete 180. But the experience is the basically the same. So yeah, so I was hooked on this. I’m like, I Yeah.
Jason Roberts (01:15) you were having more fun. was more fun than playing Call of Duty or Eve Online.
Justin Skycak (01:20) Yeah, so I beat the calculus game basically, and I’m like, okay, what are we doing next? I’m like full on addicted at this point. I can’t make it through the summer without more math.
Jason Roberts (00:00) You want resisting you to be the hardest thing that they’re gonna have to do. Not because you’re their enemy but just because They’re just gonna hey, I like what he’s trying to do. It sounds kind of crazy it’s work, but Okay, fine,
and you just try and make them feel good about it. You’re pulling them along on this journey, Like Lord of the Rings, they’re a hobbit. You gotta come on this journey with me, right? They don’t wanna do it. They’re in their hobbit hole and they’re like, You kinda inspire them and they’re like, all right, it’s easy to go on this journey with Gandalf than it is to resist him. You kinda have to create that dynamic.
Jason Roberts (00:00) parents are not aware of actually how little their kids are learning. Whatever you think your kids are learning, in most cases, they’re learning a lot less. They know less, they’re doing less work, they have fewer skills, less understanding. You know, it’s bad. know, especially…
math and physics and stuff like that
if you’re going to make everything mushy and fluffy and we’re just going to have group discussions and projects then guess what? Everybody’s going to do less work. Kids, teachers, everybody. And then at end of the day, people come out and nobody knows anything other than like that one, two kids who were mostly self-taught
Jason Roberts (00:00) I used to play, this game with myself. I would imagine myself as if I were the tutor for this bloodthirsty king for his kids. And he really cared a lot about the education for his kids. And he had already executed the previous two tutors.
You’re the new tutor and you have these two kids and you know there’s a decent chance that after the session’s done, he’d be like, come here, what’d you learn today? And then he starts to quiz them on.
this thing that they said they learned. And if he was unimpressed, he was gonna have your head taken off. Okay, so if you’re in that situation, how would you teach? You’re like, my God, okay, so I’m teaching them how to solve linear equations or something. You’re like, okay, am I going to just talk at them for an hour? No.
with no practice. They’ll totally, because if he asks them to do some, some linear equations and they can’t solve it, I’m dead. So I’m like, okay, here’s what I’m gonna do. Here’s what a linear equation is. This is what it represents. Here’s how you solve it. Now let’s, I’ll go through a couple examples, then I’m gonna have you guys do a couple of examples. And then I’m going to give you progressively more.
challenging ones, negative numbers or fractions or whatever, and we’re gonna kind of build on it, progressive them, we get a lot of practice, but I’m gonna keep raising the level. But it’s always going to be you going through, as a student going through the process, the procedure of performing the skill that I’m trying to get you to acquire. This would be the same situation we go on of your tennis lesson or.
learning how to play the violin or whatever, you know, you’re the coach, the instructor, the teacher is going to have you performing the skill and giving you feedback on what needs to be improved, what you got right, what needs to be adjusted to correctly execute the skill. And so that’s what I did. and, and of course, when you take that attitude and sort of, it’s a way of defining, of describing in a sort of elaborate.
in maybe somewhat bizarre deranged way of extreme accountability. If your ass is on the line, your life is on the line, how are you going to do this thing? Well, you’re probably going to be a lot more serious about it. And you’re probably going to do it a different way than if it just didn’t really matter.
Jason Roberts (00:00) when people make decisions purely for the money.
it often leads you in the wrong direction. You end up in a place like why am I so dissatisfied with my life? Why am I so frustrated? It’s like, you’re not doing what you really, really want to be doing. I mean, this stuff can be overplayed. mean, obviously you have to make a living. You have to be realistic about what you can do to actually pay for, rent or mortgage or, you know, get by. but it’s important to just, to really always be thinking,
Justin Skycak (00:03) Yeah.
Jason Roberts (00:30) you know, looking at your North Star like, what is it that I really, really want to do? Go do that.
Justin Skycak (00:35) again, it just comes down to playing the long game, right? need to be making some progress in the short-term, but your goal is not to optimize the short-term outcome of salary, of prestige or whatever.
Do the short term well enough that you can continue playing the long game. That you don’t like run out of money or like you’re on the street or whatever. the long game, that’s what you always want to be optimizing towards. Even if it feels slow in the short term or there’s some pain in the short term, it’s like, whatever, as long as you’re making progress towards the long game, ⁓ then you’re good.
Jason Roberts (01:09) Yeah, well, yeah, 100%.
The thing about it is you get too distracted with the short term and you can lose sight of the long term goal, right? And you get these short term dopamine hits, hey, I made some, I made more money or whatever. Right. And then you look up a year or two years, 10 years later, and you’re like, where am I? And it’s like, well, I mean, you’re making pretty good money and you got a pretty good situation, but you don’t, you’re not really happy with it.
Justin Skycak (01:20) It’s like shiny object syndrome, all this.
Yeah.
Jason Roberts (00:00) you got to get inside the trade. what that means is I came from a world of high frequency trading.
What would happen is some of these companies would hire really highly educated, smart people with PhDs in physics and computer science and math they would get a group of these people and they would say, here’s our, here’s our, massive historical database of historical time series data of all the trades that happened every minute or every second or every hundred over the second for the last 10 years, go write some algorithms that can predict
what’s going to happen, where the price is going to go in the next 30 seconds or minute or 10 minutes or whatever. And they were almost doomed to fail. typically what would work better is if you’d have a professional trader who would spend years and years trading this stuff manually and understood how you made money.
with a particular kind of trade. It’s like, look, when you’re trading this, when you do this kind of trade, these are the factors to consider. These are the forces that are at play. These are the things you’re to watch out for. This is how you can lose a lot of money. that’s this hard, hard won experience from a lot, from winning and, losing on a lot of trades and learning and understanding. And an emotional instinctual reaction to, yeah, I would not buy at that point.
my experience has been, if you have been the domain expert yourself, you understand exactly how this works.
get inside the trade. Don’t, don’t just automate it. Like do the thing and to understand what the hell is going on. Really understand it. Have, emotional scars from it. Then automate it.
Justin Skycak (01:16) Yeah, and I remember the…
There is some.
Jason Roberts (00:00) version one is the expert just demonstrates the thing over and over and over and talks about it. And the student doesn’t get any practice on it. That’s a fail. ⁓
Or two, I’m not going to tell them how to do it. I’m to just say, go and do this stuff. and they’re just flopping around. That’s highly inefficient.
Justin Skycak (00:16) Yeah, and it becomes so obvious when you think of it ⁓ in terms of sports, Like, imagine you sign up for lessons with a tennis coach and the whole time they’re just, they’re showing you all these techniques and stuff. And then like, and then it’s done. Hour passes, like you haven’t, you’re just holding your tennis racket the whole time. You haven’t hit a ball or anything. You haven’t even swung it.
the other failure mode is they just say like, okay, you two versus you two, go play each other and I’m going to go run an errand. I’ll be back in an hour.
Jason Roberts (00:45) This is a dereliction of duty. This is not doing your job. Right? So why in a math class would we expect the situation of the teacher just talking the lecture, right? Which, know, from universities in particular, but typically most schools have done with this sort of a lecture teacher gets up and just talks. Sometimes they cold call on people.
and if they assign homework, it’s like, okay, I’m gonna give you, I’m gonna talk about tennis for an hour and then I want you to go practice by yourself.
You know, and maybe I’ll have you take a video of a couple things, I’ll give feedback. Like that, no, that is stupid. anyway, anyone who has actually tried to acquire, seriously tried to acquire skills in something, sports, music, art, anything like that, where it was important to develop these skills.
They understand that this is basically how it’s done. I mean, there’s a little variation. You can change things up a little bit and whatever, but that’s the core of what a super efficient learning process would be.
Justin Skycak (01:44) guided instruction with rapid feedback cycles of explicit direct instruction on what the student is supposed to be doing.
what’s the proper technique, followed immediately after where the student is actually going and hitting the ball, going through their reps, getting really solid on the skill. They do that on some more skills. The next session, they pull some of those skills together, compound movements.
Justin Skycak (00:00) Enjoyment is a second order optimization. First is performance improvement. Like, get that right. And then without lowering your rate of improvement of progress, just try to make it as enjoyable as possible.
Jason Roberts (00:12) when a parent is spending their own money, or an adult if they were doing it themselves, and you’re paying for guitar lessons or something.
Like that money matters, right? Like the accountability, if it’s not working, you’re going to be like, this is bullshit.
extreme accountability means you’re focused on a result. And, and the result, is a combination of two things. It’s getting better at the thing, but also not making
it’s so painful that the kid or the adult doesn’t want to do it anymore.
If you hire a trainer to get you in shape, the trainer’s thinking, okay, well, I need to keep you consistent, making progress, showing you that you’re making progress and closer to the goals and making it so that it’s not so painful you don’t want to come back tomorrow or next week. Right? Now, if either of those things are false, if it’s too painful, it’s a fail. They’re going to stop coming after a week or two or three or depending on how much…
pain they’re willing to take or how much suffering they’re willing to endure or how stubborn they are. But eventually it’d be like, uh, or two if they, there’s like, man, I’ve been working out with this guy for like three months and I think I’ve lost a pound.
this isn’t moving things in the right direction for I don’t know why I need to find someone to get some results because it’s got a lot of money. It’s a lot of time, you know, whatever. And even if it’s fun, he’s like, oh, he’s like funny guy. All these crazy seal stories from his time at the seals. And be like, are you getting any stronger? No, you lose your weight? No. OK, well.
Justin Skycak (01:35) So it’s like it.
you can’t have that and not have performance improvement. You can have performance improvement and not really have so much enjoyment. And it’s like, it’s a real thing. People will benefit from it.
but you’re gonna increase your surface area if you focus on the enjoyment part after. But the enjoyment part is like the icing on the cake there.
Jason Roberts (01:58) 100%.
Jason Roberts (00:00) You optimize too much for the fun and then you’ve taught the students that things are fun and they can’t really be hard.
Justin Skycak (00:00) Yeah.
Jason Roberts (00:10) you gotta be all business.
keep them focused, listening, taking their homework seriously, you know, whatever. you can’t go from being this lax teacher that the kids don’t respect,
You can’t say, right, now I’m serious. I come back from Christmas break, like maybe the principal came down and said, look, I mean, I’m looking at these tests. This stuff doesn’t look good. You got to bang, bang. then the teacher’s like, OK. You’re like, teacher, you’re like, decided you’re to be a hard ass or something.
Justin Skycak (00:37) Yeah,
there’s a directionality to it. You can’t recover from being a pushover. Once you’re a pushover at the beginning and then you become a hard-ass, like, they just hate you more. Yeah.
Jason Roberts (00:45) The kids are like, yeah, right. They just rolled around. I had
happened to me when I was in high school. I can’t remember this teacher. And she, she was like, started out like that. And she’s like, she then she realized she was a young teacher, a first year teacher or something. She just graduated from college and she was going to be our friends and didn’t take her seriously. And because she wasn’t a serious person, you know, and so we didn’t respect her. didn’t like hate her. And then when she started to be like a hard ass, then we hated her. Right.
And she was one of our biology, there were two biology teachers. The other biology teacher who was older, she was all business. There’s no messing around with Susan Radford was her name. She was all business and you’re like, pay attention, you did what you’re supposed to do. And she would lighten up a little as year went on. And then she could lighten up and then you loved her.
Justin Skycak (01:14) Yeah.
Jason Roberts (01:36) Because the most important thing for a teacher is not that they love you, is that they respect you and they do what they’re supposed to do and they take their work seriously. That’s the important thing, especially like middle school and high school, because then kids testing boundaries and they’re 15, 16, you don’t want to be in school. You want to go mess, you want to do anything but sit in your fricking biology class. Right? Even if you’re a good student, you’re like, I don’t want to be here. Right? So that model.
Justin Skycak (01:57) Yeah.
Jason Roberts (02:03) And any really good teacher understands this fundamentally. Not that you can’t do some fun things, but you have to instill respect from day one. I understand you are not to be messed with. And then you can lighten up over time. But there are boundaries, there are expectations, and there are consequences, and there are rewards.
So if you, if you, transpose that onto a learning app and you’re like, we’re fine. And this and dance and baloney and, know, and the kids are like, ⁓ you know, whatever. then you’re like, all right, now we’re going to start learning hard. The kids are like, this is stupid. I it used to be fun. No, it. You know, but.
Justin Skycak (02:43) Yeah.
Jason Roberts (02:47) We can go from the teacher that was a hard ass, and we can lighten it up a little bit. People are like, oh, think my daughter likes it a little more. It’s kind of fun. That’s how it’s going to go, I think.
Jason Roberts (00:00) if you really want your life to be an A, you’re like, I want my life to be awesome. And even if you don’t have an absolute specific plan, get closer to doing the thing.
that makes you happy or that you feel aligned or whatever, what terminology you want to use. Like this is the thing that I like, I enjoy doing.
Justin Skycak (00:14) Yeah.
Jason Roberts (00:00) We’re trying to ultimate.
online learning system, basically what that would do is replicate the effectiveness of the best possible human tutor that you can imagine that has almost superhuman abilities to understand exactly what you know and don’t know and what you should be working on. Okay, if that’s what you’re trying to do, then you need to continually keep that person in mind and try and mimic.
their behavior in that situation. And this is a long process and we’re getting closer all the time, but still we have a lot we can do, but you know.
Jason Roberts (00:00) you don’t really learn by watching somebody else. You can become familiar with something, but you don’t learn it until you actually start performing it.
Justin Skycak (00:00) Enjoyment is a second order optimization. First is performance improvement. Like, get that right. And then just, without lowering your rate of improvement of progress, just try to make it as enjoyable
Justin Skycak (00:00) Maximize progress subject to the constraint that the pain is less than the quitting threshold. And you just run that over and over. And the quitting threshold, that can vary over time too. Once you start seeing progress, you’re much more willing
endure some pain for even more progress
Jason Roberts (00:00) you have to be careful when you make the transition that you don’t lose soul and become corporate. you can’t sacrifice or give up the essence, which when things become about the money,
and you’re just trying to make everybody feel good, can turn into that. The money will come if you deliver value for people. It’s as simple as that. And since we don’t have any outside investors, we can make that choice, that conscious choice. So we’re not going to give in to…
easy shortcuts and things to just make it nice for, but that while sacrificing results, quality of the education.
Jason Roberts (00:00) The whole project-based learning can be taken way too far. Projects are built on a foundation of skills. You can’t do projects without skills and have it be efficient at all. mean, you can. It’s just going to be incredibly inefficient because the students don’t really know what they’re doing.
Jason Roberts (00:00) we’ve heard from a lot of families tutors and teachers that kids
have not mastered their math facts. They don’t know the multiplication tables. They’re finger counting. Not finger counting in fourth or fifth grade. They’re finger counting in 10th grade. And if you can’t do, you know, even multiplication, if you don’t know multiplication tables, you’re gonna really struggle even with basic algebra.
Justin Skycak (00:00) a lot of people realize later, who go into software engineering, like they don’t really care about math in school, And then eventually they build up the foundations of coding and then they realize…
that if they just knew all their math, then they could be doing so much more. the earlier you make this happen, somebody gets interested in coding, sees how important math is to doing non-trivial coding. in life the get the motivation to skill up on both fronts. just imagine a kid graduating high school.
not only do they know pre-calculus with like coding applications but also calculus linear algebra multivariable calc and differential equations basically your core engineering math all the coding applications and they come into college and they’re just blowing the socks off of
anybody who gives them an opportunity to do some research, an internship, they already get the basics of everything. They’re ready to actually make serious impact. It’s so rare to see in an undergraduate researcher or an intern, right? you can’t count on them You just like throw them a toy problem.
whatever, but like if you can actually make a serious impact at a young age, because you have the skills to do so, then you can just compound that into a massive compression of time and figuring out what you’re interested in and everything.
Justin Skycak (00:00) I’m doing a hard thing and I’m not sure I’m able to do it at the beginning, but then we actually get through reasonably quickly and now I can do it and that’s really cool. But also do things ever stop getting hard? Like when’s it going to become easy? what they see is the short game.
And the short game is on loop, it’s hard, it’s hard, it’s hard. They don’t always see the compounding so what I would have to do sometimes is I show them like the next assignment and they’re just like, that looks hard.
then I show them do you remember back when we did logistic regression, breadth first search and you had that same reaction right there. What’s your reaction to this now? And they’re like, no, I could that’s a component of this. could, I could code that up in 10 minutes. what’s the big deal? Like, no, no, no. Remember you were saying just like three or four months ago that you were, you were groaning in the same exact way. And I guarantee you.
three or four months from now, this back propagation, this Dijkstra’s algorithm, whatever, you’re have the same reaction. gonna be muscle memory. You’re gonna have new superpowers and you’re not gonna realize that you have them until I show them to you.
Jason Roberts (00:00) it’s important when you’re in a position where you know how things work to tell younger people, this is the situation. This is the level of talent, the level of skill you’re gonna have to have if this is the thing you want to do. Now if you don’t wanna do it, that’s fine. There’s a place for everyone, everyone has a place. Not everybody has to be a math genius. Not everybody has to learn abstract algebra.
But if you want to do the things that you say you want to do or from this list, then okay, let, let us lay out a plan to help you be on schedule to get that. So you don’t find yourself in a situation where you’re getting blown out of the water
Jason Roberts (00:00) You need to understand. look, there’s a lot of things you can do in life that are just not super competitive. It’s not that hard and just kinda go in and just do your thing and it’ll be fine. But there are other things that are just super, super hard. And there’s a lot of competition.
So you better know what the game is.
Jason Roberts (00:00) we’re trying to increase optionality for kids. when they’re 12, 13, 14, even 15, 16, they don’t really know who they are what they want to do.
But you want to keep those options open. As an adult, as a parent, or even as a teacher, you’re trying to help kids keep as many doors open as possible so when they get a better idea of who they are what they want to do, that they can get through the door.
Jason Roberts (00:00) If you don’t make the decisions, the decisions get made for you. That’s just how it is. Either you go into the world and you make decisions to make things happen, or decisions be made, and they will happen to you.
Turns out if you make things happen, you make intentional decisions, make things happen, you tend to get something that’s much closer to what you really want.
Jason Roberts (00:00) heard from a lot of families tutors and teachers that kids
have not mastered their math facts. They don’t know the multiplication tables. They’re finger counting. Not finger counting in fourth or fifth grade. They’re finger counting in 10th grade. And if you can’t do, you know, even multiplication, if you don’t know multiplication tables, you’re gonna really struggle even with basic algebra.
speaker-1 (00:00) You know, and I hear from teachers and chief academic officers and tutors, and they’re just apoplectic about the situation because you get kids who are in sixth, seventh, eighth, tenth grade. Finger counting.
don’t know the multiplication tables, it’s like, how the heck are gonna factor quadratic when you don’t know the multiplication tables? Can’t do it, not really. And then guess what? Now we can’t factor quadratic, now can’t do algebra.
speaker-0 (00:27) And if you do manage to grind through, ⁓ just figuring out these factors on the fly, it’s gonna take you way, way, way too long. It’s gonna take you like 10 minutes to factor.
speaker-1 (00:37) Which some kids can do in their head and be like,
10 seconds. So you’re not going to want to do any proms. It’s through, yeah, teacher gave us four factoring problems and it took me all night, you know? And you’re just like, dude.
speaker-0 (00:42) Yes.
When you get to calculus, if you’re taking 10 minutes to factor a quadratic, that’s only one component of a calculus optimization problem. You’re going to be spending half an hour on this problem that should take you two minutes.
speaker-1 (00:00) The secret to success in life is consistent effort. You don’t have to, whether it’s exercise or learning math or learning a language or whatever the heck it is you’re trying to do. It’s like you don’t have to do the superhuman effort thing. Just get started and then make a consistent push every day, even if it’s only 15, 20 minutes.
speaker-1 (00:00) cause people are like, ah, I don’t want to do so many problems. just want like a conceptual thing. I want an intuitive understanding. It’s like, it’s repetition. Intuition is the
of repetition. You have to do reps. That’s where the intuition comes from. Me or anybody explaining something to you that feels intuitive. You don’t have the intuition. You have to earn intuition. Intuition is earned through pain.
through failure, through suffering, through trial and error. That’s where your intuition comes from. You don’t wanna suffer, well you don’t get intuition. so, and that’s something that we, you know, obviously you wanna limit the suffering, like let’s just do the things and go through repetitions and say, okay, trial, okay, okay, right, I made a mistake there, right, okay, right, I see how this works. You do enough of the reps, whether it’s shooting free throws or…
you know, or doing math problems or whatever, it’s like you have to, you have to get the reps in, you get the intuition. But then when you get the intuition, then you can really understand how this stuff works and you can actually solve challenging problems. Because until you have the intuition, it’s hard to really see your way through innovative solutions. There’s just nothing to work with.
speaker-0 (00:00) One thing that I have seen ⁓ students, even adult students do a lot is look back at the reference too often, like thinking that it’s free to look back at the reference. When in reality, if you are trying to recall something from your head
and you look back at the reference instead of trying your best to lift that weight off of your, long-term memory. If you look back at the reference, you’re basically just letting the spotter lift the weight.
speaker-1 (00:26) Yeah, so you’re weightlifting and you’re like, I can’t get it and the guy behind just lifts it up for you like, okay. He didn’t really lift the weight.
speaker-0 (00:31) Well, yeah.
Sometimes people will ask for the spotter’s help before they even get to the point of trying really hard. It’s like just the moment that it stops becoming super easy, they’re like, okay, look back at the example. When in reality, that is the moment when you are getting the most bang for buck out of recalling the information.
speaker-1 (00:52) The struggle during the active recall process is when you are strengthening the memory.
speaker-0 (00:58) Like, don’t look back at the reference right away. Like I want you to try to pull this from memory.
Coach the student not to actually look back at the reference unless they absolutely need to. Because that is a way that sometimes students kind of shoot themselves in the foot. ⁓
speaker-1 (01:13) They
shortchange the learning process. They shortchange it, right?
speaker-0 (01:17) Yeah. Or if they have like the, worked example up in another tab and then they’re looking at that while solving the problem. Like, is, that is the worst. Never, never do that.
speaker-1 (00:00) It’s painful, but sometimes it’s good to do sort of a breadth first search instead of just like, hey, we’ve got some of the work, so let’s just build on that. We’re like, let’s go tackle these other things that are really, really different. That’d be a ton of work, but it forces us to generalize, to solve those concrete problems and then generalize it and pull it into the model, pull it into the UI, pull it into the user, the student experience, and so it all makes sense because.
If you go too far down the line and you have lots of users and lots of customers and your stuff, you just can’t break things again and go back. It’s it’s hardened. Like that’s where you put the road, that’s where the road is, right? Well, it’s like, well, it goes like, it’s too hard to like, you know, we’re gonna create a new highway through these, you know, through this subdivision. It’s a nightmare. So you just kind of try and figure that out early. the way you do that is you say, well,
here are all these things that we think we might want to do, or we do think we know we want to do, they’re different, it’s going to be painful, but let’s just bite the bullet and do it now. And it slows down visible progress, so perceivable velocity, product velocity, but like, why is it taking so long? like, cause we’re tackling all these really hard, massive projects that is going to pay off and we’re going to be able to release all these things. But.
speaker-0 (01:19) Playing the long game. We are just going in all directions that we think is worth pursuing.
speaker-1 (01:21) Play the long game.
You get a lot of things that are like between 60 and 90 % done. They’re just sitting there and you’re like, God, we just can’t finish with these other things because you do have emergency things going on. You do have bugs, scalability problems. have, you know, just like really important features have to roll out for a user segment. You know, like we, you know, the schools have been coming on and wanting all this stuff. It’s like, know, there’s so much stuff. You just can’t.
You can’t just blow it off. You gotta deal with it. And then it’s like, well, why isn’t this done? It’s like, you know, yeah.
speaker-0 (00:00) We talk about the benefits of pre-learning the material before you go into a normal college course. Cause, it’s a roll of the dice, whether you’re going to get a decent instructor or not.
speaker-1 (00:08) There’s so much variance in the quality of instruction. You’re going to get some people,
great research mathematicians, but horrible pedagogues. it’s you and your group of study mates on these impossible problem sets. It’s more like a
framework for making you teach yourself as opposed to providing a real scaffolded learning experience.
There’s a problem set do it or not or don’t do it.
Grader might grade it, you might get it back two or three weeks later.
You can end up for some real crash and burn situations. If you already know it all, then it’s like, ah! All right,
speaker-0 (00:41) Not only does the pre-learning minimize your risk of that bad situation happening, but if you are blowing the class out of the water and interacting with the instructor, that’s setting you up. You get a, like amazing rec letter, guess who is up for whatever
opportunities that professor has in mind.
speaker-1 (00:57) you should apply for. We got a summer program. whatever. They’re like, ⁓ I got a kid. I got a
speaker-0 (01:02) You just
get a reputation for being the smart kid and it doesn’t matter if you’re being smart in real time or if you’re smart because you’ve already built up a large knowledge base. You’re just you’re a smart kid either way and you get the smart kid opportunities and that can compound into a virtuous cycle.
speaker-1 (01:18) I mean, that’s an incredible position to be in because, for any of us who’ve been like a math or physics major, and especially if you went to a place that had a lot of top-notch students,
you’re learning stuff for the first time and they are going at a breakneck pace and they are not playing games and they don’t get retakes and there are no study sessions, you know, it’s just boom, here you go. And then the average score is a 27 on the midterm. It’s like, jeez, you know, it’s brutal. And then you find out that like a bunch of the kids had actually, oh yeah, I took this at the state university when I was in high school. And you’re just like, what?
What, you guys, wait, half you guys already taken this? This is bullshit. You know, it’s like, we’re in a Spanish class and you got like a bunch of the kids who actually speak Spanish at home. You’re like, why are you in Spanish one? Your Spanish is, I don’t really write it.
speaker-0 (02:09) It’s like you get gaslit into thinking you’re dumb and everyone’s just like learning so much faster than you and then the glass shatters you realize they already came in.
speaker-1 (02:19) Oh, you guys all got the cheat codes? Oh, great. The cheat code is learn the material before you take the course.
speaker-0 (02:23) Yeah.
The point of learning ahead of time is not to sit there bored in class. It’s so that you can actually grapple with the hardest problems and actually extract learning out of those in an efficient way. You can be the go-to person for everyone needs help with the class. You’re that person. You’re getting reps on teaching this material to your friends. You’re making connections You’re the,
front running for any opportunities that the professor has in mind, whether it’s research with them, one of their buddies, internship with a company they have a relationship with a fellowship, you just you never know what it’s gonna be. But if you’re the go to person for the subject knowledge, you just get pulled into all these interesting
advantageous opportunities that just compound one thing into another. Guess what? Got a great internship? Well, your next is probably going to be even cooler because now you have this experience that nobody else has.
speaker-1 (03:21) the it’s snowball effect. The compounding effect.
speaker-0 (03:25) Exactly. And this really buys you a lot of time to figure out what want to do.
You get ahead, you get opportunities, you have time. I can afford to go like, my soul is not connecting with this job. I’m going to go try this other thing that I’ve interested in. Do that for a bit, eventually things merge together into your little niche. And it just
buys you more time to find that because if it takes you too long to find that, then you never actually do find it because you have to pick something.
speaker-1 (03:50) Gotta pick a major, gotta pick a job, you know.
speaker-1 (00:00) The cheat code is learn the material before you take the course.
speaker-0 (00:05) The point of learning ahead of time is not to sit there bored in class. It’s so that you can actually grapple with the hardest problems and actually extract learning out of those in an efficient way. You can be the go-to person for everyone needs help with the class. You’re that person. You’re getting reps on teaching this material to your friends. You’re making connections You’re the,
front running for any opportunities that the professor has in mind, whether it’s research with them, one of their buddies, internship with a company they have a relationship with a fellowship, you just you never know what it’s gonna be. But if you’re the go to person for the subject knowledge, you just get pulled into all these interesting
advantageous opportunities that just compound one thing into another. Guess what? Got a great internship? Well, your next is probably going to be even cooler because now you have this experience that nobody else has.
speaker-1 (00:57) the it’s snowball effect. The compounding effect.
speaker-0 (01:00) Exactly. And this really buys you a lot of time to figure out what want to do.
You get ahead, you get opportunities, you have time. I can afford to go like, my soul is not connecting with this job. I’m going to go try this other thing that I’ve interested in. Do that for a bit, eventually things merge together into your little niche. And it just
buys you more time to find that because if it takes you too long to find that, then you never actually do find it because you have to pick something.
speaker-1 (01:26) Gotta pick a major, gotta pick a job, you know.
speaker-0 (00:00) quite a courses leave off versus the level that you have to be at to implement the stuff. It’s not an absurdly high level, but it’s just you need to be comfortable with some of this programming logic, not just the syntax.
speaker-1 (00:14) have to have a certain automaticity with it. It’s one thing it’s like, well, I listened to a lecture and I watched a video and I did a 15 minute project with dictionaries. great start, but you’re not, you have not even come close to reaching problem solving level automaticity with these skills.
speaker-0 (00:00) helpful more practice math facts like bring them out of the problem solving context and drill them more frequently. Kind of like if you’re in athletics, you don’t just exercise your skills by playing games
There are some skills that you just have to be really, really solid on like shooting free throws. You don’t practice like shooting free throw during a game No, you actually go to the line and practice doing that. That’s kind of like, ⁓ math facts, automaticity practice.
speaker-1 (00:00) lot of students come to us in pre-algebra and algebra and they don’t know their multiplication tables. And they’re not very good at fractions. And that is very common. Schools not doing a very good job of that.
they’ve drank some cool aid. like, we don’t have to memorize anything anymore. It’s like.
speaker-0 (00:16) Yeah, that’s totally false.
speaker-1 (00:17) That’s a whole nother discussion. It’s totally wrong. It’s like, you don’t have to practice your free throws. You’ll just like, you just know it, you know, just go play basketball. It’s like, what are you talking about? It’s so dumb.
speaker-1 (00:00) so variance in the quality of instruction. even if you go to an elite school, You’re going to get some people,
great research mathematicians, but horrible pedagogues. really it’s you and your group of study mates and that who kind of go, hey, let’s all work together on these impossible problem sets. you basically teach yourselves, It’s more like a
framework for making you teach yourself as opposed to a real scaffolded learning experience.
speaker-1 (00:00) for any of us who’ve been a math or physics major, and especially if you went to a place that we had a lot of top-notch students,
you’re learning stuff for the first time and they are going at a breakneck pace and they are not playing games and they don’t get retakes and there are no study sessions, you know, it’s just boom, here you go. And then the average score is a 27 on the midterm. It’s like, jeez, you know, it’s brutal. And then you find out that like a bunch of the kids had actually, oh yeah, I took this at the state university when I was in high school. And you’re just like, what?
What, you guys, wait, half you guys already taken this? This is bullshit. You know, it’s like, we’re in a Spanish class and you got like a bunch of the kids who actually speak Spanish at home. You’re like, why are you in Spanish one? Your Spanish is, I don’t really write it.
speaker-0 (00:49) It’s like you get gaslit into thinking you’re dumb and everyone’s just like learning so much faster than you and then the glass shatters you realize they already came in.
speaker-1 (00:59) Oh, you guys all got the cheat codes? Oh, great. Okay, okay, okay. Now I get it. I get it. But anyway, the cheat code really is you’re saying, what’s the cheat code? The cheat code is learn the material before you take the course.
speaker-0 (01:02) Yeah.
speaker-0 (00:00) The point of learning ahead of time is not to sit there bored in class. It’s so that you can actually like legitimately grapple with the hardest problems and actually extract learning out of those in an efficient way. You can be the go-to person for everyone needs help with the class. You’re making connections You’re
front running for any opportunities that the professor has in mind, whether it’s research with them, research with one of their buddies, whether it’s internship with a company they have a relationship with a fellowship, you just you never know ⁓ what it’s gonna be. But if you are the go to person for the subject knowledge, you just get pulled into all these interesting
advantageous opportunities that just compound one thing into another. Guess what? Got a great internship? Well, your next internship is probably going to be even cooler because now you have this experience that nobody else has.
speaker-1 (00:49) it’s snowball effect. The compounding effect.
speaker-1 (00:00) The secret to success in life is consistent effort. You don’t have to, whether it’s exercise or learning math or learning a language or whatever the heck it is you’re trying to do. It’s like you don’t have to do the superhuman effort thing. Just get started and then make a consistent push every day, even if it’s only 15, 20 minutes.
speaker-1 (00:00) cause people are like, ah, I don’t want to do so many problems. just want like a conceptual thing. I want an intuitive understanding. It’s like, Intuition is the
of repetition. You have to do reps. That’s where the intuition comes from. Me or anybody explaining something to you that feels intuitive. You don’t have the intuition. You have to earn intuition. Intuition is earned through pain.
through failure, through suffering, through trial and error. That’s where your intuition comes from. You don’t wanna suffer, well you don’t get intuition. you know, obviously you wanna limit the suffering, like let’s just do the things and go through repetitions and say, okay, trial, okay, okay, right, I made a mistake there, right, okay, right, I see how this works. You do enough of the reps, whether it’s shooting free throws or…
you know, or doing math problems or whatever, it’s like you have to, you have to get the reps in, you get the intuition. But then when you get the intuition, then you can really understand how this stuff works and you can actually solve challenging problems. Because until you have the intuition, it’s hard to really see your way through innovative solutions. There’s just nothing to work with.
speaker-0 (00:00) One thing that I have seen students, even adult students do a lot is look back at the reference too often, like thinking that it’s free to look back at the reference. When in reality, if you are trying to recall something from your head
and you look back at the reference instead of trying your best to lift that weight off of your, long-term memory. If you look back at the reference, you’re basically just letting the spotter lift the weight.
speaker-1 (00:25) Yeah, so you’re weightlifting and you’re like, I can’t get it and the guy behind just lifts it up for you like, okay. He didn’t really lift the weight.
speaker-0 (00:30) Well, yeah.
Sometimes people will ask for the spotter’s help before they even get to the point of trying really hard. It’s like just the moment that it stops becoming super easy, they’re like, okay, look back at the example. When in reality, that is the moment when you are getting the most bang for buck out of recalling the information.
speaker-1 (00:51) The struggle during the active recall process is when you are strengthening the memory.
speaker-0 (00:57) Like, don’t look back at the reference right away. Like I want you to try to pull this from memory.
Coach the student not to actually look back at the reference unless they absolutely need to. Because that is a way that sometimes students kind of shoot themselves in the foot. ⁓
speaker-1 (01:12) They
shortchange the learning process. They shortchange it, right?
speaker-0 (01:15) Yeah. Or if they have like the, worked example up in another tab and then they’re looking at that while solving the problem. Like, is, that is the worst. Never, never do that.
speaker-1 (00:00) You know, and I hear from teachers and chief academic officers and tutors, and they’re just apoplectic about the situation because you get kids who are in sixth, seventh, eighth, tenth grade. Finger counting.
don’t know the multiplication tables, it’s like, how the heck are gonna factor quadratic when you don’t know the multiplication tables? Can’t do it, not really. And then guess what? Now we can’t factor quadratic, now can’t do algebra.
speaker-0 (00:27) And if you do manage to grind through, ⁓ just figuring out these factors on the fly, it’s gonna take you way, way, way too long. It’s gonna take you like 10 minutes to factor.
speaker-1 (00:37) Which some kids can do in their head and be like,
10 seconds. So you’re not going to want to do any proms. It’s through, yeah, teacher gave us four factoring problems and it took me all night, you know? And you’re just like, dude.
speaker-0 (00:41) Yes.
When you get to calculus, if you’re taking 10 minutes to factor a quadratic, that’s only one component of a calculus optimization problem. You’re going to be spending half an hour on this problem that should take you two minutes.
]]>I know that sounds counterintuitive but hear me out.
The “aha” moment – the relief of “omg it finally makes sense now” when something finally clicks into place – indicates that you were missing prerequisite information.
You built up a ton of knowledge around this missing prerequisite, so much that it created extreme tension, and you felt all that cognitive pressure get released when you filled in the missing prerequisite.
The “aha” moment is a great feeling and it does represent important learning.
But it also indicates you spent a lot of time being confused.
If you had that missing prerequisite in place beforehand, you wouldn’t have spent hours or days ruminating, bothered by it, searching for it.
Everything would have felt somewhat obvious. Like, “we just put the prerequisites together, what’s the big deal.”
And the time you saved not being confused, you would put towards learning more advanced material.
A few final notes:
– Here, I’m talking about “aha” moments internal to the curriculum. If you start out with an inefficient curriculum, build a swiss-cheese knowledge base, and then switch to a more efficient curriculum that fills in your pre-existing gaps, then of course you’ll experience “aha” moments from that. But once the gaps are filled in, the “aha” moments will come much less frequently.
– Additionally, I’m talking about foundational knowledge. Knowledge that humanity as a whole knows, but that you don’t. Once you get to the true edge of a field, you don’t have an efficient knowledge structure to learn from, and you’re roughing it on your own with discovery learning. So of course you’re going to develop some swiss-cheese knowledge and experience “aha” moments from filling in your gaps. That’s how it goes with discovery learning. But the difference is that when you fill in one of these gaps, it will be a significant contribution to the field. That’s why actual experts/researchers will often talk so romantically about their own “aha” moments.
Discussed ~2:06:44 in this podcast.
]]>And that can feel like an equalizer.
Like you’re suddenly at the same level as the pros.
But it’s not an equalizer.
It’s a multiplier.
When you have a high level of mastery, these tools scale your expertise.
When you don’t… well, it does level you up, but not nearly as much as it levels up people who actually know what they’re doing.
The distance between you and them actually increases.
It takes you from a 1 to 10, it takes them from 10 to 100. The distance goes from 9 to 90.
Because multipliers don’t just multiply your level, they also multiply the distance between levels.
Discussed ~3:07:12 in this podcast.
]]>If you go to tennis lessons and the coach just talks at you for an hour, I don’t care what your attention span is – you’re not getting better at tennis.
It’s the same in math or any other academic subject.
Discussed ~2:58:02 in this podcast.
]]>And one of the most common mistakes that leads people to quit their compounding too early is expecting results to grow in direct proportion to input.
But that’s not how compound growth works. That’s the linear growth mindset.
And if you don’t understand the difference, you may never commit to something long enough to reach your potential.
We’re so used to linear conversions, where you multiply the input by a conversion rate to get the output.
But if you look beneath the surface, so much of life is about building systems – companies, relationships, etc.
And systems typically have some kind of self-sustaining nature to them, a kind of momentum where the more they grow, the easier it is to continue growing.
That’s compound growth.
A key mathematical feature of compound growth is that, at the beginning, it may look slow compared to linear growth, but it will catch up after a while and then far outperform linear growth by the end.
If you don’t understand this phenomenon then you will be constantly tempted to quit because you don’t think you’re making enough progress.
For instance, say you compound 1% growth, 100 times.
And you look around at people who you think have compounded 1000 times, and you think, “I should be getting 10% the results that they are, because I’m 10% of the way along in the journey – I’ve taken 100 steps compared to their 1000.”
And you haven’t even made it 1% of the way to where they are, so you assume something is wrong, for whatever reason you’re going too slow to ever make it to their level, and you quit.
But in reality, you could very well be on the same trajectory as them!
I know it sounds insane, but even if you are on the same trajectory as them, your results at 10% of the way through their journey may still feel negligible compared to theirs.
These are two data points on the same compound growth curve.
This is another reason why it’s so important to measure your growth by looking at how far you’ve come, instead of constantly looking at how far you still have to go to reach the most accomplished people you look up to.
]]>But what we found is: if you make sure students have all their prerequisites in place leading into a lesson, then they’re prepared for it as-is.
What you experience as a tutor, when you have to explain things a million different ways for a student to get it, is really that there’s some prerequisites missing, and you’re trying to rapid-fire cover the space to hit those missing things.
Which, I mean, if you don’t have a mastery learning system that guarantees prerequisites are in place, that’s not a bad heuristic.
Discussed ~1:08:38 in this podcast.
]]>The latest episode covered plenty of MA updates – not just status updates but also a glimpse into our agentic coding workflows.
Some timestamps:
More info about the big unlock that won me over to purely agentic coding: Most of My Coding is Now Agentic
]]>What changes is not the amount of work. It’s the amount of free time you have.
People often say “it’s too late for me,” but usually what that really means is: “I have less time now, more responsibilities now, and it’s harder to carve out the hours.”
That’s real. But the work itself is still doable if you can make time for it.
Discussed ~2:34:35 in this podcast.
]]>The transcript below is provided with the following caveats:
Justin Skycak: Sure. My name is Justin Skycak. I’m the chief quantum director of analytics at Math Academy. We’re an adaptive online math learning platform, elementary school through university. The goal is to cover all of math and math-adjacent stuff, and we take a full nerd approach to learning.
Just trying to optimize it every moment, make sure the student is working on the learning tasks that are the most efficient use of their time. We’re all about learning efficiency.
And what I’m working on right now, what we’re all working on, and has consumed my life for the past several months of nine years, is taking us from a workshop to a factory transition where we can really turn out a lot of amazing content at the same level of quality that we have been turning out for years.
In the podcast, we’ll get into a lot of stories about our own, probably your learning background, my learning background, how we found where we’re at right now. But nowadays it’s really startup early stage company, grind mode.
And that’s just my life, totally engulfed by this tornado. In a good way, it brings together a lot of stuff, but nevertheless, there’s not a whole lot of free time for learning just for fun anymore. It’s like, let’s go, let’s go, let’s go.
Justin Skycak: Well, there’s that saying, right, nothing succeeds like success, which, I mean, on the surface sounds kind of dumb, but really it’s whether it’s building a company, working on a project, learning math, learning anything.
The more that as you experience these small wins, you get a lot of motivation to continue doing it, and that motivation, at least for me, that’s been one of the best sources of motivation, just knowing that, hey, this is working and it can work even better if I keep doing it and do it more efficiently and just see what it looks like.
Justin Skycak: What was the main driver here? This was kind of a chance event. And it was kind of like, you know, I always liked math class growing up in middle school and it was my favorite class. But am I going to go learn math outside of class? That thought just never really occurred to me. I was not that much into math, but at some point in high school actually, the summer after 10th grade, I had just finished up pre-calculus and it was a summer. And I was actually getting kind of excited about learning calculus because calculus is like, you know, when you watch a sci-fi movie and there’s always a scientist on the board. There’s got to be the integral signs, the derivatives, I mean, just at a very surface level. It’s like, what’s advanced math for somebody who’s going in through high school or learning pre-calc or algebra? It’s calculus. That’s the first thing, at least for me, that came to mind.
And knowing that, I was just, you know, hey, I’m going to learn this next year. I just finished pre-calculus. Calculus is next up. We did a little bit of calculus at the end of pre-calculus and I was getting ready to go. I was really interested in it. I knew I was going to go into STEM in some form. And it was just like, I was going to learn ANSI. So I’m like, why don’t I, I don’t know how this is going to go, but why don’t I just try and teach it myself? What’s the way, I mean, I guess I’ll learn next year at school, but if I could just teach it to myself over the summer, that’d be awesome.
It was just this idea. Nobody really planted that in my head. It was just one of those things where I was just thinking one day, why don’t I just try and do this? But that ended up turning into this kind of chance event spark that led to the trajectory that I’m in now in my obsession with efficient learning.
And so how it ended up is I found this pretty well-sequenced calculus course. It was almost like a proto Math Academy type thing. It was in some online course collection, the National Repository of Online Courses. You’d go in and there’d be a couple, it was broken up into maybe 50 or 60 or more lessons, the H.P. calculus course. You go in, you read a little bit on some slide of information and then you solve some problems. And they were very concrete problems. You got a lot of reps and stuff. So it was a great course.
Unfortunately, it no longer exists. It was sold to somebody and revamped and whatever. So you can’t just go learn it from there anymore. But this ended up working really, really well. Actually way better than I thought it was going to go. I thought I was going to spend the whole summer just grinding calculus and maybe one hour a day and come out and have learned the first third or something. But actually, just like we were talking about, success breeds success, having this small win stack up, got through lesson one, got through lesson two, solving problems. Hey, this is not a big deal. This is actually, I like it. This is fun.
And it was actually so fun that I used to play more video games and so math took the place of video games. And it was just, it almost felt like progressing through levels in a video game. And so I just kind of leaned into it and I just pulled up in my room working on calculus problems for eight hours a day or something. So I was obsessed.
And so my parents are just like, my mom’s an art major in school. My dad, business major, just logistics for health care company. They don’t, and nobody in my family really looks technical or mathematical or anything. So it’s like talking with other parents and other like, you know, what’s your kid doing over the summer? Oh, he’s just in his room learning calculus. It’s like, oh, almost as if I was in the basement playing a video game. And nobody says like, oh, yeah, my kid’s in the basement playing a video game.
And I was in the air. The song is great. I was trying to tell you that to go out and the other stuff. What did, what did we really confused, honestly, it was because it’s like, I mean, it started out like, wow, my kid is learning, taking this education into his own hands and he’s prepping for the upcoming school year. That’s great. And then just like, where is he still, he’s still deserving. It’s just like working on this problem. It’s kind of weird.
And I mean, they had never learned this kind of stuff themselves and nobody had STEM careers or anything. But so it was one of those things that was like, well, it seems like it’s a good thing. But it also seems like he’s addicted and obsessed to it. But it also seems like a good thing. So what do you do? They didn’t really intervene or anything, which I think ended up being a good thing.
I was very, very unbalanced that summer.
Justin Skycak: The 3000 hours. After that summer, I spent maybe 30 days, eight hours a day, just grinding this calculus course. And by the end of this, I was just so sucked in. I want math. Give me more math. What’s next? I can’t, it’s like, it just felt like I had found this hack or something. Nobody told me, you can learn outside of class. And I’m like, okay, let’s do this. This is working out amazingly well. I’m having fun. This is going to be great for the future. I don’t really know exactly what it’s going to do, but it seems like it’s going to do good things.
And I was like, okay, let’s keep this going. Let’s learn what’s after calculus. I finished one month into the summer and I’m like, okay, I don’t know what’s after calculus. I need to look up online. What course comes after calculus? What’s math above that? Is there math above that? I’m sure there’s math above that. I don’t know what it is.
I stumbled into MIT OpenCourseWare because that has a nice, you know, sequencing of courses. It’s like, well, what comes after calculus? We’ll just look at the MIT undergrad math major. That kind of tells you what to do. And so that’s when I started learning linear algebra and multivariable calculus. And I would just work through the, I mean, it was much less structured than this original AP calculus course that I had found. Because MIT OpenCourseWare, at the time, that was really just a collection of lecture videos and some problem sets.
Now, I know they’ve built out some of their foundational courses a little more since then, augmented it with more learning materials, but at that time, it was very, very rough writing. Here’s 20 lectures and here’s all the problem sets. Go. And so I just did that and I found some PDFs of the textbooks and I was just doing that.
I did that the rest of summer into the school year when I got to school. That following, like, 11th grade, this whole self-study experience was making me realize how much time I was wasting in school, just kind of sitting there in class. And so I was like, well, I’m not going to go back to that. I’m going to keep leaning into this thing. So I would take the math books with me to all my classes and just be doing that instead, while trying to look like I’m paying attention and stuff. Not going to confiscate stuff like that.
But yes, I kept that going the following year. And, you know, this 3000 hours thing, it was way more time than I really needed if I were working efficiently on learning this stuff. But I was, I mean, as most teenagers, I think I know everything. I know how to learn. I did not do my research in how to learn efficiently. I’m just like, okay, I’m going to go do it. And so I ended up hitting my head on basically every sort of ledge there was to hit my head on along the way.
I was kind of just discovering a lot of these principles. In hindsight, really obvious principles of learning science. But I remember there was a day or so that I initially came into it thinking, why am I working all these problems? I can just watch all the videos and learn it that way. And then I took a day and I just blew through what I thought was a third of the course watching videos. I’m like, this is such a hack. I don’t have to do the problems. I can just do the videos.
And then the following day, I just tried to work out some problems and realized I don’t retain anything from it. And so that was just the learning of like, okay, I have to do problems. And so every, basically every single cognitive science principle, I violated an issue during my self-study and kind of learned from that.
Justin Skycak: You know, I mean, on one hand, I think there was a little bit of value to having tried some inefficient forms of learning and only to realize, hey, this doesn’t work. And I think in any discipline that you are seriously going into, that’s going to be your main thing that you’re doing. It’s worth exploring the space of, you know, try things out, what works, what doesn’t.
But I just think about that spending 3,000 hours to get from calculus all the way up there. I think I got through about real analysis, abstract algebra, you know, junior level math major courses and also some physics and stuff. If I had, you know, there’s just a lot of time that I’d waste just kind of being confused, doing inefficient techniques that did not contribute to this holistic view of learning.
A little taste of, you know, did it wrong. Okay, correct to this. There’s maybe some small amount, maybe a couple percent of the time, I would say was kind of useful in exploring the space. But that came to a point where I was really like, I just want to go fast and learn this stuff. And it was trudging through mud. And the higher I got, the more things would come back to bite me, like not doing a proper review system and just working on problems that were way too hard.
A lot of this stuff, I mean, some stuff I figured out on the fly, but a lot of it, I really did not learn until years later how to do this really, really efficiently. And if there’s one thing that I wish I had more during my teenage years and just as a student, it would have been guidance on just how, just tell me, just tell me how to do it.
I just want to learn math and I’m fighting my way through getting my head on all these ledges. Maybe some amount, a small amount of that is kind of shaping my view of understanding what works and what doesn’t. But most, the vast majority of that time, for me at least, felt like it had just gone to waste.
And there was some instances of, I also, part of my goal in this was, I want to just learn the core body of math so I can contribute to do some cool research projects. And there came a point around maybe 2000 hours in. I’m just like, you know, I think I’ve learned enough and I’m going to do research. And what was this based on? This was just based on the sheer volume of time that went into learning, not based on how far I had gone.
And so I ended up doing a research project, which was just kind of on my own. It had to do with finding a generalized formula for partial for actions to become. And I spent a whole summer on it. And I used, it was kind of fun. But what ended up happening is I was kind of groping around in the dark in a direction that I would later find out was just made completely trivial by some results in complex analysis.
I mean, it’s a high-level undergrad thing, but it’s not exactly super advanced math. And so it’s like, well, if I just. You’re trying to be discovered stuff that we’re already known.
Justin Skycak: Cognitive science actually was not really on my radar until, well, I guess after high school, sort of at the beginning of college, I was sort of interested in neuroscience, computational neuroscience, and that led to kind of interest in AI and stuff. And that, I mean, it was kind of cognitive science adjacent.
But what really flipped the switch for me was once I was responsible for other people’s work. Starting out, I did a lot of tutoring in high school also and I really liked tutoring and just math education in general. At one point, actually during college, I had a full-time data science job and after graduating college, I was like, you know, screw it. We’re not doing data science anymore. We’re doing math education and we’re going to find some way to incorporate all of the coding and whatever that I had enjoyed into math education.
But it was really once I had a classroom of students who were kind of depending on me to set up some kind of structure that is going to result in them learning this material. That’s when I realized a lot of things that were, there’s some of the less obvious stuff, like the importance of spaced review and interleaving and stuff.
And that’s about the time that I also met Jason Roberts, founder of Math Academy. He and his wife Sandy founded it together. And I was teaching in the original school program there. He had done a lot of, you know, just reading around the subject of cognitive science. It was incorporating those principles.
And I was like, oh, wow, this is really interesting. And for once, it’s like, okay, this is the guidance that I was saying that I didn’t have. I was like, I was groping around, discovered maybe a third of it on my own. It was like two thirds at least that was just like, oh my goodness, that makes total sense. I go like, okay, do that, do that, do that.
And it was working in the classroom too. The more that we would leverage these cognitive science principles, the better the students would learn the material. And then at some point after that, actually around maybe three years ago or so, Jason was like, hey, we need to write up about how the science of Math Academy works. Can you just make a little information for the web page? And I started on that and looked up about 400 pages later that we had the Math Academy, what it was big book and how we apply the cognitive science learning, for instance.
Justin Skycak: Oh, I was just gonna say, I totally agree with, I’ve had that same experience with, you know, high-level professors. You sign up for the course. You’re like, wow, this professor’s really good at research. I’m going to learn so much.
And then you get in there and then you get served up these slam dunk problems that are supposed to, the professor thinks, oh, I’m going to create these really just a few really nice, really interesting problems and give them to the students. And you get those problems. You’re like, what is this? I don’t even know where to begin. And you just are flailing around and the professor is, you know, not really helping bridge the gap from you to them. And I’ve seen this too in a lot of students I’ve tutored as well. It’s a lot of engineering.
Justin Skycak: We’re just kind of accepted that the class moves in lockstep and there’s a normal distribution of grades at the end. And some of these grades are failing. Some of them are, just barely understands the material. A few grades are, okay, they get it. We’re just going to accept this system. But really, if you have every student working on what is the most efficient use of this.
Justin Skycak: I’ll talk a little bit about that. The Fermi Lab and CERN projects, these were projects that I did with local universities. Fermi Lab was with Indiana University South Bend, where I grew up, in the professor Ulan Levine’s lab. And then CERN Project was with a lab at Notre Dame. That’s a Core Connect program which connected some of the top high school students in the areas with working on projects that were going on with some labs that are at Notre Dame.
And the math, it was kind of fun. It helped in a lot of big ways, get my foot in the door in terms of these projects because, you know, you learn a bunch of the advanced math. You stand out. In my high school, I was like, okay, who are we going to pick? They could pick one student to send to this Core Connect program at Notre Dame. And they’re like, well, who are we going to pick? Well, Justin is just crushing the calculus course as a grade earlier or whatever. I guess, and he seems to be learning physics. I don’t know, I don’t know, send him.
That was just an obvious decision. Because that’s what happens when you get really, you know, you find your obsession. You get really ahead. People take notice and you get a lot of opportunities to pursue that further.
And with the other project, my junior year, I was working on the Fermilab project. And that was actually just, I was taking this research class in my high school where the point was to reach out to local professors and just try to get your foot in the door working on a research project with them. And part of the thing that made me seem like, hey, this kid is going to actually be able to help contribute a little and not just be a total waste of time was the fact that I already have my math foundations in place for all this stuff. And I’d taken it from myself to learn.
And when I got to these, you know, started working with these labs, math was just a nice thing. Math was just a non-issue. I remember for so many other students in my high school research class, they would go off to their labs as well during the school day. And just at the end of the school day, for an hour is the last period of the school. You go out and leave early.
A lot of other people would be struggling with some of the math involved sometimes. You know, maybe somebody would get into a physics lab and they would not only have to spin off on the context that the lab is operating and what research are we doing? What are the materials? But there’s just a big mathematical gap that was placing hard limits on this type of stuff that they could do.
Now I didn’t have that gap. And that was good. Though I did have some gaps in other areas. I mean, most of my projects had to do with experimental physics. There was, you know, math was one side of it. But there was also, you know, the Fermilab project, I was trying to make a material to better transmit sound. And I had never, I was not a very hands-on lab kind of person. That was definitely a point of, oh man, this is challenging.
But in terms of, you know, math just became a not problem. And I think to anyone who wants to do research, especially high school research, removing skill gaps is such a big advantage there. Because you can, you know, just imagine, say there’s some lab working in some setting, some research setting. If you have all your foundational skills in place, then you can just take off running and be a serious contributing member of lab.
Every little pillar of foundational skills that you don’t have in place is going to slow you down. I would say in my case, okay, the math was a pillar that I just knocked out of a park. That was a non-issue. Some of the more hands-on experimental physics stuff was definitely more of a drag factor for me because I never trained up, you know, that kind of hands-on.
Justin Skycak: You know, I could have leaned into this a lot more. I came in wanting to do theoretical physics. Okay. I was like, I don’t want to do experimental physics. I want to do physics, and I mean, you’re a high schooler. You’re like, hey, can I get involved in this physics stuff? Then they’re like, yeah, you’ll take whatever. You’ll take whatever this project.
And I’m like, I’m not about to turn down the experimental physics opportunity. And it’s like, no, I want to do theoretical, that or nothing. Get out of here, kid. What are you talking about?
I was a little, you know, doing these, I had not quite found my really good fit with these physics projects. It was a halfway good fit, right? That math pillar, okay, that was good. But the experiments, I think this is a pillar. I was just, you know, I wasn’t really interested in that kind of stuff. And I did not take it upon myself to learn this stuff outside of lab.
And it was kind of like, I just did one of the mill projects there. And this was enough to, I ended up making junior year to the Intel International Science and Engineering. The project was good enough to get to that level, but it was not good enough to get place, get a medal or result in a paper or stuff like that. Or even kind of really set a direction for me to, I want to continue looking into this more.
Justin Skycak: That’s interesting. I actually have a question for you on that.
You joined this molecular biology lab and you mentioned you had strong math and physics foundations. When you were joining the lab, were you leveraging the math and physics in your stuff?
Absolutely not. Or okay, it was just very zero biology methodology. Okay. That’s interesting.
Justin Skycak: That was fun. That’s you going and test during those people with all your questions, which I guess initially were a little bit novice questions. But then you build up this storm master that really takes, you know, put ego aside and I’m going to ask the dumb questions because I have to learn this stuff because you don’t get results.
Justin Skycak: It’s very real, you know, once he has something of value there.
Justin Skycak: Starts out with the diagnostic test. Figure out what you know, what you don’t know in whatever course you’re trying to take. It’s going to look back for missing prerequisites many years back into NASA. If you’re placing into calculus, it’s going to test you on a whole bunch of algebra. You don’t know completing the square? Well, guess what? You’re going to learn completing the square. I’m going to add that to your learning plan.
Makes this custom course for you. Now you got a custom course that you’re going to go through. This is kind of your knowledge graph, and we know what you know, what you’re ready to learn, and what you don’t know. And we’re going to serve you stuff that you are ready to learn. This is going to be under your knowledge frontier. We call it your edge of mastery. And we’re going to build up your understanding of these topics.
Now what does a single learning experience on a topic look like? Well, this is a lesson and it’s kind of a sequence of minimum effective doses of instruction and you actually solving problems. Think this whole situation is very similar to if you were to go with a personal trainer. What are they going to do? Well, they’re going to first just see, okay, what are you trying to do? What sport are you prepping for? Or are you trying to increase your vertical jump? What are you trying to do? Where are your weak points and what can we do to get you, you know, better at those?
They’re going to have exercises for you and each exercise they’re going to, you know, I’ll demonstrate it once. We’ll talk, this is what it is. Here’s how you do it. Okay, now you practice. And that’s what a lesson is like. It starts out with just a brief introduction. Then we immediately launch into problem solving. You see, okay. You got your introduction. This is the type of problem that we’re going to focus on solving. Here’s an example of how you can solve the type of problem. Okay, now you do a few problems.
If you’re just knocking it out of the park, you answer the first two questions in a row, it’s like, wait, you got it. Let’s move on to the next more challenging version. If you’re struggling a little bit, you’re like, oh, first question wrong, next one right, we’re going to make you solve a few more problems just to, you know, you got to end on two successes in a row before advancing.
Justin Skycak: Was this inspired by that sort of, this is born out of our hands-on experience teaching. And just, you know, the tutor is really, I’m glad you brought that up because the tutor is our model organism basically. We’re trying to emulate the decisions of an expert tutor who knows everything about your knowledge profile, what you know, what you don’t know, and has unlimited computational bandwidth to have you, you know, it can just make up problems, but it’s got a whole inventory of problems that can match you with. And every single answer that you do, it’s going to kind of take that into consideration to get you working on
Justin Skycak: That stuff, that whole gap. We started this stuff, I mean, it’s pretty little arms, right. There’s just no way to generate any of this information. We start the stuff back in 2016, I think-ish, is when Jason and our director of curriculum, Alex Smith, started working on this thing. I joined around in 2018. And a lot of stuff, we’re hand crafting it manually.
And to this day, everything is very, you know, it comes from a human expert just paying very close attention to, based on their experiences teaching and what they know about max efficiency learning and how to get students through material in a way that’s guaranteed to work. How do you do it?
And we built a lot of tooling over the years to make this process more and more efficient. We’re leveraging all the tooling that we can. But ultimately, it comes from a human domain expert is making sure that all of this is to their standards and this is all mapped out by them.
This is Alex Smith, our director of curriculum. He builds the forwards knowledge graph of, okay, what are the topics that we want to have students learn? What is this course, this calculus course? What are all the topics in it? We got about 300-something topics in our calculus course. Each one is an atomic unit of knowledge consisting of roughly three or four knowledge points.
It starts out with the simplest version of the problem and then layers on additional complexity as you evidence your ability to solve problems in each successive knowledge point. And he maps all this out. He connects everything up, the prerequisites. What topics do you need to have learned before we serve you this new thing?
And I kind of encode this backwards graph we call the encompassing graph. And it says what skills encompass what other skills, what sub-skills are you exercising as components of more advanced skills. And this helps with our spaced repetition algorithm, which is always, you know, when you do a review on an advanced skill, you’re kind of implicitly practicing a lot of skills.
We’re always trying to, you know, something that we do is we’re always trying to maximize your learning efficiency. If we can get you practicing your sub-skills that are, you kind of do for some review, we can get you practicing those by learning something new. Learn a new lesson that knocks us out, then the algorithm serves.
Justin Skycak: Exactly. And something about just the whole, you know, approaching this as, why not just generate this, just try to generate all the content with an LLM or something?
Well, the thing is, a lot of this content, the reason why it works so well is because it’s based on all of our learnings throughout the years of doing this manually. And while LLMs are, I mean, they are really good at traversing this whatever knowledge that they’ve been trained on, right? Everything that’s been written down on the internet, and often applying sequencing of that knowledge to kind of expand beyond the initial knowledge base, there’s a lot of stuff that you learn by just working with students hands-on and just kind of almost extracting data from reality itself. Not just a compressed version of, this is reality according to all the text that’s been written on the internet. There’s a lot of stuff that’s kind of off the
Justin Skycak: I mean, that’s one of the things that, you know, the design of minimum effective doses of instruction and practice. I mean, part of this is just inspired by, you know, cognitive science’s active practice. Let’s maximize the amount of time that you’re actively solving problems with guidance.
But another part is also just inspired by Jason Roberts, founder. He always talks about how, when he was sitting in math class, it’s not that he hated math, it’s that he hated being talked at for an hour. And so this, so many, we’ve realized that so many students, especially students with ADHD or just sort of really can’t sit still type of students, they don’t want to watch a video for an hour. They don’t want to watch an hour, hour video.
And if they do, it’s often because they just want to stare off into space and not think about it. What really keeps the students engaged with the process is actively solving problems that are at the right level of difficulty, kind of entering that flow state, right at the edge of their ability.
But I want to say, in addition to just designing the lessons internally to try to keep the experience maybe a little challenging but very achievable for students, the other part is there’s also the whole connectivity of the knowledge graph. This is stuff that you probably would not think about if you’re just trying to maybe get an LLM to dump out a knowledge graph or just ask somebody else to design a knowledge graph.
There’s a lot that comes into play in terms of structuring the knowledge graph to make students successful. And we have to think about things like quantifying the cognitive congestion of various nodes in the knowledge graph. If you have a topic that has 10 different prerequisites that are all being pulled in that you’ve never practiced pulling in together before, that’s going to inflate the cognitive load a ton.
And there’s just a million of these little optimizations that we put into to make the knowledge graphs really easy to learn from.
Justin Skycak: Exactly. It’s kind of, I think that’s a good analogy. If you take a textbook, and let’s say that this textbook was constructed not just by one author just doing a brain dump in isolation, but an author and an expert on the subject who’s also looking at all the other textbooks, all the other treatments of the subject, trying to just hit all the bases, a comprehensive course.
And they write this textbook. And then, instead of presenting it to the student in this linear sequence, just, here is the unit on, I’ll just use algebra as an example. Here’s the unit on linear equations. Okay, now let’s move up to quadratic equations. Okay, now let’s do some trigonometry.
Instead of presenting by units, you really chop it up into the atomic bits of knowledge and that way you can really trace out the student knowledge as a contiguous area in the knowledge graph. Exactly what they’re ready to learn. You can deliver the next piece of knowledge. You can kind of adapt the number of problems to the student.
And you’re also, you keep it interesting, right? You don’t have to just do one unit for an entire week or two weeks. You know, at school, I often, this is, this quarter, we’re doing trigonometry. Nothing but trigonometry for three months.
Justin Skycak: Three months. I know. You got sick of it. In addition to just getting sick of it, that’s not even an efficient way to learn the material. Ideally you want to be spacing it out, coming back, revisiting it. You don’t want to be blocking or having your practice on the consecutive material all the time. Interleaves are all the different units.
Justin Skycak: It’s an impossible task for one. That’s a good, because 700 students, how are you gonna, you can’t, one teacher cannot deliver an optimal learning experience to some hundred students simultaneously.
Justin Skycak: Well, I think it’s an area that is, it’s not a radar worth paying attention to. At this moment, there’s a lot of challenges with that sort of set. It’s one of those things that’s like, I mean, there’s a pull, right? You want to imagine, you think of efficient learning, you think of this personalization, you often imagine some tutor having a personalized conversation with the student around their interests, relating the material to their interests and stuff like that.
Now, that can potentially work really well in some cases. There’s a lot of surface area for things to go off the rails. And just to name a few cases. Sometimes students want to kind of get out of scope of a topic. You know, you’re teaching one thing and they’re like, oh, what did you do this? What did you do that? Oh, blah, blah, blah, blah.
And important to this can enrich the learning experience, but there also comes time when it’s like, you know, kid, I know you’re excited about this stuff, but we’re just talking in the abstract right now. I used to, back when I was teaching, I would have some students sometimes who would ask me tons of questions about, we’d be learning introductory calculus stuff. And then they start asking me questions about, what’s the hardest integral to solve? Or are all integrals solvable? How do you know or how do you not know?
And pretty quickly you get into a realm where you’re no longer really doing skills-based teaching. You’re just kind of talking around the subject, almost as if they were watching a YouTube video, some visualization and what’s the hardest math problem or that kind of stuff. And it can be, there’s some moment of, well, okay, you got to kind of scope it down to know, there’s a balance to be had, and you don’t want to allow the scope of what you’re doing to delete too much.
Another part is that, in addition to trying, making it non-gameable and stuff like that, there’s a challenge in that generating content on the fly is always, I mean, at all arms, there’s stochastic generators, right? And you don’t, I mean, you can, it’s gotten a lot better at all the hallucinations and stuff is definitely an order of magnitude improvement in that. If you don’t have static content underlying, it’s a little harder to, here, I dig the gamble.
And additionally, just to name one other thing. One thing we do a lot with our knowledge graph is, that’s an advantage of having this kind of static content, is analytics. And every student is being served the same exact step-by-step lesson that we have been optimizing for years to be the most, assuming the students have mastered the prerequisite leading into that lesson. This is the most efficient, most robust way of getting a student to learn the material on that topic.
And a lot of these optimizations, maybe we had three knowledge points and knowledge point number two was actually a little too aggressive and instead of 98% of the students getting through it on the first try, it’s only 80% or 70%. And then we break that up into two steps and we do all this content has gone through so much refinement, analytics refinement.
And once you kind of get rid of this idea of static content, it makes it a lot more difficult to do analytics. Not to say that this could never work, just to say there’s still lots of challenges.
Justin Skycak: I mean, it’s definitely something that I think will remain on our radar. It’s like, you know, the idea of, I mean, it’d be cool to, I mean, we talked about the idea of, you know, some people, when content is presented in a way, making analogies, or using word problems in a setting that they’re just really interested in. Like, oh, they’re interested in soccer. Okay, let’s make this word problem. We’re doing all our word problems about soccer. Let’s just make a fun, that can definitely elevate the learning experience.
There’s other elements, just to say, not that you’ve suggested this, but a common confusion, though, on this point is some people think that students need a million different explanations of a topic until one clicks. And that’s what I’m used to hearing in terms of suggestions for using other lines of the content.
But what we found is that if you just make sure students have all of their prerequisites in place, leading into the lesson that you’re asking them to learn, then they’re just prepared for it. And oftentimes, what you experience as a tutor, when you have to explain things a million different ways for a student to get it, is really, there’s something missing and you’re just trying to cover the space. What is this missing thing? I don’t know. I mean, I’m just rapid fire to try to hit it, which, if you don’t have a mastery learning system that is guaranteed for you, because it’s simply in place, that’s not a bad heuristic.
And we are just on the topic of framing things in ways that are really exciting to students. One of the things that we actually have coming out in one of our upcoming courses is projects that are just based in really interesting scenarios, at least that we think are very interesting. It was a little interesting to people who are interested in astrophysics, modeling, planning out civilization on Mars, translating writings from long lost civilizations, stuff like that.
We got a lot of cool projects like that. But the most, where this path leads, is kind of matching, okay, what kind of projects are you really interested in, to the student?
Justin Skycak: Imagine that you wanna learn math. You know you gotta do some form of spaced repetition because the spacing effect is one of the few free lunches you get in learning. If you space out the material and serve it at the right time, you can kind of minimize the amount of review that you have to do while maintaining a certain level of retention.
How do you do that? Well, option one, this is a thought process that kind of leads to this algorithm. Maybe you say, okay, I’m doing all these math problems and I gotta make up Anki formulas for them, or Anki flashcards. And the first thing I think about is, well, what do I put on my flashcard?
On one hand, there’s some formulas and stuff in math, but so much of math is, if you just memorize the formulas, that doesn’t mean you know how to apply them or how to solve problems with them, right? It’s like, well, how do you do Anki for actual problem solving?
Well, okay, the solution to that is just put problems on flashcards and just have a system that, instead of serving you the same problem, it just replaces it with a slightly different problem. And those are kind of your repetitions that you’re doing.
Now what you run into pretty quickly in the setting is this thing, the review hell basically. You already get this with flashcards, right? The ones that they can go through in two seconds, three seconds, they pile up. You got so many flashcards to do. And even though you can get through them so quickly, now you’ve just made this mathematical problem solving, right? Now it takes you a minute and two minutes to go through each flashcard. And now you’ve got this massive backlog of review to do.
What do you do? You start thinking about it. Okay, say you’re in this situation, you got this massive backlog, right? And you’ve got to practice solving a linear equation. You’ve got to practice solving a quadratic equation. You’ve got to practice fact. Wait a second. If I solve this quadratic by factoring it into linear pieces and then solving each equation arising from the linear piece, I have effectively practiced solving linear equations.
If I solve a linear equation, 2x plus three equals seven, I am practicing subtracting and dividing or that kind of stuff. You’re like, wait, I don’t have to do all these cards. If I do this card, if you just take a card at random, okay, I did this card and I can remove these other cards from the pile because I’ve effectively reviewed those as sub-skills.
That starts to get into compressing your reviews, because the next thing that you want to do after that is like, wait, I don’t want to just draw cards at random from the pile that I have to do. I don’t even want to necessarily draw the card that is most overdue. I want to draw the card that hits the biggest number of due repetitions with the one problem. What’s the one golden problem that is going to make the biggest dent in my review pile?
And that’s where you kind of get into the knowledge graph. FIRE, it was an acronym for fractional implicit repetition. The repetitions, you’re getting implicit repetitions on sub-skills that you’re practicing as a component of each more advanced skill. And oftentimes these sub-skills are fractional in the sense that, well, you solved this problem, it encompassed a hundred percent of the cases of the sub-skill.
Maybe quadratic equation, you factor it, you solve the linear components. Okay, you’ve handled the Ax plus B equals C equations. Or maybe it’s a quadratic equation, x squared plus, leading coefficient of one. You’ve handled the linear equations that are like x plus two equals three, stuff like that. But what about when there’s a coefficient on the x?
Maybe there’s some cases that you don’t cover, so you kind of track what skills are, what fraction of the sub-skills are being encompassed. And then as you do each review problem, these repetitions, they propagate through the graph and trickle down. Almost like you can visualize it as strikes of lightning, trying to just, this thing covers a bunch of stuff below.
Justin Skycak: That’s one of the secret ingredients in this stew of maximum efficiency learning. I mean, each of these things, you know, make sure the student has prerequisites in place before you ask them to learn the new thing. Optimize the new thing into a sequence of steps that are very bite-sized for the student to learn.
Move the student, make this minimum effective doses of guided instruction and problem solving. Don’t make them do 30 problems of the same thing when you know, okay, they just nailed the first two, ready to move on. But at the same time, give them more practice. That’s the spaced repetition, periodic quizzes. All of this kind of combines together. Each of these is a multiplier on the learning efficiency.
Justin Skycak: Well, I think, absolutely, there are knowledge graphs for any subject that is hierarchical. Computer science, physics, anything math-adjacent, for sure is a really clear candidate for a knowledge graph.
Now, this is actually on our, you know, we just released a mathematical methods for physics course. And we have a machine learning course that is almost ready. We are actively expanding out our knowledge graph beyond math proper.
Now, how far can we expand it, how into biology or history? I mean, some just get flatter and flatter the further you move from math. But my gut says you can take this probably area, things that transfer over to some degree.
I think it’s kind of like in math, the hierarchy is kind of strict, whereas you have to learn linear equations before you learn to solve a quadratic equation by factoring. There’s just no, you have to, it’s a practice sub-skill, there’s no way around it.
And I think in other subjects like history, there’s sometimes not necessarily one right answer to, you have to learn this before that, but there may be ways that work out nicely. I don’t know, I have not really thought a whole lot about beyond hierarchical subjects too much, but I know if a subject is hierarchical, you can teach it with a knowledge graph.
Justin Skycak: You had mentioned just the experience of learning biology and archaeology and the different experience. Last fall, I started kind of dipping my toe into learning biology, trying to leverage these cognitive science principles, mastery learning, this kind of stuff. And just kind of saying, what does that even look like?
And I don’t know that I figured out the most efficient way just yet, but I guess I can tell you a little bit about what I kind of stumbled into. And I’d be curious to know how that matches or does not match with your own experience and what’s worked for you.
With biology, I took a similar approach. My wife is doing a PhD in virology, and a lot of that kind of comes down to genetics. And I wanted to be able to talk to her about that kind of work. And that just have it be a tutoring session, which is like, oh, it isn’t Justin, this is how genetics 101. No, I want to actually be able to talk at an intelligent level in the conversation and understand her work on a higher level.
A lot of this was just, you know, it kind of came down to understanding, identified this model of, okay, I need to understand what exactly is a cell and what’s going on inside of it that allows it to function in the body. A lot of this, like you said, was just compressing. There’s just a lot of stuff, a lot of shit going on inside the cell, right?
It’s like, how do you, on all these things that the cell is interacting with other cells or manufacturing proteins or whatever, a lot of it comes down to being automatic, understanding this is about being automatic in the process of it. What’s doing what, who’s doing what, where.
And I kind of turned this into a problems style. Just minimum effective dose of this part of the process of the cell and then, okay, ask me questions about this. What happens if this happens in the ribosome while it’s producing a protein? What ends up happening? Does the protein still get produced or is there a failure? Things like that. A lot of these kind of, almost the equivalent of mathematical problems, questions.
And that ended up working pretty well and got, now it worked pretty well when I was doing it and then I kind of fell off the wagon and have not done my spaced repetition. So I’m very rusty on it now.
But at my peak, I was actually able to have a lot of conversations with my wife about how the genetic variants of whatever are causing things to happen, viruses infecting the cell in this way, launching this kind of attack. Oh, this screws up that mechanism inside the cell. That’s why it’s like that. That’s why this happens.
The one thing that I was not doing though, I’d be interested to hear what you think on this, is I was learning this from an all along and I was asking it to, it started initially trying to ask me, what are your interests? Tell me more about this and that. And I was just like, my interest is in learning this stuff as fast and efficiently as possible. I don’t care.
It was like, because it was trying to, I knew what it was doing. This is what you kind of default to sometimes as a tutor. Oh, what are you interested in? Why don’t we make this relevant? And I’m like, I already have it relevant. I want to be able to talk to my wife about this stuff at a high level. Let’s go. Just tell me stuff.
I want this to just burn it into my brain. I’m not going to become a biologist. I don’t need to know all the little things of, I don’t want to explore the space. Just tell me how it goes, what works. And I want to capture this automaticity as quickly as possible.
And so I was periodically having to tell it, don’t make, I don’t want to hear, and don’t ask me, and don’t offer analogies unless I specifically ask for it. Because it’s kind of slowing down the experience.
Justin Skycak: Through information at a lower level of scale. But it kind of broke your abstraction almost.
Justin Skycak: Definitely. It makes, one of the nice things about developing math curriculum and mostly carries over to computer science and physics is that things don’t really change. In math, it’s just evergreen. There is new stuff that you find that builds on top in directions I hadn’t gone before, but on the foundations, generally it’s locked in place. But I guess, right, as you kind of go out into the fuzzier subjects, there’s these complete overhauls of mental models of what’s happening.
And I remember I was
Justin Skycak: You know, one of my favorite, I think one of the most quoteable figures in the study of talent development, John Wooden, legendary basketball coach. He has so many quotes about sports that apply perfectly to learning in general.
And just any sort of skills-based learning, at least mathematics, physics, coding, where you actually, you know, you’re having to perform manipulations on whether it’s objects of information or physical objects or whatever. And you’re performing these actions, and you’re having to build up to more and more and more complex actions. This all kind of follows similar suit.
Words like, you know, all these cognitive science principles of mastery learning, get your prerequisites actions in place before you do. Well, a lot of these are kind of, they’re actually more obvious in sports, I think. I often like to go to figure skating for sports analogies, because it’s just so dependent on underlying skills.
Before we get you doing skating backwards, let’s have you skating forwards. Before we get you skating on one leg, let’s get you standing on one leg. That’s mastery learning. It’s also spaced repetition. Just because you were able to stand on one leg last week does not mean that you still know how to do it. We have to continue practicing this.
Things like layering more advanced skills to lock in the fundamental skills. It’s like, well, you’re good at standing on one leg, bouncing on one leg on your skate. Your ability to have that has saturated because you’ve been practicing in an easy environment. But once you can, if somebody tries to push you a little bit, or you’re trying to spin around on one leg, this is going to force you to develop robustness in the underlying skills.
Same thing happens in math. In calculus, there’s a joke that calculus is where students actually learn algebra because they are held to account for applying all the algebraic techniques.
And there’s interleaving. Instead of, you know, if you’re going to do a skating practice, let’s not just practice stopping with your right leg the whole time. Let’s also practice stopping with your left leg. And not just do stopping, let’s do a whole bunch of other techniques.
There’s a kind of retrieval practice, even in sports, I think. It’s like, well, you think of it in learning, it’s like on a test or quiz, you’re retrieving something from memory with no reference. You’re not getting a warm start to it by solving some easier problems leading up to it. You’re just right on the dot, under time constraints, you got to pull it out of your brain.
Well, in sports, similar thing. You got to develop a level of, you know, maybe you can do a spin on your figure skate if you practice a few times beforehand, and then you can do the spin as you get your balance. But really, you want to get to the point where you don’t need that practice leading up, stuff like that. I think this all carries over really, really well.
Justin Skycak: Exactly. If you’re playing hockey and basketball, basketball veteran, now it’s because if you’re having to think about how to dribble or how to AOB spin, you can’t, you’re not going to be seeing all your teammates around.
Justin Skycak: Absolutely. I picked that. We touched on this last time, talking about how in college, you can get a professor who’s just widely recognized in their field and you get really excited and you’re like, oh my God, I’m going to learn so much. And then they come and they’re just like proof there and proof on the chalkboard. You don’t even know what they look like because they don’t learn to face it. It’s on the board.
And they don’t really grade the quizzes or there are no quizzes. Homework sets are graded. Did you do the homework? Okay, good. Good enough. Let me get back to my research. Don’t bother me, kid.
That could totally, I think it totally holds. And I think a lot of times the people who, I mean, we’re talking kind of about in generalizations here, but often the people who rise to the top of the discipline, they’ve often gotten a lot of stuff for free. A lot of cognitive machinery or just life situation or whatever. There are things that they often did not have to figure out a solution for.
You can think of a mathematician who just, you know, I never really had challenges writing my work down. It always came naturally. What do you mean, you struggled to write your work down? You just do it. Or stuff like, you know, I never, these concepts, what do you mean, you can’t just read a textbook and then it clicks for you? And then you do the probably just read the chapter and do the problems. But what do you mean the problem is hard?
A lot of these people can take bigger leaps in generalization of things like that. Some got a GPU in their head, right? Instead of a CPU, they’re just running on some different machinery. Sometimes not necessarily different machinery, but just more optimized in different ways. Individual variation and working memory capacity and your attention rates, this. Unfortunately, this is a reality of learning that people are kind of built with different specs like this.
And a lot of learning how to teach effectively comes from, I think, having to really push the limits of your own specs and make you kind of punch above your weight in terms of where you would otherwise be. Anyway, I would totally agree.
Justin Skycak: Well, I actually have two kind of funny, concrete stories of watching this play out. One is very brief, and it’s just, you know, I saw a differential equations class, intro differential equations class, being like, the guy wanted to teach the differential equation, introduction to it, is like, well, the solution to a differential equation is just the kernel of the linear operator. That’s all it is.
This is like, the students have not done differential equations before. They have not really gone over kernels and linear operators aside from maybe a little bit in linear algebra. Just trying to jump to the highest level of abstraction. And the students are all just like, what?
And I was actually tutoring a bit for this class. And the circuit was so out of touch with thinking, oh, the students are working on their proofs. They’re learning a lot. Meanwhile, everybody’s just like, what is this stuff? And I’m trying to fill in all the gaps. And it’s just terrible.
Another instance is, you know, I actually worked with my sister-in-law a couple of years ago, and she’s taking real analysis. And real analysis is the stereotypical math major record course, right? Because that’s where it transitions from a lot of, you’re used to, you know, linear algebra, multivariable, tau, differential equations, it’s kind of concrete and maybe taking methods of proof course, kind of dip your toe in the water of abstract proof writing.
But real analysis comes and it’s like, if you’re not 100% solid on this proof writing, you’re going to struggle. And the instructor would basically, he would serve up maybe five homework problems every week. And those five homework problems were at the highest level of abstraction.
It’s like, we’re not going to skip over proving that a set is closed or open or stuff. We’re going to jump right into theorems of, prove this closure property of the n-dimensional sphere under these conditions. And man, can we just work in one dimension first with some actual numbers, not stuff like that?
And so, Chris, the homework would not even get really graded. It’s kind of a for-completion thing. If you attempt it, it’ll look like you attempted it, then you have points. And then there were no quizzes, there were no assessments, except for a midterm and the fun is this. The midterm was given in the middle of the year, and it was handed back, I think, a week before the final. So there’s no signal.
And I’m told that during class, the way he would run it is he was like, write at the room up on the board, write down the roof, okay, you guys have any questions? He would mistake the student’s silence. He would say, oh, great. Okay, this is easy. You guys already know this. Okay, let’s move on. Meanwhile, everybody’s like, we don’t even know what questions that. And otherwise he was a nice guy, like you have said. They’re nice. Very nice.
Justin Skycak: Know who you’re teaching it to. Absolutely. Another thing that strikes me about how you’re sitting in the front row, asking a bunch of questions. In addition to kind of identifying these missing prerequisites for, you know, a lot of these prerequisites shouldn’t even be prerequisites, like a notation change or whatever. It’s like, why would you do that?
But I’m sure there’s other parts that are legitimate missing prerequisites that are just kind of skimmed over, forcing you to backbuild. I think another thing that probably made this approach work really well is that you’re effectively simulating minimum effective doses of instruction and active, you know, you’re generating what you’re thinking about it, kind of answer it, trying to get your own question answered.
It’s like, right. I’m checking the class. I pay for this. I pay for this. I’m here.
Justin Skycak: Get the knowledge out of them, forcefully almost. This is a broken model of reality, how learned. You don’t teach a student by talking at them for an hour. That’s going to overload any, unless the student already knows the material already, they’re going to be overloaded with their working memory, their cognitive load.
You got to be breaking this stuff up, I mean, filling in the productions, but also having students, why would you talk at a student for even 10 minutes or more before having them do an exercise? And some classes do a kind of very, try to move in this direction by doing clicker exercises and stuff, but that’s only baby steps in this direction to what it needs to be.
Justin Skycak: Here’s, I think this, I think a lot of people who would clash with this, or if there’s a line drawn in the sand on this, there’s a battle between two sides, I think the point of contention really boils down to what does it mean to understand something?
Okay, there’s a kind of conceptual, you know, you watch a video on, say, neural networks, right? You don’t know really how they, you’ve heard neural networks kind of underpin a lot of modern AI, and you’re interested in it, and so you watch a video on YouTube, and you learn about back propagation, and you learn about regularization, and some of these things, and training tests, whatever.
And so you get a little familiar with this stuff. Do you understand it? You definitely don’t have automaticity. You haven’t worked any problems in it. You haven’t set up anything on your own. Do you understand it? Well, there’s some kind of, I guess, you’re familiar with some of the terms.
I personally, whenever I talk about understanding, I mean understanding to the level of conceptual, understanding to the highest degree, conceptualizing. There’s a difference between how maybe a basketball player conceptualizes strategy versus somebody who just watches basketball games.
And because the basketball player knows at the fundamental level these strategies that work or don’t work because of these movements that happen. When they’re thinking about these strategies and stuff, the parts of their brain that are actually activating movements and things. It’s almost like simulating a simulation in their head of all the underlying things versus somebody who’s just seen this at a high level.
And so I think, I mean, you can acquire maybe some familiarity with just the, okay, backdrop is something that involves hardcore multivariable calculus and you somehow figure out how to adjust the weights of the network. I mean, if you call that conceptual understanding, then I guess you can get that without having automaticity on the concrete computations.
But if you talk about backdrop in terms of, how do I optimize? Say this issue is happening in the neural network, this is what the loss curve looks like and this is what layer three reads as, what’s wrong with it. I would say that’s a level of conceptual understanding that you can’t get unless you have actually worked through the computations.
Justin Skycak: I’ve got a concrete example for this again in the machine learning setting. It comes from when I was teaching this really advanced applied math and computer science elective sequence within Math Academy’s original Pasadena program.
We got students from the point of, you know, they came in having their core engineering math in place. They learned linear algebra, multivariable calc, and everything. And we scaffold them all the way up through coding exercises, building machine learning models from scratch, no libraries. It’s like re-implementing papers in 90s artificial intelligence.
And one of the things that I remember just really stood out to me as validating this kind of, you have to learn the movements in order to really understand what’s going on, is when we were talking about exploding gradients and vanishing gradients in neural networks.
I remember having talked about this previously with some people who are interested in machine learning but didn’t really let the math actually go through the computations. And it was always a kind of fuzzy area. It’s like, what do you mean? Why do they explode or vanish? Okay, I guess the activation function, the derivative of the activation function, if it is that the slope is too high or it’s too low or is it if it’s unbounded, they just kind of, something about the activation function, then it’s bad and then that stuff happens. Alright, I understand it, let’s move on. Let’s move on to some cool stuff.
But with these kids, I made them work out some simplified scenarios, maybe a two, three layer neural net of two nodes in each layer with some nice numbers. The computation took maybe three, four minutes or whatever, but they could see very clearly the effect of different activation functions.
And so I would like, okay, what if we use, let’s just say we use an exponential function as the exit or just even a linear function with, you know, quadratic or something where the derivative is continually increasing. And then they would see in their computations, wait, when I chain rule it and I take the derivative of this thing and then I multiply it, it just keeps getting bigger and bigger.
And then I’m like, what do you mean? Why did you give me a bad activation function? Why did you do that? And then I was like, ah, alright, you see the problem. Why don’t I give you an activation problem that doesn’t have that property?
And then it’ll just go flat and they’re like, okay, great, thanks. And then when they do that, they work computations. They’re, wait, wait a second. They just get one round in, and they already anticipate how it’s going to play out. They’re like, wait, this thing is already basically the derivative is saturating. It’s going to zero and all the terms are going to go to zero. I don’t even have to work out the rest. I already know how it’s going to play out because you just chain rule, multiply, chain rule, multiply, goes to zero, goes to zero.
And then at that point, I guess they get it. They get the exploding and the vanishing.
Justin Skycak: I think the experience can vary a lot depending on whether you’re kind of skilling up on known knowledge that’s just, this is figured out, just learning, versus kind of at the edge of the frontier. I think it’s looking really fuzzy and what’s even true and stuff like that.
I think what the confusion represents, that moment where it clicks in place and you just have that insight, you know, oh my goodness, this is it. And then it clicks in place.
I think when you are going through the knowledge graph of just stuff that is known as the foundational material, I mean, I think what that amounts to is really you are missing some prerequisites and you build up. Maybe there’s this topic sits a bunch, or sits atop maybe five different prerequisites, or maybe say 50, 100 ancestor topics, prerequisites and stuff.
And there’s one learning path in there that you haven’t really, you know, it’s just missing. You haven’t filled out. See, you got most of it filled out, but there’s one pillar that’s kind of incomplete and you’re banging your head on the stop. What is wrong? And then at some point, whether you kind of stumble into it or somebody directs you to it or whatever, you kind of realize, oh my goodness, wait, that’s the, oh, this works that way. And that’s how it works. And then it kind of clicks more into place. You kind of fill in that missing pillar.
And it feels amazing, right? It’s the click, it’s the aha moment. But when you’re in that foundational body of knowledge, it’s like, well, there’s a lot of time that you had kind of wasted being confused. If you just had that pillar in place beforehand, you wouldn’t have had to sit there for hours or days just ruminating over this thing. And then you would have the pillar, you wouldn’t have experienced a click and aha moment because it would have just been like, it was a deal. We just put these prerequisites together. What’s hard, man? That’d be it.
And I think when you’re in the foundational body of knowledge, these aha moments are actually something you want to try to avoid. I mean, if you’re in a well-structured print, you won’t actually have, I know it sounds a little counterintuitive.
But I remember somebody, we actually had at Math Academy, we were talking to, this was years ago, there was somebody we were talking to about recommendations for growth on social media and stuff. And they had recommended something like, well, they had to give it a nice name to these aha moments or snap moments, and talk about these things. And then I was just like, wait, that’s the opposite. We tried to smooth it out so you don’t have that, like, you’re climbing out of a hole and you’re like, I made it. I made it. No, it just makes it smooth.
Now, eventually you run out of those knowledge that’s known, right? And you get to the end of the frontier where things are fuzzy. And that’s kind of, I think, this is more what you’re talking about. It’s like you’re kind of in a setting where this has not been fully mapped out.
You’re going to run into situations where you just have missing prerequisites, because everybody has missing prerequisites because nobody knows what are the prerequisites. That’s kind of what we’re trying to figure out. We’re trying to map out how to extend this body of knowledge further.
And part of this mapping it out means identifying what reframes are prerequisites are necessary. And I think as a researcher, that sounds like it would be a very valuable moment where you’re kind of sitting in confusion. Why am I confused? It’s because there’s some missing prerequisite. Holy crap. If I figure out what this is, resolve this, everybody around is confused too. If I can figure this out, that’s going to be a high-value contribution.
That’s my take.
Justin Skycak: It’s like, generally doing hard things is good, right? It’s good to push yourself to do hard things, but also when things don’t need to be hard, why make it harder? You’ll get to the hard things that really matters to spend your time, spend your confusion on things that, when resolving, it’s like, spend your confusion on things that have high ROI and that confusion.
If you’re confused, you get an aha moment, a click moment, how about make it something that actually is a contribution to a field instead of just like, oh, this is a result in this math course that I skipped. Don’t spend, okay.
Justin Skycak: We actually came up with the solution to this problem at Math Academy, because we were getting a lot of adult learners asking this class, basically immediately as we kind of emerged on the scene on X, Twitter.
A lot of adult learners are all asking the same question. It’s like, what do I do? Do I need to kind of start up? I’m not even at calculus. I took pre-calculus in high school and I forgot all of it. Should I just start with fractions, fourth grade math? What do I do?
And then our answer to that was basically, why don’t we design a sequence that is the, you know, we’ll start it with fractions. You can go down as far low as you need. You can place into the sequence. You can place out of the material already know.
But it’s going to strip away all the stuff that is there for, you know, in schools, school, Common Core standards, but does not actually really get leveraged in university math.
Just to name an example, in geometry, there’s this inscribed circle theorems, where you’re like, okay, I got a circle. I got an angle in the circle. And what’s the, if the angle, the vertex lies on the circumference of the circle. What’s the measurement? If you know the arc intercepted by this thing. It’s one of those things. It’s like, okay, maybe it’ll come up a little bit if you do the physics of lenses and stuff, but aside from that, you’re just never going to see this kind of. Not relevant to machine learning, not relevant to most physics, not relevant to basically anything else.
And there’s a ton of these topics that just, they’re not very foundational for high school math. I think it was maybe a quarter or a third of the traditional school sequence. You can just kind of strip away and it’s really not a big deal.
But you get your foundations, the really important stuff that underpins a lot of the serious math and physics and computers and machine learning, whatever people want to do.
And we made this three-course sequence. We call it the Mathematical Foundation sequence. And our director of curriculum, Alex Smith, was really the one who put all this together and made it happen.
But the sequence is all together, it’s this three-course sequence. You take the three courses and you emerge on the other side ready for university math, like linear algebra and multivariable calculus, or a composite course like Math for Machine Learning that stitches stuff together.
And to go from this math foundation sequence, the very bottom to the very top. Assuming that you really forgot how to add fractions, you are starting from almost zero. To fill it out all the way takes about 15,000 minutes.
Now it might seem like a lot, but you actually do it. How long is that actually for days? Well, if you just do every weekday for an hour, for a year, you can fill in all your foundations, if you’re starting from the very bottom.
And now if you decide to do, I’m going to do just two hours per weekday. Guess what? Only half a year. Three hours per weekday. Maybe you want to split that up into a morning session and evening session or a couple of times a day. But if you get really, really serious, guess what? Just a few months.
And that’s assuming that you’re starting from the very bottom. Most people don’t start just chromatic fractions. They remember some algebra at least, how to solve a linear equation, stuff like that. Really, if you’re serious about this, you can get it done in several months.
The path is there and we’ve had people do this and come out the other side. Actually, the person I referred to earlier who had signed up that got me course because he wanted to read this robotics book and do robotics. And he just had no, way above him. He went through that sequence and he came up to the other end and it was, it’s just world changing. Well, this is good.
Justin Skycak: Definitely ties into what we were talking about earlier and the end of the pipeline where I mentioned that we were talking about having projects as just really cool projects as this anchoring goal for learners.
Where, you know, you can look and do, wait, I just take these math courses and then I get to training a convolutional neural network to classify digits, really? Or even cooler applications, like I use a neural net to approximate some lengthy physics computation and run it really fast, or the clustering of language symbols and then a long loss of whatever. We got a ton of different projects like that coming out in our upcoming machine learning course.
And we want to kind of extend this to the whole curriculum, not just machine learning, but actually do it in just your routine calculus course. Why not have calculus with a bunch of really cool projects that involve, you know, be integrated with a little physics and have one of the projects be figuring out the viability of some situation for setting up a colony on Mars or something, or figuring out what type of fuel is best for a boosted race car or figuring out what ingredients are going to work or not going to work for a certain recipe given chemical property, whatever, stuff like that.
A goal is that if we can cover the space with enough really, really cool stuff and then allow people to anchor on particular projects that they want to do. And hopefully not just the highest level projects, but also intermediate projects leading up to there. Have some projects in algebra, have some projects in calculus, some projects in linear algebra, multivariable calculus, math, or machine learning, all this sort of stuff. There’s kind of a real application that seems really, really interesting.
Justin Skycak: This is the only thing. The only thing. There was one paper that we actually are, is going to form the basis of one of the machine learning projects, which, I remember, I mentioned that I had, at least high school, when I was teaching this advanced machine learning crack within Math Academy’s original school program, the kids reproduce papers and there was one paper back in the nineties artificial intelligence.
Like you say, it’s a cool result. And it’s surprisingly, you know, it’s just evolving neural networks to play games, like tic-tac-toe or checkers or whatever. And you actually don’t even need to do back propagation because it’s an evolutionary procedure. And if you can just forward propagate, I mean, that’s really just the hard part is not that much math involved. And then you can do it.
I think having paper reproduction is going to be a really cool thing as well. And I’m sure there’s tons, tons of those papers.
Justin Skycak: I guess one of those things where, as you’re kind of going, we’ve made this distinction between when you’re in the knowledge graph, the known knowledge graph, versus when you’re building on top of the knowledge, right? You’re at the frontier of things.
And I think, just from a pure efficiency standpoint, when you’re going through the known knowledge graph, any sort of creative thing that you will do, unfortunately, it’s not actually that creative because it can be rendered trivial by a lot of results. You could study the statistical properties of random walks as a solo creative project or stuff, but I guess that’s already figured out. Whatever result you’re going to have on that is probably known already.
Now not to say that there is zero value to this because there is motivational value to it, right? If the choices are, either I’m going to lock in, and if you’re going to draw a dichotomy for yourself, it’s like, okay, either go monk mode for all my studies until I reach the frontier and only then will I allow myself any sort of creative exploration. That’s just mechanically going to be the most efficient way to get the frontier.
But if that’s going to force you off the rails or you’re just like, man, I quit. I’m done. Well, okay, just do what you need to do to stay on the rails. And if that involves going down some rabbit hole or whatever, just have some kind of fun with it, experience some cases of the rewards that you want to get once you get to the frontier of it, then okay, do that.
Do what you need to do to make it fun. And everyone’s going to be different in this capacity and how much of that they feel like they want to meet or what type of rabbit holes they want to go down or whatever. But I think the real point where this exploration and everything becomes very valuable, a high ROI sort of thing, is really once you reach the frontier.
Justin Skycak: This approach of doing a, go to some kind of interesting project at the frontier and do what you can and then backfill to try to unlock more creativity or more ideas or whatever to do with this thing to understand it better.
I totally see how this can work for people who have some amount, a large chunk of foundational knowledge already in place. And I think it kind of depends on the distance between what they’re trying to do and what they’re doing.
Because this may seem like a caricature, but this happens a lot where there’s somebody who doesn’t even know how to solve a linear equation. And they want to do like, wow, this AI stuff is cool. How do I do something? And it’s like, well, I mean, I guess you can download and run a transformer model or something.
But if the gap is that big, when it comes into backfilling prerequisites, you’re going to be like, well, okay, I need to learn back propagation. What is that? That’s squiggly simple. Oh, is that calculus? It’s like, if the gap to fill in is not very, very big, then you can be efficient in terms of backfill. Oh, I just need to learn, learn, learn. Okay, new unlock.
But the gap is gigantic, where you’re essentially missing all of algebra, all of calculus, then I think it gets to a point where it can be inefficient to try to trace your way all the way back down to fractions or algebra.
Justin Skycak: I’m like, oh my god, I’m going to level. And it’s like, well, guess what? You’re going to need to learn this stuff anyway. It’s not like there’s three secret topics in all of 300 algebra topics that you need to learn to unlock machine learning. It’s like, no, you got to learn basically all of the algebra. It’s foundational.
And I think for these foundational, when a body is foundational like that, if you try to just go and back, you’re going to be ultimately just kind of trying to construct your own algebra graph. And this is a known thing. Just take the algebra. Take an algebra course, and you can get through it a lot faster.
But I agree, as you reach closer to, okay, you’re kind of within striking distance of this thing, you can kind of see the full path between the project that you’re doing and the math that you need, the backfilling strategy becomes less
Justin Skycak: That’s, I think that’s right. There’s also the variable, how much does person understand how to learn effectively? That plays the same role as a gap in knowledge.
Justin Skycak: I have a very concrete example. But before I say the example, I will say there have been a number of adult students who I’ve seen go from, even in their thirties, just go from, you know, I don’t really, I want to get involved in some kind of cutting edge thing, or maybe not even cutting edge. I just want to become a software engineer.
And they start this in the thirties or even their forties, or I’m sure there’s people who do it even later in life. If you’re willing to put in the work, it’s the same amount of work. The amount of work does not change whether you start this at the age of 15 and emerge where you want to be at the age of 20 or 25 or whatever. It’s the same amount of work if you started at 25 or if you started at 35 or whatever.
And some of the things that makes it more challenging for, well, I think sometimes people think, oh, it’s too late for me to do this kind of thing. And I think that tends to be more of just like, I’m not willing to put in the work given that I have less free time.
You know, you go through life, you got a family, you got kids, you got more responsibility. But it’s just harder to make time to do things aside from it. Especially, you need to have a job too, a separate job. It’s like, where’s the time of the day to learn? When you were a teenager, you’re just like, you just come from school. What are you doing? Play video games or go to the basketball court with friends. You got all the time in the world to do this stuff.
But the amount of work does not change, even if your time to do it. But the time that you devote to doing the work is within your control. You can choose to be more efficient in various areas of your life to open up time. And it’s not going to be easy. And there’s always going to be lots of constraints and stuff. And probably not going to open up as much time as you did when you were younger. But the takeaway, if you put in the work, you will get to the result. If you find some way to put in that work.
And I’ve seen this happen in particular. Back when I was working in data science, I started out as an intern for six months before converting to full time. And there was another guy who was actually interning with me. And he was, I think he was actually in his thirties. Yes, somewhere in his thirties, or maybe 30 years old at the time.
His story was kind of that in high school, he wasn’t really paying attention to what he wanted to really get out of career-wise. He was more concerned about, I just want to have kids, have a family. And that was his main goal. And that’s a fine main goal. And he did that. He accomplished that. Had kids, had a family, and he was working as a receiver at an emergency center, if you 911 who answers the phone.
And he wanted to get into a kind of career that would offer more money and better balance and stuff and just to get more to his family and stuff like that. So he was like, I want to go into software engineering, want to get into data science. And this was, you know, he was in his late 20s when he started even thinking about this. And he didn’t know any math, no coding background, none of the stuff.
And so he just signed up at his community college. And in his late 20s, one of the oldest students there. And he’s like, whatever, this is, I got to learn the stuff. So I’m going to learn the stuff.
And at the same time, he’s got his job that he’s holding down as well. He’s got his family and stuff. This is not an easy sort of, he’s pressed for time. But he’s making it happen. And so he goes there, he gets his computer science degree from community college. And then gets an internship at this company. And so he and I interned around the same time. And he also converted to full time as well.
And he took it very seriously and really got good at the kinds of software engineering that are doing the data flows and stuff. He got the nickname the Lord of the Data Flows over there because he was so into it.
I remember talking to him. He’s like, man, I really need this to work. There’s like, he was at a point where there was no, what’s the alternative plan? He wasn’t thinking about alternative plans. He’s like, no, this is what I’m going to make happen. There’s no hedging of, oh, this doesn’t work out, I guess I’ll do this or whatever. I’m sure there were, he planned these, but he’s not thinking about, focusing on the thing in front of him. Making it happen.
And one thing that I think is true is, initially, the heart of this journey, especially if you’re embarking on it later in life, the hardest part is kind of the upskilling phase when you’re trying to prepare yourself to get paid to do something that’s more aligned with what you want. Because at that moment, you’re not good enough to get paid for it yet. So probably you have to be doing a different thing to get paid. And that’s going to eat up a lot of your time.
You have to spend time doing this other thing and spend time preparing for this new thing. It’s like, you’re just pressed for time. Once you can make that transition to actually earn a living from the thing that’s in the direction that you want to do, it’s like a total phase change because now you have maybe eight hours of your life every week that you’re actually doing the thing that’s closer to what you want to do, if you’re improving in that direction.
And it’s like, you get a rock after this. And if you can just make it to that point, then
Justin Skycak: It’s kind of like, you let life ossify around you. You let the concrete harden. You got to break the concrete. Yes. But once the concrete is, okay, it’s the prize, you have more concrete.
There was a student that we had, well, that we had, started last year. He was talking about how he posted some video talking about the experience of just getting a habit, sitting, doing that. And he was talking about how when he started, he just was not really into it. Math is, I think he wants to go into software engineering or data science. Something in that direction, in the tech direction.
He was doing a degree in computer science. And there was a bunch of math courses that he was just like, I can’t do it. I’ve got it through all my dimensions. And he started on this and initially, he was saying, every day I wake up and my brain just tells me, don’t do it, man. Don’t do it. Don’t start working on the course. This is not going to be fun. You should run away from it.
But it’s every day. And he just, there was, at the beginning, there was just a lot of, shut up, brain, I’m doing this. We’re going to do it in the morning, where you’re really awake enough. I’m going to give you to, before you start yelling, I mean, I don’t like this. We’re just already going to be doing this.
And he said that this continued for weeks. I can’t remember. He said 30 or 60 days or something like that. And he was like, man, I just accepted that this feeling is probably just never going to go away. And then a week after he just accepted that, I’m just, this is what we’re doing. He said he started just not having all those bad thoughts.
But that, just way to, it’s like, you wake up and you’d be waiting for the brain to go, do it, man, don’t do it. Don’t you dare start us on math problems. And it’s like the brain, it just given in. It was just like, okay, I guess this is what we’re doing. Fine.
If you get yourself, you just go through the habit enough to get your brain to just accept that this is happening. This is the path of least resistance is really, you’re going to be
Justin Skycak: Not that, though? I’m a little bit familiar with it. I have the book on my reading list, ultra learning book. I’ve read through a blog for the big students journey through this. Rusty on all the details of it.
But this comes out very very quickly when I talk to people. I’m like, oh, this is very similar to the ultra learning thing. Oh, my God. That’s right. I need to leave this thing again.
Justin Skycak: When I was doing my own version of the ultra-learning marathon, 3000-hour study, very fun. I was actually not really going in my own interest areas. This was, I think, a difference between me and my program and the ultra-learning program, and I started out some life.
And I also did not have the very, my goal was actually not to go through and retire. It was not a specific project that I wanted to do. I started with realizing that, like I said, earth-shattering realization that the pace of school is common. And if I just take this into my own hands, then I can wait back.
That paired with the idea of all this, I’m sorry, initially it was like, I started out, I was like, I’m going to learn confidence. That was the mission goal. Once I got through that, and I was like, I’m just going to, all this math stuff, it seems like it’s going to be really useful for me in the future. And I’m enjoying it. It’s a lot of fun. I’m just improving on the stuff.
I didn’t have a free air. I was just like, okay, I just want to learn. This is a subject, this, this, this, because I can. And so it’s not like I was sitting there, like, okay, I learned calculus, I learned knowledge, but I want to go into different ways. I don’t really have that.
Oh, that’s cool. It’s like, what about the cross pollination between different equations and multivariable and the partials? I wasn’t going out in Wikipedia, going down the rabbit hole and stuff and that. It was a very structured kind of like, I’m gonna, just tell me what to do. Just find a resource that just tells me what to do and what to do and what’s the mission way possible.
And initially, I mean, I was coming into it, I wanted to do this, but I wasn’t always going on that. I was getting my head, my head ball was, I wanted to learn this stuff really, really, I want to get as far as I can.
Justin Skycak: One thing I want to say is that sometimes, if anyone has a particular goal, like machine learning, I want to reimplement this paper, whatever, I never want to say, don’t even try it. Go, try it, you know, give it a whirl. If you are close enough, within striking distance, that you can do it, then what might I say?
I’m just saying, if somebody is like, man, I tried it, but I can’t do this thing, since I don’t know what the hell do I do, it moved in my just to help to do it, and that’s what I’m like, okay, no, stop, stop. You’re doing it. Get your foundations in place. This is going to make all the difference.
Justin Skycak: That’s interesting. When I was doing the biology, I had kind of viewed it as the Math Academy approach to just what you do is, you literally go in for a certain term. And so I kind of do that as, okay, I just want to spin up on biology as fast as possible, which is a mastery of the subject.
But now that you bring it up, it’s true. I kind of scope it down in particular to genetics, genetics, that’s the lay area. And I think overall, there is a reasonable amount of scoping and top-down plan whenever you have it.
And so that’s something that we’ve actually created courses on math that specifically works like Math for Machine Learning. What we’ve got, that’s for physics. It’s like, if somebody wants to do machine, you’re going to have to learn linear algebra, all type of work out of this, probably the statistics, but you don’t need to learn all of these things. There’s a lot of multivariable calculus, like the use of the there, the average is there, it’s there. At the end, that’s not the type of, the most empirical, the CPA.
You really need up to the change of hyperplaning, stuff like that. Which is really, I mean, it’s a large, tropical, but this is not the whole thing. There’s a lot of reasonable scoping to have in any people. That’s kind of almost a hybrid, I guess. It sounds almost like a midway between higher subjects, mastery learning versus the only prerequisite chaining, ultra-learning, just kind of that. Scope it down, top down, and then fill it up.
Justin Skycak: The interesting thing is, a lot of these people with ADHD think that, you know, just a lot of the education system is not cut out, and the way I’d like, you know, that there’s not a hearing from, that’s not, and only one of those things is true. The education system, as it stands, is not threatening to people with attention challenges. People who can’t sit still, actually, I mean, it’s really not friendly to anyone, but especially with the potential for the challenges.
And it’s like, nobody likes being talked about in a subject, but a lot of, I’m kind of similar to you with that, I think. I can try it up, six hours of writing, I just don’t know how much I’ve been talking about. And it’s like, what if you can take somebody who has, you know, more, it is not like you and me as a opposite way. It, just the hint at the talk and the power of how it just kind of limits the amount of, you know, how much can you communicate with somebody before they start doing something?
And initially that might seem like something, some kind of disadvantage. Well, I just keep the system, the power it takes to go through the electronic, you know, quite really. If you look into, you start going down the rabbit hole, it’s not as a learning, it’s like, well, that’s not even a good thing to do in the first place, to be talked about for now.
It’s just people have, some other people have a higher talent, but it’s not good for any. What’s good is these bit of effective doses. Just like, okay, just tell me a little bit of what to do and then let me start doing it, and then active, active, active. Or you stay in a sort of passive state or just not doing anything. It’s like, you’re not, if you go to tennis, it wasn’t so much a question to talk about for you right now, or I don’t care if you have ADHD or not, you’re not getting better at tennis.
If you have ADHD, then you might just be expressing or to this company, come on, what’s going on? Just do something. But either way, the learning’s not happening.
I think a lot of times, the neurodivergence, it’s not just, it makes more sensitive to things that are, there’s an issue. If you’re able to, the issue more, or earlier on.
I mentioned, I have very similar to you in the, the passage of time is just, it’s almost hard to keep on attractive and just to sync with this company. Our founder of Math Academy, Jason Roberts, he is actually humbling the exact opposite of his ADHD perspective. So a lot of Math Academy is built in terms of what would Jason have won? What did Jason hate about the application growing up?
And those other things, like being talked at for a long time, or actually doing problems, hated him. He’s got these stories, I like about him, just being a descendant of the class, who never sits still. He’s just like, Jason, hate, mention, what are you doing? It’s like, well, the problem isn’t Jason. The problem is that he’s being talked at for too long without doing anything.
And so Jason actually has three kids who are all, one was just very highly computationally, mathematically gifted, and just blows through all this stuff. And he has another kid with a discount, he’ll be up, and then another kid who is just kind of using more, give you a typical honors math.
And so the kind of approach that we take is kind of making sure that this efficient learning cycle is working for all the students. And all these kids, he and I, you and I are so different too, and we just agree with all the same things. It’s just whether you come at it from, okay, kid with ADHD, it can’t, it doesn’t want to sit still for 15 minutes for an explanation, or you come at it from just calculate the learning efficiency of this setting, this is that set, you get to the same answer.
Justin Skycak: Absolutely. It’s just kind of, right, the feeling of, it’s like, the feeling is often the real kind of, the being done. Often the issue is not that you’re done, it’s just that it’s some other settings that are not, you know, causing the information to flow, whether that’s a prerequisite gap or just not wanting to participate in the process, or whether it’s you’re just feeling that this is not, you don’t identify with this stuff, this is not something good at. It’s all this kind of blockage that’s gonna be put up, and it’s separate from the actual, how good you
Justin Skycak: That’s a really good question. All right, I’m not a research mathematician, so I don’t want to overstep in my answer to this, making claim that I don’t actually know what I’m talking about. I think this question would be best posed to somebody who has a track record of mathematical publications.
Well, let me generalize the scope here a little bit about amateurs contributing to the edge of the ills in general. One thing, at least in my experience, and that I hear from a lot of the people who confirmed, is that this tooling that exists nowadays, LLM stuff, OpenAI, or whatever, all these kinds of tools, they’re kind of multipliers on what you’re able to do in initiative.
And now, if you’re not able to do a whole bunch, you don’t have technical chops, and then you ask some stuff, ask questions, the LLMs have to code up stuff for you that you couldn’t otherwise do. This can feel sort of like equalizing, like, oh, hey, I’m on the same caliber as everybody. I don’t have to learn how to code. I can alternative vibe to everything. I don’t have to learn how to go to goal, hack into science and stuff. I can just tell the LLM to generate course stuff like that.
It’s like, well, okay, that is, I guess, level you. But now think about how much it’s leveling up people who actually know these foundations of what’s going on. And the thing is, when you have a high level of mastery in something, at least in my experience, these tools can scale you. You are the zero to one, and then these tools put the zeros behind them, like 10, 100, 2000.
And if you don’t really know what you’re doing, you start being
Justin Skycak: The other, the edge, absolutely. A lot of the efficiency gains that I’ve seen personally, you see, these tools have been, it’s more about delegating work that I already know how to do, because that I can automate more and just free me up to do additional stuff.
But it’s not a way to get needed to awful at work that I don’t know how to do because it has the same failure modes. I’ve done this before, I’ve made this mistake before, trying to, I have a junior employee, human employee, and you got a project that come out of this, I was pretty cool. You haven’t really smoked it, you haven’t gotten a piece of it or anything, and it’s like, this sounds like a good thing for you to do.
And then you look up a day later, a week later, just, wait, what the hell was happening? This is not the whole. And then you have,
Justin Skycak: Here’s our options here, what do you think? And then you can, because evaluate on a totally little delivered.
One of the big unlocks, I’d say, that I’ve experienced recently, and this was the agentic coding stuff, was kind of just scoping stuff down enough that I can get it into, that it fits into my own working memory of what we’re doing here. That was one of the challenges that I experienced before.
It had to have this whole plan for, okay, here’s the feature that we’re gonna build, here’s the thing that we’re gonna do. You have to get to do too much, and then it’s gonna go off in directions that you don’t anticipate, because you have not verified everything that’s gonna be doing, because you can’t keep it in your working memory.
But if you split this up into, if you are able to sign up, get the expert feedback, and I’m gonna do it. Okay. This all looks good. I seriously looked at every little thing, it looks good. You can keep the reins tight like that, then it’s amazing.
Justin Skycak: You know, there’s one thing that I think about as soon as I wake up, late into the night, all the time. It’s what I’m working on, and I think I mentioned at the very beginning, when he asked, what am I working on right now, it’s taking Math Academy from workshop to a factory transition.
It’s a huge team-lift. Me, Jason, Alex, Dr. Coagulant, just figuring out how to take the, it’s like, it interfaces with learning with all the AI, everything, how to deliver this learning experience, and in particular, this content, a whole courses at a much higher level of scale. How to turn out more courses, factory style, but not just like, hey, I’ll build a course of work, and then, then it’s like, how do you make the experts, take the human expertise and then apply these tools as multipliers to just speed up that level of expertise without a traffic standards.
And we’ve made so much progress on this in the past few months, and I’m really excited about this. But this is really our kind of, you know, the Tesla, Elon sleeping on the factory floor, kind of stuff. But it’s just production hell of, oh my god, this is, we got ourselves into this game, and now we have to scale on and really deliver.
And just, it’s gonna suck for the next however long it takes to make this factory work. That’s kind of where we’re kind of doing the people. That’s where we are. That’s what I’m doing at every waking hour, basically.
Justin Skycak: Well, fantastic. Thank you, Assistant. Thank you for having me for two days. This was great. This was a lot of fun talking to you. And, you know, like I said earlier, I came in with reflections today, and that only happens when you have a really good conversation with somebody. It’s the testament to you as an interviewer, man.
This is too much effort into this. Kind of have great questions, and also answer, including so many people’s questions that he said to you. I never imagined that we would be addressing this many people’s questions that are possible, that we were able to.
The following prompt was used to generate this transcript.
You are a grammar cleaner. All you do is clean grammar, remove single filler words such as “yeah” and “like” and “so”, remove any phrases that are repeated consecutively verbatim, and make short paragraphs separated by empty lines. Do not change any word choice, or leave any information out. Do not summarize or change phrasing. Please clean the attached text. It should be almost exactly verbatim. Keep all the original phrasing. Do not censor.
I manually ran this on each segment of a couple thousand characters of text from the original transcript.
]]>If something comes easy to you then you should lean into it even harder.
You have no idea how many doors can open up for you just by being insanely good at something.
If you have a winning lottery ticket then for God’s sake cash it in!
]]>