Specialties

Educational Knowledge Graphs • Physics of Learning • Math Education • Math Content

Educational Knowledge Graphs

Since 2019, I've solved every problem that Math Academy has faced while constructing an educational knowledge graph for all of 4th grade through university-level mathematics and building a fully automated & personalized learning system around it:

Constructing and annotating the graph manually yet efficiently using command-line tools
Validating graph integrity and automatically correcting issues (or supplying relevant information if a human-expert decision is required)
Initializating student knowledge profiles via adaptive diagnostics (and minimizing the number of diagnostic questions needed)
Modeling student knowledge profiles via spaced repetition -- canonical spaced repetition operates on discrete independent flashcards, but I developed a novel mathematical framework that extends it to hierarchical bodies of knowledge where repetitions on advanced topics trickle down to simpler topics that are implicitly practiced
Caching student knowledge profiles to reduce compute time/resources and enable visualizations throughout the system UI, and continually recalibrating cached knowledge states
Automatically selecting personalized learning tasks that optimize learning efficiency -- students learn 4x as much content in the same amount of time, as compared to a traditional classroom
Triggering remedial interventions when repeated confusion is detected
Enabling flexible configuration settings to tailor the system's behavior to student goals -- for example, if you're taking a calculus course, do you want to be given material from lower grades that we think you don't know yet? If so, do you want to be given all of that material or only material that is foundational for calculus?
Predicting when a student will finish their course (via Monte Carlo simulation, and continually recailbrating estimates)
Validating system behavior -- single decisions in real time as well as sequences of decisions whose effects manifest over longer periods of time
Analyzing student performance to pinpoint specific pieces of content that need further refinement
Avoiding downtime during production graph revisions by cloning topics, revising the clones, and then instantaneously "knife-switching" the revisions back into the production graph -- this enables a tight feedback loop connecting user feedback and content revisions
Aligning incentives of adversarial students in a gamified learning environment by calibrating awards and penalties
Automatically generating human-interpretable justifications for all decisions made by the system -- why did I get a particular learning task? Why did I get a penalty?
Creating an overall student performance score, along with various subscores, analogous to a credit score
Conceiving of and creating a successful prototype for our very first knowledge graph visualization (and constructing/managing the underlying data for later iterations of the knowledge graph visualization)

Physics of Learning

I developed a novel mathematical framework, Fractional Implicit Repetition (FIRe), that extends spaced repetition to hierarchical bodies of knowledge where repetitions on advanced topics trickle down to simpler topics that are implicitly practiced. FIRe is the core model that forms the brain of Math Academy's fully automated and personalized online learning system.

I also developed novel techniques for selecting personalized learning tasks that optimize learning efficiency. As a result, Math Academy students learn 4x as much content in the same amount of time, as compared to a traditional classroom.

The specifics of these algorithms are proprietary and are undergoing continual refinement, but a high-level summary of what they accomplish is provided below.

Background Context: Canonical Spaced Repetition

After a student learns a topic, their memory of it begins to decay roughly exponentially with time. Consequently, in order to retain knowledge of the topic, they must review it again. The process of reviewing previously-learned material is called spaced repetition. Loosely speaking, a "repetition" is a successful review at the appropriate time. As discovered by Ebbinghaus, after each successful repetition, the rate of memory decay decreases.

The main challenge of spaced repetition is choosing the optimal time to wait before performing the next repetition. Too late, and the knowledge has decayed so much that the next review may not be successful (and even if it is successful, it may take an inordinate amount of time to refresh on the forgotten material before attempting the knowledge evaluation task). Too early, and the next repetition is so easy that it's not worth the opportunity cost (it would be more efficient to focus on learning new topics).

Canonical spaced repetition leverages some simplifying assumptions:

Topics are entirely independent (like vocabulary flashcards).
Every repetition occurs at the optimal time.
Each repetition involves the same amount of successful practice.

Novel Algorithm: Fractional Implicit Repetition (FIRe)

FIRe generalizes canonical spaced repetition to hierarchical bodies of knowledge where repetitions on advanced topics trickle down to simpler topics that are implicitly practiced. It consists of two pieces: fractional spaced repetition, and implicit repetition flow.

In fractional spaced repetition, we remove simplifying assumptions $2$ and $3$ of canonical spaced repetition. Now, a topic receives a sequence of fractional repetitions, where each repetition

has a general weight (not necessarily equal to $1$ repetition),
is started and completed at arbitrary times (neither of which is necessarily the optimal time),
and is either passed or failed.

Implicit repetition flow extends fractional spaced repetition to remove simplifying assumption $1$ of canonical spaced repetition. Now, the topics are connected in a knowledge graph with "encompassing" edges. Each encompassing edge has a weight that represents the fraction of central skills of the simpler topic that are implicitly exercised whenever the more advanced topic is practiced.

Using the FIRe algorithm, we can determine the following quantities at the time of completing a repetition:

the topic's effective repetition number, and
the number of days until the topic's next repetition should occur.

Novel Algorithms: Minimal Cover Construction and Knowledge Gradient Maximization

The FIRe algorithm can be used to construct a student's knowledge profile. It remains to select optimal learning tasks based on that profile.

When selecting a new topic to learn, two constraints must be satisfied:

The student must have completed all prerequisites of the topic.
The topic must be at an appropriate priority rank in the curriculum. (For example, even if a student has completed all the prerequisites for a topic in the next course, we generally do not want to select that topic until they have completed most if not all of their current course.)

The selection candidates are the highest-priority topics that are not known but whose prerequisites are known (i.e. they are on the "frontier" of knowledge). Once selection candidates have been determined, learning efficiency can be maximized using the following techniques:

Minimal Covers. When there exist topics with due repetitions, select the smallest set of candidates that covers those topics with due repetitions.
Importances. When no repetitions are due and we wish to select a new topic to learn, choose the candidate that has the greatest importance. Importance is a product of two factors: the gradient (the rate of increase in aggregate memory across all topics that will result from doing a repetition on the candidate topic), and the lookahead (which quantifies how central the topic is to opening up new learning paths in the knowledge graph).

Math Education

I worked hands-on with 300+ students over the course of a decade (2013-2023) and have seen just about every single success mode and failure mode when it comes to learning math in all sorts of settings:

Group and one-on-one tutoring at franchises, through agencies, and independently: homework help, test prep, and enrichment (primary school through college)
Public and private school classes: regular, honors/AP, and even radical acceleration (calculus in 8th grade & university math during high school)

Additionally, within a radically accelerated math program, I developed a quantitative computer science track that scaffolded high school students up to doing masters/PhD-level coursework (reproducing academic research papers in artificial intelligence). More info here.

At the same time, I've also had first-hand experience with general dysfunction surrounding education including failing students, cheating rings, unreasonable parents, grade inflation, "no-fail" policies, administrative bureaucracy, and teacher credentialing / professional development that is centered around political ideology rather than the science of learning.

While my years of teaching and tutoring were certainly memorable, I don't do it manually anymore. Instead, I'm working on Math Academy's adaptive, fully-automated online learning platform. Our goal is to do everything an expert tutor would do, but better, and at scale.

Math Content

As a hobby, I used to write about math, including several textbooks. Despite little to no search optimization, this content ranks in the top results for many common search queries, some of which are provided below:

I also wrote hundreds of Math Academy's first lessons, which are admittedly much better scaffolded than anything I've written independently. And although my day-to-day is now mostly engineering/analytics, I still do a final review on every single lesson that Math Academy releases to users. We currently have multiple thousands of fully-scaffolded lessons that we continually improve as we gather feedback and data from users.