1
00:00:00,000 --> 00:00:02,940
What if I told you that data doesn't actually

2
00:00:02,940 --> 00:00:06,160
exist? Oh boy. Starting right out of the gate

3
00:00:06,160 --> 00:00:08,339
with that. I mean it. Like the numbers or the

4
00:00:08,339 --> 00:00:10,820
photos or even the text messages saved on your

5
00:00:10,820 --> 00:00:13,220
phone right now. What if they aren't physical

6
00:00:13,220 --> 00:00:16,160
things stored in little microscopic boxes but

7
00:00:16,160 --> 00:00:19,239
are actually, you know... active invisible spells

8
00:00:19,239 --> 00:00:21,960
just waiting for a command. It sounds totally

9
00:00:21,960 --> 00:00:24,660
like science fiction or, I don't know, mysticism.

10
00:00:24,739 --> 00:00:27,899
Right. Because we are so conditioned to think

11
00:00:27,899 --> 00:00:32,159
of computers as these rigid mechanical factories.

12
00:00:32,420 --> 00:00:34,219
You know, you punch in a number, the gears turn

13
00:00:34,219 --> 00:00:37,240
and a predictable widget pops out. Exactly. We

14
00:00:37,240 --> 00:00:39,679
treat programming like handing a list of mundane

15
00:00:39,679 --> 00:00:43,060
chores to a very fast, very literal filing clerk.

16
00:00:43,259 --> 00:00:45,159
Right. A clerk who has no imagination whatsoever.

17
00:00:45,679 --> 00:00:48,020
But today, we are going to completely shatter

18
00:00:48,020 --> 00:00:50,780
that factory floor. Welcome to the Deep Dive.

19
00:00:50,780 --> 00:00:54,399
We are opening up a literal grimoire today. A

20
00:00:54,399 --> 00:00:57,179
grimoire. I like that. It really is. It's the

21
00:00:57,179 --> 00:01:00,359
1996 second edition of a legendary MIT textbook.

22
00:01:00,579 --> 00:01:03,159
It's called Structure and Interpretation of Computer

23
00:01:03,159 --> 00:01:05,439
Programs. Written by Harold Abelson and Gerald

24
00:01:05,439 --> 00:01:08,099
J. Sussman with Julie Sussman. Yeah. And if you

25
00:01:08,099 --> 00:01:09,659
hang around software engineers, you only have

26
00:01:09,659 --> 00:01:11,420
to say two words, and they know exactly what

27
00:01:11,420 --> 00:01:14,329
you mean. The Wizard Book. The Wizard Book. And

28
00:01:14,329 --> 00:01:16,329
that nickname comes directly from the cover art,

29
00:01:16,430 --> 00:01:18,670
which is just iconic in the field. It really

30
00:01:18,670 --> 00:01:21,430
is. It features the sorcerer in classic robes,

31
00:01:21,430 --> 00:01:24,469
and he's holding a glowing magical sphere. And

32
00:01:24,469 --> 00:01:28,569
on the sphere, it says the words Evol and Apply.

33
00:01:29,030 --> 00:01:32,290
Right. And across from him, there's a woman gesturing

34
00:01:32,290 --> 00:01:34,230
toward a burst of light containing the Greek

35
00:01:34,230 --> 00:01:36,670
letter Lambda. Yeah, which we will definitely

36
00:01:36,670 --> 00:01:38,230
get into. But I mean, it doesn't look like a

37
00:01:38,230 --> 00:01:40,069
computer science textbook at all. Not at all.

38
00:01:40,150 --> 00:01:42,569
It looks like an arcane text on how to bend the

39
00:01:42,569 --> 00:01:44,849
fabric of reality. Which is incredibly fitting,

40
00:01:44,930 --> 00:01:47,269
really. Because for years, this book was the

41
00:01:47,269 --> 00:01:50,469
terrifying, mind -expanding foundation of MIT's

42
00:01:50,469 --> 00:01:52,670
entry -level computer science course. Course

43
00:01:52,670 --> 00:01:57,370
6001, right? Exactly. 6 .001. But I think we

44
00:01:57,370 --> 00:01:59,530
should make something crystal clear for you listening

45
00:01:59,530 --> 00:02:02,719
right now. Yes, please. Whether you are a senior

46
00:02:02,719 --> 00:02:05,620
software architect or someone who has literally

47
00:02:05,620 --> 00:02:07,879
never written a single line of code in your life,

48
00:02:08,080 --> 00:02:11,819
this deep dive is for you. 100%. This is not

49
00:02:11,819 --> 00:02:14,199
a coding manual. We are not going to teach you

50
00:02:14,199 --> 00:02:17,060
how to build a web app today. Far from it. It's

51
00:02:17,060 --> 00:02:20,319
a profound philosophical treatise, really, on

52
00:02:20,319 --> 00:02:23,139
how human beings can manage impossible complexity.

53
00:02:23,560 --> 00:02:25,800
It's about organizing thought. Yes. Organizing

54
00:02:25,800 --> 00:02:28,740
thought. The overarching theme is actually laid

55
00:02:28,740 --> 00:02:31,439
bare right in the foreword, which was written

56
00:02:31,439 --> 00:02:34,879
by Alan J. Perlis. And he drops a massive bombshell

57
00:02:34,879 --> 00:02:37,460
on the very first page. He does. He states, and

58
00:02:37,460 --> 00:02:40,620
I quote, Computer science is not a science, and

59
00:02:40,620 --> 00:02:42,860
its significance has little to do with computers.

60
00:02:43,280 --> 00:02:46,219
I absolutely love that line. It's so good. But

61
00:02:46,219 --> 00:02:47,979
it makes you wonder, if it's not about computers,

62
00:02:48,240 --> 00:02:51,180
what is it about? Well, my read on Perlis is

63
00:02:51,180 --> 00:02:53,560
that the computer revolution is actually a revolution

64
00:02:53,560 --> 00:02:56,479
in human expression. Okay, unpack that. He calls

65
00:02:56,479 --> 00:02:58,960
it the emergence of procedural epistemology.

66
00:02:59,340 --> 00:03:01,560
Epistemology being the study of knowledge. Right.

67
00:03:02,020 --> 00:03:03,979
Classical mathematics is declarative, right?

68
00:03:04,039 --> 00:03:06,280
Yeah. It provides a framework for dealing with

69
00:03:06,280 --> 00:03:09,560
what is. Like stating facts. Exactly. It's about

70
00:03:09,560 --> 00:03:12,979
true statements. But computation provides a framework

71
00:03:12,979 --> 00:03:17,030
for dealing with how to. Ah. So if math is the

72
00:03:17,030 --> 00:03:20,509
study of absolute truths, this procedural epistemology

73
00:03:20,509 --> 00:03:23,909
is the study of active processes. Yes. You aren't

74
00:03:23,909 --> 00:03:26,569
just stating facts anymore. You are conjuring

75
00:03:26,569 --> 00:03:29,650
abstract processes that evolve and manipulate

76
00:03:29,650 --> 00:03:33,009
information. The book leans so heavily into this

77
00:03:33,009 --> 00:03:36,349
metaphor of magic. Like, the programs we write

78
00:03:36,349 --> 00:03:39,270
are carefully composed spells. Spells written

79
00:03:39,270 --> 00:03:41,930
in esoteric languages directing invisible spirits.

80
00:03:42,229 --> 00:03:45,949
It's so poetic. But Perlis uses another analogy

81
00:03:45,949 --> 00:03:48,030
in his preface that really stuck with me. He

82
00:03:48,030 --> 00:03:51,129
compares the computer to a violin. Oh, the phonograph

83
00:03:51,129 --> 00:03:53,530
versus the violin. That frames the entire philosophy

84
00:03:53,530 --> 00:03:55,889
of the book. Right. So he says, imagine a novice

85
00:03:55,889 --> 00:03:58,349
who tries a phonograph first. You put the needle

86
00:03:58,349 --> 00:04:00,830
down and perfect music plays. It's effortless.

87
00:04:01,030 --> 00:04:03,530
Completely effortless. Then that same novice

88
00:04:03,530 --> 00:04:05,789
picks up a violin, scrapes the bow across the

89
00:04:05,789 --> 00:04:08,310
strings, and it just screeches. It sounds terrible.

90
00:04:08,759 --> 00:04:11,219
awful. And the novice immediately declares that

91
00:04:11,219 --> 00:04:13,400
the violin is a terrible, inflexible machine

92
00:04:13,400 --> 00:04:16,079
that isn't fit for purpose. But the violin isn't

93
00:04:16,079 --> 00:04:18,459
inflexible at all. No, it just requires mastery.

94
00:04:18,699 --> 00:04:20,660
A phonograph can only ever play what someone

95
00:04:20,660 --> 00:04:22,759
else has already recorded. Right. It's a closed

96
00:04:22,759 --> 00:04:26,060
system. Exactly. But a violin, once you master

97
00:04:26,060 --> 00:04:30,399
it, can create infinite original music. And most

98
00:04:30,399 --> 00:04:33,079
of the software we interact with daily, you know,

99
00:04:33,079 --> 00:04:34,860
our social media apps, our operating systems,

100
00:04:34,959 --> 00:04:37,500
those are phonographs. They are prepackaged.

101
00:04:37,879 --> 00:04:40,259
Completely prepackaged, closed experiences. You

102
00:04:40,259 --> 00:04:43,379
can only do what the programmer anticipated you

103
00:04:43,379 --> 00:04:45,620
would do. Right. But the programming language

104
00:04:45,620 --> 00:04:49,060
itself, the raw tool of computation, that is

105
00:04:49,060 --> 00:04:51,480
the violin. And Perlis points out that when we

106
00:04:51,480 --> 00:04:53,920
learn to program at this really deep level, we

107
00:04:53,920 --> 00:04:57,000
are learning to build evolving organisms. He

108
00:04:57,000 --> 00:04:59,399
explicitly contrasts languages here, which is

109
00:04:59,399 --> 00:05:02,000
fascinating. Yeah, he says a language like Pascal

110
00:05:02,000 --> 00:05:04,939
is for building pyramids. Right, pyramids. Static,

111
00:05:04,980 --> 00:05:07,740
imposing, rigid structures where every single

112
00:05:07,740 --> 00:05:09,519
block has to be perfectly planned in advance.

113
00:05:09,860 --> 00:05:11,959
But Lisp, which is the language used throughout

114
00:05:11,959 --> 00:05:14,779
the wizard book, is for building organisms. Dynamic,

115
00:05:14,779 --> 00:05:17,459
fluctuating systems. Yeah, systems made of simpler

116
00:05:17,459 --> 00:05:19,459
organisms fitting together, constantly evolving

117
00:05:19,459 --> 00:05:21,360
and reacting to things. And that transition,

118
00:05:21,500 --> 00:05:24,420
you know, from pyramid builder to organism breeder.

119
00:05:25,319 --> 00:05:27,740
That is the core journey of the book. We have

120
00:05:27,740 --> 00:05:30,420
to learn to read the sheet music of this masterpiece.

121
00:05:30,819 --> 00:05:33,000
And that is exactly what our mission is today.

122
00:05:33,240 --> 00:05:36,139
We are going to meticulously go through the core

123
00:05:36,139 --> 00:05:39,360
concepts, chapter by chapter. To uncover the

124
00:05:39,360 --> 00:05:42,319
secrets of how you build these robust, complex

125
00:05:42,319 --> 00:05:46,000
organisms from absolute primitive spells. But,

126
00:05:46,000 --> 00:05:47,759
you know, to build an organism, you can't just

127
00:05:47,759 --> 00:05:50,560
start with a fully formed... arm or leg. No,

128
00:05:50,800 --> 00:05:53,279
you have to understand the cellular biology first.

129
00:05:53,459 --> 00:05:55,980
Right, you need the basic building blocks. Chapter

130
00:05:55,980 --> 00:05:59,240
one. The fundamental spells, the active verbs

131
00:05:59,240 --> 00:06:01,800
of our magical language. The authors argue that

132
00:06:01,800 --> 00:06:04,319
every powerful programming language, no matter

133
00:06:04,319 --> 00:06:07,819
how modern or archaic it is, has to provide three

134
00:06:07,819 --> 00:06:10,560
basic mechanisms. Okay, let's break those down.

135
00:06:10,639 --> 00:06:13,240
First one. First, you need primitive expressions.

136
00:06:13,870 --> 00:06:15,430
These are the simplest entities the language

137
00:06:15,430 --> 00:06:17,730
cares about. So like basic numbers. Exactly.

138
00:06:17,790 --> 00:06:19,769
Basic numbers. That's your raw material. Okay.

139
00:06:19,810 --> 00:06:22,290
What's the second mechanism? Then you need means

140
00:06:22,290 --> 00:06:24,569
of combination. You have to be able to put those

141
00:06:24,569 --> 00:06:26,870
primitive elements together to build compound

142
00:06:26,870 --> 00:06:29,529
elements. So basic math operations. Addition,

143
00:06:29,529 --> 00:06:32,329
multiplication. Right. And finally, the most

144
00:06:32,329 --> 00:06:35,009
important piece of the puzzle, means of abstraction.

145
00:06:35,870 --> 00:06:38,060
Abstraction. That's a heavy word. What does that

146
00:06:38,060 --> 00:06:40,319
mean in this context? It means once you've combined

147
00:06:40,319 --> 00:06:43,360
things into a complex compound element, you need

148
00:06:43,360 --> 00:06:46,199
a way to give it a name and treat it as a single

149
00:06:46,199 --> 00:06:49,600
new unit. Ah, okay. So I don't have to keep repeating

150
00:06:49,600 --> 00:06:52,300
the long, complex combination. I just use the

151
00:06:52,300 --> 00:06:55,379
name. Precisely. Now, the language they use to

152
00:06:55,379 --> 00:06:58,990
explore all of this is a dialect of lisp. called

153
00:06:58,990 --> 00:07:02,430
Scheme. Scheme. And Lisp is ancient in computing

154
00:07:02,430 --> 00:07:05,269
terms. Very ancient. I think only Fortran is

155
00:07:05,269 --> 00:07:07,110
older. It was invented by John McCarthy in the

156
00:07:07,110 --> 00:07:10,750
late 1950s. That's wild. But it survives because

157
00:07:10,750 --> 00:07:13,589
of its radical, fundamental elegance. It really

158
00:07:13,589 --> 00:07:15,449
does. Though I have to admit, looking at Lisp

159
00:07:15,449 --> 00:07:17,389
code for the first time feels like looking at

160
00:07:17,389 --> 00:07:19,750
alien algebra. Oh, it completely departs from

161
00:07:19,750 --> 00:07:21,850
customary mathematical convention. Yeah, Lisp

162
00:07:21,850 --> 00:07:24,540
uses prefix notation. Which... throws everyone

163
00:07:24,540 --> 00:07:26,660
off at first. It totally does, because it means

164
00:07:26,660 --> 00:07:29,240
the operator always comes first. Right. Like,

165
00:07:29,300 --> 00:07:32,180
if you or I want to add 2 and 3, we write 2,

166
00:07:32,279 --> 00:07:35,540
then a plus sign, then 3. But in Lisp, you open

167
00:07:35,540 --> 00:07:38,079
a parenthesis, write the plus sign, then the

168
00:07:38,079 --> 00:07:40,560
2, then the 3, and then you close the parenthesis.

169
00:07:40,779 --> 00:07:43,720
It looks strange to our eyes, but the advantages

170
00:07:43,720 --> 00:07:46,000
are immediate when you actually start writing

171
00:07:46,000 --> 00:07:48,980
real programs. How so? Well, prefix notation

172
00:07:48,980 --> 00:07:51,459
accommodates procedures that take any number

173
00:07:51,459 --> 00:07:53,860
of arguments without any ambiguity whatsoever.

174
00:07:54,240 --> 00:07:55,980
Okay, give me an example. You can write open

175
00:07:55,980 --> 00:08:00,100
parenthesis, plus sign, 21, 35, 12, 7, close

176
00:08:00,100 --> 00:08:02,680
parenthesis. And the machine just adds them all

177
00:08:02,680 --> 00:08:05,899
up. Because the leftmost element inside the parenthesis

178
00:08:05,899 --> 00:08:08,060
is always the action. Exactly. And everything

179
00:08:08,060 --> 00:08:10,220
that follows is just the stuff being acted upon.

180
00:08:10,490 --> 00:08:12,930
It makes sense. But the real power I saw was

181
00:08:12,930 --> 00:08:16,110
how perfectly it nests. Yes. The nesting is where

182
00:08:16,110 --> 00:08:18,490
it gets incredibly powerful. You can put a combination

183
00:08:18,490 --> 00:08:21,610
inside a combination infinitely deep. And that

184
00:08:21,610 --> 00:08:24,629
nested structure becomes explicitly drawn out

185
00:08:24,629 --> 00:08:27,110
by the parentheses. It's so visual. You can write

186
00:08:27,110 --> 00:08:30,670
an expression that means add the result of multiplying

187
00:08:30,670 --> 00:08:33,549
3 and 5 to the result of subtracting 6 from 10.

188
00:08:33,730 --> 00:08:36,029
Right. The parentheses force you to articulate

189
00:08:36,029 --> 00:08:37,929
the exact tree structure of that computation.

190
00:08:38,570 --> 00:08:41,029
It's literally like Russian nesting dolls of

191
00:08:41,029 --> 00:08:43,690
math. That's a perfect analogy. You open the

192
00:08:43,690 --> 00:08:46,809
big doll, the addition, and there are two smaller

193
00:08:46,809 --> 00:08:49,889
operations inside it. The multiplication and

194
00:08:49,889 --> 00:08:52,129
the subtraction. And sitting behind all this

195
00:08:52,129 --> 00:08:55,350
nesting, this endless combination, is the heartbeat

196
00:08:55,350 --> 00:08:57,629
of the list machine. What the authors call the

197
00:08:57,629 --> 00:09:02,690
REPL. R -E -P -L. The Read Evil Printlip. It

198
00:09:02,690 --> 00:09:04,909
is the core cycle of the interpreter. Walk me

199
00:09:04,909 --> 00:09:07,269
through that cycle. So it reads the expression

200
00:09:07,269 --> 00:09:10,740
you typed. That's the R. Okay, reading the text

201
00:09:10,740 --> 00:09:12,960
on the screen. Then it evaluates that expression

202
00:09:12,960 --> 00:09:16,019
to find its meaning. That's the E. The evaluation

203
00:09:16,019 --> 00:09:18,860
part seems like the heavy lifting. It is. Then

204
00:09:18,860 --> 00:09:21,019
it prints the result, the P. And then it loops.

205
00:09:21,080 --> 00:09:23,059
It just sits there waiting for you to type something

206
00:09:23,059 --> 00:09:26,360
else. Exactly. Now, to give us a mental framework

207
00:09:26,360 --> 00:09:28,620
for how this evaluation actually happens under

208
00:09:28,620 --> 00:09:31,559
the hood, the book introduces the substitution

209
00:09:31,559 --> 00:09:34,419
model of evaluation. The substitution model?

210
00:09:35,080 --> 00:09:36,899
Let me see if I have this right, because it feels

211
00:09:36,899 --> 00:09:39,200
a lot like middle school algebra. It is very

212
00:09:39,200 --> 00:09:42,539
similar. So, if I define a procedure called square

213
00:09:42,539 --> 00:09:46,320
that takes a number, let's call it x, and multiplies

214
00:09:46,320 --> 00:09:49,720
x by x. Okay, simple enough. Later, I ask the

215
00:09:49,720 --> 00:09:52,559
machine to evaluate the square of 5. Under the

216
00:09:52,559 --> 00:09:54,820
substitution model, the machine simply takes

217
00:09:54,820 --> 00:09:57,210
the text of my procedure. finds the parameter

218
00:09:57,210 --> 00:10:00,549
x, and swaps in the number 5. Right. So x times

219
00:10:00,549 --> 00:10:04,590
x just literally becomes 5 times 5, which reduces

220
00:10:04,590 --> 00:10:07,190
to 25. You've completely captured the essence

221
00:10:07,190 --> 00:10:09,250
of it. It's just text replacement. Basically.

222
00:10:09,509 --> 00:10:12,909
To evaluate a complex combination, the machine

223
00:10:12,909 --> 00:10:15,809
evaluates all the subexpressions first, and then

224
00:10:15,809 --> 00:10:18,730
applies the operator to those evaluated arguments.

225
00:10:18,929 --> 00:10:21,919
Okay. And if the operator is a custom procedure

226
00:10:21,919 --> 00:10:24,799
you built, like your square function, it literally

227
00:10:24,799 --> 00:10:27,139
substitutes the arguments into the body of your

228
00:10:27,139 --> 00:10:28,860
procedure. It's just a process of continuous

229
00:10:28,860 --> 00:10:30,820
reduction. We're shrinking the equation until

230
00:10:30,820 --> 00:10:33,159
there's only one number left. Exactly. But of

231
00:10:33,159 --> 00:10:35,480
course, the authors throw a massive curveball

232
00:10:35,480 --> 00:10:38,000
early on. Oh, they do. They introduce recursion.

233
00:10:38,299 --> 00:10:40,720
Defining a procedure in terms of itself. Which

234
00:10:40,720 --> 00:10:43,440
is so trippy. If creating a procedure is like

235
00:10:43,440 --> 00:10:46,519
writing a recipe for the machine, recursion feels

236
00:10:46,519 --> 00:10:48,840
like writing a line in the recipe that says,

237
00:10:48,980 --> 00:10:52,279
now go read this exact recipe again. Yes. My

238
00:10:52,279 --> 00:10:54,700
immediate thought was, how does the computer

239
00:10:54,700 --> 00:10:57,860
not just spin out of control into an infinite

240
00:10:57,860 --> 00:11:00,460
loop and eventually just explode? It's a very

241
00:11:00,460 --> 00:11:02,940
common reaction. And the authors use this exact

242
00:11:02,940 --> 00:11:05,200
confusion to draw a really crucial distinction.

243
00:11:05,679 --> 00:11:08,100
Between what? Between the syntax of a procedure,

244
00:11:08,340 --> 00:11:11,409
which is... just how the text is written on the

245
00:11:11,409 --> 00:11:14,289
screen and the shape of the computational process

246
00:11:14,289 --> 00:11:16,789
it actually generates in memory. The syntax versus

247
00:11:16,789 --> 00:11:19,149
the shape of the process. Right. A procedure

248
00:11:19,149 --> 00:11:21,529
might be recursive in its syntax, meaning the

249
00:11:21,529 --> 00:11:24,149
code text literally calls its own name. Okay.

250
00:11:24,629 --> 00:11:27,230
But the process that unfolds in the machine's

251
00:11:27,230 --> 00:11:29,570
memory can be entirely different. Wait, so a

252
00:11:29,570 --> 00:11:31,750
recursive syntax doesn't always mean a recursive

253
00:11:31,750 --> 00:11:33,970
process? Exactly. We can illustrate this with

254
00:11:33,970 --> 00:11:36,309
a classic example from the book, Calculating

255
00:11:36,309 --> 00:11:38,620
a Factorial. Ah, factorials. Remind me how that

256
00:11:38,620 --> 00:11:41,860
works. So the factorial of 6 is 6 times 5 times

257
00:11:41,860 --> 00:11:45,100
4 times 3 times 2 times 1. If we design this

258
00:11:45,100 --> 00:11:48,740
as a recursive process, the machine thinks, to

259
00:11:48,769 --> 00:11:51,809
find the factorial of 6, I need to multiply 6

260
00:11:51,809 --> 00:11:54,610
by the factorial of 5. But it doesn't know the

261
00:11:54,610 --> 00:11:56,970
factorial of 5 yet. Exactly. I don't know the

262
00:11:56,970 --> 00:11:59,009
factorial of 5 yet, so I'll put the multiply

263
00:11:59,009 --> 00:12:01,669
by 6 operation on hold. It leaves the trail of

264
00:12:01,669 --> 00:12:04,860
brokrums. Yes. It pauses its current task. Saves

265
00:12:04,860 --> 00:12:07,000
its place and dives deeper into the rabbit hole.

266
00:12:07,440 --> 00:12:10,139
Now I need the factorial of 4, so put the 5 on

267
00:12:10,139 --> 00:12:12,179
hold. And it builds up this massive chain of

268
00:12:12,179 --> 00:12:14,159
deferred operations. It dives all the way down

269
00:12:14,159 --> 00:12:16,919
to 1. Okay, and then what? Once it hits the bottom,

270
00:12:17,039 --> 00:12:20,179
the 1, it has to walk all the way back up that

271
00:12:20,179 --> 00:12:22,779
chain, executing all those deferred multiplications.

272
00:12:22,840 --> 00:12:24,940
Doing all the math it put on hold. That is a

273
00:12:24,940 --> 00:12:28,340
true recursive process. If shape expands as it

274
00:12:28,340 --> 00:12:31,100
builds the chain and then shrinks as it calculates

275
00:12:31,100 --> 00:12:33,059
the answer. It physically takes up more space.

276
00:12:33,610 --> 00:12:36,129
Crucially, yes. It requires an expanding amount

277
00:12:36,129 --> 00:12:38,149
of memory to keep track of all those deferred

278
00:12:38,149 --> 00:12:40,350
operations. So what's the alternative? How do

279
00:12:40,350 --> 00:12:42,850
you calculate a factorial without leaving that

280
00:12:42,850 --> 00:12:45,330
massive memory -hogging trail of breadcrumbs?

281
00:12:45,590 --> 00:12:48,850
You use an iterative process. You use state variables.

282
00:12:49,110 --> 00:12:51,370
State variables? How does that change the shape?

283
00:12:51,649 --> 00:12:53,730
Instead of deferred operations, you keep a running

284
00:12:53,730 --> 00:12:57,090
product and a counter. Ah, so like step one,

285
00:12:57,309 --> 00:12:59,870
the product is one, the counter is one. Right.

286
00:12:59,929 --> 00:13:02,190
Step two, the product is two, the counter is

287
00:13:02,190 --> 00:13:05,399
two. Step 3. The product is 6, the counter is

288
00:13:05,399 --> 00:13:08,840
3. You are passing the current complete state

289
00:13:08,840 --> 00:13:11,860
to the next step every single time. So the instructions

290
00:13:11,860 --> 00:13:14,580
are just multiply the running product by the

291
00:13:14,580 --> 00:13:16,879
counter, increase the counter, and call the procedure

292
00:13:16,879 --> 00:13:19,779
again. Exactly. Now notice the syntax is still

293
00:13:19,779 --> 00:13:22,299
recursive. The procedure is still calling itself

294
00:13:22,299 --> 00:13:25,720
by name at the end of every step. Oh, I see.

295
00:13:25,879 --> 00:13:29,220
But the process is iterative. At any given moment,

296
00:13:29,610 --> 00:13:32,250
Those two variables contain the entire state

297
00:13:32,250 --> 00:13:35,070
of the computation. So if I tripped over the

298
00:13:35,070 --> 00:13:37,309
power cord and unplugged the machine halfway

299
00:13:37,309 --> 00:13:39,289
through, I wouldn't lose some invisible chain

300
00:13:39,289 --> 00:13:41,330
of deferred operations. No, you wouldn't. I could

301
00:13:41,330 --> 00:13:43,009
just look at the two numbers written on the screen.

302
00:13:43,070 --> 00:13:46,210
Let's say product is 120, counter is 5, plug

303
00:13:46,210 --> 00:13:48,350
the machine back in, and resume perfectly from

304
00:13:48,350 --> 00:13:50,830
that exact spot. That is the defining feature

305
00:13:50,830 --> 00:13:54,009
of an iterative process. It runs in constant

306
00:13:54,009 --> 00:13:57,620
memory space. it never expands. That is brilliant.

307
00:13:58,000 --> 00:14:00,059
And one of the really beautiful things about

308
00:14:00,059 --> 00:14:02,899
Scheme is that its interpreter is specifically

309
00:14:02,899 --> 00:14:06,019
designed to execute these iterative processes

310
00:14:06,019 --> 00:14:08,779
without ever running out of memory, no matter

311
00:14:08,779 --> 00:14:11,019
how many times the procedure calls itself. Okay,

312
00:14:11,059 --> 00:14:13,279
so we have our basic spells. We have numbers,

313
00:14:13,340 --> 00:14:16,399
we can name things, we can nest operations, and

314
00:14:16,399 --> 00:14:18,600
we can write loops that run forever in constant

315
00:14:18,600 --> 00:14:21,139
space. That's a solid foundation. Most people

316
00:14:21,139 --> 00:14:22,600
would stop there and say, great, I can write

317
00:14:22,600 --> 00:14:26,029
software now. But... The wizard book shifts gears

318
00:14:26,029 --> 00:14:28,690
into something called higher order procedures.

319
00:14:29,009 --> 00:14:30,870
And this is where the training wheels completely

320
00:14:30,870 --> 00:14:33,669
come off. Up to this point, our spells have only

321
00:14:33,669 --> 00:14:37,370
acted on raw materials. Just numbers. Nouns.

322
00:14:37,370 --> 00:14:39,990
Right. Yeah. But true abstraction requires more

323
00:14:39,990 --> 00:14:42,830
power than that. Often you'll find the exact

324
00:14:42,830 --> 00:14:45,730
same pattern of computation used in completely

325
00:14:45,730 --> 00:14:48,090
different situations. Give me an example. You

326
00:14:48,090 --> 00:14:49,909
might write a loop to sum the integers from 1

327
00:14:49,909 --> 00:14:53,399
to 10. 1 plus 2 plus 3. Exactly. Then later,

328
00:14:53,519 --> 00:14:55,899
you write a loop to sum the cubes of the integers

329
00:14:55,899 --> 00:14:59,960
from 1 to 10. So 1 cubed plus 2 cubed plus 3

330
00:14:59,960 --> 00:15:02,600
cubed. The structure of the loop is identical.

331
00:15:02,720 --> 00:15:04,899
The addition part is identical. The only thing

332
00:15:04,899 --> 00:15:07,399
that changes is the tiny little action you perform

333
00:15:07,399 --> 00:15:09,559
on the number right before you add it to the

334
00:15:09,559 --> 00:15:11,299
pile. Right. In many conventional languages,

335
00:15:11,600 --> 00:15:13,860
you'd be forced to write two completely separate

336
00:15:13,860 --> 00:15:17,009
procedures for that. Lots of repeated code. But

337
00:15:17,009 --> 00:15:19,710
not in LISP. Not in LISP. In LISP, procedures

338
00:15:19,710 --> 00:15:21,789
are first -class citizens. First -class citizens,

339
00:15:21,909 --> 00:15:24,210
meaning what exactly? They have the exact same

340
00:15:24,210 --> 00:15:26,710
rights and privileges as numbers do. Meaning

341
00:15:26,710 --> 00:15:30,309
I can pass a verb into another verb. I don't

342
00:15:30,309 --> 00:15:32,389
just hand the machine a number. I can hand the

343
00:15:32,389 --> 00:15:34,929
machine a tool. Yes. You can define a general

344
00:15:34,929 --> 00:15:36,769
sum procedure. It takes the starting number,

345
00:15:36,909 --> 00:15:39,389
the ending number, and it takes a third argument,

346
00:15:39,570 --> 00:15:41,610
a procedure that tells it what to do to each

347
00:15:41,610 --> 00:15:44,399
term. So I can pass my cube procedure directly

348
00:15:44,399 --> 00:15:46,919
into the sum procedure as an argument. It's like,

349
00:15:46,960 --> 00:15:50,019
imagine a factory. Instead of giving a factory

350
00:15:50,019 --> 00:15:52,240
worker a pile of wood and telling them to build

351
00:15:52,240 --> 00:15:55,279
a chair, you hand them a pile of wood and you

352
00:15:55,279 --> 00:15:58,440
also hand them a custom built saw and you say,

353
00:15:58,500 --> 00:16:01,559
use whatever saw I hand you on this wood. That's

354
00:16:01,559 --> 00:16:03,379
a great way to visualize it. And then the next

355
00:16:03,379 --> 00:16:05,440
day you hand them the same wood, but a drill

356
00:16:05,440 --> 00:16:09,549
instead of a saw. The factor worker, which is

357
00:16:09,549 --> 00:16:11,929
our sum procedure, doesn't need to know how the

358
00:16:11,929 --> 00:16:13,750
tool works. They just need to know they have

359
00:16:13,750 --> 00:16:16,250
to apply it. The flexibility is just staggering.

360
00:16:16,529 --> 00:16:18,649
You can even write procedures whose entire job

361
00:16:18,649 --> 00:16:21,429
is to manufacture and return brand new procedures.

362
00:16:21,649 --> 00:16:23,889
We are totally blurring the traditional line

363
00:16:23,889 --> 00:16:27,289
between active processes and passive data. A

364
00:16:27,289 --> 00:16:29,909
procedure is no longer just a rigid rule for

365
00:16:29,909 --> 00:16:32,250
manipulating data. It can be tossed around, passed

366
00:16:32,250 --> 00:16:34,809
as an argument, and returned as a result. Just

367
00:16:34,809 --> 00:16:36,590
like a simple number. Which perfectly sets up

368
00:16:36,590 --> 00:16:38,370
the most mind -bending transition in the book.

369
00:16:38,549 --> 00:16:40,929
Chapter two. Yes. Moving on from procedures to

370
00:16:40,929 --> 00:16:43,350
data. Because if procedures are the active verbs

371
00:16:43,350 --> 00:16:45,450
of our language and we can pass verbs around

372
00:16:45,450 --> 00:16:48,549
like nouns, what are the nouns? What actually

373
00:16:48,549 --> 00:16:51,730
is data? This introduces the concept of the illusion

374
00:16:51,730 --> 00:16:55,190
of data. In our journey from basic cells to complex

375
00:16:55,190 --> 00:16:59,269
organisms, we've mastered processes. Now we must

376
00:16:59,269 --> 00:17:01,730
tackle compound data. Because the real world

377
00:17:01,730 --> 00:17:04,670
isn't modeled with isolated numbers. No, we need

378
00:17:04,670 --> 00:17:07,049
to represent complex phenomena with multiple

379
00:17:07,049 --> 00:17:10,450
interconnected parts. And the authors use rational

380
00:17:10,450 --> 00:17:12,990
numbers fractions as their prime example here.

381
00:17:13,210 --> 00:17:15,369
Because a fraction intrinsically has two parts.

382
00:17:15,950 --> 00:17:18,410
A numerator and a denominator. They are glued

383
00:17:18,410 --> 00:17:21,250
together. To manage this complexity, they introduce

384
00:17:21,250 --> 00:17:23,910
the principle of data abstraction. Okay, data

385
00:17:23,910 --> 00:17:26,410
abstraction. The goal is to build a wall, an

386
00:17:26,410 --> 00:17:28,970
abstraction barrier, between how a compound data

387
00:17:28,970 --> 00:17:31,549
object is used and how it is physically constructed

388
00:17:31,549 --> 00:17:33,450
under the hood. It reminds me of a corporate

389
00:17:33,450 --> 00:17:36,559
org chart, honestly. Esso. Imagine the CEO of

390
00:17:36,559 --> 00:17:38,359
the math department wants to add two fractions

391
00:17:38,359 --> 00:17:40,759
together. The CEO shouldn't be digging around

392
00:17:40,759 --> 00:17:42,559
in the basement trying to figure out if the numerator

393
00:17:42,559 --> 00:17:45,400
is stored in filing cabinet A or written on a

394
00:17:45,400 --> 00:17:47,920
sticky note on someone's desk. Right. The CEO

395
00:17:47,920 --> 00:17:49,960
shouldn't care about those details. The CEO just

396
00:17:49,960 --> 00:17:52,279
wants to say, give me the numerator. Exactly.

397
00:17:52,380 --> 00:17:54,819
So we build a constructor procedure. Let's call

398
00:17:54,819 --> 00:17:57,759
it make rat, which takes two numbers and glues

399
00:17:57,759 --> 00:17:59,700
them together into a rational number. OK. And

400
00:17:59,700 --> 00:18:02,299
we build selector procedures, numera and denum,

401
00:18:02,299 --> 00:18:05,609
which. Extract those pieces. And the CEO only

402
00:18:05,609 --> 00:18:08,230
ever talks to Numer and Denim. Right. The CEO's

403
00:18:08,230 --> 00:18:11,490
high -level addition procedure uses those selectors,

404
00:18:11,529 --> 00:18:14,769
does the math, and uses MakeRat to output the

405
00:18:14,769 --> 00:18:17,430
new fraction. And the beauty of this is, if the

406
00:18:17,430 --> 00:18:20,430
IT department goes into the basement a year later

407
00:18:20,430 --> 00:18:24,109
and completely changes how fractions are stored

408
00:18:24,109 --> 00:18:26,579
in the computer's memory. The CEO doesn't have

409
00:18:26,579 --> 00:18:28,500
to change a single line of their high -level

410
00:18:28,500 --> 00:18:31,019
addition code. Deferring decisions about data

411
00:18:31,019 --> 00:18:33,619
representation until the last possible moment

412
00:18:33,619 --> 00:18:37,160
preserves maximum flexibility in your system

413
00:18:37,160 --> 00:18:40,559
architecture. It's so clean. But then the authors

414
00:18:40,559 --> 00:18:43,279
push this concept to an extreme that completely

415
00:18:43,279 --> 00:18:45,839
challenges our fundamental understanding of computation.

416
00:18:46,420 --> 00:18:48,900
They ask a seemingly simple question. What is

417
00:18:48,900 --> 00:18:51,279
meant by data? I remember reading this section

418
00:18:51,279 --> 00:18:53,460
and literally putting the book down to stare

419
00:18:53,460 --> 00:18:55,630
at the wall. It has that effect. Because their

420
00:18:55,630 --> 00:18:57,910
answer is that we don't actually need underlying

421
00:18:57,910 --> 00:19:01,470
data structures to create data. They define data

422
00:19:01,470 --> 00:19:04,049
purely by its behavior. But how is that physically

423
00:19:04,049 --> 00:19:06,390
possible? I mean, if I have a fraction 1 over

424
00:19:06,390 --> 00:19:09,130
2, those numbers 1 and 2 have to live somewhere

425
00:19:09,130 --> 00:19:11,390
in the machine, right? You would think so. If

426
00:19:11,390 --> 00:19:13,589
you don't have a physical data structure, like

427
00:19:13,589 --> 00:19:16,910
a physical box in memory, how does the pair even

428
00:19:16,910 --> 00:19:20,250
exist? Well, in Lisp, the primitive glue used

429
00:19:20,250 --> 00:19:23,210
to pair two things together is called cons, C.

430
00:19:24,920 --> 00:19:28,700
Okay, cons. You cons one, two together. Then

431
00:19:28,700 --> 00:19:30,759
you extract the first part with a command called

432
00:19:30,759 --> 00:19:34,859
car, C -A -R, and the second part with C -D -R

433
00:19:34,859 --> 00:19:38,039
-C -D -R. Weird names, but okay. They are historical

434
00:19:38,039 --> 00:19:41,400
artifacts from early hardware. Normally, we imagine

435
00:19:41,400 --> 00:19:44,000
cons allocating a tiny physical box in memory

436
00:19:44,000 --> 00:19:46,079
with two compartments. Right, a literal container.

437
00:19:46,400 --> 00:19:48,740
But the authors demonstrate the procedural representation

438
00:19:48,740 --> 00:19:53,039
of pairs. They show that cons, car, and CDR can

439
00:19:53,039 --> 00:19:55,220
be implemented using nothing but functions. Okay,

440
00:19:55,279 --> 00:19:56,599
you have to walk me through the mechanics of

441
00:19:56,599 --> 00:19:58,420
that. How does a function act as a pair of numbers?

442
00:19:58,880 --> 00:20:02,000
Imagine cons doesn't build a box at all. When

443
00:20:02,000 --> 00:20:04,319
you call cons and hand it the numbers 1 and 2,

444
00:20:04,539 --> 00:20:06,980
it doesn't store them in memory compartments.

445
00:20:07,039 --> 00:20:10,160
What does it do? Instead, it manufactures and

446
00:20:10,160 --> 00:20:13,660
returns... A brand new procedure. A verb. A verb.

447
00:20:13,799 --> 00:20:16,880
And inside the internal scope of this new verb,

448
00:20:17,079 --> 00:20:20,180
those numbers one and two are trapped. Trapped.

449
00:20:20,240 --> 00:20:22,279
They're suspended in the environment of that

450
00:20:22,279 --> 00:20:25,319
function. This new procedure just sits there,

451
00:20:25,420 --> 00:20:28,039
completely dormant, waiting for you to interact

452
00:20:28,039 --> 00:20:30,319
with it. It's just waiting for a command. Exactly.

453
00:20:30,460 --> 00:20:32,859
When you want the first number, you use car.

454
00:20:33,220 --> 00:20:36,599
But car isn't reaching into a physical box. Car

455
00:20:36,599 --> 00:20:39,319
is actually sending a message to this dormant

456
00:20:39,319 --> 00:20:41,579
procedure. Oh, wow. It sends a message saying,

457
00:20:41,660 --> 00:20:43,799
give me your first item. The doorman procedure

458
00:20:43,799 --> 00:20:46,440
wakes up, hears the message, and replies with

459
00:20:46,440 --> 00:20:48,599
the one. And if you send it the CDR message.

460
00:20:48,819 --> 00:20:51,079
It replies with the two. That is unbelievable.

461
00:20:51,180 --> 00:20:54,019
The data isn't a passive container at all. The

462
00:20:54,019 --> 00:20:57,000
data is an active, intelligent entity that reacts

463
00:20:57,000 --> 00:20:59,059
to messages. This is known as message passing.

464
00:20:59,500 --> 00:21:02,819
And the abstraction barrier is so flawless that

465
00:21:02,819 --> 00:21:05,160
the high -level math program adding those fractions

466
00:21:05,160 --> 00:21:08,039
literally cannot tell if the pair it's interacting

467
00:21:08,039 --> 00:21:11,660
with is a physical block of memory or a cleverly

468
00:21:11,660 --> 00:21:14,099
disguised function intercepting messages. It

469
00:21:14,099 --> 00:21:16,880
destroys the boundary between procedure and data.

470
00:21:17,259 --> 00:21:19,359
Nouns are just verbs waiting to be activated.

471
00:21:19,559 --> 00:21:22,000
It's all just processes all the way down. Indeed.

472
00:21:22,180 --> 00:21:24,980
So, okay, once we have this magical cons glue,

473
00:21:25,140 --> 00:21:27,339
whether it's made of boxes or functions, what

474
00:21:27,339 --> 00:21:30,019
do we actually build with it? We leverage a mathematical

475
00:21:30,019 --> 00:21:32,980
property called closure. Closure. And we should

476
00:21:32,980 --> 00:21:35,319
be careful here. This is mathematical closure,

477
00:21:35,400 --> 00:21:37,599
not the lexical closures we just touched on,

478
00:21:37,660 --> 00:21:40,660
where a function traps variables in its scope.

479
00:21:40,720 --> 00:21:42,660
Right. Different kind of closure. Mathematical

480
00:21:42,660 --> 00:21:44,740
closure means that when you combine two things

481
00:21:44,740 --> 00:21:47,319
using an operation, the result can be combined

482
00:21:47,319 --> 00:21:50,000
again using that exact same operation. Like addition.

483
00:21:50,279 --> 00:21:53,160
If I add two and three, the result is five. And

484
00:21:53,160 --> 00:21:55,500
five is still a number. I can immediately feed

485
00:21:55,500 --> 00:21:58,059
five back into another addition operation. The

486
00:21:58,059 --> 00:22:01,769
system doesn't break. Exactly. Because cons satisfies

487
00:22:01,769 --> 00:22:04,369
the closure property, we don't just have to glue

488
00:22:04,369 --> 00:22:07,509
two raw numbers together. We can take a pair

489
00:22:07,509 --> 00:22:10,150
and glue it to another pair. Okay. And then glue

490
00:22:10,150 --> 00:22:12,509
that resulting structure to a third pair. We

491
00:22:12,509 --> 00:22:15,869
can build sequences, linked lists, deeply branching

492
00:22:15,869 --> 00:22:18,950
trees. We're moving from single cells to complex

493
00:22:18,950 --> 00:22:21,710
tissues now. And we can build higher order procedures

494
00:22:21,710 --> 00:22:24,329
to operate on these massive structures uniformly.

495
00:22:24,690 --> 00:22:26,930
Like what? We can map a function across every

496
00:22:26,930 --> 00:22:29,589
single leaf of a giant tree or filter a list

497
00:22:29,589 --> 00:22:32,309
based on a specific condition or accumulate all

498
00:22:32,309 --> 00:22:35,670
the values into a single result. The book visualizes

499
00:22:35,670 --> 00:22:38,250
this beautifully with a picture language. Oh,

500
00:22:38,309 --> 00:22:40,329
that's a wonderful section. They show how you

501
00:22:40,329 --> 00:22:43,029
can define a basic painter, which is just a procedure

502
00:22:43,029 --> 00:22:45,990
that draws lines inside a frame. Right. But because

503
00:22:45,990 --> 00:22:48,750
of Clojure, you can write an operation that takes

504
00:22:48,750 --> 00:22:51,750
a painter. scales it down to half size, and places

505
00:22:51,750 --> 00:22:54,150
it next to a copy of itself. And the output of

506
00:22:54,150 --> 00:22:57,349
that operation is not just raw pixels on a screen.

507
00:22:57,789 --> 00:23:00,970
It is a new, more complex painter. Which means

508
00:23:00,970 --> 00:23:03,089
you can feed it right back into the same scaling

509
00:23:03,089 --> 00:23:05,430
operation. You can take the two -image painter,

510
00:23:05,670 --> 00:23:08,009
shrink it, and place it next to itself to make

511
00:23:08,009 --> 00:23:12,359
four. Then 16. You can add rotations. The authors

512
00:23:12,359 --> 00:23:15,380
show how you can recreate those mind -bending

513
00:23:15,380 --> 00:23:18,180
MC Escher drawings. Yes, like the one where the

514
00:23:18,180 --> 00:23:20,559
fish interlock and shrink infinitely toward the

515
00:23:20,559 --> 00:23:23,059
center of the page. And you do it using just

516
00:23:23,059 --> 00:23:26,000
a handful of perfectly closed operations. Infinite

517
00:23:26,000 --> 00:23:28,759
complexity arising from very simple rules. It

518
00:23:28,759 --> 00:23:31,319
is a stunning demonstration of abstract modeling.

519
00:23:31,579 --> 00:23:34,960
But, you know, we've built these beautiful, hierarchical,

520
00:23:34,960 --> 00:23:38,230
infinitely complex structures. But there is a

521
00:23:38,230 --> 00:23:40,369
glaring limitation to everything we've done so

522
00:23:40,369 --> 00:23:43,130
far. There is. The Escher drawings are frozen.

523
00:23:43,269 --> 00:23:45,529
They don't move. They are completely timeless.

524
00:23:46,170 --> 00:23:47,950
In the functional programming we've been doing

525
00:23:47,950 --> 00:23:50,609
so far, if you evaluate a mathematical expression,

526
00:23:50,849 --> 00:23:53,809
it always yields the exact same result, regardless

527
00:23:53,809 --> 00:23:55,950
of when you ask. But the real world isn't static.

528
00:23:56,230 --> 00:23:59,009
No. The real world changes over time. If we want

529
00:23:59,009 --> 00:24:01,470
our computational organisms to model a dynamic

530
00:24:01,470 --> 00:24:04,289
world, we have to allow our programs to change.

531
00:24:04,490 --> 00:24:07,190
We have to introduce the concept of state. Chapter

532
00:24:07,190 --> 00:24:10,690
3. State. Meaning an object has state if its

533
00:24:10,690 --> 00:24:13,410
behavior is influenced by its own history. To

534
00:24:13,410 --> 00:24:15,910
achieve this, the author's hand is a dangerous

535
00:24:15,910 --> 00:24:20,490
new spell. Assignment. The set command. S -E

536
00:24:20,490 --> 00:24:23,869
-T exclamation point. Which is a great visual

537
00:24:23,869 --> 00:24:26,569
warning sign in the code. It really is. We are

538
00:24:26,569 --> 00:24:29,730
finally giving our objects local memory. Consider

539
00:24:29,730 --> 00:24:32,190
modeling a bank account. Without assignment,

540
00:24:32,369 --> 00:24:34,869
an account is just a static number. Okay. If

541
00:24:34,869 --> 00:24:37,609
you have an account with $100 and you evaluate

542
00:24:37,609 --> 00:24:40,849
a withdrawal of $10, a purely functional program

543
00:24:40,849 --> 00:24:43,369
doesn't change the account. What does it do?

544
00:24:43,490 --> 00:24:46,430
It takes the $100, subtracts $10, and returns

545
00:24:46,430 --> 00:24:48,690
a brand new, completely separate entity representing

546
00:24:48,690 --> 00:24:53,369
$90. But the original $100 account... Still exists,

547
00:24:53,630 --> 00:24:55,809
perfectly intact in the mathematical ether. It's

548
00:24:55,809 --> 00:24:57,730
like snapping a photograph of the account at

549
00:24:57,730 --> 00:24:59,690
different moments, but the account itself isn't

550
00:24:59,690 --> 00:25:02,920
a living thing. Right. But with Set... We can

551
00:25:02,920 --> 00:25:05,059
change the value of the balance variable in place.

552
00:25:05,240 --> 00:25:07,079
We can tell the machine, update the balance to

553
00:25:07,079 --> 00:25:08,900
be whatever the balance used to be, minus 10.

554
00:25:09,160 --> 00:25:11,319
And the account now remembers its new balance.

555
00:25:11,619 --> 00:25:14,240
It creates a much more intuitive model of the

556
00:25:14,240 --> 00:25:17,519
real world. But the cost is devastating, isn't

557
00:25:17,519 --> 00:25:22,420
it? It is. By introducing set, we permanently

558
00:25:22,420 --> 00:25:25,940
lose. a mathematical property called referential

559
00:25:25,940 --> 00:25:29,000
transparency. Let's ground that term. What does

560
00:25:29,000 --> 00:25:31,759
referential transparency actually mean for the

561
00:25:31,759 --> 00:25:34,119
programmer typing at the keyboard? It means you

562
00:25:34,119 --> 00:25:36,819
can substitute a procedure call with its calculated

563
00:25:36,819 --> 00:25:39,480
value without changing how the program behaves.

564
00:25:39,759 --> 00:25:42,000
Okay, give me an example. In our old, timeless

565
00:25:42,000 --> 00:25:45,559
world, if you evaluate square 5, it is always

566
00:25:45,559 --> 00:25:47,759
25. You could literally go through your code,

567
00:25:47,920 --> 00:25:51,500
erase square 5, write 25 in pencil, and the program

568
00:25:51,500 --> 00:25:53,519
would run identically. But with our new bank

569
00:25:53,519 --> 00:25:55,700
account, that's impossible. Completely impossible.

570
00:25:55,940 --> 00:25:58,599
If I run withdraw 10, the result depends on time.

571
00:25:58,740 --> 00:26:00,700
The first time I run it, it might return 90.

572
00:26:00,839 --> 00:26:02,940
If I run the exact same line of code a second

573
00:26:02,940 --> 00:26:05,680
time, it returns 80. Because the underlying variables

574
00:26:05,680 --> 00:26:07,880
are mutating. I can't replace the code with a

575
00:26:07,880 --> 00:26:10,240
fixed number. We are no longer doing pure mathematics.

576
00:26:10,440 --> 00:26:12,720
We have let the demon of time into our machine.

577
00:26:13,019 --> 00:26:16,200
We broke the x -ray machine. We can't just trace

578
00:26:16,200 --> 00:26:18,839
the code on a piece of paper easily anymore using

579
00:26:18,839 --> 00:26:21,859
the substitution model. So how do the authors...

580
00:26:22,089 --> 00:26:25,170
Fix our mental model because substitution is

581
00:26:25,170 --> 00:26:27,890
dead. They replace substitution with a much heavier

582
00:26:27,890 --> 00:26:30,210
framework, the environment model of evaluation.

583
00:26:30,710 --> 00:26:33,250
I remember this requiring a lot of physical drawing

584
00:26:33,250 --> 00:26:36,589
of boxes and arrows on paper to understand. It

585
00:26:36,589 --> 00:26:38,950
requires visualizing environments, which are

586
00:26:38,950 --> 00:26:41,210
sequences of frames. Frame. Think of a frame

587
00:26:41,210 --> 00:26:44,250
as a secure physical box in the computer's memory.

588
00:26:44,369 --> 00:26:47,250
Inside this box is a ledger that holds the associations

589
00:26:47,250 --> 00:26:50,299
between variable names and their current. mutable

590
00:26:50,299 --> 00:26:52,339
values. So when I call my bank account procedure,

591
00:26:52,599 --> 00:26:54,799
the machine doesn't just substitute text anymore.

592
00:26:55,000 --> 00:26:57,740
No. It creates a new sealed box, a new frame.

593
00:26:58,119 --> 00:27:01,059
Inside that box, it writes down balance equals

594
00:27:01,059 --> 00:27:03,559
100. Exactly. And the withdraw procedure lives

595
00:27:03,559 --> 00:27:05,720
inside this specific environment. So when it

596
00:27:05,720 --> 00:27:08,759
uses set, it reaches into its specific box and

597
00:27:08,759 --> 00:27:11,440
crosses out the 100 and writes 90. And because

598
00:27:11,440 --> 00:27:14,099
that frame is encapsulated, no other part of

599
00:27:14,099 --> 00:27:16,680
the massive program can accidentally see or mutate

600
00:27:16,680 --> 00:27:19,470
that specific balance variable. It gives us true

601
00:27:19,470 --> 00:27:22,829
modularity. But modularity with state leads us

602
00:27:22,829 --> 00:27:26,150
into a terrifying thicket. It really does. Because

603
00:27:26,150 --> 00:27:28,430
in the real world, things don't happen one by

604
00:27:28,430 --> 00:27:31,890
one in a neat, orderly line. Things happen simultaneously.

605
00:27:32,490 --> 00:27:35,230
Concurrency. If we model the world with independent

606
00:27:35,230 --> 00:27:38,069
objects that have internal state, What happens

607
00:27:38,069 --> 00:27:40,650
when multiple processes try to interact with

608
00:27:40,650 --> 00:27:44,049
the exact same object at the exact same moment?

609
00:27:44,170 --> 00:27:46,349
The classic race condition. Imagine you and I

610
00:27:46,349 --> 00:27:49,369
share a joint built account with $100. I go to

611
00:27:49,369 --> 00:27:51,650
an ATM in New York and you go to an ATM in London.

612
00:27:51,849 --> 00:27:54,950
At the exact same millisecond, we both hit the

613
00:27:54,950 --> 00:27:58,049
button to withdraw $10. If the timing is perfectly

614
00:27:58,049 --> 00:28:00,390
misaligned, the consequences are disastrous.

615
00:28:00,809 --> 00:28:02,890
Walk me through the disaster. The New York ATM

616
00:28:02,890 --> 00:28:06,000
reads the balance. 100. A millisecond later,

617
00:28:06,039 --> 00:28:08,460
before New York updates anything, the London

618
00:28:08,460 --> 00:28:11,559
ATM reads the balance. 100. Okay, so New York

619
00:28:11,559 --> 00:28:13,740
calculates the new balance as 90, and London

620
00:28:13,740 --> 00:28:16,119
calculates the new balance as 90. New York writes

621
00:28:16,119 --> 00:28:19,200
90 into the shared frame. A microsecond later,

622
00:28:19,420 --> 00:28:22,539
London overwrites that 90 with its own 90. Oh,

623
00:28:22,539 --> 00:28:26,319
wow. So we collectively withdrew $20, but the

624
00:28:26,319 --> 00:28:29,359
bank's internal state only reflects a $10 withdrawal.

625
00:28:29,779 --> 00:28:32,839
Free money for us, bad news for the bank. The

626
00:28:32,839 --> 00:28:35,299
state variable has been corrupted by time. To

627
00:28:35,299 --> 00:28:37,640
solve this, you have to introduce incredibly

628
00:28:37,640 --> 00:28:40,380
tedious traffic control mechanisms. You hear

629
00:28:40,380 --> 00:28:42,960
programmers talk about mutexes. Mutually exclusive

630
00:28:42,960 --> 00:28:45,559
locks. Right. It's like the physical key attached

631
00:28:45,559 --> 00:28:48,099
to a giant hubcap for a gas station bathroom.

632
00:28:48,339 --> 00:28:50,920
That's a good visual. If the New York ATM has

633
00:28:50,920 --> 00:28:53,799
the key, it locks the bathroom door. It goes

634
00:28:53,799 --> 00:28:56,279
in, reads the balance, does the math, writes

635
00:28:56,279 --> 00:28:58,480
the new balance, walks out, and hands the key

636
00:28:58,480 --> 00:29:01,640
to London. But managing those locks across thousands

637
00:29:01,640 --> 00:29:04,900
of interacting processes, ensuring processes

638
00:29:04,900 --> 00:29:08,920
don't freeze while waiting for a key, is notoriously

639
00:29:08,920 --> 00:29:11,279
difficult. It introduces immense complexity.

640
00:29:11,700 --> 00:29:13,799
Which is why I was so struck by the author's

641
00:29:13,799 --> 00:29:16,640
alternative. Yes. After spending pages showing

642
00:29:16,640 --> 00:29:18,980
how complex time and state are, they introduce

643
00:29:18,980 --> 00:29:21,740
a paradigm shift. They ask, what if we just didn't

644
00:29:21,740 --> 00:29:24,059
use time? They introduce streams. This felt like

645
00:29:24,059 --> 00:29:26,799
stepping completely outside of reality. How do

646
00:29:26,799 --> 00:29:30,059
streams work? A stream is a data structure that

647
00:29:30,059 --> 00:29:33,059
represents the entire history of a variable past,

648
00:29:33,279 --> 00:29:36,420
present, and future as a single continuous mathematical

649
00:29:36,420 --> 00:29:39,180
list. Okay, wait. Instead of updating a single

650
00:29:39,180 --> 00:29:41,720
balanced variable step -by -step using assignments,

651
00:29:42,019 --> 00:29:44,759
a stream models the bank account as an unbroken

652
00:29:44,759 --> 00:29:48,000
sequence of balances. Yes. But how can a computer

653
00:29:48,000 --> 00:29:51,039
store the future? A bank account could theoretically

654
00:29:51,039 --> 00:29:54,170
have an infinite number of transactions. If I

655
00:29:54,170 --> 00:29:56,549
ask the computer to build an infinite list, it

656
00:29:56,549 --> 00:29:59,130
will instantly run out of memory and crash. It

657
00:29:59,130 --> 00:30:00,829
avoids crashing through the magic of delayed

658
00:30:00,829 --> 00:30:03,609
evaluation. Delayed evaluation. A stream looks

659
00:30:03,609 --> 00:30:06,130
and feels exactly like the list we built earlier,

660
00:30:06,329 --> 00:30:08,630
but it doesn't compute all its elements at once.

661
00:30:08,750 --> 00:30:10,829
When you create a stream, it computes the first

662
00:30:10,829 --> 00:30:13,069
element and then packages the rest of the computation

663
00:30:13,069 --> 00:30:16,970
into a func. Let's ground func. What is it mechanically?

664
00:30:17,130 --> 00:30:19,650
A func is simply a procedure with no arguments

665
00:30:19,650 --> 00:30:21,190
that hasn't been called yet. It's a promise.

666
00:30:21,680 --> 00:30:24,420
Exactly. It's a promise. The stream says, here

667
00:30:24,420 --> 00:30:27,819
is your first balance, and here is a sealed envelope.

668
00:30:28,140 --> 00:30:30,799
Inside the envelope are the instructions to calculate

669
00:30:30,799 --> 00:30:33,720
the next balance, but do not open it until you

670
00:30:33,720 --> 00:30:36,220
actually need it. It's extremely lazy evaluation.

671
00:30:36,380 --> 00:30:39,240
It only works when forced. Exactly. Because of

672
00:30:39,240 --> 00:30:42,099
this laziness, you can write programs that manipulate

673
00:30:42,099 --> 00:30:44,920
infinite series. You can write a stream that

674
00:30:44,920 --> 00:30:46,660
represents all the prime numbers in the universe.

675
00:30:46,799 --> 00:30:49,539
And the machine doesn't crash? No. Because it

676
00:30:49,539 --> 00:30:51,740
only calculates the very first prime number two

677
00:30:51,740 --> 00:30:54,140
and hands you an envelope containing the recipe

678
00:30:54,140 --> 00:30:56,299
for finding the next one. And then? You ask for

679
00:30:56,299 --> 00:30:58,440
the next prime, it opens the envelope, finds

680
00:30:58,440 --> 00:31:00,960
the three, and hands you a new envelope. It turns

681
00:31:00,960 --> 00:31:04,779
the crank exactly one time. But how does this

682
00:31:04,779 --> 00:31:07,640
save us from the horrors of the race condition

683
00:31:07,640 --> 00:31:10,900
at the APM? Because there is no set. Nothing

684
00:31:10,900 --> 00:31:14,059
is ever being overwritten or mutated. The stream

685
00:31:14,059 --> 00:31:16,640
of bank balances is just a pure mathematical

686
00:31:16,640 --> 00:31:19,339
function of the stream of withdrawal requests.

687
00:31:19,819 --> 00:31:22,359
We have recovered referential transparency. Yes.

688
00:31:22,480 --> 00:31:25,180
We've completely bypassed time. Time is no longer

689
00:31:25,180 --> 00:31:27,319
a sequence of mutating variables that we have

690
00:31:27,319 --> 00:31:29,900
to lock and protect. Time is just treated as

691
00:31:29,900 --> 00:31:32,599
another spatial dimension laid out completely

692
00:31:32,599 --> 00:31:35,319
flat like a strip of film. We can look at the

693
00:31:35,319 --> 00:31:37,779
whole history at once. It's a God's eye view

694
00:31:37,779 --> 00:31:40,400
of computation. But the authors don't stop at

695
00:31:40,400 --> 00:31:42,809
giving us a God's eye view. They take us to the

696
00:31:42,809 --> 00:31:46,150
logical extreme of procedural epistemology. They

697
00:31:46,150 --> 00:31:48,589
show us how to become gods ourselves. Chapter

698
00:31:48,589 --> 00:31:51,890
4. This takes us into the realm of metalinguistic

699
00:31:51,890 --> 00:31:54,930
abstraction. Metalinguistic. We've bent time.

700
00:31:55,130 --> 00:31:58,430
We've turned data into active procedures. But

701
00:31:58,430 --> 00:32:00,710
all of this has been within the confines of the

702
00:32:00,710 --> 00:32:03,470
scheme language. What if the language itself

703
00:32:03,470 --> 00:32:06,240
isn't suited to our problem? The authors argue

704
00:32:06,240 --> 00:32:08,619
that the most fundamental idea in programming

705
00:32:08,619 --> 00:32:11,500
is that the evaluator, the complex software that

706
00:32:11,500 --> 00:32:13,740
reads your code and figures out what it means,

707
00:32:13,859 --> 00:32:16,779
is just another program. An evaluator is just

708
00:32:16,779 --> 00:32:19,299
an interpreter. And they are saying we can write

709
00:32:19,299 --> 00:32:22,480
a Lisp interpreter using Lisp code to run Lisp

710
00:32:22,480 --> 00:32:24,960
programs. It is called a metacircular evaluator.

711
00:32:25,039 --> 00:32:27,160
And its core mechanism is a beautiful endless

712
00:32:27,160 --> 00:32:30,660
cycle between two procedures, evil and apply.

713
00:32:31,130 --> 00:32:33,369
The two words on the sorcerer's sphere from the

714
00:32:33,369 --> 00:32:35,670
book's cover. The yin and yang of computation.

715
00:32:36,150 --> 00:32:38,769
Okay, how do they interact? Evil's job is to

716
00:32:38,769 --> 00:32:41,569
take a raw expression and the current environment

717
00:32:41,569 --> 00:32:45,049
and reduce it to a procedure and a set of arguments.

718
00:32:45,250 --> 00:32:47,849
It preps everything. Right. It then hands those

719
00:32:47,849 --> 00:32:51,509
over to Apply. Apply takes that procedure, strips

720
00:32:51,509 --> 00:32:54,109
away the abstraction barrier, and reduces it

721
00:32:54,109 --> 00:32:56,730
to a new, simpler set of expressions. And what

722
00:32:56,730 --> 00:32:58,930
does it do with those? It hands those new expressions

723
00:32:58,930 --> 00:33:03,269
back to Evil. Evil to apply, apply to evil, cycling

724
00:33:03,269 --> 00:33:06,069
downward, unpacking the Russian nesting dolls

725
00:33:06,069 --> 00:33:08,750
until it hits the absolute primitive bedrock

726
00:33:08,750 --> 00:33:11,609
basic numbers and built -in hardware math. The

727
00:33:11,609 --> 00:33:13,769
philosophical implication is what Alan Turing

728
00:33:13,769 --> 00:33:16,509
proved decades ago. The concept of a universal

729
00:33:16,509 --> 00:33:19,130
machine. A universal machine. Our evaluator is

730
00:33:19,130 --> 00:33:22,569
a program, but it acts as a machine. If we feed

731
00:33:22,569 --> 00:33:25,009
this evaluator a set of rules describing a different

732
00:33:25,009 --> 00:33:27,430
programming language, our evaluator configures

733
00:33:27,430 --> 00:33:29,329
itself to emulate that other language. It is

734
00:33:29,329 --> 00:33:31,490
a master blueprint that can fold itself into

735
00:33:31,490 --> 00:33:33,609
any machine you can imagine. Exactly. But my

736
00:33:33,609 --> 00:33:35,829
immediate pushback when reading this was, why?

737
00:33:36,210 --> 00:33:38,609
Why go to the immense trouble of building a scheme

738
00:33:38,609 --> 00:33:41,230
evaluator inside scheme? I already have scheme.

739
00:33:41,589 --> 00:33:43,890
If your goal is just to run standard scheme,

740
00:33:44,130 --> 00:33:47,470
it is redundant. But the power of metalinguistic

741
00:33:47,470 --> 00:33:51,440
abstraction is linguistic experimentation. If

742
00:33:51,440 --> 00:33:54,339
you own the evaluator, you are no longer bound

743
00:33:54,339 --> 00:33:57,440
by the rules of the language designers. You can

744
00:33:57,440 --> 00:33:59,680
change the fundamental laws of physics of your

745
00:33:59,680 --> 00:34:02,880
universe. We can invent entirely new programming

746
00:34:02,880 --> 00:34:06,180
paradigms. The book demonstrates this by tweaking

747
00:34:06,180 --> 00:34:09,159
the evaluator to perform lazy evaluation globally.

748
00:34:09,460 --> 00:34:11,219
Like the streams we just talked about. Right.

749
00:34:11,300 --> 00:34:14,960
In standard scheme, arguments are evaluated eagerly

750
00:34:14,960 --> 00:34:17,769
before being passed to a function. But with a

751
00:34:17,769 --> 00:34:20,670
few changes to our evil procedure, we can force

752
00:34:20,670 --> 00:34:23,030
the language to wrap every single argument in

753
00:34:23,030 --> 00:34:25,590
a thunk, a delayed promise. So instead of streams

754
00:34:25,590 --> 00:34:28,190
being this special, weird data structure we have

755
00:34:28,190 --> 00:34:30,570
to build manually, the entire language becomes

756
00:34:30,570 --> 00:34:33,510
lazy by default. We've invented a normal order

757
00:34:33,510 --> 00:34:35,590
language. But they push it even further. They

758
00:34:35,590 --> 00:34:38,510
introduce a language extension called non -deterministic

759
00:34:38,510 --> 00:34:42,309
computing. Using an operator called AM. A -M

760
00:34:42,309 --> 00:34:45,110
-B. The AM operator. I have to say, this was

761
00:34:45,110 --> 00:34:47,510
the moment my brain officially melted. It stands

762
00:34:47,510 --> 00:34:50,730
for ambiguous. And it is a phenomenal paradigm

763
00:34:50,730 --> 00:34:54,530
shift. The AM operator essentially implements

764
00:34:54,530 --> 00:34:57,210
the multiverse theory of quantum mechanics into

765
00:34:57,210 --> 00:34:59,289
your code. It really does. You give it a set

766
00:34:59,289 --> 00:35:02,289
of choices. For example, you say X is the AM

767
00:35:02,289 --> 00:35:05,630
of 1, 2, and 3. Normal language would throw an

768
00:35:05,630 --> 00:35:07,849
error. Or maybe just pick the first one. But

769
00:35:07,849 --> 00:35:10,469
the AM evaluator doesn't pick. No. It literally

770
00:35:10,469 --> 00:35:12,889
splits the execution into parallel realities.

771
00:35:13,190 --> 00:35:16,010
It explores the universe where X is 1, the universe

772
00:35:16,010 --> 00:35:19,150
where X is 2, and the universe where X is 3 simultaneously.

773
00:35:19,469 --> 00:35:22,909
And you combine this branching multiverse with

774
00:35:22,909 --> 00:35:25,429
constraints. You assert that a certain condition

775
00:35:25,429 --> 00:35:28,889
must be true in a valid universe. The book uses

776
00:35:28,889 --> 00:35:31,050
logic puzzles to show this, like the multiple

777
00:35:31,050 --> 00:35:33,269
dwelling problem. Oh, that's a classic. Five

778
00:35:33,269 --> 00:35:35,250
people live on five different floors of an apartment.

779
00:35:35,599 --> 00:35:37,840
And you have a list of arbitrary rules. Baker

780
00:35:37,840 --> 00:35:39,800
doesn't live on the top floor. Cooper doesn't

781
00:35:39,800 --> 00:35:42,199
live on the bottom floor. Fletcher isn't adjacent

782
00:35:42,199 --> 00:35:44,599
to Smith. In a normal language, you would have

783
00:35:44,599 --> 00:35:48,619
to write massive, complex, nested loops to test

784
00:35:48,619 --> 00:35:51,400
every single combination. But with AM, you don't

785
00:35:51,400 --> 00:35:53,780
write the search algorithm at all. You simply

786
00:35:53,780 --> 00:35:56,619
state the choices. Each person lives on an AM

787
00:35:56,619 --> 00:35:58,679
of floors one through five. And then you list

788
00:35:58,679 --> 00:36:01,179
the constraints. And the language does the searching.

789
00:36:01,340 --> 00:36:04,059
If the evaluator explores a universe where Baker

790
00:36:04,059 --> 00:36:06,440
is on the top floor. It hits your constraint,

791
00:36:06,639 --> 00:36:10,280
realizes this reality is invalid, and that universe

792
00:36:10,280 --> 00:36:13,980
dies. But here's the critical part. How does

793
00:36:13,980 --> 00:36:17,179
it keep searching without losing its mind? Right.

794
00:36:17,280 --> 00:36:19,480
How does it track all these dead universes? It

795
00:36:19,480 --> 00:36:21,760
uses continuations. Continuation. A continuation

796
00:36:21,760 --> 00:36:24,280
is essentially a highly advanced save state.

797
00:36:24,480 --> 00:36:27,340
Every time the AM operator faces a choice, the

798
00:36:27,340 --> 00:36:29,639
evaluator takes a complete snapshot of the entire

799
00:36:29,639 --> 00:36:32,380
program state. and saves it. Okay. When a constraint

800
00:36:32,380 --> 00:36:35,039
fails and a universe dies, the evaluator simply

801
00:36:35,039 --> 00:36:37,920
backtracks. It hits load game on the most recent

802
00:36:37,920 --> 00:36:41,000
save state, rewinds time, and takes the next

803
00:36:41,000 --> 00:36:42,880
available fork in the road. It automatically

804
00:36:42,880 --> 00:36:45,400
collapses the multiverse until it finds the one

805
00:36:45,400 --> 00:36:48,179
reality where all constraints are satisfied and

806
00:36:48,179 --> 00:36:50,360
it prints the answer. You just described the

807
00:36:50,360 --> 00:36:52,659
rules of the puzzle and the language solves it.

808
00:36:52,739 --> 00:36:55,260
It feels like pure magic. They conclude this

809
00:36:55,260 --> 00:36:57,639
metalinguistic journey with an even more radical

810
00:36:57,639 --> 00:37:01,239
departure. logic programming. We abandon the

811
00:37:01,239 --> 00:37:04,159
imperative how to computing entirely and return

812
00:37:04,159 --> 00:37:06,800
to declarative what is knowledge. We build an

813
00:37:06,800 --> 00:37:09,179
engine where we just declare relationships. We

814
00:37:09,179 --> 00:37:11,260
tell it John is a father of Mary and Mary is

815
00:37:11,260 --> 00:37:13,480
the mother of Susan. Then we ask who is the grandfather

816
00:37:13,480 --> 00:37:16,099
of Susan. And the engine uses mathematical logic,

817
00:37:16,260 --> 00:37:18,400
pattern matching, and deduction to figure out

818
00:37:18,400 --> 00:37:21,090
the answer. We have traveled from simple numbers

819
00:37:21,090 --> 00:37:24,969
to state -driven objects to lazy streams to multiverses

820
00:37:24,969 --> 00:37:28,230
to automated logic engines. But this raises the

821
00:37:28,230 --> 00:37:30,530
ultimate grounding question. Yes, it does. All

822
00:37:30,530 --> 00:37:33,070
these infinite multiverses, these deferred promises,

823
00:37:33,329 --> 00:37:36,670
these environments, they are beautiful abstractions.

824
00:37:36,809 --> 00:37:39,289
But eventually, these abstract thoughts have

825
00:37:39,289 --> 00:37:41,719
to run on physical atoms. They have to hit a

826
00:37:41,719 --> 00:37:44,900
processor made of silicon and copper wire. Which

827
00:37:44,900 --> 00:37:47,460
brings us to the final descent of the book, stripping

828
00:37:47,460 --> 00:37:50,059
away the magic to find the mechanics. Chapter

829
00:37:50,059 --> 00:37:52,059
5 explores computing with register machines.

830
00:37:52,519 --> 00:37:54,599
We drop out of the clouds of philosophy and hit

831
00:37:54,599 --> 00:37:57,960
the bare metal. We are no longer talking about

832
00:37:57,960 --> 00:38:00,460
environments or infinite timelines. We are talking

833
00:38:00,460 --> 00:38:03,500
about physical boxes. A register machine is a

834
00:38:03,500 --> 00:38:05,840
model of a computer at the hardware level. What

835
00:38:05,840 --> 00:38:07,840
are the components? The boxes are registers'

836
00:38:08,139 --> 00:38:10,940
tiny physical memory banks that hold sequences

837
00:38:10,940 --> 00:38:14,079
of bits. Okay, memory banks. The data paths are

838
00:38:14,079 --> 00:38:15,940
the physical wires connecting these registers,

839
00:38:16,199 --> 00:38:18,519
allowing bits to flow from one box to another.

840
00:38:18,820 --> 00:38:21,099
And how do they know when to flow? The controllers

841
00:38:21,099 --> 00:38:23,619
are the physical sequences of buttons that dictate

842
00:38:23,619 --> 00:38:26,320
which wires are active at any given microsecond.

843
00:38:26,500 --> 00:38:28,760
The authors show us how to build a theoretical

844
00:38:28,760 --> 00:38:31,639
machine to compute the greatest common divisor,

845
00:38:31,659 --> 00:38:35,539
the GCD, of two numbers. And it is breathtakingly

846
00:38:35,539 --> 00:38:38,679
granular. You define register A, register B,

847
00:38:38,860 --> 00:38:41,719
and a temporary register T. You literally write

848
00:38:41,719 --> 00:38:44,039
out the step -by -step controller sequences.

849
00:38:44,400 --> 00:38:46,360
The controller checks a physical sensor to see

850
00:38:46,360 --> 00:38:49,599
if register B is zero. If the sensor is triggered,

851
00:38:49,780 --> 00:38:52,619
it opens a gate to route the contents of register

852
00:38:52,619 --> 00:38:55,639
A to the output. And if it's not zero? If the

853
00:38:55,639 --> 00:38:58,199
sensor is not triggered, it calculates the remainder.

854
00:38:59,070 --> 00:39:02,329
opens a gate to move B into A, opens another

855
00:39:02,329 --> 00:39:05,010
gate to move the remainder into B, and loops

856
00:39:05,010 --> 00:39:07,110
back to the start. It just switches opening and

857
00:39:07,110 --> 00:39:09,610
closing. It's so mechanical. But reading this,

858
00:39:09,690 --> 00:39:12,269
I hit a massive intellectual roadblock. What

859
00:39:12,269 --> 00:39:14,489
was it? It bridges all the way back to our very

860
00:39:14,489 --> 00:39:17,530
first discussion about recursion. Ah, the problem

861
00:39:17,530 --> 00:39:20,309
of recursion on finite hardware. Exactly. If

862
00:39:20,309 --> 00:39:23,190
a register machine only has a fixed physical

863
00:39:23,190 --> 00:39:27,190
set of boxes, let's say, 10 registers soldered

864
00:39:27,190 --> 00:39:29,730
onto a motherboard, how on earth does it compute

865
00:39:29,730 --> 00:39:32,789
a recursive process? We established that a recursive

866
00:39:32,789 --> 00:39:35,130
process, like calculating a complex factorial

867
00:39:35,130 --> 00:39:38,190
or traversing a giant tree, builds up an expanding

868
00:39:38,190 --> 00:39:41,050
chain of deferred operations. It needs an exponentially

869
00:39:41,050 --> 00:39:43,489
growing amount of memory to remember its past

870
00:39:43,489 --> 00:39:46,289
states. How can a machine with only 10 physical

871
00:39:46,289 --> 00:39:49,369
boxes remember an infinitely deep recursive chain?

872
00:39:49,690 --> 00:39:51,650
This is the moment the hardware magic is revealed.

873
00:39:52,050 --> 00:39:54,050
To execute recursion, the physical machine relies

874
00:39:54,050 --> 00:39:56,690
on one critical hardware component. The stack.

875
00:39:56,889 --> 00:39:59,409
The stack. The stack is what provides the illusion

876
00:39:59,409 --> 00:40:01,650
of infinite memory. How does it work? It is a

877
00:40:01,650 --> 00:40:04,090
specialized memory structure that operates on

878
00:40:04,090 --> 00:40:06,630
a last -in, first -out principle. Think of a

879
00:40:06,630 --> 00:40:09,610
physical stack of heavy plates at a buffet. Right,

880
00:40:09,650 --> 00:40:11,269
the ones that are pushed down into a spring.

881
00:40:11,489 --> 00:40:13,769
You push a new plate onto the top of the pile,

882
00:40:13,849 --> 00:40:16,269
pushing the others down. When you need a plate,

883
00:40:16,349 --> 00:40:19,230
you pop the top one off. So how does the register

884
00:40:19,230 --> 00:40:22,190
machine use this to solve recursion? When the

885
00:40:22,190 --> 00:40:24,630
machine is calculating a factorial and realizes

886
00:40:24,630 --> 00:40:27,110
it needs to dive into a recursive subproblem,

887
00:40:27,429 --> 00:40:29,929
it knows it can't just overwrite its current

888
00:40:29,929 --> 00:40:33,289
registers. It needs those specific values later

889
00:40:33,289 --> 00:40:36,329
to finish the math. So before it dives down the

890
00:40:36,329 --> 00:40:38,269
rabbit hole, it saves the current contents of

891
00:40:38,269 --> 00:40:40,349
its registers by physically pushing them onto

892
00:40:40,349 --> 00:40:42,510
the top of the stack. It sets its current reality

893
00:40:42,510 --> 00:40:45,469
aside. It safely packs its bags. Having secured

894
00:40:45,469 --> 00:40:48,329
its state on the stack, It is now free to overwrite

895
00:40:48,329 --> 00:40:50,869
its main registers to solve the smaller subproblem.

896
00:40:51,030 --> 00:40:53,349
It dives deeper. Pushing new states onto the

897
00:40:53,349 --> 00:40:55,730
stack every time it recurses. Okay, so it gets

898
00:40:55,730 --> 00:40:58,349
to the bottom, solves the base case. Then what?

899
00:40:58,530 --> 00:41:01,949
It needs to resume its previous work. So it pops

900
00:41:01,949 --> 00:41:04,530
the top values off the stack, loading them back

901
00:41:04,530 --> 00:41:07,050
into the main registers. Restoring its previous

902
00:41:07,050 --> 00:41:10,250
state perfectly. It restores, does the deferred

903
00:41:10,250 --> 00:41:12,829
multiplication, pops the next state, restores,

904
00:41:12,829 --> 00:41:15,010
multiplies, all the way back to the surface.

905
00:41:15,250 --> 00:41:17,550
The stack is the physical bridge that allows

906
00:41:17,550 --> 00:41:21,090
finite silicon hardware to execute infinitely

907
00:41:21,090 --> 00:41:25,110
deep abstract recursive processes. It's the secret

908
00:41:25,110 --> 00:41:27,289
sauce that makes the math possible in the real

909
00:41:27,289 --> 00:41:30,050
world. And then for the grand finale of the entire

910
00:41:30,050 --> 00:41:32,869
book, the authors pull off an incredible synthesis.

911
00:41:33,230 --> 00:41:35,650
The explicit control evaluator. They literally

912
00:41:35,650 --> 00:41:37,989
take the metacircular evaluator we built in Chapter

913
00:41:37,989 --> 00:41:41,590
4, that massive Lisp -in -List program with its

914
00:41:41,590 --> 00:41:43,809
eval -apply cycle and its complex environment

915
00:41:43,809 --> 00:41:46,510
frames, and they translate the entire thing into

916
00:41:46,510 --> 00:41:48,750
the granular register machine instructions of

917
00:41:48,750 --> 00:41:51,250
Chapter 5. They bridge the gap between high -level

918
00:41:51,250 --> 00:41:54,289
Lisp and raw machine code. They design a physical

919
00:41:54,289 --> 00:41:56,949
register machine whose sole dedicated purpose

920
00:41:56,949 --> 00:42:00,030
is to read, evaluate, and execute LISP programs.

921
00:42:00,289 --> 00:42:02,550
They cast the abstract evaluator into silicon.

922
00:42:02,769 --> 00:42:04,889
And as they do this, they clarify the fundamental

923
00:42:04,889 --> 00:42:07,309
difference between an interpreter and a compiler,

924
00:42:07,469 --> 00:42:09,489
which I found fascinating. The explicit control

925
00:42:09,489 --> 00:42:11,610
evaluator we just described is an interpreter,

926
00:42:11,809 --> 00:42:14,710
right? An interpreter traverses your source code

927
00:42:14,710 --> 00:42:17,570
structure, analyzing it and executing it on the

928
00:42:17,570 --> 00:42:21,539
fly, line by line. It's interactive. If you type

929
00:42:21,539 --> 00:42:24,519
a command into the REPL, the interpreter executes

930
00:42:24,519 --> 00:42:27,320
it instantly. But the drawback is that the interpreter

931
00:42:27,320 --> 00:42:29,500
has to figure out the syntax every single time?

932
00:42:29,760 --> 00:42:33,260
Yes. If you have a loop that runs a million times,

933
00:42:33,380 --> 00:42:36,260
the interpreter re -analyzes the structure of

934
00:42:36,260 --> 00:42:38,960
that loop a million times. It's flexible, but

935
00:42:38,960 --> 00:42:41,380
it's slow. A compiler, on the other hand, is

936
00:42:41,380 --> 00:42:43,639
a translator. It takes your entire high -level

937
00:42:43,639 --> 00:42:46,480
Lisp program and analyzes it fully before anything

938
00:42:46,480 --> 00:42:49,610
runs. It translates the abstract... abstractions

939
00:42:49,610 --> 00:42:52,409
directly into a highly optimized raw sequence

940
00:42:52,409 --> 00:42:54,349
of machine instructions. It does all the heavy

941
00:42:54,349 --> 00:42:56,409
lifting of figuring out the environments and

942
00:42:56,409 --> 00:42:58,610
the variable lookups ahead of time. Once compiled,

943
00:42:58,750 --> 00:43:00,889
the program runs incredibly fast because it's

944
00:43:00,889 --> 00:43:03,869
just pure native machine code. And modern computing

945
00:43:03,869 --> 00:43:06,349
ecosystems rely on both, right? We interpret

946
00:43:06,349 --> 00:43:08,869
while we are actively developing, experimenting,

947
00:43:08,949 --> 00:43:11,050
and debugging because we need that interactive

948
00:43:11,050 --> 00:43:13,510
REPL. And we compile when we're ready to deploy

949
00:43:13,510 --> 00:43:16,409
the software and need raw, unadulterated performance.

950
00:43:17,039 --> 00:43:19,320
So let's step back and look at the epic descent

951
00:43:19,320 --> 00:43:21,980
we've just made. It's been quite a journey. We

952
00:43:21,980 --> 00:43:24,360
started in chapter one with the magical fundamental

953
00:43:24,360 --> 00:43:27,400
spells of computation, learning how to combine

954
00:43:27,400 --> 00:43:30,280
primitive procedures and write recursive processes.

955
00:43:30,739 --> 00:43:33,159
In chapter two, we realized that data itself

956
00:43:33,159 --> 00:43:36,440
is an illusion, constructed from those very same

957
00:43:36,440 --> 00:43:39,730
procedural spells using message passing. In Chapter

958
00:43:39,730 --> 00:43:42,510
3, we grappled with the curse of time, state,

959
00:43:42,730 --> 00:43:45,869
and concurrency, and explored how lazy streams

960
00:43:45,869 --> 00:43:49,269
can map infinite time to flat space. In Chapter

961
00:43:49,269 --> 00:43:51,829
4, we learned to rewrite the laws of the universe

962
00:43:51,829 --> 00:43:54,750
by building our own evaluators and multiverses.

963
00:43:54,809 --> 00:43:57,250
And finally, in Chapter 5, we stripped away the

964
00:43:57,250 --> 00:44:00,289
magic to find the cold, hard clockwork of the

965
00:44:00,289 --> 00:44:02,829
register machine and the stack beneath it all.

966
00:44:02,969 --> 00:44:05,829
It is a remarkable, exhausting intellectual journey.

967
00:44:06,400 --> 00:44:08,639
But it brings us perfectly full circle to Alan

968
00:44:08,639 --> 00:44:11,039
Perlis' Forward. We didn't just learn the syntax

969
00:44:11,039 --> 00:44:14,260
of a 1950s programming language today. We learned

970
00:44:14,260 --> 00:44:16,760
a procedural epistemology. We learned how to

971
00:44:16,760 --> 00:44:19,280
manage impossible complexity by building abstraction

972
00:44:19,280 --> 00:44:21,219
barriers and defining conventional interfaces.

973
00:44:21,739 --> 00:44:24,760
We learned how to build Perlis' evolving organisms

974
00:44:24,760 --> 00:44:28,440
rather than static pyramids. If you can internalize

975
00:44:28,440 --> 00:44:30,880
these concepts, it permanently changes how you

976
00:44:30,880 --> 00:44:33,039
approach complex problems in your everyday life.

977
00:44:33,369 --> 00:44:35,449
Whether you are organizing a massive corporate

978
00:44:35,449 --> 00:44:38,429
structure, modeling a financial system, or just

979
00:44:38,429 --> 00:44:41,349
trying to understand human language. You start

980
00:44:41,349 --> 00:44:43,550
seeing abstraction barriers and evaluation loops

981
00:44:43,550 --> 00:44:47,050
everywhere. It provides vocabulary for describing

982
00:44:47,050 --> 00:44:49,989
complex systems of any kind. Which leaves me

983
00:44:49,989 --> 00:44:52,909
with one final, deeply provocative thought for

984
00:44:52,909 --> 00:44:56,010
you, the listener, to mull over as we end this

985
00:44:56,010 --> 00:44:58,449
deep dive. Lay it on us. Throughout this entire

986
00:44:58,449 --> 00:45:01,869
journey, we've seen a recurring pattern. As systems

987
00:45:01,869 --> 00:45:04,349
become sufficiently complex, the rigid lines

988
00:45:04,349 --> 00:45:06,269
we draw to make ourselves comfortable begin to

989
00:45:06,269 --> 00:45:08,610
blur and dissolve. Yes, they do. Data becomes

990
00:45:08,610 --> 00:45:11,190
procedures waiting for a message. Procedures

991
00:45:11,190 --> 00:45:13,190
become data to be passed around as arguments.

992
00:45:13,530 --> 00:45:15,949
Our metacircular evaluator is just a program

993
00:45:15,949 --> 00:45:18,369
written in text, yet it acts as a machine. And

994
00:45:18,369 --> 00:45:20,489
the hardware register machine itself is just

995
00:45:20,489 --> 00:45:23,090
a physical evaluator, casting those same logical

996
00:45:23,090 --> 00:45:26,389
rules into copper and silicon. So, if an evaluator

997
00:45:26,389 --> 00:45:30,000
is just a program, And hardware is just an evaluator

998
00:45:30,000 --> 00:45:33,019
made of sand and wire. Where exactly does the

999
00:45:33,019 --> 00:45:35,900
software end and the machine begin? That blur

1000
00:45:35,900 --> 00:45:38,619
is the ultimate truth of computation. Is the

1001
00:45:38,619 --> 00:45:40,800
computer even a real physical thing, or is it

1002
00:45:40,800 --> 00:45:42,760
just the lowest level of language we happen to

1003
00:45:42,760 --> 00:45:44,719
be speaking today? Maybe the cold mechanical

1004
00:45:44,719 --> 00:45:48,000
factory floor was the illusion all along. Thank

1005
00:45:48,000 --> 00:45:49,860
you for joining us on this deep drive into the

1006
00:45:49,860 --> 00:45:51,239
architecture of thought itself.