EPISODE TRANSCRIPT
Deep Learning for Finance: Creating Machine & Deep Learning Models for Trading in Python (Sofien Kaabar)
Automatically transcribed from the episode audio. Some words and names may be inaccurate.
Listen to the episodeDownload textDownload captions
So in 2010, the global stock market just evaporated by, I mean, a trillion dollars in just 36 minutes. Right. The flash crash. Yeah, exactly. And there was no bomb. You know, there was no declaration of war. There wasn't even like a massive geopolitical news event that day. It was entirely algorithmic. It was just these silent strings of code making millions of decisions a second, reacting to each other in this insane cascading feedback loop. Yeah. And then. Just as quickly as that trillion dollars vanished, the algorithms recalibrated and the market just bounced back. Like nothing happened. Right. So today we're opening up that black box. We want to see exactly how those supercomputers learn to print money and occasionally how they almost destroy the entire system. Because the financial industry really relies on this perception that quantitative trading is sort of this dark art. Oh, absolutely. Wall Street benefits immensely for making these concepts seem like they're strictly the domain of rocket scientists or quantum physicists.
But at its core, algorithmic trading is really just the mechanization of human behavior and statistical probabilities. And that mechanization is our mission for you today in this deep dive. We are pulling back the curtain on this totally inaccessible mystery box. And we have a great guide for this. We really do. We're using this incredibly expansive, just meticulous text, Deep Learning for Finance. It's a 2024 book by Sofian Kabar. It's extremely comprehensive. Yeah. And we aren't just doing a high level overview here.
We're going concept by concept, translating the raw math into the actual mechanics of modern trading. We're going to start from the absolute fundamental building blocks, you know, the raw, messy financial data. Right. And walk you all the way to the bleeding edge of autonomous artificial intelligence. Because the goal here is to interrelate these highly complex concepts so that they actually make mechanical sense. Make them accessible. Exactly. We're going to explore how an algorithm actually thinks, how it processes the noise of the market, and how it arrives at a decision to buy or sell. But we're going to do it without sacrificing the mathematical rigor that makes those decisions possible. Right. Because before a machine can learn anything, it needs a universe to learn from. It needs raw material. The data.
The data. And the source material focuses really heavily on demystifying data science and Python right out of the gate. We have to begin with how data is gathered and crucially how it's fundamentally altered before an algorithm ever even like glances at a price tag. There's this foundational rule in computer science, right? Garbage in, garbage out. Oh, yeah. Classic. If the universe you present to your algorithm is flawed, every subsequent prediction it makes, no matter how sophisticated your multilayer neural network is. It's going to be equally flawed. It's built on sand. Precisely.
I mean, in 2012, a firm called Knight Capital lost $440 million in 45 minutes. Wow. Yeah. And it was just because of a flaw in how their software interacted with the market data. It bankrupted them. So understanding the raw material, it isn't some administrative task. It's literally the foundation of survival. Okay. So looking at the text, it keeps bringing up OHLC data as the primary raw material. Right. Open, high, low, close. Yeah. But is that just the raw ticker price, like the numbers we see scrolling at the bottom of the screen on a financial news network? Not quite. I mean, that scrolling ticker is usually just the last traded price, just a single data point. OK. OHLC gives you the open, high, low and close prices for a specific given time period, let's say a day or even just a five minute window.
So it's more of a snapshot of the action. Exactly. It gives the algorithm a multidimensional view of the price action rather than just a single point. It tells the machine where the battle between the buyers and sellers started, right, the open. Right. How far the buyers pushed it up, the high. How far the sellers pushed it down, the low. And who ultimately won at the end of the period, which is the close. But the data goes way beyond just numbers on the spreadsheet now, right? The book breaks down several types of data used in modern quant finance. We've got the numerical data, like OHLC. But then there's categorical data. Right, like labeling specific market regimes. Is it a bull market, a bear market, high volatility? Yeah. And then there's text data, which is just massive right now with natural language processing. NLP is everywhere. I mean, algorithms are literally reading news headlines in milliseconds. Or they're parsing the specific vocabulary of a central bank speech to gauge sentiment, which is crazy to me. It gets even wilder when you bring in visual data. Right, visual. Like... Charts? Literally images of charts.
Neural networks can actually be trained to recognize chart patterns by feeding them image files. No way. Yeah. So instead of teaching a computer to identify a stop sign for like a self -driving car, you're training it to visually identify a descending channel on a stock chart. That's incredible. Yeah. And the book also mentions audio data becoming prevalent. It is. Algorithms are starting to parse the tone of voice or, you know, the micro hesitations, even pitch changes of a CEO during a live quarterly earnings call. Just to detect if they're lying? Basically, yeah.
Or anxious. Basically, yeah. To detect concealed anxiety or on the flip side, overconfidence. It's extracting data from human emotion. Which is completely mind blowing. But OK, to process any of this, Kabbar outlines six critical steps of data science. gathering, pre -processing, exploration, visualization, analysis, and interpretation. The core workflow. Right. And to anchor this really abstract process, the book gives a very specific, practical example using the VIX. Right, the VIX. So the VIX is widely known as the fear index. I hear that term a lot on the news. Yeah, it tracks the implied volatility of the S &P 500 using options pricing. So it doesn't measure price directly. It measures how much traders are willing to pay for insurance against massive price swings. So it's basically a gauge of panic.
Exactly. When the market is panicking, the VIX spikes violently. But when the market is complacent, you know, just grinding higher day after day, the VIX slowly bleeds downward. Okay, so the source walks through using a Python library called... pandas data reader to gather this VIX data. Right. It's an API call. Yeah. You just write a few lines of code to connect to a data source and you request the VIX data from like January 1st, 1990, all the way to early 2023. And in seconds, you have over 30 years of digitized market fear just sitting on your screen. It's incredibly powerful. But when you look closely at that downloaded data, it isn't perfect. The text highlights this massive roadblock, which is handling invalid data. Specifically, in that 30 -year data set, the author found 292 NAN values. N -A -N. Not a number. Yeah, finding those blank spots in a historical timeline, that's the rule. It's not the exception, yeah. Really? Yeah. I think historical data, it's pretty solid. You'd hope so, but data gets corrupted during transmission all the time. Exchanges suffer micro outages or maybe there are unexpected market holidays that mess up the formatting. Oh, OK. But here's the issue. You absolutely cannot feed a machine learning algorithm a blank space. Because it's math. Right. If a neural network tries to multiply a weight by not a number, the math just breaks.
The code throws an error and the entire system halts. So it has to be pre -processed. So the text gives three specific mathematical ways to handle missing data, like those 292 NANDs in the VIX, or, say, missing quarterly GDP numbers from some economic database. What are the three options it outlines? Well, first, you can just delete the row entirely. Second, you can assume the missing cell equals the exact value of the previous cell, so just carrying it forward. Or third, you can calculate a mean, an average of the surrounding cells to kind of smooth it over.
And I really want to emphasize the philosophical weight of that choice because it seems like a mundane administrative task, right? Just cleaning up a spreadsheet. Yeah, it sounds like housekeeping. But it's not. Pre -processing is literally altering the fabric of reality that the algorithm is going to learn from. Wait, how do you mean? Well, let's say you choose to fill a missing GDP number with the mathematical average of the previous quarter and the next quarter. Okay. You are commanding the algorithm to learn from an economic trajectory that never actually happened. You've artificially erased a potential shock and smoothed out the volatility of the past. Okay, hold on. I'm thinking about that first option now, deleting the row.
Right. If it's Tuesday and Wednesday's data is corrupted, so we just delete Wednesday. Thursday is now sitting right next to Tuesday in the spreadsheet. It is. The algorithm is going to think Thursday happened right after Tuesday. Doesn't that completely destroy the timeline of the market? Like in a system where momentum and the distance between events really matter. Aren't we blinding the machine to the actual passage of time? It's a really valid concern. And it highlights a crucial vulnerability in proof processing. Now, when you use a Python library like Pandas and you invoke a function like Dropna to delete those NANs. Right, Dropna. The remaining data isn't just mindlessly mashed together like a text document.
The timestamps attached to those prices, which is called the index, they remain intact. Oh, OK. So Thursday still mathematically knows it's Thursday. Exactly. Advanced models can be engineered to look at the timestamp gap rather than just sequential steps. Whew, okay. However, a lot of standard models do process data sequentially. And for those models, deleting a row absolutely warps the temporal distance between events. You are, in effect, artificially accelerating time. See, basically every method of dealing with missing data is a compromise. Always. You're either losing time by deleting it, you're repeating the past artificially by carrying the last number forward, or you're hallucinating this smooth, fake reality by taking an average. And you have to understand the specific biases you're introducing when you make that choice. Because the algorithm, it's going to treat whatever you hand it as the absolute, unquestionable truth. It doesn't know you manipulated it. Right. If you fill missing data with averages, your model will eventually assume the market is much safer and smoother than it really is.
And that leads to taking on excessive risk when things get chaotic. Man. OK, so we've gathered our raw material. We've made our philosophical choices about the missing pieces. And we now have a pristine pre -processed data set. Ready to go. But raw prices are just a historical ledger. We have to translate that data into a language the machine can actually mathematically parse. Yes. We're moving from the hard computer science of data gathering into the subjective psychological history of market analysis. We have to figure out how to translate human panic and greed into stationary statistics. And the absolute gateway to that translation and arguably the most important statistical concept in all of quantitative finance is stationarity. Stationarity. Yes. Machine learning models require data with stable statistical properties.
If the environment is constantly shifting beneath their feet, they literally cannot learn meaningful rules. Let's break that down for everyone. When we say stable statistical properties, we are talking about things like a stable mean, right? Like a stable average over time. Correct. Consider the raw price of the S &P 500. Over a 30 -year horizon, it trends massively upward, right? From around 300 in the early 1990s to well over 4 ,000 today. Right. If you feed an algorithm those raw prices, it really struggles to find recurring patterns because the baseline just keeps drifting away. A pattern that occurred when the price was at 500 might look mathematically completely different to the machine than the exact same pattern when the price is at 4 ,000. Because the scales totally warped. Exactly. Stationary data is data where the mean and the variance, so the average and the spread, they don't care about the time at which they're measured. They remain relatively flat. Okay, so the text mentions using the augmented Dickey -Fuller test, the ADF test. Yes. And this is to mathematically prove if a data set is stationary. It basically spits out a p -value, and if that value is low enough, you can trust your data is stable. That's the threshold, yeah. And to force the raw VIX data into stationarity, the author uses this process called differencing. Differencing is crucial. It transforms the data from absolute values into relative changes. How so? Instead of telling the algorithm that the price today is $1 .50 and yesterday it was $1 .48, you just tell the algorithm the price changed by positive two. Oh, okay. Right? And then the next day, it might change by negative one. By looking only at the daily changes, the data loses that massive upward trend. It begins to hover tightly around a mean of zero. And boom, the data becomes stationary.
Exactly. And once we have that stable mathematical foundation, we can start applying technical analysis to create the actual features we'll feed our model. Right, the inputs. And technical analysis is fascinating to me because it's built on three core, almost philosophical beliefs. Let's hear them. One, history repeats itself. Two, the market discounts everything, meaning the current price already reflects all public information, news, earnings, geopolitical tension. It's all baked into the price already. The efficient market hypothesis, essentially. Yeah. And three, the market moves in waves, not straight lines. And Khabar divides the application of those beliefs into two distinct camps. You have charting and you have indicators. Charting is what immediately comes to mind when you picture like a day trader, right? Drawing angled lines on a screen, connecting the bottoms of a price chart to find support or connecting the tops to find resistance. Yep.
Classical technical analysis. The text talks about ascending and descending channels where the price bounces back and forth between two parallel lines. And there's a really fascinating psychological bias the author mentions here called the invisible hand. The invisible hand bias is so interesting because it illustrates how charting can become a self -fulfilling prophecy. Right. When human traders look at an ascending channel, so a stock bouncing upward between two upward sloping lines, they're heavily influenced by market axioms like Martin Zweig's famous quote, the trend is your friend until the end when it bends. Classic trader speak. Right. Because thousands of human traders see the exact same line on their screens and they hold the same ingrained belief, whenever the price drops to touch that bottom support line, an invisible hand of bullish pressure enters the market. Because they all buy at the exact same time. Exactly. They automatically hit the buy button because they collectively believe the trend will hold. Okay, but this is where I struggle with the premise of this part of the book. I'm looking at my notes and traditional chart reading. Literally drawing lines on a screen based on where your human eyes think a trend should be. It feels a bit like financial astrology.
It gets that criticism a lot. Right. Because two different analysts could look at the exact same S &P chart and draw totally conflicting support lines. Like one guy includes the wick of a candlestick and the other just uses the body. True. So why in the world does a meticulous book about math -driven deep learning dedicate space to subjective human charting? It's a fair question. Drawing lines by hand is inherently subjective. And you're completely correct that a machine learning model cannot ingest a trader's visual interpretation of a chart. A neural network doesn't have a gut feeling about a support zone.
It doesn't. But the text includes charting to establish the underlying psychology that drives those market waves. To actually capture that psychology for the machine, though, we rely on the second camp of technical analysis. Indicators. Okay, like the moving average. Or the RSI. Yes. Indicators digitize that subjective human psychology into objective mathematical formulas. Ah, I see. Take the RSI, the Relative Strength Index. It's just a formula that measures the speed and change of price movements on a scale from 0 to 100.
Okay. The industry standard looks at the last 14 periods. When the RSI gets above 70, human traders collectively look at the chart. They feel that the asset has gone up way too far, way too fast. They declare it overbought, and they start selling. Right. And then the opposite is true. Yep. When it falls below 30, they declare it oversold, and they expect a bounce, so they buy. Awesome. Okay, so the algorithm isn't learning to draw subjective lines on a chart. It's ingesting the RSI data as a raw number. Precisely. The deep learning models, they do not care about the visual elegance of a chart pattern, but they can easily ingest that 14 period RSI value as a stationary statistical feature. Because it's just a number between zero and 100. Exactly.
The model can process 10 million historical data points and mathematically learn exactly when human traders are statistically likely to panic because the RSI hit 15 or get irrationally exuberant because it hit 85. Wow. We are turning messy human emotional cycles into crap. cold, hard numbers that a neural network can process and optimize. Okay, that bridges the gap perfectly for me. So we have digitized human trading psychology. We've forced those emotional waves into stationary statistics using differencing. Our raw materials are completely prepped. They're mathematically sound. We're ready for the algorithm. Yes. Now we enter the machine. We step into the realm of introductory machine learning. And crucially...
how we evaluate if the machine is actually doing anything useful. Right. To build a machine learning model, the author sets up a very strict framework. We need features, our x values. Those are the inputs we feed into the algorithm. Like the RSI. Like the RSI, exactly. And we need a target, our y value. That's the specific outcome we're trying to predict. So the text uses a specific 4x example here. It uses lagged daily price differences on the UER -USD currency pair. The euro against the U .S. dollar? Right. It uses those daily differences as the features, the X values. And it uses those to try and predict the future returns, which is the Y value. So we're basically looking at how much the euro to dollar exchange rate changed over the last few days to guess what it's going to do tomorrow. But before deploying some massively complex algorithm to solve this, Khabar introduces a really vital concept. It prevents quantitative researchers from fooling themselves. Oh, the dummy regression model. Yes, the dummy model. It sounds insulting, honestly, but it's actually this really elegant reality check. It really is. A dummy model is just a basic benchmark algorithm that uses incredibly simple, naive rules. In predicting a time series, a common dummy model simply predicts that Whatever happened today will happen again tomorrow. Wait, really? That's it. That's it. If the euro was up today, the dummy model blindly predicts the euro will be up tomorrow. The rule is simple.
If your computationally expensive, highly complex artificial intelligence cannot beat the performance of this completely naive dummy model. Your AI adds absolutely zero value. Exactly. You scrap it. It's worthless. It's the ultimate gatekeeper. I love that. So the text evaluates these models using a few key performance metrics. We have accuracy, which, you know, is pretty straightforward. And then we have root mean squared error, or RMSE. RMSE is really critical because of how it treats errors mathematically. How does it treat them? By squaring the difference between the predicted price and the actual price, the formula heavily penalizes large mistakes compared to small ones. Okay, let me make sure I get this.
Think of it this way. If your model misses a price prediction by one penny, the squared error is tiny. One squared is just one. But if it misses by $10, 10 squared is 100. Squaring that error explodes the penalty. I see. So it trains the model to avoid catastrophic misses at all costs. Exactly. It forces the model to be robust against massive outliers. Okay. And here is the specific example from the book that literally stopped me in my tracks when I read it. Oh, the baseline model. Yes. Kabar runs a baseline machine learning model on the US dollar USD testing data using all these clean features we just talked about.
And the final result is 49 .33 % accuracy. 49 .33%. I mean, this isn't just an academic exercise, right? This 49 % accuracy model might be the exact foundational code managing someone's 401k right now. It very well could be. Why on earth are we spending hours learning Python, writing complex code, processing data, and building sophisticated machine learning models if they are statistically worse than just flipping a quarter? We built a robot that loses to a coin flip? The frustration is entirely warranted. I get it. But the context of that 49 .33 % reveals the core challenge of quantitative finance. First, we really have to clarify what that metric actually measures. Okay.
It's directional accuracy on a daily horizon, meaning did it go up or did it go down tomorrow? It does not account for the magnitude of the move. Okay. So it just cares about green or red. Right. If it was wrong on 10 days where the market barely moved at all, just chopped sideways, but it correctly predicted one day where the market rallied massively, the model might actually be wildly profitable despite a terrible accuracy rate. Oh. Okay, that makes a lot of sense. Furthermore, this is a raw model output. There are no transaction costs applied, no risk management frameworks, and no stop losses. So it's literally just raw, unfiltered directional guessing without any actual trading logic applied. Exactly. But more importantly, the author deliberately includes this mediocre result to teach the most dangerous concept in all of machine learning. Overfitting. Overfitting, yes. Overfitting is like a student who memorizes the answers to the practice test, right? But they don't actually learn the subject. That's a perfect analogy. A data scientist could easily build a machine learning model that achieves 99 % accuracy on past data. You achieve that by making the model incredibly complex, giving it tons of parameters, and just allowing it to memorize the historical data set completely.
It just remembers what happened in 2008. Right. It will capture every tiny random fluctuation, and it'll look like a flawless money printing machine on paper. But the moment you deploy it in the real world on unseen data, it will utterly fail. Because it learned the random noise of the past, not the underlying signal. Exactly. The mathematical bias -variance tradeoff explains this perfectly. A model with high complexity has low bias. It fits the training data perfectly, but massive variance when exposed to new unseen data. So a 49 .33 % accuracy on testing data, meaning data the model hasn't seen during its training phase. That's actually a feature, not a bug for this introductory part of the book.
Yes. A 49 % baseline on unseen test data without overfitting is the honest starting point for optimization. It proves the model is attempting to generalize the real rules of the market, even if it's currently failing to beat a coin flip. It's the raw clay. It is. It proves you haven't cheated by just memorizing the past. Okay. Which brings us to the revolution. Because if we want to beat that coin flip, if we want to extract real tradable signal from the chaotic noise of the market, shallow machine learning just isn't enough. No, it hits a wall. We have to change the very architecture of the algorithm to discover relationships that human analysts literally cannot physically see. We're talking about deep learning.
The transition from machine learning to deep learning is fundamentally about depth and nonlinear complexity. In the text, Kabar focuses on artificial neural networks, or ANNs. Specifically, he utilizes multilayer perceptrons, MLPs, and long short -term memory models, which are called LSTMs. And he uses these to forecast S &P 500 returns. Let's talk about the architecture of these networks, because this is where the magic really happens. The text explains that these deep models contain what are called hidden layers. Yes. So you have an input layer where the stationary data like our RSI indicators goes in. You have an output layer where the prediction comes out. And in between is this vast interconnected web of hidden layers. Right. A shallow machine learning model might just look at the direct linear relationship between yesterday's price and today's price.
But a deep model with all its hidden layers applies activation functions that allow it to catch nonlinear relationships. Let's visualize how those layers actually stack up. Like layer one might look at basic price momentum, right? Sure. Then layer two might take the output of layer one and combine it with volatility data. Then layer three might take the output of layer two and analyze how momentum and volatility interact with the RSI on Tuesdays during high market volume. Exactly. By the time the data cascades through all the hidden layers and reaches the output, the network has synthesized patterns so complex, so multidimensional, that a human mathematician couldn't possibly graph them on a screen. It's beyond human comprehension at that level. And the long short -term memory model, the LSTM, takes this a step further. How so? Traditional neural networks process data in isolated snapshots. But financial markets are sequential. The order of events matters deeply.
LSTMs contain specialized memory cells with gates. Gates? Yeah. An input gate, an output gate, and crucially, a forget gate. This allows the model to process a sequence of time, explicitly deciding which information from 10 days ago is important enough to keep in its memory, and which information is just noise that should be forgotten entirely. Wow. So it's actively pruning its own memory. But to make these massive networks actually function profitably, the text introduces optimization satellites. Yes. These aren't the neural network itself, but they're techniques and formulas that kind of hover around the network to refine the data and the training process. Things like fractional differentiation. Fractional differentiation is an incredibly elegant mathematical solution to the stationarity problem we discussed earlier.
Right, because standard differencing... where we just subtract yesterday's price from today's price is kind of like having severe amnesia every morning. That's a good way to put it. You only know how today changed from yesterday, but you completely lose the memory of what the price was five years ago. You lose the overarching trend of the whole market. So how does fractional differencing fix that? Well, instead of integer differencing, which applies a weight of one to today and minus one to yesterday and completely ignores everything before that. Fractional differencing, say an order of 0 .5, applies a slowly decaying mathematical weight to all past prices. It considers yesterday heavily, the day before slightly less, and the price from five years ago only a tiny fraction. It mathematically flattens the curve just enough to pass the augmented Dickey -Fuller test for stationarity, but it retains the long -term memory of the overarching market trend.
So it gives the deep learning model the best of both worlds. Exactly. The short -term stability and the long -term context. The book also covers forecasting thresholds and continuous retraining as vital optimization satellites. And continuous retraining is pretty intuitive, right? The market is a living, breathing ecosystem. You can't train a model on the low inflation environment of 2018 and expect it to profitably trade the high interest rate market of 2024 without continuously updating its internal weights. It would get crushed. But this brings up the mystery box dilemma again.
If a multilayer... LSTM neural network finds a highly complex nonlinear hidden pattern, but we have literally no idea what that pattern actually is because it's buried in the math of a thousand hidden nodes. How can a hedge fund possibly trust it with hundreds of millions of dollars? That's terrifying. Right. How do you give real capital to a black box? Institutional funds grapple with this exact dilemma every single day. The regulatory and financial risk of deploying an opaque model is massive. The answer is you do not trust the black box blindly. You mitigate the risk of the unknown using rigorous validation frameworks.
So you cross -validate it. The book mentions time series cross -validation. Yes. Cross -validating time series data is like training a weather forecaster. Okay. How so? With normal data, you might train a model on random data points and test it on other random points. But time series data has a strict chronology. You cannot train your weather forecaster on random days from the summer and test their predictive accuracy on random days in the winter. Right, because time only moves forward. Exactly. You have to train them on the data from, say, 1990 to 2010, and then test their predictive power on the unseen year of 2011.
Then you roll the window forward. So then you'd train on 1991 to 2011 and test on 2012. Precisely. And by rolling that window chronologically forward through decades of market history, you basically proved that the model can consistently adapt to unseen data without peering into the future to cheat. Exactly. The text is unambiguous about this. Deep learning in finance is fundamentally an exercise in massive, structured, and validated trial and error. You are structurally weeding out market noise from true alpha. You might not know exactly what mathematical correlations the hidden layers are prioritizing, but your rigorous chronological cross -validation framework mathematically proves that the model's logic works reliably over time. All right, let's turn the dial up even further. Everything we've discussed so far, shallow machine learning, deep learning, LSTMs, it all culminates in predicting a price or at least a direction, right? Yes, forecasting. But what if the algorithm didn't just predict the price? What if it could actually learn how to trade that price on its own, managing its own portfolio? We are moving into the absolute bleeding edge of the book here. Deep reinforcement learning.
This represents a massive paradigm shift in artificial intelligence. With supervised deep learning, you provide the model with the answers, the historical target prices. And you say. Adjust your internal weights until you learn how to predict this answer. Right. You give it the test key. With reinforcement learning, there are no answers provided. You create a framework where an autonomous agent interacts with an environment, learning an optimal policy of behavior entirely through trial and error. The text lays out the framework for this autonomous agent really clearly. You have states, actions, and rewards. Let's unpack the states first. The state is basically the current environment the agent finds itself in. And the book mentions using cue tables to represent this. I need a bit of help translating cue tables into plain English. Sure. Think of a cue table as a massive, multidimensional cheat sheet that the agent constantly updates. Okay. A cheat sheet. It's a matrix.
The rows of the matrix represent every possible state the market could be in. For example, a state where the RSI is at 70, the MACD is crossing over, and the price is touching an upper Bollinger Band. Got it. And the columns of the matrix represent the actions the agent can take, buy, sell, or hold. And what goes inside the intersecting cells of that matrix? The expected future reward. Those are the Q values. When the agent is in a specific state and takes a specific action, it receives a reward or a penalty from the environment. Based on whether the trade made money or lost money.
Exactly. It then uses a mathematical formula, it's called the Bellman equation, to update the Q value in that specific cell. Over time, the agent looks at its cheat sheet, finds the state it's currently in, and simply chooses the action with the highest Q value. Okay. And the rewards are the ultimate goals you program into the system. You can tell it to just maximize absolute profit per trade or, more intelligently, you can tell it to maximize the Sharpe ratio, which rewards risk -adjusted returns, forcing the agent to prioritize smooth equity curves rather than wild, volatile gains. Yes. You can even severely penalize the agent for maximum drawdowns, teaching it to be terrified of losing capital. And initially, the agent begins by taking completely random actions. It buys at the wrong time, loses money, and receives a negative reward, a penalty. The Q value for that action drops. Makes sense. But then it sells at the right time, makes money, gets a positive reward, and the Q value increases.
Over millions of simulated iterations, it populates that entire queue table and learns a comprehensive policy of behavior that maximizes its total cumulative reward. But the text outlines a massive hurdle for these agents, the exploration versus exploitation dilemma. Ah, yes. The classic RL problem. The algorithm constantly has to make a choice. Does it stick with the trading strategy that has generated a steady profit so far? which is exploitation? Or does it randomly try a completely new, untested strategy, just in case it yields a massive undiscovered payout, which is exploration? This is managed mathematically using what is called an epsilon greedy strategy. Epsilon greedy. Yeah. Early in the training, the epsilon value is high, meaning the agent ignores its cue table and explores randomly most of the time. It tries everything.
As training progresses, the epsilon value decays and the agent begins to exploit the knowledge it's gathered. So it settles down. Right. But if the epsilon decays too fast, the agent exploits too early, locking into a mediocre strategy and missing a vastly more profitable one hidden in the data. If it explores too much, it never establishes a consistent strategy. You know, it sounds incredibly similar to training a puppy with treats. If you tell a puppy to sit and it sits, you give it a treat. Positive reinforcement. It updates its internal cue table and repeats the action. It is a bit like training a puppy. But imagine if the living room floor was constantly shifting, the rules of gravity randomly reversed every few weeks, and the puppy occasionally got a massive pile of treats for chewing up the couch due to sheer luck. Oh, my God. That is the reality of financial markets. They are chaotic, non -stationary environments heavily driven by random noise.
And that's exactly where my concern lies. What if the reinforcement learning agent makes a completely random, statistically terrible trade like buying at the absolute peak of a speculative bubble, but because of sheer dumb luck, the market glitches higher for a single millisecond? The agent closes the trade, makes a massive profit, and receives a huge reward. It happens. Doesn't the algorithm learn the exact wrong lesson? Doesn't the noise of the market just completely destroy the reinforcement learning process? The author explicitly outlines these exact challenges, acknowledging that Reinforcement learning is exceedingly fragile in financial contexts. Exceedingly fragile? Yes. RL agents are notorious for overfitting to specific historical noise rather than generalized patterns. Furthermore, they struggle immensely with reward function design. How so? Well, if you tell an RL agent to simply maximize profit by the end of the year, it might take catastrophic leveraged risks to get there. It might blow up the account 99 times out of 100 during training just to... hit a massive lucky home run once. Yikes. And going back to the black box problem, the reasoning behind an RL agent's decisions is even more opaque than a standard deep learning model. Oh, absolutely.
If an autonomous agent suddenly decides to aggressively short the entire S &P 500, you have zero idea if it found a genius hidden correlation in the macro environment or if it just learned a flawed lucky lesson from a random noise spike three years ago. This is exactly why reinforcement learning is not yet dominating quantitative trading execution the way standard supervised machine learning is. It is the bleeding edge, but it bleeds easily. So if the deep neural networks and the autonomous reinforcement agents all have these profound structural limitations when fed standard price data, if they all eventually struggle with the chaotic noise of the market, then maybe the ultimate edge doesn't come from building a more complicated algorithm. Where does it come from? Maybe the edge comes from feeding a slightly simpler algorithm entirely unique data, which brings us to advanced strategies and unconventional inputs in the book. When every hedge fund in the world has access to the same LSTM architectures and the same OHLC price data, the algorithm itself is no longer the competitive advantage. The data is. The data is the moat. Exactly. This is where a quantitative researcher truly earns their keep.
The text introduces the Commitments of Traders Report, commonly known as the COT Report. Released by the CFTC, the Commodity Futures Trading Commission in the U .S. It's a weekly government publication that breaks down the aggregate open positions of different types of institutional traders in the futures market. It provides a literal map of where the big money is positioned. Follow the money. Exactly. And the text highlights two main groups detailed in the report. commercial traders and non -commercial traders. Let's define those groups because analyzing them kind of digitizes human economic behavior on a massive macro scale.
Commercial traders are the hedgers. And the text notes, their open positions in the futures market are usually negatively correlated to the actual price of the asset. Think of a massive corporate airline. Right. Their entire business model is heavily exposed to the fluctuating price of jet fuel. Right. They use the energy futures market to lock in prices and protect their profit margins. They are not speculating to make a quick trade. They are hedging their operational risk. So when the price of oil skyrockets, what do they do? The airline aggressively sells futures contracts to lock in current rates before it goes even higher. They're selling increases as the price increases, which is a negative correlation. And on the other side, the non -commercial traders are the hedge funds, the massive speculators, the trend followers. Their positions are generally positively correlated. Right. They are the ones trying to ride the wave. If oil is going up, they are buying oil futures to profit from the momentum. So the book outlines three specific algorithmic architectures to process this weekly COT data and extract actionable forecasts. There is the indirect one -step, the MPF direct, and the MPF recursive models. Yes. How do those models actually function mechanically? Like, what's the difference? Well, the indirect one -step model involves predicting the future COT data first, so guessing what the hedgers and speculators will do next week, and then using that internal prediction to forecast the actual market price.
Okay, kind of a two -step dance. Exactly. Now, the MPF Direct, which stands for multi -period forecasting, bypasses that middle step. It attempts to predict the market price several steps into the future directly from the current week's COT data. Straight to the point. Right. And the recursive one? The MPF recursive model predicts the price one step ahead and then feeds its own prediction back into itself as a new input to predict the second step, just cascading forward. That's super clever. The author also covers predicting Bitcoin volatility using deep learning, utilizing a rolling 10 -day standard deviation as a proxy target for volatility. Crypto is perfect for these models.
Yeah. And crucially, the book provides the Python code to generate real -time visual outputs of the LSTM model's internal architecture to monitor the training process. Generating visual maps of the architecture is a vital practice for avoiding that mystery box anxiety we talked about. You want to see what it's doing. It allows the researcher to actually see the layers, the node connections, and the shape of the data tensors as they flow through the network during training, ensuring the model is structured correctly before it ever touches real capital. But the COT data is the real strategic pivot here. And this is where the mechanics confuse me a little bit. The COT report is a slow, bureaucratic release. It comes out once a week on Fridays. And the data is already a few days old by the time it's even published.
It's lagging data. Yeah. But modern markets are trading millions of times a second. How on earth does an advanced, hyperfast, deep learning algorithm use slow, weakly lagging government data to trade in a high frequency environment? Isn't there a massive temporal mismatch there? What's brilliant about this approach is how you combine entire different time frames to create a structural edge. You are absolutely correct. You cannot use weekly data for high frequency execution. But the text explains that the COT data analysis acts as a macro level guide, an overarching fundamental bias. Oh, so it's like using a climate model to determine the general season rather than a radar to predict today's rainstorm. That is an.
Excellent way to frame it. You run your deep learning models on the slow weekly COT data. Let's say your LSTM model processes the historical patterns and predicts that over the next two weeks, commercial hedgers are going to massively increase their short positions on the Canadian dollar. Okay. The algorithm sees a fundamental institutional shift coming. You don't execute a high -frequency trade based just on that weekly signal. Instead, you use that slow data to filter the execution of your fast algorithms. So the slow AI gives orders to the fast AI. Precisely.
You program a rule for your fast five -minute chart algorithm. For the next two weeks, because the macro COT model anticipates heavy institutional selling, you are only allowed to execute on short signals for the Canadian dollar. Ignore all technical bisignals, no matter how good the RSI looks. Wow. You are utilizing the deep learning analysis of slow, unconventional data to provide a fundamental tailwind for your fast technical execution. It keeps your high -frequency bots from fighting the overarching institutional tide. That is incredibly elegant. You're marrying fundamental macro data with quantitative micro execution. which actually sets us up beautifully for the conclusion of this journey. It all comes together. It really does, because ultimately, no matter how deep the machine learning is, no matter how many LSTM hidden layers you build, and no matter how cleverly you filter your timeframe biases, these algorithms operate in a reality governed by human emotion and real -world economics. We are looking at the final chapter of the text, Market Drivers and Risk Management. Cobber makes a vital editorial choice to end the book by grounding all this high level mathematics in the messy reality of the trading floor. A human element. Yes. He outlines core market drivers that algorithms struggle to anticipate. Economic intuition and the interpretation of unprecedented news. When the FOMC, the Federal Open Market Committee of the U .S. Central Bank, makes an unexpected, massive shift in interest rate policy. it instantly reprices every algorithmic model across stocks, bonds, commodities, and currencies globally. A human being steps to a podium in Washington, D .C., says a few cautiously worded sentences about inflation, and a million lines of optimized Python code suddenly have to recalibrate instantly. And they often fail in the process. Because the underlying reality, the cost of capital that drives the entire mathematical system, just fundamentally shifted. Right. The text also delves into behavioral finance. It highlights the enduring power of cognitive biases.
It discusses anchoring, where traders irrationally fixate on a past price. We all do that. We do. And confirmation bias, where they actively seek out only the information that validates their existing failing trades. And overconfidence. These human biases dictate the market anomalies that algorithms attempt to exploit, but they also create the unpredictable panics that destroy algorithms. And finally, the text hammers home risk management, the absolute non -negotiable necessity of setting hard stop losses, defining profit targets, utilizing trailing stops, and prioritizing capital preservation above all else.
And honestly, looking at the arc of this entire deep dive, I see a profound irony here. How so? Well... We just spent this entire time and the author spent an entire meticulous book building mathematically flawless, emotionless, highly sophisticated, deep learning robots in Python. Right. We spent hours discussing stationary data, multilayer perceptrons, fractional differentiation and autonomous reinforcement agents. We built the ultimate quantitative machine. The perfect robot. Yeah. Only to end the book talking about human panic. central bank press conferences and the absolute necessity of manually setting a hard stop loss just in case the ai gets it completely wrong it feels like we spent a decade building an indestructible autonomous spaceship but we are still relying on a guy with a canvas parachute just in case the engine explodes the irony you're observing is the exact synthesis of quantitative finance the algorithms the python libraries the neural network architectures they are perfectly objective mathematically sound tools yeah But what they are attempting to measure, forecast and navigate is fundamentally subjective. They are trying to map human behavior, geopolitical chaos and economic fear. It is math attempting to predict madness. The markets are made of people and even the algorithms are written by people carrying the structural biases of their creators.
Without strict, non -negotiable risk management, without an understanding of the macro drivers like the FOMC, and without respect for behavioral biases, even the most beautifully optimized, multi -layer LSTM neural network will eventually encounter a black swan human panic it hasn't seen in its training data. And it will blow up the account. It will blow up the account. The sophisticated math only works long term if it is protected by simple risk management. That brings everything full circle. Let's recap this incredible journey. We started with the absolute raw materials, downloading historical VIX data, and philosophically debating the mechanics of altering time itself by dropping missing data rows or filling them with averages. The pre -processing phase. Yeah. And then we explored how to force drifting market data into stationarity using differencing and the Dickey -Fuller test. We learned how to translate subjective human charting psychology into objective statistical indicators like the RSI. Setting up the features. Right.
And then we built basic dummy machine learning models, learned why a 49 % coin flip accuracy on unseen data is essential to avoid the trap of overfitting, and then plunged into the complex hidden layers and memory gates of deep learning LSTMs. The architecture. We trained a reinforcement learning agent that occasionally gets lucky for the wrong reasons. We used slow macro COT data to filter fast micro markets. And we ended by realizing that even the most advanced AI needs a stop loss because human panic remains undefeated.
It really is a remarkable synthesis of computer science, statistical probability, and behavioral psychology, rigorously documented in code. And for you listening... Whether you are an aspiring quantitative researcher, a casual investor managing your own portfolio, or just somewhat fascinated by how artificial intelligence is mapping human behavior, understanding how these models ingest reality changes how you view the modern financial world. It really does. The next time you look at a stock chart, you aren't just seeing a squiggly line of prices. You are seeing the exact data points. that thousands of hidden algorithmic layers are currently digesting and fighting over in milliseconds. The market becomes infinitely more fascinating when you can see the architecture operating beneath the surface. But before we go, I want to leave you with a final provocative thought to mull over, something that builds on everything we've discussed today, but pushes it one step further. Oh, this should be good.
We talked extensively about how these deep learning models learn from historical data and human biases to predict what the market will do next. But if every major quantitative hedge fund and institutional bank in the world is eventually running very similar deep learning and reinforcement models. Which they are. And they are all training on the exact same historical data sets, ingesting the exact same stationary features, and learning the exact same human behavioral biases. Are these algorithms actually predicting the market anymore?
That's a great question. Or by acting in unison at light speed? Are they mathematically creating the market's future movements as one massive automated self -fulfilling prophecy? Have we used advanced math to clear the murky waters of human finance only to realize that the machine itself is now deciding where the waves will crash? Fascinating to think about. Thanks for joining us on this deep dive. Keep questioning the data.