> ## Content Index > Fetch the complete content index at: https://kycuong.com/llms.txt > Use this file to discover other available public pages before exploring further. # Diving into Deep Learning - URL: https://kycuong.com/deep-learning/ - Published: 2024-11-28T04:38:00.000Z - Updated: 2025-08-13T00:08:26.000Z - Description: We have a new technological primitive. - Author: Ky-Cuong Huynh # Summary - *(Update: This series is currently on-pause while I'm focused elsewhere within AI.)* - *Overview*: This is the index page for my learnings and reflections as I work through the fast.ai course, "[Practical Deep Learning for Coders](http://course.fast.ai/?ref=kycuong.com)". - *Background*: Artificial intelligence has improved exponentially over the past decade, with breakthroughs in data availability, compute performance, and algorithms. We have a new primitive that can and will be used \~everywhere. It's worth learning. - *Motivation*: Learning from the past, we should be wary of the dual-use nature of these new capabilities. We need to proactively counteract malicious actors, prevent problematic feedback loops, and think ahead for handling the risks of much more capable systems that will emerge over the next \~5 years. - *Context*: I'm focusing on deep learning because it is currently the most generally capable approach to AI we have. It's the foundation behind many of the headlines we see today. Theoretically, the Universal Approximation Theorem tells us that \~any continuous function can be approximated by a neural network. That gives cross-domain problem-solving power. - *Conclusion*: To build fundamental knowledge, I am working through the fast.ai course as a starting point and chronicling my learnings as I go for the benefit of others. # The World Has Changed In just a decade's time, we've gone from laughing at clunky chatbots to [superhuman capabilities](https://unchartedterritories.tomaspueyo.com/p/ai-does-it-best?ref=kycuong.com) across a broad range of tasks. We now have omnipresent hardware acceleration and exabytes of data to enable what was once science fiction. We now have [semi-autonomous vehicles](https://waymo.com/?ref=kycuong.com), an [unprecedented revolution](https://erictopol.substack.com/p/learning-the-language-of-life-with) in the life sciences, [weather forecasts](https://doi.org/10.48550/arXiv.2307.10128?ref=kycuong.com) orders of magnitude faster than before, and [conversational AI assistants](https://openai.com/index/hello-gpt-4o/?ref=kycuong.com) far beyond what Star Trek imagined. There also are faddish developments. AI is now everywhere, even in places where it's [not that useful](https://www.citationneeded.news/ai-isnt-useless/?ref=kycuong.com). It makes the "[Excel but with Stories](https://www.threads.net/@simonfeilder/post/DBI79BtvP7L/that-old-excel-now-has-stories-meme-but-its-everything-and-ai?ref=kycuong.com)" meme look quaint. Still, when you zoom out, it's clear that we have a new technological primitive, a foundational capability or tool that can be used to build other ones. # The Dual Use Problem After realizing all this, I began to get worried. You see, I have a long-standing interest in military history and technology. One of the fundamental themes there is the [dual-use problem](https://doi.org/10.1007%2Fs40592-014-0004-9?ref=kycuong.com). Knowledge and technology that can be used to heal or build can also be used to kill and destroy. How well are we countering the misuse of new capabilities by malicious actors? Or addressing the problematic feedback loops that can arise in the interactions between humans and algorithms? What of the [complex problems](https://cset.georgetown.edu/publication/through-the-chat-window-and-into-the-real-world-preparing-for-ai-agents/?ref=kycuong.com) that will emerge from autonomous systems interacting with each other? I worry the answer is not very, especially given the harms that are already possible *even without any further research advancements*. For example, a machine learning model used to generate candidate drug molecules can also be used to [discover chemical weapons](https://doi.org/10.1038/s42256-022-00465-9?ref=kycuong.com) far more potent than the ones that exist today. This has come up before with [fertilizer development + explosives](https://cen.acs.org/articles/91/i8/WEAPON-CHOICE.html?ref=kycuong.com), [microbiology + bioweapons](https://biodefensecommission.org/germ-warfare/?ref=kycuong.com), and [nuclear energy + nuclear power](https://www.goodreads.com/book/show/6452798-command-and-control?ref=kycuong.com). I am particularly concerned with how [AI lowers barriers](https://www.dhs.gov/publication/fact-sheet-and-report-dhs-advances-efforts-reduce-risks-intersection-artificial?ref=kycuong.com) when it comes to CBRN (chemical, biological, radiological, nuclear) weapon development. Current nonproliferation regimes (systems to prevent these weapons from falling into unwanted hands) focus on supply chain chokepoints. These are the difficult-to-conceal equipment and inputs needed to make new weapons; the heavily guarded locations of current ones; the list of known experts with weaponization knowledge; and so on. Advanced AI models throw these constraints out the window. They lower the level of expertise needed and enable the discovery of novel precursors: a harmless-looking chemical compound here, a normal DNA or RNA sequence there. This makes it especially difficult to monitor and combat malicious non-state actors. Looking even only at pure software use-cases of AI, there is still cause for concern: - Social media optimizing for engagement and [amplifying political polarization](https://www.brookings.edu/blog/techtank/2021/09/27/how-tech-platforms-fuel-u-s-political-polarization-and-what-government-can-do-about-it/?ref=kycuong.com) and [disinformation](https://www.algotransparency.org/?ref=kycuong.com) in the process - [Risk prediction scores](https://www.propublica.org/article/bias-in-criminal-risk-scores-is-mathematically-inevitable-researchers-say?ref=kycuong.com) that create a feedback loop for recidivism - [Forgery of all content types](https://hbr.org/2019/03/how-will-we-prevent-ai-based-forgery?ref=kycuong.com) becoming trivial, enabling unprecedented information warfare - Just plain incorrect results, as with [Epic’s sepsis predictions](https://www.statnews.com/2022/10/24/epic-overhaul-of-a-flawed-algorithm/?ref=kycuong.com) We cannot simply dismiss these as "policy problems" that "someone else" will figure out. That's abandoning responsibility that fundamentally begins with us, the technologists pushing the frontiers of possibility. We have the best understanding of runtime reality. If there is a gap in policy, then we need to be the first ones to point it out and propose solutions. We are the ones best positioned to do so and we are the ones most directly responsible for the downstream consequences of not doing so. These problems become more complex with [multimodal](https://www.technologyreview.com/2024/05/08/1092009/multimodal-ais-new-frontier/?ref=kycuong.com) and [agentic](https://www.forbes.com/sites/bernardmarr/2024/11/15/the-third-wave-of-ai-is-here-why-agentic-ai-will-transform-the-way-we-work/?ref=kycuong.com) AI. And all of this is before we even consider how [widespread automation](https://www.youtube.com/watch?v=WSKi8HfcxEk&ref=kycuong.com) is upending society and the [imminent threat](https://www.metaculus.com/questions/5121/date-of-artificial-general-intelligence/?ref=kycuong.com) of [unaligned artificial general intelligence](https://80000hours.org/problem-profiles/artificial-intelligence/?ref=kycuong.com) (AGI). All in all, I am convinced that I need to learn more. Way more. # Why Deep Learning First? Let's back up and define some terms: - Artificial intelligence (AI): as a field of study, the academic discipline that focuses on "the study and construction of agents that do the right thing" ([Artificial Intelligence: A Modern Approach](https://www.goodreads.com/book/show/27543.Artificial%5FIntelligence?ref=kycuong.com), 4th edition, p.22). As a noun, a machine system that takes rational action to accomplish a given objective and so demonstrates [intelligence](https://en.wikipedia.org/wiki/Intelligence?ref=kycuong.com). - [Machine learning](https://www.coursera.org/in/articles/ai-vs-deep-learning-vs-machine-learning-beginners-guide?ref=kycuong.com) (ML): an approach to (and subset of) AI that focuses on systems that can learn from examples of problems and their solutions, and then generalize to solving previously-unseen problems without explicit instructions on how to do so. - [Deep learning](https://www.nvidia.com/en-us/glossary/deep-learning/?ref=kycuong.com) (DL): a subset of machine learning that uses artificial neural networks to solve problems. These modeled after biological brains, and "deep" in how multi-layered they are. I'm starting with deep learning specifically because it's the most powerful approach to artificial intelligence we have today. It dominates the [state-of-the-art (SOTA) results](https://paperswithcode.com/sota?ref=kycuong.com) currently out there. ![](https://storage.ghost.io/c/94/ed/94ed0f0d-b3b0-43d6-a446-97c21216eb1a/content/images/2022/11/Deep_Learning_Icons_R5_PNG.jpg-672x427.png) ([Source](https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/?ref=kycuong.com)) There's a key theoretical result underpinning all this: the [Universal Approximation Theorem](http://neuralnetworksanddeeplearning.com/chap4.html?ref=kycuong.com), which loosely states that "neural networks with a single hidden layer can be used to approximate any continuous function to any desired precision". Translation: even a very simple model of organic computation is surprisingly capable across many different problem domains. Empirically, artificial neural networks with many layers of neurons tend to be most effective. They encode [layers of concepts](https://doi.org/10.48550/arXiv.1311.2901?ref=kycuong.com) from the data on which they're trained. For an introduction to the underlying technical fundamentals, see [this video playlist](https://www.youtube.com/watch?v=aircAruvnKkhttps://www.youtube.com/watch?v=aircAruvnKk&list=PLZHQObOWTQDNU6R1%5F67000Dx%5FZCJB-3pi&ref=kycuong.com). At an intuitive level, this checks out. We're mirroring the only known form of intelligence we know of today: biological brains. If you're doubtful that we understand the brain well enough to be able to model it to any useful degree, it's time to update your beliefs. See [this discussion](https://slatestarcodex.com/2017/09/05/book-review-surfing-uncertainty/?ref=kycuong.com) of predictive coding and the second chapter of [this book](https://www.goodreads.com/book/show/45024007-the-singularity-is-nearer?ref=nav%5Fsb%5Fss%5F4%5F15) as a starting point. # Learning in Public Why am I blogging as I go? A few reasons: (1) It's a [habit tracker](https://jamesclear.com/habit-tracker?ref=kycuong.com) to keep me consistently learning and doing. Seeing past progress is very satisfying and drives momentum. (2) It's an opportunity to capture knowledge just as I understood it. Future-me will inevitably need to review this material at some point. How great would it be to already have a library of knowledge structured from first principles and hands-on examples? (3) I want to help others learn. By giving a top-down, progressive view of what I've just learned, I can sidestep the [Curse of Knowledge](https://www.wikiwand.com/en/Curse%5Fof%5Fknowledge?ref=kycuong.com). Having just extended my own [knowledge tree](https://unchartedterritories.tomaspueyo.com/p/the-tree-of-knowledge?ref=kycuong.com), I know what branches are needed to explain a new concept to others. I'll be less likely to give a bottoms-up explanation that requires knowledge the other person doesn't have yet. It's why many courses have a learning assistant model for peers helping peers. Into the future we go!