Deep Learning with JAX

Book excerpt: Accelerate deep learning and other number-intensive tasks with JAX, Google’s awesome high-performance numerical computing library. The JAX numerical computing library tackles the core performance challenges at the heart of deep learning and other scientific computing tasks. By combining Google’s Accelerated Linear Algebra platform (XLA) with a hyper-optimized version of NumPy and a variety of other high-performance features, JAX delivers a huge performance boost in low-level computations and transformations. In Deep Learning with JAX you will learn how to: • Use JAX for numerical calculations • Build differentiable models with JAX primitives • Run distributed and parallelized computations with JAX • Use high-level neural network libraries such as Flax • Leverage libraries and modules from the JAX ecosystem Deep Learning with JAX is a hands-on guide to using JAX for deep learning and other mathematically-intensive applications. Google Developer Expert Grigory Sapunov steadily builds your understanding of JAX’s concepts. The engaging examples introduce the fundamental concepts on which JAX relies and then show you how to apply them to real-world tasks. You’ll learn how to use JAX’s ecosystem of high-level libraries and modules, and also how to combine TensorFlow and PyTorch with JAX for data loading and deployment. Purchase of the print book includes a free eBook in PDF and ePub formats from Manning Publications. About the technology Google’s JAX offers a fresh vision for deep learning. This powerful library gives you fine control over low level processes like gradient calculations, delivering fast and efficient model training and inference, especially on large datasets. JAX has transformed how research scientists approach deep learning. Now boasting a robust ecosystem of tools and libraries, JAX makes evolutionary computations, federated learning, and other performance-sensitive tasks approachable for all types of applications. About the book Deep Learning with JAX teaches you to build effective neural networks with JAX. In this example-rich book, you’ll discover how JAX’s unique features help you tackle important deep learning performance challenges, like distributing computations across a cluster of TPUs. You’ll put the library into action as you create an image classification tool, an image filter application, and other realistic projects. The nicely-annotated code listings demonstrate how JAX’s functional programming mindset improves composability and parallelization. What's inside • Use JAX for numerical calculations • Build differentiable models with JAX primitives • Run distributed and parallelized computations with JAX • Use high-level neural network libraries such as Flax About the reader For intermediate Python programmers who are familiar with deep learning. About the author Grigory Sapunov holds a Ph.D. in artificial intelligence and is a Google Developer Expert in Machine Learning. The technical editor on this book was Nicholas McGreivy. Table of Contents Part 1 1 When and why to use JAX 2 Your first program in JAX Part 2 3 Working with arrays 4 Calculating gradients 5 Compiling your code 6 Vectorizing your code 7 Parallelizing your computations 8 Using tensor sharding 9 Random numbers in JAX 10 Working with pytrees Part 3 11 Higher-level neural network libraries 12 Other members of the JAX ecosystem A Installing JAX B Using Google Colab C Using Google Cloud TPUs D Experimental parallelization