Clik here to view.
Randomized prior functions in PyTorch
Reading Time: 4 minutesTrained MLP with 2 hidden layers and a sine prior. I was experimenting with the approach described in “Randomized Prior Functions for Deep Reinforcement Learning” by Ian Osband...
View ArticleClik here to view.
Benford law on GPT-2 language model
Reading Time: 1 minuteI wrote some months ago about how the Benford law emerges from language models, today I decided to evaluate the same method to check how the GPT-2 would behave with some sentences...
View ArticleUncertainty Estimation in Deep Learning (PyData Lisbon / July 2019)
Just sharing some slides I presented at the PyData Lisbon on July 2019 about the talk “Uncertainty Estimation in Deep Learning“: Uncertainty Estimation in Deep Learning from Christian Perone
View ArticleClik here to view.
Listening to the neural network gradient norms during training
Training neural networks is often done by measuring many different metrics such as accuracy, loss, gradients, etc. This is most of the time done aggregating these metrics and plotting visualizations on...
View ArticleTalk: Bayesian modelling for COVID-19 seroprevalence studies
I just published the slides of the talk below presented at the Machine Learning Porto Alegre meetup in Brazil: Bayesian modelling for COVID-19 seroprevalence studies from Christian Perone The post...
View ArticleClik here to view.
Tutorial on using LLVM to JIT PyTorch fx graphs to native code...
In 2009 I started playing with LLVM for some projects (data structure jit, for genetic programming, jit for tensorflow graphs, etc), and in these projects I realized how powerful LLVM design was at...
View ArticleClik here to view.
Couple of recent publications in uncertainty estimation and autonomous vehicles
Just sharing some recent publications I’ve been involved recently: L2M: Practical posterior Laplace approximation with optimization-driven second moment estimation ArXiv:...
View ArticleClik here to view.
Feste: composing NLP tasks with automatic parallelization and batching
I just released Feste, a free and open-source framework with a permissive license that allows scalable composition of NLP tasks using a graph execution model that is optimized and executed by...
View ArticleClik here to view.
Appreciating the complexity of large language models data pipelines
Erik Desmazieres’s “La Bibliothèque de Babel”. 1997. We have been training language models (LMs) for years, but finding valuable resources about the data pipelines commonly used to build the datasets...
View ArticleClik here to view.
Thoughts on Riemannian metrics and its connection with diffusion/score...
Different gaussian curvature surfaces. Image by Nicoguaro. We are so used to Euclidean geometry that we often forget the importance of curved geometries and how to measure things that don’t live on...
View ArticleClik here to view.
Generalisation, Kant’s schematism and Borges’ Funes el memorioso – Part I
Introduction Portrait of Immanuel Kant by Johann Gottlieb Becker, 1768. One of the most interesting, but also obscure and difficult parts of Kant’s critique is schematism. Every time I reflect on...
View ArticleMemory-mapped CPU tensor between Torch, Numpy, Jax and TensorFlow
This is just a fun experiment to answer the question: how can I share a memory-mapped tensor from PyTorch to Numpy, Jax and TensorFlow in CPU without copy and making sure changes done in memory by...
View ArticleTorch Titan distributed training code analysis
I really like to peek into different ML codebases for distributed training and this is a very short post on some things I found interesting in Torch Titan: Disable and control of Python’s garbage...
View ArticleClik here to view.
The geometry of data: the missing metric tensor and the Stein score [Part II]
Credit: ESA/Webb, NASA & CSA, J. Rigby. / The James Webb Space Telescope captures gravitational lensing, a phenomenon that can be modeled using differential geometry. Note: This is a continuation...
View ArticleClik here to view.
Notes on Gilbert Simondon’s “On the Mode of Existence of Technical Objects”...
Happy new year ! This is the first post of 2025 and this time it is not a technical article (but it is about philosophy of technology ) Gilbert Simondon (1924-1989). Photo by LeMonde. This is a short...
View Article