Performance bump or schism in scientific computing? The most popular data science tool, Pandas (a NumFOCUS project), is switching backends from NumPy (NumFOCUS) to Arrow (Apache). However, the rest of the NumFOCUS scientific computing toolset (e.g. sklearn and scipy) is built on top of NumPy. 🎥 Practical Summary Video 🏎️ Performance You’ll hear a lot…

As a multi-language bridge to Julia and a next-gen arXiv — 1) Reflecting on the rebrand At its annual rstudio::conf last week, RStudio announced that it is rebranding as Posit and prioritizing a multi-language approach. Given that the company raised a $161M round of financing last year, the first time since 2005 that it has taken venture money, it was evident that something big was brewing…

Visualization for rapidly finding generalizable models — The aiqcboomerang plot visualizes various performance metrics across each split (train, validation, test) for every model in an experiment. When the points of a model trace are tightly clustered/ precise, it means that the model has discovered patterns that generalize across each population. 🧮 How to evaluate many tuned models Imagine that you’ve just trained a large…

In the context of MVC application development & data science — This post is a prologue for Dash is Deeper than Dashboards. When Node.js was released in 2009, there was widespread speculation that — given JavaScript’s pre-existing front end dominance and Node’s new backend capabilities — it was just a matter of time until we all became fullstack JS developers. …

The missing link that makes Python a fullstack language — Prologue: to set the stage, reference Brief History of Reactive Web Dev, which has been abstracted out of this post. 💡 Why is Dash so revolutionary? Dash’s reactive front end is the missing link that makes Python a truly compelling full-stack language. It enables each layer of the application stack (e.g. front end and backend) to…

Handling Numeric, Unsupervised, & Multi-Dimensional Data — The go-to data science tools for stratification are great at handling supervised classification datasets. However, they aren’t designed to handle numeric, unsupervised, and multi-dimensional datasets out-of-the-box. This is where AIQC excels. Stratification Leads to Generalization In previous blog posts, we extensively covered the importance of evaluating models with validation and test splits/ folds:

Why simpler models are more generalizable — If a student memorizes every definition in a textbook, have they really grasped the theories within? Or have they proven they can store the information in the book within their biological neural network, and recall that information when needed? Learning is about being able to generalize — to grasp the…

The fundamental problem with deep learning libraries is that they are designed for a single in-memory run whose sole aim is to minimize loss — when in reality — it takes many runs to tune an architecture, the workflow needs to be persisted, and the training loss is just the…

Post-processing is the data wrangling that stands between the output of an algorithm and insight. — Decoder Analogy If you held a vinyl record up to your ear, you wouldn’t expect to hear music — no matter how much dust you blew off it. The information that the record contains is encoded in a machine-readable format. …