DimWit - Tensor programming with wit
We present DimWit, a system for writing readable numerical and machine learning code. DimWit's core feature is a trait-based implementation of the named tensor notation, which improves readability and type-safety compared to standard tensors. We illustrate how to write machine learning models using DimWit, and discuss our experience training GPT-2.
In this session, we motivate why typed tensor computations are needed in machine learning and how named tensor notation helps us to write readable and type-safe code. We discuss the ongoing efforts in the Python community and argue that Scala already today can deliver what many Python library authors would like Python to support eventually. We present DimWit, a Scala library that implements named tensor notation as its core abstraction. We show how DimWit, which is built on top of Python’s JAX, can achieve efficiency on par with state-of-the-art libraries. We will also discuss our efforts to train GPT-2 (a 124M parameters) from scratch in Scala. We hope that this talk will show that Scala can be a better alternative to Python for expressing numerical computation and writing programs that are easier to reason about.
Repository: https://github.com/dimwit-dev/dimwit
PhD candidate at the University of Zurich in computer vision, with a focus on applied deep learning in document recognition and echocardiography.
Scala enthusiast working on Scala-based open-source machine learning projects such as DimWit, DeepWit, and ScalaGrad.
I am a lecturer at the Department of Computer Science at ETH Zurich. Besides teaching, I am doing research in medical data science. I have helped to create various open source libraries in this domain, such as Scalismo and most recently DimWit.