1. Finale

Congrats (to me), this is the end of the tutorial!

I used to count how many papers are about deep learning in ISMIR 2015 - 2017. It increased from, say, 3% to 30% within 2-3 years. After then, it was pointless to count it because essentially almost every machine learning papers in ISMIR is using deep learning.

These days, I feel like LLMs are what deep learning were back then. It’s fascinating. The promise of deep learning was to rely on data and reduce domain specialties. When the model structure (once CNNs and RNNs, then transformers) were unified, I thought the promise was already fulfilled. Recently, I’ve been humbled again by observing even model weights can be unified with minimal adaptations, across modalities and domains.

Until just recently, it frustrated the GPU poors to be in the research field. With LLMs, this polarization gets so much worse that it somehow made almost everyone equal(ly poor). Unless you have 300k H100s (as of 2024 in Meta), you can’t train a strong foundational model anyway. But we can still finetune open-source LLMs, with a relatively small amount of compute, with the help of LoRA. Even the developers of LLMs have limited knowledge of what LLMs can or cannot, will or won’t be able to do. This is an exciting time to be a researcher!

I truly hope this light, casual, and minimal tutorial to be efficient, fun, non-intimidating, and helpful, to make you embark upon your LLM journey.

Happy LLM!

2024 May, Keunwoo Choi