a discussion to introduce a mini-series about the integration of music and Artificial Intelligence.
print(‘Hello world!’) My name is Aarohi Gupta. If you’ve read any of my other blogs (this is an open invitation to read them if you wish to), you’d know that I enjoy music and computer science.
The following section touches upon my hobbies, scroll down to read more about AI in particular.
Music: Currently, I enjoy composing and producing original music of genres ranging from Indian classical/folk, western classical, hip-hop, RnB, hard rock, pop (and more!). I am both a vocalist and an instrumentalist, and I spend much of my time experimenting with novel forms of music.
AI: While I’m interested in several other fields of comp sci (web development, AR/VR), I’ll be focusing on AI here, as that’s what my mini-series will be about. Like most people, I started learning about AI solely because of how popular the word was. The terms ‘deep learning’ and ‘big data’ were written in newspapers so often that I was naturally intrigued. But, as my knowledge about this subject increased, I found myself falling deeper in love with it. One program that aided in augmenting said love was the ‘InspiritAI AI Scholars’ program.
What was the AI Scholars program? What did I do?
While the first half of the course comprised of learning about what ML is and the techniques it uses, we worked on group projects the latter half. The project I worked on (thank you Tyler Bonnen for being such an amazing instructor) was one that detects human emotions from images of faces by using computer vision concepts.
We used a modified version of the fer2013 dataset comprising five emotion labels. The objective was to see the difference between the accuracy of a KNN, MLP, CNN and VGG (pre-trained CNN for image classification). I trained the ML algorithms with both pixels and facial landmarks as features to see what would provide the best results. The difference between the CNNs and the MLPs was that the CNNs were extracting features that they deemed relevant by themselves, while we were feeding either pixels or landmarks as features to the MLPs. The result could accurately label the emotion on images on this dataset 70% of the time (read this blog to read more about the project)
What is the aim of this mini-series?
As an aspiring musician and a programmer, I am fascinated by the intersection of AI and music. Whether it’s AI models that generate baroque music (see Carykh’s amazing video doing just this) or a deep learning system that locates image regions from a video that produce sounds and separate these sounds into a set of components that represents the sound from each pixel (PixelPlayer by the MIT CSAIL team), I am inspired by the vast scope of intersection in these fields.
Through this mini-series, I wish to discuss a project that I’m currently working on: the chord-inator (ok ok, the name is a work in progress) which transcribes chords from an audio file.
I often look up the chords of the songs I am asked to sing impromptu. However, chord transcriptions are usually available only for popular songs. For those songs that don’t readily have transcriptions, I can always transcribe them by ear, leaning on the many years of formal training I possess. However, for beginner students, not knowing the chords can also discourage them.
The ML model will be trained on a dataset that contains guitar sounds and has a subsection with chords. Once the database is cleaned, there would be .wav files of major, minor, diminished and seventh chords of every key. This series will discuss the different approaches that can solve this problem and go further in-depth for some of the technologies/techniques (MFCCs, CNNs, HMM).
What do I hope to achieve with this series?
I hope to increase the conversation surrounding AI and music, and how the two fields can be integrated. I believe that the two fields can benefit from each other, to make something beautiful.
I look forward to writing more about my endeavours, and I hope you’ll enjoy reading them too!
Aarohi Gupta is a Student Ambassador in the Inspirit AI Student Ambassadors Program. Inspirit AI is a pre-collegiate enrichment program that exposes curious high school students globally to AI through live online classes. Learn more at https://www.inspiritai.com/.