Sound effects are often made by foley artists for movies as sounds such as booming thunder, car crashes, creepy footsteps down a hall and creaking doors are often created inside a special studio that adds to the magic of cinema.

This skill requires, human talent and creativity and now, a computer program can create its own sounds, fit for a movie.

Like Us on Facebook

Now, researchers from MIT's Computer Science and Artificial Intelligence Lab now created this deep learning algorithm that can study and work through creating sound effects, after this artificial intelligence watches a silent movie and applies its own sounds, simultaneously generating sounds as the film plays.

Researchers successfully carried out this experiment by teaching the algorithm using a video made by MIT researchers of them hitting objects with a drumstick. The algorithm was able to learn how these sounds were produced and coordinated them with its corresponding visuals. This resulted in a dataset that is readily available for future research, that it is aptly called "Greatest Hits".

The AI successfully learned from this instructional video, where the computer program was able to generate the correct sound effects for its new silent movie, that even confused people who watched the videos into hearing real or artificially made sounds generated by an algorithm. In reality, this algorithm does not just chooses the correct sound from the database but it can synthesize its own waveform to match the video.

This new sound effects AI is far from perfect though, as it still becomes confused by movements on the video that appear to look like hits producing sounds but are actually not. where they still cannot produce ambient sounds such as distant traffic or chirping birds.

According to lead author of the paper, MIT's Andrew Owens, being able to predict sound is a very crucial step in being able to predict consequences in physical interactions with the world.

This new study will be presented during the Computer Vision and Pattern Recognition conference happening this month.

TagsMIT, AI, computer algorithm, algorithm sound effects, MIT AI sound effects