Visual Rhythm
and Beat


Everyone has a dance

Paper

“Visual Rhythm and Beat” by Abe Davis and Maneesh Agrawala, SIGGRAPH 2018

Abstract


We present a visual analogue for musical rhythm derived from an analysis of motion in video, and show that alignment of visual rhythm with its musical counterpart results in the appearance of dance. Central to our work is the concept of visual beats — patterns of motion that can be shifted in time to control visual rhythm. By warping visual beats into alignment with musical beats, we can create or manipulate the appearance of dance in video. Using this approach we demonstrate a variety of retargeting applications that control musical synchronization of audio and video: we can change what song performers are dancing to, warp irregular motion into alignment with music so that it appears to be dancing, or search collections of video for moments of accidentally dance-like motion that can be used to synthesize musical performances.

Links


More updates comin soon. Feel free to send inquiries or links to content you'd like to see dancified to EveryoneHasADance@gmail.com

Thanks! -Abe

Videos and Results:

Overview Video:

A video giving an overview of the project and what we can do.

Psy's Best Friend:

This example was a lot of fun to create. I found this Youtube channel for Dancing Nathan -- a dog that does this strange flailing trick on a chair, which kind of looks like off-beat dancing. I like to think that in his head he imagines that he and Psy are, like, super tight, and that he's dancing with Psy...


Barney Don't Need No Education:

This was one of the earliest retargeting examples I tried, and has always been a favorite of mine...

If you're happy and you know it play my jam:

More details soon: