Research


Publications

Visual Rhythm and Beat

Visual Rhythm and Beat

SIGGRAPH 2018
Abe Davis and Maneesh Agrawala

Make arbitrary video dance to arbitrary music.

Structural Health Monitoring from the Window Seat of a Passenger Airplane

Structural Health Monitoring from the Window Seat of a Passenger Airplane

IWSHM 2017
Abe Davis, Justin G. Chen, Oral Buyukozturk, Fredo Durand, Doug L. James
  Winner, SHM In Action 2017

Monitoring the vibration modes of an airplane wing from the window seat using a mobile phone.

Computational Video Editing for Dialogue-Driven Scenes

Computational Video Editing for Dialogue-Driven Scenes

SIGGRAPH 2017
Mackenzie Leake, Abe Davis, Anh Truong and Maneesh Agrawala

Computational tools for fast, high-level, editing of multiple video takes for dialogue driven scenes.

Visual Vibrometry: Estimating Material Properties from Small Motions in Video

Visual Vibrometry: Estimating Material Properties from Small Motions in Video

PAMI 2016
Abe Davis*, Katherine L. Bouman*, Justin G. Chen, Michael Rubinstein, Oral Buyukozturk, Fredo Durand, William T. Freeman

Improving on our CVPR 2015 paper, we adding additional analysis, showed you could do damping estimation, and infer hidden structure (e.g., tell whether a container is leaking).

Smaller Than the Eye Can See: Vibration Analysis with Video Cameras

Smaller Than the Eye Can See: Vibration Analysis with Video Cameras

19th World Conference on Non-Destructive Testing 2016 (WCNDT)
Oral Buyukozturk, Justin G. Chen, Neal Wadhwa, Abe Davis, Fr├ędo Durand, and William T. Freeman

Including visual vibration analysis of earthquakes!

Computational Bounce Flash for Indoor Portraits

Computational Bounce Flash for Indoor Portraits

SIGGRAPH Asia 2016
Lukas Murmann, Abe Davis, Jan Kautz, Fredo Durand

A robotic bounce flash that makes portraiture lighting super easy

Visual Vibration Analysis

Visual Vibration Analysis

MIT EECS PhD Thesis
Abe Davis
  Winner of the MIT Sprowls thesis award
  ACM SIGGRAPH Thesis Award, Honorable mention

My dissertation on visual vibration analysis and its applications.

"Image-Space Modal Bases for Plausible Manipulation of Objects in Video"

"Image-Space Modal Bases for Plausible Manipulation of Objects in Video"

"Interactive Dynamic Video" in my dissertation
SIGGRAPH Asia 2015
Abe Davis, Justin Chen, Fredo Durand

This paper describes an improved version of the demo I showed at the end of my TED 2015 talk (with some improvements since TED) and demonstrates applications related to low-cost special effects.

Video Camera-based Vibration Measurement for Condition Assessment of Civil Infrastructure

Video Camera-based Vibration Measurement for Condition Assessment of Civil Infrastructure

International Symposium Non-Destructive Testing in Civil Engineering
Chen, J. G.; Davis, A.; Wadhwa, N.; Durand, F.; Freeman, W. T.; Buyukozturk, O.

More work with folks from the Civil Engineering department on NDT of structures using visual vibration analysis.

Long Distance Video Camera Measurements Of Structures

Long Distance Video Camera Measurements Of Structures

10th International Workshop on Structural Health Monitoring (IWSHM 2015)
Justin G. Chen, Neal Wadhwa, Abe Davis, Fredo Durand, William T. Freeman, Oral Buyukozturk

Work with folks from the Civil Engineering department looking at how distance effects visual vibration analysis for NDT of structures. I presented this at IWSHM 2015.

Light Field Reconstruction Using Sparsity in the Continuous Fourier Domain

Light Field Reconstruction Using Sparsity in the Continuous Fourier Domain

TOG / SIGGRAPH 2015
Lixin Shi, Haitham Hassanieh, Abe Davis, Dina Katabi, Fredo Durand

1) We show that light fields are really only sparse in the continuous frequency domain, and 2) We show how to leverage that sparsity in an optimization over continuous frequencies to reconstruct even highly non-Lambertian light fields.

Visual Vibrometry: Estimating Material Properties from Small Motions in Video

Visual Vibrometry: Estimating Material Properties from Small Motions in Video

CVPR 2015 (Oral presentation)
Abe Davis*, Katherine L. Bouman*, Justin G. Chen, Michael Rubinstein, Fredo Durand, William T. Freeman

The Visual Microphone: Passive Recovery of Sound from Video

The Visual Microphone: Passive Recovery of Sound from Video

SIGGRAPH 2014
Abe Davis, Michael Rubinstein, Neal Wadhwa, Gautham J. Mysore, Fredo Durand, William T. Freeman
  Discover Magazine Best Science Breakthroughs of 2014, #32

This project received a lot of press. Check out the project webpage for some examples. Also check out the Code.

Caperture

Caperture

iOS App
Abe Davis

I built on my masters thesis to create an AR-based iOS app for capturing light fields. Caperture no longer exists, but it was one of the first AR apps on a mobile phone (years before apple introduced ARKit).

Unstructured Light Fields

Unstructured Light Fields

Eurographics 2012
Abe Davis, Marc Levoy, Fredo Durand

I turned this into a free iOS app called Caperture.

Laser Speckle Photography for Surface Tampering Detection

Laser Speckle Photography for Surface Tampering Detection

IEEE Conference on Computer Vision and Pattern Recognition, (CVPR 2012)
YiChang Shih, Abe Davis, Samuel W. Hasinoff, Fredo Durand, and William T. Freeman
  Google Student Travel Award

Use lasers to detect tampering with a surface. We can detect whether someone has lightly touched a surface even if they were wearing gloves! This system also uses AR for ultra-precise rephotography.

Fast High-Dimensional Filtering Using the Permutohedral Lattice

Fast High-Dimensional Filtering Using the Permutohedral Lattice

Eurographics 2010
Andrew Adams, Jongmin Baek, Abe Davis (as Myers Abraham Davis)
  Runner-up, Best Paper Award