Research


Publications

Check out my research group's webpage for a more up-to-date list.

You will also find more info about my group there, which is probably what you want anyway...



Eventfulness for Interactive Video Alignment

Eventfulness for Interactive Video Alignment

SIGGRAPH 2023 (Journal Paper)
Jiatian Sun, Longxiulin Deng, Triantafyllos Afouras, Andrew Owens and Abe Davis

More info/video/website/code coming soon!

FactorMatte: Redefining Video Matting for Re-Composition Tasks

FactorMatte: Redefining Video Matting for Re-Composition Tasks

SIGGRAPH 2023 (Journal Paper)
Zeqi Gu, Wenqi Xian1, Noah Snavely, Abe Davis

Video matting for scenes with complex interactions!

Ray Conditioning: Trading Photo-consistency for Photo-realism in Multi-view Image Generation

Ray Conditioning: Trading Photo-consistency for Photo-realism in Multi-view Image Generation

ICCV 2023
Eric Ming Chen, Sidhanth Holalkere, Ruyu Yan, Kai Zhang, Abe Davis

A nice clean light-weight trick for photorealistic viewpoint control over generative image models!

InStitches: Augmenting Sewing Patterns with Personalized Material-Efficient Practice

InStitches: Augmenting Sewing Patterns with Personalized Material-Efficient Practice

CHI 2023
Mackenzie Leake, Kathryn Jin, Abe Davis Stefanie Mueller.

ReCapture: AR-Guided Time-lapse Photography

ReCapture: AR-Guided Time-lapse Photography

UIST 2022
Ruyu Yan, Jiatian Sun, Longxiulin Deng, Abe Davis

AR-based guidance for hand-held time lapse capture.

A Mathematical Foundation for Foundation Paper Pieceable Quilts

A Mathematical Foundation for Foundation Paper Pieceable Quilts

SIGGRAPH 2021
Mackenzie Leake, Gilbert Bernstein, Abe Davis, Maneesh Agrawala

Computational tools for designing paper-pieceable quilts!

Crowdsampling the Plenoptic Function

Crowdsampling the Plenoptic Function

ECCV 2020 (Selected for Oral Presentation)
Zhengqi Li, Wenqi Xian, Abe Davis, and Noah Snavely

Learn from Internet photos to create interactive space-time views of popular attractions! With self-supervision!

Visual Chirality

Visual Chirality

CVPR 2020 (Selected for Oral Presentation)
Zhiqiu Lin, Jin Sun, Abe Davis, Noah Snavely
  Nominated for Best Paper

How does reflection change what we learn from images? Despite widespread use in data augmentation, people had not looked closely at this question before our work.

Head-mounted Augmented Reality for Guided Surface Reflectance Capture

Head-mounted Augmented Reality for Guided Surface Reflectance Capture

CVPR Workshop on Computer Vision for Augmented and Virtual Reality
Harald Haraldsson, Søren Skovsen, Ser-Nam Lim, Steve Marschner, Serge Belongie, Abe Davis

Decoupled Localization and Sensing with HMD-based AR for Interactive Scene Acquisition

Decoupled Localization and Sensing with HMD-based AR for Interactive Scene Acquisition

CVPR Workshop on Computer Vision for Augmented and Virtual Reality
Søren Skovsen, Harald Haraldsson, Abe Davis, Henrik Karstoft, Serge Belongie

Modal Imaging of Portsmouth, New Hampshire Bridge for Video-Based Structural Health Monitoring

Modal Imaging of Portsmouth, New Hampshire Bridge for Video-Based Structural Health Monitoring

IWSHM 2019
Abe Davis, Justin G. Chen, Abe Davis, Travis M. Adams, Hao Sun, Erin S. Bell, Fredo Durand, Oral Buyukozturk

Comparing modal images of a bridge with an FEM model of that bridge.

Visual Rhythm and Beat

Visual Rhythm and Beat

SIGGRAPH 2018
Abe Davis and Maneesh Agrawala

Make arbitrary video dance to arbitrary music.

Structural Health Monitoring from the Window Seat of a Passenger Airplane

Structural Health Monitoring from the Window Seat of a Passenger Airplane

IWSHM 2017
Abe Davis, Justin G. Chen, Oral Buyukozturk, Fredo Durand, Doug L. James
  Winner, SHM In Action 2017

Monitoring the vibration modes of an airplane wing from the window seat using a mobile phone.

Computational Video Editing for Dialogue-Driven Scenes

Computational Video Editing for Dialogue-Driven Scenes

SIGGRAPH 2017
Mackenzie Leake, Abe Davis, Anh Truong and Maneesh Agrawala

Computational tools for fast, high-level, editing of multiple video takes for dialogue driven scenes.

Visual Vibrometry: Estimating Material Properties from Small Motions in Video

Visual Vibrometry: Estimating Material Properties from Small Motions in Video

PAMI 2016
Abe Davis*, Katherine L. Bouman*, Justin G. Chen, Michael Rubinstein, Oral Buyukozturk, Fredo Durand, William T. Freeman

Improving on our CVPR 2015 paper, we adding additional analysis, showed you could do damping estimation, and infer hidden structure (e.g., tell whether a container is leaking).

Smaller Than the Eye Can See: Vibration Analysis with Video Cameras

Smaller Than the Eye Can See: Vibration Analysis with Video Cameras

19th World Conference on Non-Destructive Testing 2016 (WCNDT)
Oral Buyukozturk, Justin G. Chen, Neal Wadhwa, Abe Davis, Frédo Durand, and William T. Freeman

Including visual vibration analysis of earthquakes!

Computational Bounce Flash for Indoor Portraits

Computational Bounce Flash for Indoor Portraits

SIGGRAPH Asia 2016
Lukas Murmann, Abe Davis, Jan Kautz, Fredo Durand

A robotic bounce flash that makes portraiture lighting super easy

Visual Vibration Analysis

Visual Vibration Analysis

MIT EECS PhD Thesis
Abe Davis
  Winner of the MIT Sprowls thesis award
  ACM SIGGRAPH Thesis Award, Honorable mention

My dissertation on visual vibration analysis and its applications.

"Image-Space Modal Bases for Plausible Manipulation of Objects in Video"

"Image-Space Modal Bases for Plausible Manipulation of Objects in Video"

"Interactive Dynamic Video" in my dissertation
SIGGRAPH Asia 2015
Abe Davis, Justin Chen, Fredo Durand

This paper describes an improved version of the demo I showed at the end of my TED 2015 talk (with some improvements since TED) and demonstrates applications related to low-cost special effects.

Video Camera-based Vibration Measurement for Condition Assessment of Civil Infrastructure

Video Camera-based Vibration Measurement for Condition Assessment of Civil Infrastructure

International Symposium Non-Destructive Testing in Civil Engineering
Chen, J. G.; Davis, A.; Wadhwa, N.; Durand, F.; Freeman, W. T.; Buyukozturk, O.

More work with folks from the Civil Engineering department on NDT of structures using visual vibration analysis.

Long Distance Video Camera Measurements Of Structures

Long Distance Video Camera Measurements Of Structures

10th International Workshop on Structural Health Monitoring (IWSHM 2015)
Justin G. Chen, Neal Wadhwa, Abe Davis, Fredo Durand, William T. Freeman, Oral Buyukozturk

Work with folks from the Civil Engineering department looking at how distance effects visual vibration analysis for NDT of structures. I presented this at IWSHM 2015.

Light Field Reconstruction Using Sparsity in the Continuous Fourier Domain

Light Field Reconstruction Using Sparsity in the Continuous Fourier Domain

TOG / SIGGRAPH 2015
Lixin Shi, Haitham Hassanieh, Abe Davis, Dina Katabi, Fredo Durand

  1. We show that light fields are really only sparse in the continuous frequency domain, and 2) We show how to leverage that sparsity in an optimization over continuous frequencies to reconstruct even highly non-Lambertian light fields.
Visual Vibrometry: Estimating Material Properties from Small Motions in Video

Visual Vibrometry: Estimating Material Properties from Small Motions in Video

CVPR 2015 (Selected for Oral Presentation)
Abe Davis*, Katherine L. Bouman*, Justin G. Chen, Michael Rubinstein, Fredo Durand, William T. Freeman

The Visual Microphone: Passive Recovery of Sound from Video

The Visual Microphone: Passive Recovery of Sound from Video

SIGGRAPH 2014
Abe Davis, Michael Rubinstein, Neal Wadhwa, Gautham J. Mysore, Fredo Durand, William T. Freeman
  Discover Magazine Best Science Breakthroughs of 2014, #32

This project received a lot of press. Check out the project webpage for some examples. Also check out the Code.

Caperture

Caperture

iOS App
Abe Davis

I built on my masters thesis to create an AR-based iOS app for capturing light fields. Caperture no longer exists, but it was one of the first AR apps on a mobile phone (years before apple introduced ARKit).

Unstructured Light Fields

Unstructured Light Fields

Eurographics 2012
Abe Davis, Marc Levoy, Fredo Durand

I turned this into a free iOS app called Caperture.

Laser Speckle Photography for Surface Tampering Detection

Laser Speckle Photography for Surface Tampering Detection

IEEE Conference on Computer Vision and Pattern Recognition, (CVPR 2012)
YiChang Shih, Abe Davis, Samuel W. Hasinoff, Fredo Durand, and William T. Freeman
  Google Student Travel Award

Use lasers to detect tampering with a surface. We can detect whether someone has lightly touched a surface even if they were wearing gloves! This system also uses AR for ultra-precise rephotography.

Fast High-Dimensional Filtering Using the Permutohedral Lattice

Fast High-Dimensional Filtering Using the Permutohedral Lattice

Eurographics 2010
Andrew Adams, Jongmin Baek, Abe Davis (as Myers Abraham Davis)
  Runner-up, Best Paper Award