ArtWalks via Latent Diffusion Models

Abstract

ArtWalks presents a creative application of multiple techniques in image and video generation. We explore the creation of a dynamic bridge between between two paintings, seamlessly transitioning between artworks adding motion to static images. We build a two stage generative process, creating both abstract-conceptual interpo lation as well as spatio-temporal interpolation. We first hallucinate intermediate artworks using a generative diffusion model, then interpolate between resulting frames using a motion generation network, thus obtaining a complete video out of static paintings.