Transport-Based Neural Style Transfer for Smoke Simulations
Computer Graphics Lab., ETH Zurich

Volcanic smoke simulation. Left: stylized output by our transport-based neural style transfer; right: close-up views of low-resolution base input and ours.
abstract
Artistically controlling fluids has always been a challenging task. Optimization techniques rely on approximating simulation states towards target velocity or density field configurations, which are often handcrafted by artists to indirectly control smoke dynamics. Patch synthesis techniques transfer image textures or simulation features to a target flow field. However, these are either limited to adding structural patterns or augmenting coarse flows with turbulent structures, and hence cannot capture the full spectrum of different styles and semantically complex structures. In this paper, we propose the first Transport-based Neural Style Transfer (TNST) algorithm for volumetric smoke data. Our method is able to transfer features from natural images to smoke simulations, enabling general content-aware manipulations ranging from simple patterns to intricate motifs. The proposed algorithm is physically inspired, since it computes the density transport from a source input smoke to a desired target configuration. Our transport-based approach allows direct control over the divergence of the stylization velocity field by optimizing incompressible and irrotational potentials that transport smoke towards stylization. Temporal consistency is ensured by transporting and aligning subsequent stylized velocities, and 3D reconstructions are computed by seamlessly merging stylizations from different camera viewpoints.
video
Fast Forward
Two Minute Papers
SIGGRAPH Asia 2019 – Technical Papers Trailer
download
Paper | Supplemental Material | Code | SlideBibtex
@article{kim19tnst,
title={{Transport-Based Neural Style Transfer for Smoke Simulations}},
author={Kim, Byungsoo and C. Azevedo, Vinicius and Gross, Markus and Solenthaler, Barbara},
journal={ACM Transactions on Graphics (TOG)},
volume={38},
number={6},
pages={188},
year={2019},
publisher={ACM}
}
Acknowledgement
The authors would like to thank Fraser Rothnie for his artistic contributions. This work was supported by the Swiss National Science Foundation under Grant No. 200021_168997.