Transport-Based Neural Style Transfer for Smoke Simulations

Byungsoo Kim, Vinicius C. Azevedo, Markus Gross, Barbara Solenthaler
Computer Graphics Lab., ETH Zurich
ACM Transaction on Graphics (SIGGRAPH ASIA 2019), arXiv:1905.07442

Volcanic smoke simulation. Left: stylized output by our transport-based neural style transfer; right: close-up views of low-resolution base input and ours.

abstract

Artistically controlling fluids has always been a challenging task. Optimization techniques rely on approximating simulation states towards target velocity or density field configurations, which are often handcrafted by artists to indirectly control smoke dynamics. Patch synthesis techniques transfer image textures or simulation features to a target flow field. However, these are either limited to adding structural patterns or augmenting coarse flows with turbulent structures, and hence cannot capture the full spectrum of different styles and semantically complex structures. In this paper, we propose the first Transport-based Neural Style Transfer (TNST) algorithm for volumetric smoke data. Our method is able to transfer features from natural images to smoke simulations, enabling general content-aware manipulations ranging from simple patterns to intricate motifs. The proposed algorithm is physically inspired, since it computes the density transport from a source input smoke to a desired target configuration. Our transport-based approach allows direct control over the divergence of the stylization velocity field by optimizing incompressible and irrotational potentials that transport smoke towards stylization. Temporal consistency is ensured by transporting and aligning subsequent stylized velocities, and 3D reconstructions are computed by seamlessly merging stylizations from different camera viewpoints.


video


Fast Forward

Two Minute Papers

SIGGRAPH Asia 2019 – Technical Papers Trailer

download

Paper | Supplemental Material | Code | Slide | ACM Digital Library (recorded video soon)

Bibtex


@article{kim19tnst,
  title={{Transport-Based Neural Style Transfer for Smoke Simulations}},
  author={Kim, Byungsoo and C. Azevedo, Vinicius and Gross, Markus and Solenthaler, Barbara},
  journal={ACM Transactions on Graphics (TOG)},
  volume={38},
  number={6},
  pages={188},
  year={2019},
  publisher={ACM}
}
							

Acknowledgement

The authors would like to thank Fraser Rothnie for his artistic contributions. This work was supported by the Swiss National Science Foundation under Grant No. 200021_168997.