S4: Self-Supervised learning of Spatiotemporal Similarity
G. Tkachev, S. Frey, and T. Ertl. IEEE Transactions on Visualization and Computer Graphics, (2021)
Abstract
We introduce an ML-driven approach that enables interactive example-based queries for similar behavior in ensembles of spatiotemporal scientific data. This addresses an important use case in the visual exploration of simulation and experimental data, where data is often large, unlabeled and has no meaningful similarity measures available. We exploit the fact that nearby locations often exhibit similar behavior and train a Siamese Neural Network in a self-supervised fashion, learning an expressive latent space for spatiotemporal behavior. This space can be used to find similar behavior with just a few user-provided examples. We evaluate this approach on several ensemble datasets and compare with multiple existing methods, showing both qualitative and quantitative results.
%0 Journal Article
%1 tkachev2021selfsupervised
%A Tkachev, Gleb
%A Frey, Steffen
%A Ertl, Thomas
%D 2021
%J IEEE Transactions on Visualization and Computer Graphics
%K EXC2075 PN6
%T S4: Self-Supervised learning of Spatiotemporal Similarity
%X We introduce an ML-driven approach that enables interactive example-based queries for similar behavior in ensembles of spatiotemporal scientific data. This addresses an important use case in the visual exploration of simulation and experimental data, where data is often large, unlabeled and has no meaningful similarity measures available. We exploit the fact that nearby locations often exhibit similar behavior and train a Siamese Neural Network in a self-supervised fashion, learning an expressive latent space for spatiotemporal behavior. This space can be used to find similar behavior with just a few user-provided examples. We evaluate this approach on several ensemble datasets and compare with multiple existing methods, showing both qualitative and quantitative results.
%@ 10.1109/TVCG.2021.3101418
@article{tkachev2021selfsupervised,
abstract = {We introduce an ML-driven approach that enables interactive example-based queries for similar behavior in ensembles of spatiotemporal scientific data. This addresses an important use case in the visual exploration of simulation and experimental data, where data is often large, unlabeled and has no meaningful similarity measures available. We exploit the fact that nearby locations often exhibit similar behavior and train a Siamese Neural Network in a self-supervised fashion, learning an expressive latent space for spatiotemporal behavior. This space can be used to find similar behavior with just a few user-provided examples. We evaluate this approach on several ensemble datasets and compare with multiple existing methods, showing both qualitative and quantitative results.},
added-at = {2021-12-08T17:10:32.000+0100},
author = {Tkachev, Gleb and Frey, Steffen and Ertl, Thomas},
biburl = {https://puma.ub.uni-stuttgart.de/bibtex/287f3a38e8d43c2711bff361e7c7e8a59/simtech},
interhash = {347d569c3aae7cd4934c93a157da3cfe},
intrahash = {87f3a38e8d43c2711bff361e7c7e8a59},
isbn = {10.1109/TVCG.2021.3101418},
journal = {IEEE Transactions on Visualization and Computer Graphics},
keywords = {EXC2075 PN6},
timestamp = {2023-07-31T05:41:57.000+0200},
title = {S4: Self-Supervised learning of Spatiotemporal Similarity},
year = 2021
}