2024
Fleck, Philipp
Situated Analytics: Data-, Prototyping- and Application -Concepts for Augmented Reality PhD Thesis
2024.
@phdthesis{fleck_diss2024,
title = {Situated Analytics: Data-, Prototyping- and Application -Concepts for Augmented Reality},
author = {Philipp Fleck},
url = {https://repository.tugraz.at/publications/d6x22-3ys36},
doi = {10.3217/d6x22-3ys36},
year = {2024},
date = {2024-07-01},
urldate = {2024-07-01},
abstract = {Situated Analytics can become equally helpful for everyday tasks as for complex industrial procedures. It helps to understand data within its environment and allows users to interactively explore such data. However, the current state of the art is not scalable or applicable beyond laboratory conditions. We introduce solutions for data processing, application prototyping, and user interactions in Augmented Reality. Sensors and other Internet of Things devices are present in a physical environment, produce data (for example, by measuring environmental conditions) and allow for user interactions. We introduce Data Locomotion mechanics to automatically match the virtual data to the physical device, allowing us to create digital twins. Users and developers can author situated visualizations using a unityfied programming model manner across devices. We apply our findings to games and add in-game visualizations for analytical tasks using introspection and code injection. Due to the distributed natures of the presented concepts and implementations, co-located collaboration is always available, and multiple users can take actions such as authoring or interacting with the virtual content. Finally, we report on user evaluations to derive important lessons learned for creating situated visualization applications.},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
Situated Analytics can become equally helpful for everyday tasks as for complex industrial procedures. It helps to understand data within its environment and allows users to interactively explore such data. However, the current state of the art is not scalable or applicable beyond laboratory conditions. We introduce solutions for data processing, application prototyping, and user interactions in Augmented Reality. Sensors and other Internet of Things devices are present in a physical environment, produce data (for example, by measuring environmental conditions) and allow for user interactions. We introduce Data Locomotion mechanics to automatically match the virtual data to the physical device, allowing us to create digital twins. Users and developers can author situated visualizations using a unityfied programming model manner across devices. We apply our findings to games and add in-game visualizations for analytical tasks using introspection and code injection. Due to the distributed natures of the presented concepts and implementations, co-located collaboration is always available, and multiple users can take actions such as authoring or interacting with the virtual content. Finally, we report on user evaluations to derive important lessons learned for creating situated visualization applications.
Stranner, Marco; Fleck, Philipp; Schmalstieg, Dieter; Arth, Clemens
Instant Segmentation and Fitting of Excavations in Subsurface Utility Engineering Honorable Mention Journal Article
In: IEEE Transactions on Visualization and Computer Graphics, vol. 30, no. 5, pp. 2319-2329, 2024, ISSN: 1941-0506.
@article{10458391,
title = {Instant Segmentation and Fitting of Excavations in Subsurface Utility Engineering},
author = {Marco Stranner and Philipp Fleck and Dieter Schmalstieg and Clemens Arth},
url = {https://youtu.be/s151P3kJD-4?si=foLmog5wjDANvQIj
https://www.youtube.com/watch?v=00VmKBgAvek
},
doi = {10.1109/TVCG.2024.3372064},
issn = {1941-0506},
year = {2024},
date = {2024-05-01},
urldate = {2024-05-01},
journal = {IEEE Transactions on Visualization and Computer Graphics},
volume = {30},
number = {5},
pages = {2319-2329},
abstract = {Using augmented reality for subsurface utility engineering (SUE) has benefited from recent advances in sensing hardware, enabling the first practical and commercial applications. However, this progress has uncovered a latent problem – the insufficient quality of existing SUE data in terms of completeness and accuracy. In this work, we present a novel approach to automate the process of aligning existing SUE databases with measurements taken during excavation works, with the potential to correct the deviation from the as-planned to as-built documentation, which is still a big challenge for traditional workers at sight. Our segmentation algorithm performs infrastructure segmentation based on the live capture of an excavation on site. Our fitting approach correlates the inferred position and orientation with the existing digital plan and registers the as-planned model into the as-built state. Our approach is the first to circumvent tedious postprocessing, as it corrects data online and on-site. In our experiments, we show the results of our proposed method on both synthetic data and a set of real excavations.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Using augmented reality for subsurface utility engineering (SUE) has benefited from recent advances in sensing hardware, enabling the first practical and commercial applications. However, this progress has uncovered a latent problem – the insufficient quality of existing SUE data in terms of completeness and accuracy. In this work, we present a novel approach to automate the process of aligning existing SUE databases with measurements taken during excavation works, with the potential to correct the deviation from the as-planned to as-built documentation, which is still a big challenge for traditional workers at sight. Our segmentation algorithm performs infrastructure segmentation based on the live capture of an excavation on site. Our fitting approach correlates the inferred position and orientation with the existing digital plan and registers the as-planned model into the as-built state. Our approach is the first to circumvent tedious postprocessing, as it corrects data online and on-site. In our experiments, we show the results of our proposed method on both synthetic data and a set of real excavations.
Kernbauer, Thomas; Tschulik, Maximilian; Fleck, Philipp; Arth, Clemens
Spatial Augmented Reality for Heavy Machinery Using Laser Projections Proceedings Article
In: 2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 235-243, 2024.
@inproceedings{10536239,
title = {Spatial Augmented Reality for Heavy Machinery Using Laser Projections},
author = {Thomas Kernbauer and Maximilian Tschulik and Philipp Fleck and Clemens Arth},
doi = {10.1109/VRW62533.2024.00047},
year = {2024},
date = {2024-03-01},
booktitle = {2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)},
pages = {235-243},
abstract = {Operating heavy machinery is challenging and can pose safety hazards for the operator and bystanders. Although commonly used augmented reality (AR) devices, such as head-mounted or head-up displays, can provide occupational support to operators, they can also cause problems. Particularly in off-highway scenarios, i.e., when driving machines in bumpy environments, the usefulness of current AR devices and the willingness of operators to wear them are limited. Therefore, we explore how laser-projection-based AR can help the operator facilitate their tasks and enhance safety. For this, we present a compact hardware unit and introduce a flexible and declarative software system. Furthermore, we examine the calibration process to leverage a camera projector setup and outline a process for creating images suitable for display by a laser projector from a set of line segments. Finally, we showcase its ability to provide efficient instructions to operators and bystanders and propose concrete applications for our setup.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Operating heavy machinery is challenging and can pose safety hazards for the operator and bystanders. Although commonly used augmented reality (AR) devices, such as head-mounted or head-up displays, can provide occupational support to operators, they can also cause problems. Particularly in off-highway scenarios, i.e., when driving machines in bumpy environments, the usefulness of current AR devices and the willingness of operators to wear them are limited. Therefore, we explore how laser-projection-based AR can help the operator facilitate their tasks and enhance safety. For this, we present a compact hardware unit and introduce a flexible and declarative software system. Furthermore, we examine the calibration process to leverage a camera projector setup and outline a process for creating images suitable for display by a laser projector from a set of line segments. Finally, we showcase its ability to provide efficient instructions to operators and bystanders and propose concrete applications for our setup.
Kernbauer, Thomas; Fleck, Philipp; Arth, Clemens
PanoTherm: Panoramic Thermal Imaging for Object Detection and Tracking Proceedings Article
In: Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, pp. 98–109, SciTePress, Portugal, 2024, (Publisher Copyright: © 2024 by SCITEPRESS – Science and Technology Publications, Lda.; 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications : VISIGRAPP 2024, VISIGRAPP 2024 ; Conference date: 27-02-2024 Through 29-02-2024).
@inproceedings{9e1b426d34f44b4a9af09ec0e634c50d,
title = {PanoTherm: Panoramic Thermal Imaging for Object Detection and Tracking},
author = {Thomas Kernbauer and Philipp Fleck and Clemens Arth},
doi = {10.5220/0012330400003660},
year = {2024},
date = {2024-01-01},
booktitle = {Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications},
volume = {4, VISAPP},
pages = {98–109},
publisher = {SciTePress},
address = {Portugal},
abstract = {Visible-light cameras are used traditionally in object detection and tracking. Thermal imaging can equally be used for this purpose, however at the cost of additional calibration efforts, expenses, and limitations concerning the field of view. Still, thermal imaging is advantageous in various scenarios and basically the only plausible technology to apply in harsh outdoor environments, in which the use of standard RGB cameras is prohibitive due to low-light conditions or in complete darkness. While panoramic imaging using visible light cameras is becoming more popular for advanced photography or action recording, limited work has been done on developing panoramic thermal cameras. In this work, we present the first panoramic thermal camera targeting the constant 360◦ monitoring of the environment. We describe the calibration and stitching process in detail and demonstrate how to use the camera in a vehicular scenario. Finally, we give an example of the detection and tracking of objects and discuss the advantages and disadvantages of thermal imaging for this purpose.},
note = {Publisher Copyright: © 2024 by SCITEPRESS – Science and Technology Publications, Lda.; 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications : VISIGRAPP 2024, VISIGRAPP 2024 ; Conference date: 27-02-2024 Through 29-02-2024},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Visible-light cameras are used traditionally in object detection and tracking. Thermal imaging can equally be used for this purpose, however at the cost of additional calibration efforts, expenses, and limitations concerning the field of view. Still, thermal imaging is advantageous in various scenarios and basically the only plausible technology to apply in harsh outdoor environments, in which the use of standard RGB cameras is prohibitive due to low-light conditions or in complete darkness. While panoramic imaging using visible light cameras is becoming more popular for advanced photography or action recording, limited work has been done on developing panoramic thermal cameras. In this work, we present the first panoramic thermal camera targeting the constant 360◦ monitoring of the environment. We describe the calibration and stitching process in detail and demonstrate how to use the camera in a vehicular scenario. Finally, we give an example of the detection and tracking of objects and discuss the advantages and disadvantages of thermal imaging for this purpose.
Heinemann, Mika Benjamin; Kernbauer, Thomas; Fleck, Philipp; Arth, Clemens
Spotlight Control for Real-Time Targeting Journal Article
In: International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG), pp. 299-306, 2024.
@article{Heinemann24,
title = {Spotlight Control for Real-Time Targeting},
author = {Mika Benjamin Heinemann and Thomas Kernbauer and Philipp Fleck and Clemens Arth},
url = {https://api.semanticscholar.org/CorpusID:271410112},
doi = {10.24132/CSRN.3401.32},
year = {2024},
date = {2024-01-01},
journal = {International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG)},
pages = {299-306},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2023
Reyes-Aviles, Fernando; Fleck, Philipp; Schmalstieg, Dieter; Arth, Clemens
Bag of World Anchors for Instant Large-Scale Localization Journal Article
In: IEEE Transactions on Visualization and Computer Graphics, vol. 29, no. 11, pp. 4730-4739, 2023, ISSN: 1941-0506.
@article{10268998,
title = {Bag of World Anchors for Instant Large-Scale Localization},
author = {Fernando Reyes-Aviles and Philipp Fleck and Dieter Schmalstieg and Clemens Arth},
url = {https://www.youtube.com/watch?v=K5iybmbYORY&t=35s
},
doi = {10.1109/TVCG.2023.3320264},
issn = {1941-0506},
year = {2023},
date = {2023-11-01},
urldate = {2023-11-01},
journal = {IEEE Transactions on Visualization and Computer Graphics},
volume = {29},
number = {11},
pages = {4730-4739},
abstract = {In this work, we present a novel scene description to perform large-scale localization using only geometric constraints. Our work extends compact world anchors with a search data structure to efficiently perform localization and pose estimation of mobile augmented reality devices across multiple platforms (e.g., HoloLens 2, iPad). The algorithm uses a bag-of-words approach to characterize distinct scenes (e.g., rooms). Since the individual scene representations rely on compact geometric (rather than appearance-based) features, the resulting search structure is very lightweight and fast, lending itself to deployment on mobile devices. We present a set of experiments demonstrating the accuracy, performance and scalability of our novel localization method. In addition, we describe several use cases demonstrating how efficient cross-platform localization facilitates sharing of augmented reality experiences.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In this work, we present a novel scene description to perform large-scale localization using only geometric constraints. Our work extends compact world anchors with a search data structure to efficiently perform localization and pose estimation of mobile augmented reality devices across multiple platforms (e.g., HoloLens 2, iPad). The algorithm uses a bag-of-words approach to characterize distinct scenes (e.g., rooms). Since the individual scene representations rely on compact geometric (rather than appearance-based) features, the resulting search structure is very lightweight and fast, lending itself to deployment on mobile devices. We present a set of experiments demonstrating the accuracy, performance and scalability of our novel localization method. In addition, we describe several use cases demonstrating how efficient cross-platform localization facilitates sharing of augmented reality experiences.
Reyes-Aviles, Fernando; Fleck, Philipp; Schmalstieg, Dieter; Arth, Clemens
Compact World Anchors: Registration Using Parametric Primitives as Scene Description Journal Article
In: IEEE Transactions on Visualization and Computer Graphics, vol. 29, no. 10, pp. 4140-4153, 2023, ISSN: 1941-0506.
@article{9797054,
title = {Compact World Anchors: Registration Using Parametric Primitives as Scene Description},
author = {Fernando Reyes-Aviles and Philipp Fleck and Dieter Schmalstieg and Clemens Arth},
doi = {10.1109/TVCG.2022.3183264},
issn = {1941-0506},
year = {2023},
date = {2023-10-01},
journal = {IEEE Transactions on Visualization and Computer Graphics},
volume = {29},
number = {10},
pages = {4140-4153},
abstract = {We present a registration method relying on geometric constraints extracted from parametric primitives contained in 3D parametric models. Our method solves the registration in closed-form from three line-to-line, line-to-plane or plane-to-plane correspondences. The approach either works with semantically segmented RGB-D scans of the scene or with the output of plane detection in common frameworks like ARKit and ARCore. Based on the primitives detected in the scene, we build a list of descriptors using the normals and centroids of all the found primitives, and match them against the pre-computed list of descriptors from the model in order to find the scene-to-model primitive correspondences. Finally, we use our closed-form solver to estimate the 6DOFtransformation from three lines and one point, which we obtain from the parametric representations of the model and scene parametric primitives. Quantitative and qualitative experiments on synthetic and real-world data sets demonstrate the performance and robustness of our method. We show that it can be used to create compact world anchors for indoor localization in AR applications on mobile devices leveraging commercial SLAM capabilities.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We present a registration method relying on geometric constraints extracted from parametric primitives contained in 3D parametric models. Our method solves the registration in closed-form from three line-to-line, line-to-plane or plane-to-plane correspondences. The approach either works with semantically segmented RGB-D scans of the scene or with the output of plane detection in common frameworks like ARKit and ARCore. Based on the primitives detected in the scene, we build a list of descriptors using the normals and centroids of all the found primitives, and match them against the pre-computed list of descriptors from the model in order to find the scene-to-model primitive correspondences. Finally, we use our closed-form solver to estimate the 6DOFtransformation from three lines and one point, which we obtain from the parametric representations of the model and scene parametric primitives. Quantitative and qualitative experiments on synthetic and real-world data sets demonstrate the performance and robustness of our method. We show that it can be used to create compact world anchors for indoor localization in AR applications on mobile devices leveraging commercial SLAM capabilities.
Fleck, Philipp; Calepso, Aimée Sousa; Hubenschmid, Sebastian; Sedlmair, Michael; Schmalstieg, Dieter
RagRug: A Toolkit for Situated Analytics Journal Article
In: IEEE Transactions on Visualization and Computer Graphics, vol. 29, no. 7, pp. 3281-3297, 2023, ISSN: 1941-0506.
@article{9729627,
title = {RagRug: A Toolkit for Situated Analytics},
author = {Philipp Fleck and Aimée Sousa Calepso and Sebastian Hubenschmid and Michael Sedlmair and Dieter Schmalstieg},
url = {https://github.com/philfleck/ragrug
https://youtu.be/mFxSdvQhSVU
https://youtu.be/G4kl2jU0qlA?si=vxDVJ6uilmxXjWYd
https://www.youtube.com/watch?v=dJWCHzPvNVg
https://github.com/philfleck/ragrug/blob/master/RagRug_VR22_Tutorial_20220314.pdf
https://files.icg.tugraz.at/f/9fad7480101647a5bca7/
},
doi = {10.1109/TVCG.2022.3157058},
issn = {1941-0506},
year = {2023},
date = {2023-07-01},
urldate = {2023-07-01},
journal = {IEEE Transactions on Visualization and Computer Graphics},
volume = {29},
number = {7},
pages = {3281-3297},
abstract = {We present RagRug, an open-source toolkit for situated analytics. The abilities of RagRug go beyond previous immersive analytics toolkits by focusing on specific requirements emerging when using augmented reality (AR) rather than virtual reality. RagRug combines state of the art visual encoding capabilities with a comprehensive physical-virtual model, which lets application developers systematically describe the physical objects in the real world and their role in AR. We connect AR visualizations with data streams from the Internet of Things using distributed dataflow. To this end, we use reactive programming patterns so that visualizations become context-aware, i.e., they adapt to events coming in from the environment. The resulting authoring system is low-code; it emphasises describing the physical and the virtual world and the dataflow between the elements contained therein. We describe the technical design and implementation of RagRug, and report on five example applications illustrating the toolkit’s abilities.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We present RagRug, an open-source toolkit for situated analytics. The abilities of RagRug go beyond previous immersive analytics toolkits by focusing on specific requirements emerging when using augmented reality (AR) rather than virtual reality. RagRug combines state of the art visual encoding capabilities with a comprehensive physical-virtual model, which lets application developers systematically describe the physical objects in the real world and their role in AR. We connect AR visualizations with data streams from the Internet of Things using distributed dataflow. To this end, we use reactive programming patterns so that visualizations become context-aware, i.e., they adapt to events coming in from the environment. The resulting authoring system is low-code; it emphasises describing the physical and the virtual world and the dataflow between the elements contained therein. We describe the technical design and implementation of RagRug, and report on five example applications illustrating the toolkit’s abilities.
- https://github.com/philfleck/ragrug
- https://youtu.be/mFxSdvQhSVU
- https://youtu.be/G4kl2jU0qlA?si=vxDVJ6uilmxXjWYd
- https://www.youtube.com/watch?v=dJWCHzPvNVg
- https://github.com/philfleck/ragrug/blob/master/RagRug_VR22_Tutorial_20220314.pd[...]
- https://files.icg.tugraz.at/f/9fad7480101647a5bca7/
- doi:10.1109/TVCG.2022.3157058
Calepso, Aimee Sousa; Fleck, Philipp; Schmalstieg, Dieter; Sedlmair, Michael
Exploring Augmented Reality for Situated Analytics with Many Movable Physical Referents Proceedings Article
In: Proceedings of the 29th ACM Symposium on Virtual Reality Software and Technology, Association for Computing Machinery, Christchurch, New Zealand, 2023, ISBN: 9798400703287.
@inproceedings{10.1145/3611659.3615700,
title = {Exploring Augmented Reality for Situated Analytics with Many Movable Physical Referents},
author = {Aimee Sousa Calepso and Philipp Fleck and Dieter Schmalstieg and Michael Sedlmair},
url = {https://doi.org/10.1145/3611659.3615700},
doi = {10.1145/3611659.3615700},
isbn = {9798400703287},
year = {2023},
date = {2023-01-01},
booktitle = {Proceedings of the 29th ACM Symposium on Virtual Reality Software and Technology},
publisher = {Association for Computing Machinery},
address = {Christchurch, New Zealand},
series = {VRST '23},
abstract = {Situated analytics (SitA) uses visualization in the context of physical referents, typically by using augmented reality (AR). We want to pave the way toward studying SitA in more suitable and realistic settings. Toward this goal, we contribute a testbed to evaluate SitA based on a scenario in which participants play the role of a museum curator and need to organize an exhibition of music artifacts. We conducted two experiments: First, we evaluated an AR headset interface and the testbed itself in an exploratory manner. Second, we compared the AR headset to a tablet interface. We summarize the lessons learned as guidance for designing and evaluating SitA.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Situated analytics (SitA) uses visualization in the context of physical referents, typically by using augmented reality (AR). We want to pave the way toward studying SitA in more suitable and realistic settings. Toward this goal, we contribute a testbed to evaluate SitA based on a scenario in which participants play the role of a museum curator and need to organize an exhibition of music artifacts. We conducted two experiments: First, we evaluated an AR headset interface and the testbed itself in an exploratory manner. Second, we compared the AR headset to a tablet interface. We summarize the lessons learned as guidance for designing and evaluating SitA.
2022
Fleck, Philipp; Reyes-Aviles, Fernando; Pirchheim, Christian; Arth, Clemens; Schmalstieg, Dieter
Exploring Tele-Assistance for Cyber-Physical Systems with MAUI Proceedings Article
In: Bouatouch, Kadi; Sousa, A. Augusto; Chessa, Manuela; Paljic, Alexis; Kerren, Andreas; Hurter, Christophe; Farinella, Giovanni Maria; Radeva, Petia; Braz, Jose (Ed.): Computer Vision, Imaging and Computer Graphics Theory and Applications, pp. 431–452, Springer International Publishing, Cham, 2022, ISBN: 978-3-030-94893-1.
@inproceedings{10.1007/978-3-030-94893-1_20,
title = {Exploring Tele-Assistance for Cyber-Physical Systems with MAUI},
author = {Philipp Fleck and Fernando Reyes-Aviles and Christian Pirchheim and Clemens Arth and Dieter Schmalstieg},
editor = {Kadi Bouatouch and A. Augusto Sousa and Manuela Chessa and Alexis Paljic and Andreas Kerren and Christophe Hurter and Giovanni Maria Farinella and Petia Radeva and Jose Braz},
isbn = {978-3-030-94893-1},
year = {2022},
date = {2022-01-01},
booktitle = {Computer Vision, Imaging and Computer Graphics Theory and Applications},
pages = {431–452},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {In this paper we present an improved version of MAUI [9] (MAUI - Maintenance Assistance User Interface), extending the user-study, giving detailed insight into the implementations and introducing a new User-Interface for mobile use. MAUI is a novel take on tele-assisted tasks on cyber-physical systems. In its core we do not only provide real-time communication between workers and experts, but also allow an expert to have full control over the worker's user-interface.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper we present an improved version of MAUI [9] (MAUI - Maintenance Assistance User Interface), extending the user-study, giving detailed insight into the implementations and introducing a new User-Interface for mobile use. MAUI is a novel take on tele-assisted tasks on cyber-physical systems. In its core we do not only provide real-time communication between workers and experts, but also allow an expert to have full control over the worker's user-interface.
2021
Hansen, Lasse H.; Fleck, Philipp; Stranner, Marco; Schmalstieg, Dieter; Arth, Clemens
Augmented Reality for Subsurface Utility Engineering, Revisited Journal Article
In: IEEE Transactions on Visualization and Computer Graphics, vol. 27, no. 11, pp. 4119-4128, 2021, ISSN: 1941-0506.
@article{9523841,
title = {Augmented Reality for Subsurface Utility Engineering, Revisited},
author = {Lasse H. Hansen and Philipp Fleck and Marco Stranner and Dieter Schmalstieg and Clemens Arth},
doi = {10.1109/TVCG.2021.3106479},
issn = {1941-0506},
year = {2021},
date = {2021-11-01},
journal = {IEEE Transactions on Visualization and Computer Graphics},
volume = {27},
number = {11},
pages = {4119-4128},
abstract = {Civil engineering is a primary domain for new augmented reality technologies. In this work, the area of subsurface utility engineering is revisited, and new methods tackling well-known, yet unsolved problems are presented. We describe our solution to the outdoor localization problem, which is deemed one of the most critical issues in outdoor augmented reality, proposing a novel, lightweight hardware platform to generate highly accurate position and orientation estimates in a global context. Furthermore, we present new approaches to drastically improve realism of outdoor data visualizations. First, a novel method to replace physical spray markings by indistinguishable virtual counterparts is described. Second, the visualization of 3D reconstructions of real excavations is presented, fusing seamlessly with the view onto the real environment. We demonstrate the power of these new methods on a set of different outdoor scenarios.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Civil engineering is a primary domain for new augmented reality technologies. In this work, the area of subsurface utility engineering is revisited, and new methods tackling well-known, yet unsolved problems are presented. We describe our solution to the outdoor localization problem, which is deemed one of the most critical issues in outdoor augmented reality, proposing a novel, lightweight hardware platform to generate highly accurate position and orientation estimates in a global context. Furthermore, we present new approaches to drastically improve realism of outdoor data visualizations. First, a novel method to replace physical spray markings by indistinguishable virtual counterparts is described. Second, the visualization of 3D reconstructions of real excavations is presented, fusing seamlessly with the view onto the real environment. We demonstrate the power of these new methods on a set of different outdoor scenarios.
2020
Aviles, Fernando Reyes; Fleck, Philipp; Schmalstieg, Dieter; Arth, Clemens
Improving rgb image consistency for depth-camera based reconstruction through image warping Journal Article
In: Journal of WSCG, vol. 28, no. 1-2, pp. 105–113, 2020, ISSN: 1213-6964.
@article{546bf0dcfaec4b2180ea1de88db3294e,
title = {Improving rgb image consistency for depth-camera based reconstruction through image warping},
author = {Fernando Reyes Aviles and Philipp Fleck and Dieter Schmalstieg and Clemens Arth},
doi = {10.24132/JWSCG.2020.28.13},
issn = {1213-6964},
year = {2020},
date = {2020-05-01},
journal = {Journal of WSCG},
volume = {28},
number = {1-2},
pages = {105–113},
publisher = {Vaclav Skala Union Agency},
abstract = {Indoor reconstruction using depth camera algorithms (e.g., InfiniTAMv3) is becoming increasingly popular. Simple reconstruction methods solely use the frames of the depth camera, leaving any imagery from the adjunct RGB camera untouched. Recent approaches also incorporate color camera information to improve consistency. However, the results heavily depend on the accuracy of the rig calibration, which can strongly vary in quality. Unfortunately, any errors in the rig calibration result in apparent visual discrepancies when it comes to colorization of the 3D reconstruction. We propose an easy approach to fix this issue for the purpose of image-based rendering. We show that a relatively simple warping function can be calculated from a 3D checkerboard pattern for a rig with poor calibration between cameras. The warping is applied to the RGB images online during reconstruction, leading to a significantly improved visual result.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Indoor reconstruction using depth camera algorithms (e.g., InfiniTAMv3) is becoming increasingly popular. Simple reconstruction methods solely use the frames of the depth camera, leaving any imagery from the adjunct RGB camera untouched. Recent approaches also incorporate color camera information to improve consistency. However, the results heavily depend on the accuracy of the rig calibration, which can strongly vary in quality. Unfortunately, any errors in the rig calibration result in apparent visual discrepancies when it comes to colorization of the 3D reconstruction. We propose an easy approach to fix this issue for the purpose of image-based rendering. We show that a relatively simple warping function can be calculated from a 3D checkerboard pattern for a rig with poor calibration between cameras. The warping is applied to the RGB images online during reconstruction, leading to a significantly improved visual result.
Schmalstieg, Dieter; Fleck, Philipp
Towards Embedded Visualization Authoring Conference
2020, (ACM CHI Workshop on Immersive Analytics ; Conference date: 25-04-2020 Through 26-04-2020).
@conference{c77b94de95374cafbc0e49e11054a2c5,
title = {Towards Embedded Visualization Authoring},
author = {Dieter Schmalstieg and Philipp Fleck},
year = {2020},
date = {2020-05-01},
note = {ACM CHI Workshop on Immersive Analytics ; Conference date: 25-04-2020 Through 26-04-2020},
keywords = {},
pubstate = {published},
tppubtype = {conference}
}
Fleck, Philipp; Aviles, Fernando Reyes; Pirchheim, Christian; Arth, Clemens; Schmalstieg, Dieter
MAUI: Tele-assistance for Maintenance of Cyber-physical Systems Conference
2020, (15th International Conference on Computer Vision Theory and Applications : VISAPP 2020, VISIGRAPP 2020 ; Conference date: 27-02-2020 Through 29-02-2020).
@conference{fleck_maui,
title = {MAUI: Tele-assistance for Maintenance of Cyber-physical Systems},
author = {Philipp Fleck and Fernando Reyes Aviles and Christian Pirchheim and Clemens Arth and Dieter Schmalstieg},
url = {http://www.visapp.visigrapp.org/, https://www.insticc.org/node/technicalprogram/visigrapp/2020
https://www.youtube.com/watch?v=Jgg1JrICuZo
https://arbook.icg.tugraz.at/schmalstieg/Schmalstieg_376.pdf
},
year = {2020},
date = {2020-02-26},
urldate = {2020-02-26},
pages = {800–812},
abstract = {In this paper, we present the maintenance assistance user interface (MAUI), a novel approach for providing tele-assistance to a worker charged with maintenance of a cyber-physical system. Such a system comprises both physical and digital interfaces, making it challenging for a worker to understand the required steps and to assess work progress. A remote expert can access the digital interfaces and provide the worker with timely information and advice in an augmented reality display. The remote expert has full control over the user interface of the worker in a manner comparable to remote desktop systems. The worker needs to perform all physical operations and retrieve physical information, such as reading physical labels or meters. Thus, worker and remote expert collaborate not only via shared audio, video or pointing, but also share control of the digital interface presented in the augmented reality space. We report results on two studies: The first study evaluates the benefits of our system against a condition with the same cyber-physical interface, but without tele-assistance. Results indicate significant benefits concerning speed, cognitive load and subjective comfort of the worker. The second study explores how interface designers use our system, leading to initial design guidelines for tele-presence interfaces like ours.},
note = {15th International Conference on Computer Vision Theory and Applications : VISAPP 2020, VISIGRAPP 2020 ; Conference date: 27-02-2020 Through 29-02-2020},
keywords = {},
pubstate = {published},
tppubtype = {conference}
}
In this paper, we present the maintenance assistance user interface (MAUI), a novel approach for providing tele-assistance to a worker charged with maintenance of a cyber-physical system. Such a system comprises both physical and digital interfaces, making it challenging for a worker to understand the required steps and to assess work progress. A remote expert can access the digital interfaces and provide the worker with timely information and advice in an augmented reality display. The remote expert has full control over the user interface of the worker in a manner comparable to remote desktop systems. The worker needs to perform all physical operations and retrieve physical information, such as reading physical labels or meters. Thus, worker and remote expert collaborate not only via shared audio, video or pointing, but also share control of the digital interface presented in the augmented reality space. We report results on two studies: The first study evaluates the benefits of our system against a condition with the same cyber-physical interface, but without tele-assistance. Results indicate significant benefits concerning speed, cognitive load and subjective comfort of the worker. The second study explores how interface designers use our system, leading to initial design guidelines for tele-presence interfaces like ours.
Fleck, Philipp; Schmalstieg, Dieter; Arth, Clemens
Creating IoT-ready XR-WebApps with Unity3D Proceedings Article
In: Proceedings of the 25th International Conference on 3D Web Technology, Association for Computing Machinery, Virtual Event, Republic of Korea, 2020, ISBN: 9781450381697.
@inproceedings{10.1145/3424616.3424691,
title = {Creating IoT-ready XR-WebApps with Unity3D},
author = {Philipp Fleck and Dieter Schmalstieg and Clemens Arth},
url = {https://doi.org/10.1145/3424616.3424691},
doi = {10.1145/3424616.3424691},
isbn = {9781450381697},
year = {2020},
date = {2020-01-01},
booktitle = {Proceedings of the 25th International Conference on 3D Web Technology},
publisher = {Association for Computing Machinery},
address = {Virtual Event, Republic of Korea},
series = {Web3D '20},
abstract = {The rise of IoT-ready devices is supported through well-established web concepts for communication and analytics, but interaction yet remains in the world of web browsers and screen-based 2D interaction during times of tablet and smartphone popularity. Transforming IoT interaction concepts into 3D for future exploitation with head-worn XR devices is a difficult task due to the lack of support and continued disengagement of game engines used in XR development. In this work, we present an approach to overcome this limitation, tightly including web technology into a 3D game engine. Our work leverages the versatility of web concepts to create immersive and scalable web applications in XR, without the need for deep-tech know-how about XR concepts or tiring customization work. We describe the methodology and tools in detail and provide some exemplary XR applications.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The rise of IoT-ready devices is supported through well-established web concepts for communication and analytics, but interaction yet remains in the world of web browsers and screen-based 2D interaction during times of tablet and smartphone popularity. Transforming IoT interaction concepts into 3D for future exploitation with head-worn XR devices is a difficult task due to the lack of support and continued disengagement of game engines used in XR development. In this work, we present an approach to overcome this limitation, tightly including web technology into a 3D game engine. Our work leverages the versatility of web concepts to create immersive and scalable web applications in XR, without the need for deep-tech know-how about XR concepts or tiring customization work. We describe the methodology and tools in detail and provide some exemplary XR applications.
Karner, Florian; Gsaxner, Christina; Pepe, Antonio; Li, Jianning; Fleck, Philipp; Arth, Clemens; Wallner, Jürgen; Egger, Jan
Single-Shot Deep Volumetric Regression for Mobile Medical Augmented Reality Proceedings Article
In: Syeda-Mahmood, Tanveer; Drechsler, Klaus; Greenspan, Hayit; Madabhushi, Anant; Karargyris, Alexandros; Linguraru, Marius George; Laura, Cristina Oyarzun; Shekhar, Raj; Wesarg, Stefan; Ballester, Miguel Ángel González; Erdt, Marius (Ed.): Multimodal Learning for Clinical Decision Support and Clinical Image-Based Procedures, pp. 64–74, Springer International Publishing, Cham, 2020, ISBN: 978-3-030-60946-7.
@inproceedings{10.1007/978-3-030-60946-7_7,
title = {Single-Shot Deep Volumetric Regression for Mobile Medical Augmented Reality},
author = {Florian Karner and Christina Gsaxner and Antonio Pepe and Jianning Li and Philipp Fleck and Clemens Arth and Jürgen Wallner and Jan Egger},
editor = {Tanveer Syeda-Mahmood and Klaus Drechsler and Hayit Greenspan and Anant Madabhushi and Alexandros Karargyris and Marius George Linguraru and Cristina Oyarzun Laura and Raj Shekhar and Stefan Wesarg and Miguel Ángel González Ballester and Marius Erdt},
isbn = {978-3-030-60946-7},
year = {2020},
date = {2020-01-01},
booktitle = {Multimodal Learning for Clinical Decision Support and Clinical Image-Based Procedures},
pages = {64–74},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Augmented reality for medical applications allows physicians to obtain an inside view into the patient without surgery. In this context, we present an augmented reality application running on a standard smartphone or tablet computer, providing visualizations of medical image data, overlaid with the patient, in a video see-through fashion. Our system is based on the registration of medical imaging data to the patient using a single 2D photograph of the patient. From this image, a 3D model of the patient's face is reconstructed using a convolutional neural network, to which a pre-operative CT scan is automatically registered. For efficient processing, this is performed on a server PC. Finally, anatomical and pathological information is sent back to the mobile device and can be displayed, accurately registered with the live patient, on the screen. Hence, our cost-effective, markerless approach needs only a smartphone and a server PC for image processing. We present a qualitative and quantitative evaluation using real patient photos and CT from the clinical routine in facial surgery, reporting overall processing times and registration errors.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Augmented reality for medical applications allows physicians to obtain an inside view into the patient without surgery. In this context, we present an augmented reality application running on a standard smartphone or tablet computer, providing visualizations of medical image data, overlaid with the patient, in a video see-through fashion. Our system is based on the registration of medical imaging data to the patient using a single 2D photograph of the patient. From this image, a 3D model of the patient's face is reconstructed using a convolutional neural network, to which a pre-operative CT scan is automatically registered. For efficient processing, this is performed on a server PC. Finally, anatomical and pathological information is sent back to the mobile device and can be displayed, accurately registered with the live patient, on the screen. Hence, our cost-effective, markerless approach needs only a smartphone and a server PC for image processing. We present a qualitative and quantitative evaluation using real patient photos and CT from the clinical routine in facial surgery, reporting overall processing times and registration errors.
Reyes-Aviles, Fernando; Fleck, Philipp; Schmalstieg, Dieter; Arth, Clemens
Improving rgb image consistency for depth-camera Journal Article
In: 2020.
@article{reyes2020improving,
title = {Improving rgb image consistency for depth-camera},
author = {Fernando Reyes-Aviles and Philipp Fleck and Dieter Schmalstieg and Clemens Arth},
year = {2020},
date = {2020-01-01},
publisher = {Václav Skala-UNION Agency},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2019
Stranner, Marco; Arth, Clemens; Schmalstieg, Dieter; Fleck, Philipp
A High-Precision Localization Device for Outdoor Augmented Reality Proceedings Article
In: 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 37-41, 2019.
@inproceedings{8951896,
title = {A High-Precision Localization Device for Outdoor Augmented Reality},
author = {Marco Stranner and Clemens Arth and Dieter Schmalstieg and Philipp Fleck},
doi = {10.1109/ISMAR-Adjunct.2019.00025},
year = {2019},
date = {2019-10-01},
booktitle = {2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)},
pages = {37-41},
abstract = {In contrast to indoor tracking using computer vision, which has reached a good amount of maturity, outdoor tracking still suffers from comparably poor localization on a global scale. Smartphones and other commodity devices contain consumer-grade sensors for GPS, compass and inertial measurements, which are not accurate enough for augmented reality (AR) in most situations. This restricts what AR can offer to application areas such as surveying or building constructions. We present a self-contained localization device which connects wirelessly to any AR device, such as a smartphone or headset. The device gives centimeter-level accuracy and can be built out of commercial-of-the-shelf components for less than 500 EUR. We demonstrate the performance of the localization device using a variety of position and orientation sensing benchmarks.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In contrast to indoor tracking using computer vision, which has reached a good amount of maturity, outdoor tracking still suffers from comparably poor localization on a global scale. Smartphones and other commodity devices contain consumer-grade sensors for GPS, compass and inertial measurements, which are not accurate enough for augmented reality (AR) in most situations. This restricts what AR can offer to application areas such as surveying or building constructions. We present a self-contained localization device which connects wirelessly to any AR device, such as a smartphone or headset. The device gives centimeter-level accuracy and can be built out of commercial-of-the-shelf components for less than 500 EUR. We demonstrate the performance of the localization device using a variety of position and orientation sensing benchmarks.
2018
Arth, Clemens; Fleck, Philipp; Kalkofen, Denis; Mohr, Peter; Schmalstieg, Dieter
Image processing method, mobile device and method for generating a video image database Miscellaneous
2018, (US Patent 10,163,224).
@misc{arth2018image,
title = {Image processing method, mobile device and method for generating a video image database},
author = {Clemens Arth and Philipp Fleck and Denis Kalkofen and Peter Mohr and Dieter Schmalstieg},
year = {2018},
date = {2018-12-01},
publisher = {Google Patents},
note = {US Patent 10,163,224},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
Stanescu, Ana; Fleck, Philipp; Schmalstieg, Dieter; Arth, Clemens
Semantic Segmentation of Geometric Primitives in Dense 3D Point Clouds Proceedings Article
In: 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 206-211, 2018.
@inproceedings{8699332,
title = {Semantic Segmentation of Geometric Primitives in Dense 3D Point Clouds},
author = {Ana Stanescu and Philipp Fleck and Dieter Schmalstieg and Clemens Arth},
url = {https://www.youtube.com/watch?v=MkQQ0RPNcLs},
doi = {10.1109/ISMAR-Adjunct.2018.00068},
year = {2018},
date = {2018-10-01},
urldate = {2018-10-01},
booktitle = {2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)},
pages = {206-211},
abstract = {This paper presents an approach to semantic segmentation and structural modeling from dense 3D point clouds. The core contribution is an efficient method for fitting of geometric primitives based on machine learning. First, the dense 3D point cloud is acquired together with RGB images on a mobile handheld device. Then, RANSAC is used to estimate the presence of geometric primitives, followed by an evaluation of their fit based on classification of the fitting parameters. Finally, the approach iterates over successive frames to optimize the fitting parameters or replace a detected primitive by a better fitting one. As a result, we obtain a semantic model of the scene consisting of a set of geometric primitives. We evaluate the approach on an extensive set of scenarios and show its plausibility in augmented reality applications.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper presents an approach to semantic segmentation and structural modeling from dense 3D point clouds. The core contribution is an efficient method for fitting of geometric primitives based on machine learning. First, the dense 3D point cloud is acquired together with RGB images on a mobile handheld device. Then, RANSAC is used to estimate the presence of geometric primitives, followed by an evaluation of their fit based on classification of the fitting parameters. Finally, the approach iterates over successive frames to optimize the fitting parameters or replace a detected primitive by a better fitting one. As a result, we obtain a semantic model of the scene consisting of a set of geometric primitives. We evaluate the approach on an extensive set of scenarios and show its plausibility in augmented reality applications.
2016
Fleck, Philipp; Arth, Clemens; Schmalstieg, Dieter
Scalable Mobile Image Recognition for Real-Time Video Annotation Proceedings Article
In: 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), pp. 338-339, 2016.
@inproceedings{7836531,
title = {Scalable Mobile Image Recognition for Real-Time Video Annotation},
author = {Philipp Fleck and Clemens Arth and Dieter Schmalstieg},
doi = {10.1109/ISMAR-Adjunct.2016.0110},
year = {2016},
date = {2016-09-01},
booktitle = {2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct)},
pages = {338-339},
abstract = {Traditional AR frameworks for gaming and advertising focus on tracking 2D static targets. This limits the plausible use of this solutions to certain application cases like brochures or posters, but deprives their use for dynamically changing 2D targets, such as video walls or electronic billboards used in advertising.In this demo, we show how to use a rapid, fully mobile image recognition system to introduce AR in videos playing on TV sets or other dynamic screens, without the need to alter or modify the content for trackability. Our approach uses a scalable and fully mobile concept, which requires a database with a very small memory footprint on mobiles for a video or even a collection of videos.The feasibility of the approach is demonstrated on over 16 hours of video from a popular TV series, indexing into the video and giving accurate time codes and full 6DOF tracking for AR augmentations.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Traditional AR frameworks for gaming and advertising focus on tracking 2D static targets. This limits the plausible use of this solutions to certain application cases like brochures or posters, but deprives their use for dynamically changing 2D targets, such as video walls or electronic billboards used in advertising.In this demo, we show how to use a rapid, fully mobile image recognition system to introduce AR in videos playing on TV sets or other dynamic screens, without the need to alter or modify the content for trackability. Our approach uses a scalable and fully mobile concept, which requires a database with a very small memory footprint on mobiles for a video or even a collection of videos.The feasibility of the approach is demonstrated on over 16 hours of video from a popular TV series, indexing into the video and giving accurate time codes and full 6DOF tracking for AR augmentations.
Fleck, Philipp; Schmalstieg, Dieter; Arth, Clemens
Visionary collaborative outdoor reconstruction using SLAM and SfM Proceedings Article
In: 2016 IEEE 9th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS), pp. 1-2, 2016, ISSN: 2328-7829.
@inproceedings{7551588,
title = {Visionary collaborative outdoor reconstruction using SLAM and SfM},
author = {Philipp Fleck and Dieter Schmalstieg and Clemens Arth},
doi = {10.1109/SEARIS.2016.7551588},
issn = {2328-7829},
year = {2016},
date = {2016-03-01},
booktitle = {2016 IEEE 9th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS)},
pages = {1-2},
abstract = {In this position paper, we argue about a concept for collaborative outdoor reconstruction using SLAM clients and a scalable SfM engine running in the cloud. Based on previous observations and results, we discuss issues like illumination changes, overall scalability or the dacay of buildings, having a serious impact on the practical feasibility of such a system. Revisiting ideas and insights from work on outdoor reconstruction and localization done in the last couple of years, we outline an idea for collaborative and vivid reconstruction of the world, potentially through the cameras of millions of mobile devices.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this position paper, we argue about a concept for collaborative outdoor reconstruction using SLAM clients and a scalable SfM engine running in the cloud. Based on previous observations and results, we discuss issues like illumination changes, overall scalability or the dacay of buildings, having a serious impact on the practical feasibility of such a system. Revisiting ideas and insights from work on outdoor reconstruction and localization done in the last couple of years, we outline an idea for collaborative and vivid reconstruction of the world, potentially through the cameras of millions of mobile devices.
2015
Fleck, Philipp; Arth, Clemens; Pirchheim, Christian; Schmalstieg, Dieter
[POSTER] Tracking and Mapping with a Swarm of Heterogeneous Clients Proceedings Article
In: 2015 IEEE International Symposium on Mixed and Augmented Reality, pp. 136-139, 2015.
@inproceedings{7328080,
title = {[POSTER] Tracking and Mapping with a Swarm of Heterogeneous Clients},
author = {Philipp Fleck and Clemens Arth and Christian Pirchheim and Dieter Schmalstieg},
url = {https://youtu.be/UIXsv3ROCVs?si=1C1U1BC3c--OJDkJ},
doi = {10.1109/ISMAR.2015.40},
year = {2015},
date = {2015-09-01},
urldate = {2015-09-01},
booktitle = {2015 IEEE International Symposium on Mixed and Augmented Reality},
pages = {136-139},
abstract = {In this work, we propose a multi-user system for tracking and mapping, which accommodates mobile clients with different capabilities, mediated by a server capable of providing real-time structure from motion. Clients share their observations of the scene according to their individual capabilities. This can involve only keyframe tracking, but also mapping and map densification, if more computational resources are available. Our contribution is a system architecture that lets heterogeneous clients contribute to a collaborative mapping effort, without prescribing fixed capabilities for the client devices. We investigate the implications that the clients' capabilities have on the collaborative reconstruction effort and its use for AR applications.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this work, we propose a multi-user system for tracking and mapping, which accommodates mobile clients with different capabilities, mediated by a server capable of providing real-time structure from motion. Clients share their observations of the scene according to their individual capabilities. This can involve only keyframe tracking, but also mapping and map densification, if more computational resources are available. Our contribution is a system architecture that lets heterogeneous clients contribute to a collaborative mapping effort, without prescribing fixed capabilities for the client devices. We investigate the implications that the clients' capabilities have on the collaborative reconstruction effort and its use for AR applications.
Fleck, Philipp
Multiuser SLAM Masters Thesis
2015.
@mastersthesis{fleck_masters_thesis_2015,
title = {Multiuser SLAM},
author = {Philipp Fleck},
url = {https://repository.tugraz.at/publications/axxyv-xmf37},
doi = {10.3217/axxyv-xmf37},
year = {2015},
date = {2015-03-01},
urldate = {2015-03-01},
abstract = {We present a method to combine multiple local SLAM maps into combined maps in a client-server system. The server takes care of all clients and tries to detect overlapping regions among keyframes committed by clients. The system supports different clients with different levels of complexity, such as a thin client, which is used for image acquisitions, or an autonomous SLAM client, which generates its own local map. If clients move, the combined map is refreshed to keep pace with the client’s local map. Beyond the combination of client maps, the server system can update clients to improve their local system using keyframes and poses. Allowing clients to operate in the same context, will serve as a base for future AR applications. In particular, multiple clients commit their keyframes and the server generates a per-client reconstruction, as well a combined map. Afterwards, the clients receive updates in form of new keyframes and poses, to improve and enlarge their local system.},
keywords = {},
pubstate = {published},
tppubtype = {mastersthesis}
}
We present a method to combine multiple local SLAM maps into combined maps in a client-server system. The server takes care of all clients and tries to detect overlapping regions among keyframes committed by clients. The system supports different clients with different levels of complexity, such as a thin client, which is used for image acquisitions, or an autonomous SLAM client, which generates its own local map. If clients move, the combined map is refreshed to keep pace with the client’s local map. Beyond the combination of client maps, the server system can update clients to improve their local system using keyframes and poses. Allowing clients to operate in the same context, will serve as a base for future AR applications. In particular, multiple clients commit their keyframes and the server generates a per-client reconstruction, as well a combined map. Afterwards, the clients receive updates in form of new keyframes and poses, to improve and enlarge their local system.