AIMR, AI-based decision support in mixed reality environments
The AIMR (AI based Decision Support in MR-Environments) project aims to simulate real influencing factors and test their effects on users using various sensors. For this purpose, Mixed Reality (MR) environments are enabled, enriched via various IoT sensors and the data obtained is analysed using suitable AI algorithms and corresponding results are derived. The central element for this is the construction and use of a low-cost VR-CAVE, a type of virtual reality (VR) environment in which several screens are positioned in such a way that they form a 2x3 metre room and enable a virtual presentation of 3D data. The physical cave space not only serves as a platform for the visual presentation, but also provides an environment for the fixed installation of sensor technology, which allows consistent use and analysis in various application scenarios.
Motivation
In particular, the physical freedom of movement without the need for VR headsets and navigation without controllers within the VR cave make it possible to experience simulations with several people at the same time and to consistently analyse their natural movement patterns. **In addition, physical objects such as measuring instruments can also be integrated into the room depending on the scenario, which enables the combination of real and virtual components in the sense of a mixed reality environment. The simulation environment is not only used to display statistical and spatial data, but in particular for the immersive visualisation of city models under various environmental factors. These are difficult to visualise in reality due to potential hazards, costs or other factors.
The movement data of the test subjects is recorded during the simulation and supplemented by additional IoT sensors, such as ground weight, heart rate or temperature sensors. This makes it possible to investigate the influence of the environmental visualisation on the test subjects in various scenarios. Under laboratory conditions, the cave room therefore offers the ideal environment for combining personal sensor and 3D movement data and analysing their complex interrelationships holistically.
Activities
The project centres not only on the construction of the Cave itself, but also on the creation of mixed reality scenarios. These enable comprehensive analyses of existing and newly developed AI algorithms. For this purpose, a real laboratory in the form of a 2x3 metre low-cost VR cave will be realised using 8 monitors, a touch-sensitive floor panel and additional camera and IoT hardware. The following three application scenarios are currently planned: analysing movement patterns in space, determining the degree of activation of users of MR applications and virtual surveying exercises in the Geoinformatics & Surveying degree programme.
The research and development work being carried out by i3mainz will initially focus on city models that are designed to offer users an immersive experience by providing a realistic and three-dimensional representation of urban environments. The application potential of these freely explorable city models is to be investigated in conjunction with suitable AI-based analysis methods in the context of the RAVFINIERT project in various citizen survey scenarios. With regard to the use of the Cave in the Geoinformatics & Surveying degree programme, it will be investigated whether complex surveying exercises can be prepared with students in the Cave. For example, the placement of totalisers or 3D scanners within the Cave is being considered in order to carry out virtual measuring exercises with real measuring instruments.