Multimodal interactive 3D technologies

Higher education teachers: Mihelj Matjaž
Collaborators: Riener Robert

Subject description

Prerequisits:

• Enrolment in the first year.

Content (Syllabus outline):

Psychophysiology of human visual, auditory and haptic sensing; Methods for modelling and rendering of three-dimensional stimuli of all three modalities (visual, auditory and haptic); Collision detection between complex virtual objects; Technologies for spatial presentation of synthesized 3D stimuli (stereoscopic and holographic displays, spatial sound generation, haptic robots for presentation of kinaesthetic and tactile stimuli); Technologies and methods for user movement tracking that enable interaction with the environment and navigation within the environment; Augmented reality as interaction with real and telepresence as interaction with remote environment through digital medium; Analysis of users’ psychophysiological responses and strategies for real time adaptation of virtual environment based on users’ psychophysiological state; Use of interactive 3D technologies in areas such as product design, sales and marketing, architecture and design, education, medicine, research and development.

Objectives and competences:

The course enables the student to understand interactive 3D presentations, to design interactive 3D applications and to select adequate presentation technologies. It deals with methods and technologies for interactive presentations of three-dimensional computer generated environments by conveying visual, auditory and haptic stimuli. Analysed are effects of multimodal virtual environments on human psychophysiological state (immersion, presence) and presented are methods for assessment of psychophysiological responses in real-time. Students learn to design and implement interactive 3D simulations and interactive presentations used for functional product analysis, 3D simulation based learning (medicine), marketing applications, entertainment and other applications.

Intended learning outcomes:

Understanding psychophysiology of human perception of environmental stimuli; Methods for analysis and synthesis of movement in human and robot; Synthesis of visual, auditory and haptic stimuli; Methods of affecting human’s psychophysiological responses. Finding solutions to practical problems related to human/machine (robot, computer) interactions. Skills required for problem solving, design, and implementation of solutions, critical analysis and synthesis, use of software tools. Implementation of interactive 3D applications in industry, medicine, science, …

Learning and teaching methods:

• Multimedia supported lectures,
• individual project based work.

Study materials

1. M. Mihelj, J. Podobnik, Haptics for Virtual Reality and Teleoperation, Springer, 2012.
2. M. Mihelj, D. Novak, S. Beguš, Virtual Reality Technology and Applications, Springer, 2013.
3. B. Furht, Handbook of Augmented Reality, Springer, 2011.
4. W. Sherman, A. B. Craig: Understanding Virtual Reality, Morgan Kaufmann, 2003.