Multisensorial Portable Device for Augmented Reality Experiences in Museums

PDF

ABSTRACT

This paper presents a portable device capable of enhancing the augmented reality (AR) experience to the full five senses through the stimulus of touch, taste, and smell. While most of the existing solutions for this type of devices lack portability or fail to explore all the five senses at once, in this work we introduce a new portable device that adapts to the users’ smartphone or tablet to provide a full five senses experience. This is part of a mobile five senses augmented reality system for Museums (M5SAR) project, which aims at developing an AR system to be a guide in cultural and historical events and museums, complementing or replacing the traditional orientation given by a guide, by directional signs or maps, while enhancing the users’ experience adding by new senses to the museums’ objects. The proposed device communicates via Bluetooth with an App (out of the scope of this paper) and is responsible for generating the stimuli that reproduces three senses: touch, taste, and smell, while the senses of sight and hearing are already reproduced through the users’ smartphone or tablet, allowing to explore all the human senses. The proposed hardware is powered by a rechargeable battery, giving the module portability to be used during a visit to the museum. It uses a microcontroller for the core unit, which receives instructions from the mobile application, running on the user’s smart device, and acts respectively, activating the physical interfaces to deliver the five senses experience to the user.

KEYWORDS

Augmented reality, multisensorial display, portable device, five-sense experience, museums

REFERENCES

[1] Yuan, Z., Bi, T., Muntean, G. M. & Ghinea, G. (2015). Perceived synchronization of multisemedia services. IEEE Transactions on Multimedia, 17(7), 957-966.

[2] Kovács, P. T., Rozinaj, G., Murray, N., Sulema, Y. & Rybárová, R. (2015). Application of immersive technologies for education: state of the art. Paper presented at the 2015 International Conference on Interactive Mobile Communication Technologies and Learning (IMCL), Thessaloniki, Greece. 0 200200200 400400400 600600600 800800800 MCUMCU Bluetooth BluetoothBluetooth BluetoothBluetoothBluetooth Thermal ThermalThermal ThermalThermal Vibration VibrationVibrationVibration VibrationVibration Airflow Airflow Taste Taste SmellSmellSmellSmell Energy (mWh)Energy (mWh)Energy (mWh) Energy (mWh)Energy (mWh)Energy (mWh)Energy (mWh)Energy (mWh)Energy (mWh)Energy (mWh)Energy (mWh)

[3] Wolf, M. J. (2008). The video game explosion: a history from PONG to Playstation and beyond. ABC-CLIO.

[4] Bau, O., Poupyrev, I., Israr, A., & Harrison, C. (2010). TeslaTouch: electrovibration for touch surfaces. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (pp. 283-292). ACM.

[5] Alexander, J., Marshall, M. T., & Subramanian, S. (2011). Adding haptic feedback to mobile tv. In CHI'11 Extended Abstracts on Human Factors in Computing Systems (pp. 1975-1980). ACM.

[6] Subramanian, S., Seah, S. A., Shinoda, H., Hoggan, E., & Corenthy, L. (2016). Mid-Air Haptics and Displays: Systems for Un-instrumented Mid-air Interactions. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (pp. 3446-3452). ACM.

[7] Moon, T., & Kim, G. J. (2004). Design and evaluation of a wind display for virtual reality. In Proceedings of the ACM symposium on Virtual reality software and technology (pp. 122-128). ACM.

[8] Sodhi, R., Poupyrev, I., Glisson, M., & Israr, A. (2013). AIREAL: interactive tactile experiences in free air. ACM Transactions on Graphics (TOG), 32(4), 134.

[9] Harrison, C., & Hudson, S. E. (2009). Providing dynamically changeable physical buttons on a visual display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 299-308). ACM.

[10] Geomagic (2016) Geomagic. Retrieved: Dec. 6, 2016 from http://www.geomagic.com.

[11] Benali-Khoudja, M., Hafez, M., Alexandre, J. M., Benachour, J., & Kheddar, A. (2003). Thermal feedback model for virtual reality. In International Symposium on Micromechatronics and Human Science (Vol. 1, pp. 153-158).

[12] Kaye, J. N. (2001). Symbolic olfactory display (Doctoral dissertation, Massachusetts Institute of Technology).

[13] Matsukura, H., Yoneda, T., & Ishida, H. (2013). Smelling screen: development and evaluation of an olfactory display system for presenting a virtual odor source. IEEE transactions on visualization and computer graphics, 19(4), 606-615.

[14] Onotes (2016) Onotes. Retrieved: Dec. 6, 2016 from http://www.onotes.com/.

[15] Ranasinghe, N., Cheok, A. D., Fernando, O. N. N., Nii, H., & Ponnampalam, G. (2011). Electronic taste stimulation. In Proceedings of the 13th international conference on Ubiquitous computing (pp. 561-562). ACM

[16] Ranasinghe, N., Nakatsu, R., Nii, H., & Gopalakrishnakone, P. (2012). Tongue mounted interface for digitally actuating the sense of taste. In Wearable Computers (ISWC), 2012 16th International Symposium on (pp. 80-87). IEEE.

[17] FeelReal (2016) FeelReal. Retrieved: Dec. 6, 2016 from http://feelreal.com/.

[18] Museum of Food and Drink (2016) Flavour. Retrieved: Sep. 2017 from http://www.mofad.org/flavor/

[19] Farsalinos, K. E., & Polosa, R. (2014). Safety evaluation and risk assessment of electronic cigarettes as tobacco cigarette substitutes: a systematic review. Therapeutic advances in drug safety, 5(2), 67-86.

Cite this paper

João D. P. Sardo, João A. R. Pereira, Ricardo J. M. Veiga, Jorge Semião, Pedro J. S. Cardoso, João M. F. Rodrigues. (2018) Multisensorial Portable Device for Augmented Reality Experiences in Museums. International Journal of Education and Learning Systems, 3, 60-69