Iranian Journal of Biomedical Engineering (IJBME)
نشریه علمی مهندسی پزشکی زیستی

Concealed Information Recognition with the Fusion of Physiological Communication Network of Facial Areas and Psychological Analysis

Document Type : Full Research Paper

Authors

Amin Mohammadian 1
Akram Ghorbali 2
Maryam Asadolah Tooyserkani 3
Razieh kaveh 4
kian Shahi 4

1 Research Assistant Professor, Research Center for Development of Advanced Technologies, Tehran, Iran

2 Assistant Professor, Department of Psychology, South Tehran Branch, Islamic Azad University, Tehran, Iran

3 Assistant Professor, Department of Psychology, East Tehran Branch, Islamic Azad University, Tehran, Iran

4 Researcher, Research Center for Development of Advanced Technologies, Tehran, Iran

https://doi.org/10.22041/ijbme.2022.545465.1744
Abstract
The interview analyst’s need to detect deception is a topic that has provided the conditions for providing solutions to empower them. So that, the experts and interview analysts can be assisted by automatically monitoring the subject's unsalient, unknown, or counterintuitive activities during the interview. The aim of this study was to combine quantitative and qualitative information to help improve the detection of deception. For this purpose, in addition to using the capacity of verbal and non-verbal analysis methods, thermal imaging technology and new methods of spatiotemporal analysis of the thermal patterns have been used to detect concealed information in individuals. Then, based on the study design, the database consisting of 48 truth-tellers and liars who participated in a mock scenario was collected. Then, two qualitative methods of verbal and non-verbal information analysis, including standard criteria-based content analysis (CBCA) and behavioral analysis interview (BAI) scoring, were used to identify liars and truth-tellers. In order to complete the obtained results based on these two methods, using effective connectivity analysis method, physiological network analysis of communication between different areas of the face was performed in thermal images of individuals. As a result of combining quantitative and qualitative information, the final accuracy of individuals' diagnosis increased from an average of 73.61% to 79.17%. The investigation of the agreement analysis between methods by kappa coefficient and analysis of confusion matrix information indicated the existence of complementary information in various quantitative and qualitative methods to identify concealed information in individuals.

Keywords

Concealed Information
Effective Connectivity
Thermal Image
Verbal and Nonverbal Behavioral Analysis

Subjects

  1. Dcosta, M., et al. Perinasal indicators of deceptive behavior. in 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG). 2015.
  2. Lai, V. and C. Tan. On human predictions with explanations and predictions of machine learning models: A case study on deception detection. in Proceedings of the Conference on Fairness, Accountability, and Transparency. 2019.
  3. Porter, S. and L. ten Brinke, The truth about lies: What works in detecting high‐stakes deception? Legal and criminological Psychology, 2010. 15(1): p. 57-75.
  4. Frank, M.G. and E. Svetieva, Microexpressions and deception, in Understanding facial expressions in communication. 2015, Springer. p. 227-242.
  5. Ekman, P. and W.V. Friesen, Detecting deception from the body or face. Journal of personality and Social Psychology, 1974. 29(3): p. 288.
  6. Kircher, J.C., Chapter 9 - Ocular-Motor Deception Test, in Detecting Concealed Information and Deception, J.P. Rosenfeld, Editor. 2018, Academic Press. p. 187-212.
  7. DePaulo, B.M., et al., Cues to Deception. Psychological Bulletin, 2003. 129(1): p. 74-118.
  8. Yu, R., et al., Using Polygraph to Detect Passengers Carrying Illegal Items. Frontiers in Psychology, 2019. 10 (322).
  9. Ambach, W. and M. Gamer, Chapter 1 - Physiological Measures in the Detection of Deception and Concealed Information, in Detecting Concealed Information and Deception, J.P. Rosenfeld, Editor. 2018, Academic Press. p. 3-33.
  10. Rosenfeld, J.P., et al., Chapter 6 - Effects of Motivational Manipulations on the P300-Based Complex Trial Protocol for Concealed Information Detection, in Detecting Concealed Information and Deception, J.P. Rosenfeld, Editor. 2018, Academic Press. p. 125-143.
  11. Tsiamyrtzis, P., et al., Imaging Facial Physiology for the Detection of Deceit. International Journal of Computer Vision, 2007. 71(2): p. 197-214.
  12. Pavlidis, I., N.L. Eberhardt, and J.A. Levine, Seeing through the face of deception. Nature, 2002. 415(6867): p. 35-35.
  13. Gálvez-García, G., et al., A trifactorial model of detection of deception using thermography. Psychology, Crime & Law, 2020: p. 1-22.
  14. Derakhshan, A., et al., Network physiology of ‘fight or flight’ response in facial superficial blood vessels. Physiological Measurement, 2019. 40(1): p. 014002.
  15. Derakhshan, A., et al., Identifying the Optimal Features in Multimodal Deception Detection. Multimodal Technologies and Interaction, 2020. 4(2): p. 25.
  16. ا. درخشان., م. ع. خلیل‏زاده و ا. محمدیان, آشکارسازی استرس با استفاده از سیگنال دمای میانگین ناحیه دور چشم. مهندسی برق مدرس, 1389. سال دهم (4).
  17. م. قدوسی, ز. بهمنی, و ا. محمدیان, تشخیص فریب باا ستفاده ازویدئوی حرارتی ثبت شده ازچهره فرد درحین پروتکل دروغ سنجی زمان کوتاه, در بیستمین کنفرانس مهندسی پزشکی ایران. 1392.
  18. Panasiti, M.S., et al., Thermal signatures of voluntary deception in ecological conditions. Scientific Reports, 2016. 6(1): p. 35174.
  19. Shastri, D., et al., Perinasal imaging of physiological stress and its affective potential. IEEE Transactions on Affective Computing, 2012. 3(3): p. 366-378.
  20. ک. شاهی و همکاران, دادگان چند مدالیته (سیگنال های فیزیولوژیک و تصاویر حرارتی) از افراد جهت ارائه شاخص پیوسته از سطوح مختلف برانگیختگی ذهنی, در بیست و ششمین کنفرانس ملی و چهارمین کنفرانس بین المللی مهندسی‌ زیست پزشکی ایران. 1398.
  21. Park, K.K., et al., A functional analysis of deception detection of a mock crime using infrared thermal imaging and the Concealed Information Test. Frontiers in human neuroscience, 2013. 7: p. 70-70.
  22. Burzo, M., et al., Multimodal deception detection, in The Handbook of Multimodal-Multisensor Interfaces: Signal Processing, Architectures, and Detection of Emotion and Cognition-Volume 2. 2018. p. 419-453.
  23. Abouelenien, M., R. Mihalcea, and M. Burzo. Analyzing thermal and visual clues of deception for a non-contact deception detection approach. in Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments. 2016.
  24. Derrick, D.C., et al., Border security credibility assessments via heterogeneous sensor fusion. IEEE Intelligent Systems, 2010(3): p. 41-49.
  25. Horvath, F.S., Verbal and nonverbal clues to truth and deception during polygraph examinations. Journal of Police Science & Administration, 1973.
  26. Vrij, A., Detecting lies and deceit: Pitfalls and opportunities. 2008: John Wiley & Sons.
  27. Inbau, F., et al., Criminal interrogation and confessions. 2011: Jones & Bartlett Publishers.
  28. Colwell, L.H., et al., The training of law enforcement officers in detecting deception: A survey of current practices and suggestions for improving accuracy. Police Quarterly, 2006. 9(3): p. 275-290.
  29. Horvath, F., B. Jayne, and J. Buckley, Differentiation of truthful and deceptive criminal suspects in behavior analysis interviews. Journal of Forensic Science, 1994. 39(3): p. 793-807.
  30. Vrij, A., et al., Detecting deceit via analysis of verbal and nonverbal behavior. Journal of Nonverbal behavior, 2000. 24(4): p. 239-263.
  31. Amado, B.G., et al., Criteria-Based Content Analysis (CBCA) reality criteria in adults: A meta-analytic review. International Journal of Clinical and Health Psychology, 2016. 16(2): p. 201-210.
  32. Bogaard, G., K. Colwell, and S. Crans, Using the Reality Interview improves the accuracy of the Criteria‐Based Content Analysis and Reality Monitoring. Applied Cognitive Psychology, 2019. 33(6): p. 1018-1031.
  33. Amado, B.G., R. Arce, and F. Fariña, Undeutsch hypothesis and Criteria Based Content Analysis: A meta-analytic review. The European Journal of Psychology Applied to Legal Context, 2015. 7(1): p. 3-12.
  34. Hauch, V., et al., Can credibility criteria be assessed reliably? A meta-analysis of criteria-based content analysis. Psychological Assessment, 2017. 29(6): p. 819.
  35. Oberlader, V.A., et al., Validity of content-based techniques to distinguish true and fabricated statements: A meta-analysis. Law and human behavior, 2016. 40(4): p. 440.
  36. Ghodsi, S., H. Mohammadzade, and H. Aghajan, Analysis of Brain Connectivity for Epileptic Seizure Prediction using EEG Signals. Iranian Journal of Biomedical Engineering, 2019. 13(3): p. 189-207.
  37. Granger, C.W., Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society, 1969: p. 424-438.
  38. Schiatti, L., et al., Extended Granger causality: a new tool to identify the structure of physiological networks. Physiological measurement, 2015. 36(4): p. 827.
  39. Vrij, A., Cooperation of liars and truth tellers. Applied Cognitive Psychology, 2005, 19 (1), p. 39-50.
  40. Tsiamyrtzis, P., et al., Imaging facial physiology for the detection of deceit. International Journal of Computer Vision, 2007. 71(2): p. 197-214.
  41. Rajoub, B.A. and R. Zwiggelaar, Thermal facial analysis for deception detection. IEEE transactions on information forensics and security, 2014. 9(6): p. 1015-1023.
  42. Derakhshan, A., et al. Preliminary study on facial thermal imaging for stress recognition. in Intelligent Environments (Workshops). 2014.
  43. Pollina, D.A., S.M. Senter, and R.G. Cutlip, HEMIFACIAL SKIN TEMPERATURE CHANGES RELATED TO DECEPTION: BLOOD FLOW OR THERMAL CAPACITANCE? Journal of Global Research in Computer Science, 2015. 6 (4).
  44. Mitani, A.A., P.E. Freer, and K.P. Nelson, Summary measures of agreement and association between many raters' ordinal classifications. Annals of epidemiology, 2017. 27(10): p. 677-685.e4.
  45. G, C., Cohen's kappa compute the Cohen's kappa ratio on a 2x2 matrix. 2007.
  46. Kassin, S.M. and C.T. Fong, “I'm innocent!”: Effects of training on judgments of truth and deception in the interrogation room. Law and Human Behavior, 1999. 23(5): p. 499-516.
  47. Mann, S., A. Vrij, and R. Bull, Suspects, lies, and videotape: An analysis of authentic high-stake liars. Law and human behavior, 2002. 26(3), p. 365-376.
  48. Mann, S., A. Vrij, and R. Bull, Detecting True Lies: Police Officers' Ability to Detect Suspects' Lies. Journal of applied psychology, 2004. 89(1): p. 137.
  49. Vrij, A., S. Mann, and R.P. Fisher, An empirical test of the behaviour analysis interview. Law and human behavior, 2006. 30(3): p. 329-345.
  50. Horvath, F., J. Blair, and J.P. Buckley, The behavioural analysis interview: clarifying the practice, theory and understanding of its use and effectiveness. International Journal of Police Science & Management, 2008. 10(1): p. 101-118.
  51. Akehurst, L., et al., An evaluation of a new tool to aid judgements of credibility in the medico‐legal setting. Legal and Criminological Psychology, 2017. 22(1): p. 22-46.
Iranian Journal of Biomedical Engineering (IJBME)
Volume 16, Issue 1
Spring 2022
Pages 33-50

  • Receive Date 23 December 2021
  • Revise Date 22 April 2022
  • Accept Date 14 May 2022

Home

Contact Us