Iranian Journal of Biomedical Engineering (IJBME)
نشریه علمی مهندسی پزشکی زیستی

A Novel Framework for Predicting the Score of Nasal Aesthetic based on Machine Learning Algorithms

Document Type : Full Research Paper

Authors

Maryam Ashoori 1
Reza A. Zoroofi 2
Mohammad Sadeghi 3

1 Ph.D. Student, Control and Intelligent Processing Center of Excellence, School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran

2 Professor, Control and Intelligent Processing Center of Excellence, School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran

3 Professor, Imam Khomeini Hospital Complex, Tehran University of Medical Sciences, Tehran, Iran

https://doi.org/10.22041/ijbme.2024.2014643.1867
Abstract
Currently, the rapid growth of the beauty industry, along with the development of intelligent models based on machine learning algorithms, has led to an increase in extensive research in this field. Rhinoplasty is one of the most common and demanding facial cosmetic surgeries because the nose is the most prominence element of the face, which has a great impact on its attractiveness. The purpose of this article is to present a machine learning-based framework for predicting nasal aesthetic evaluation. In this article, a  set of geometric parameters of the nose relative to the entire face are given as input and human rating as output to the popular machine learning regression algorithms. An ablation study was then carried out to examine the influence of facial shape, skin color, and texture on the beauty of the nose. Multilayer perceptron classification, K-means clustering, and grey level co-occurrence matrix were used to extract facial shape, skin color, and texture. The results show that the model based on geometric parameters has a moderate correlation with human rating, and by adding each subset of the features of face shape, color, and skin texture, the correlation of the obtained model increases until a high degree of correlation is achieved. The results also show that the random forest algorithm has the best performance among other algorithms based on the evaluation criteria of absolute mean error, root mean square error, and Pearson correlation. The results of this study show that the proposed framework can be helpful in determining the beauty of the nose.

Keywords

Nasal Aesthetic
Machine Learning
Geometric Parameters
Face Shape
Skin Color
Skin Texture

Subjects

  1. Zhao , M. Zhang , C. He , X. Xie and J. Li, "A novel facial attractiveness evaluation system based on face shape, facial structure features and skin," Cogn Neurodyn, vol. 14, no. 5, pp. 643-656, 2020.
  2. Martos, A. Valsecchi, O. Ibáñez and I. Alemán, "Estimation of 2D to 3D dimensions and proportionality indices for facial examination," Forensic Sci. Int., vol. 287, pp. 142-154, June, 2018.
  3. Rhodes, F. Proffitt, J. M. Grady and A. Sumich , "Facial symmetry and the perception of beauty," Psychon. Bull. Rev., vol. 5, no. 4, p. 659–669, December, 1998.
  4. Eaves and D. G. Leathers , Successful Nonverbal Communication: Principles and Applications, Routledge, 2015, p. 142.
  5. GoodStein, Face This: Real advice from real models, photographers and makeup artists on how to become Picture Perfect, CreateSpace Independent Publishing Platform, 2011, p. 39.
  6. -H. Liao, R. Tong, J. Geng and M. Tang, "Inhomogeneous volumetric Laplacian deformation for rhinoplasty planning and simulation system," Comput Animat Virtual Worlds, vol. 21, no. 3-4, pp. 331-341, May, 2010.
  7. -x. Wang , S.-h. Liao, X.-h. Zhu, Y. Wang, C.-x. Ling, X. Ding, Y.-m. Fang and X.-h. Zhang, "Real time 3D simulation for nose surgery and automatic individual prosthesis design," Comput Methods Programs Biomed, vol. 104, no. 3, pp. 472-479, December, 2011.
  8. M. T. Foda, "Challenging Problems in Rhinoplasty," Facial Plast Surg, vol. 32, no. 4, pp. 331-332, August, 2016.
  9. Guevara and M. Matouk, "In-office 3D printed guide for rhinoplasty," Int J Oral Maxillofac Surg, vol. 50, no. 12, pp. 1563-1565, December, 2021.
  10. Hafezi , K. Kochakzadeh and B. Naghibzadeh, "History and Status of Nose Surgery," Iranian Journal of Surgery, vol. 17, no. 2, pp. 88-94, 2009.
  11. B. Surowitz and S. P. Most, "Complications of Rhinoplasty," Facial Plastic Surgery Clinics, vol. 21, no. 4, pp. 639-651, 2013.
  12. Layliev , V. Gupta , C. Kaoutzanis, N. G. Kumar, J. Winocour, J. C. Grotting and K. K. Higdon, "Incidence and Preoperative Risk Factors for Major Complications in Aesthetic Rhinoplasty: Analysis of 4978 Patients," Aesthet Surg J, Vols. 37-7, pp. 757-767, July, 2017.
  13. "ISAPS International Survey on Aesthetic/ Cosmetic Procedures Performed in 2014," 2017.
  14. Mehta and R. K. Srivastava, "The Indian Nose: An anthropometric analysis," J Plast Reconstr, vol. 70, no. 10, pp. 1472-1482, October, 2017.
  15. J. Iyer, R. K. R. Nersisson, Z. Zhuang, A. Noel, J. Raj and I. Refayee. (2021). Machine Learning-Based Facial Beauty Prediction and Analysis of Frontal Facial Images Using Facial Landmarks and Traditional Image Descriptors, Comput. Intell. Neurosci. Available: https:// www.hindawi.com/journals/cin/2021/4423407/
  16. G. Cula, K. J. Dana, F. P. Murphy and B. K. Rao, "Skin texture modeling," Int J Comput Vis, vol. 62, no. 1, pp. 97-119, April, 2005.
  17. S. Atiyeh and S. N. Hayek, "Numeric expression of aesthetics and beauty,"  Aesthetic Plast Surg, vol. 32, no. 2, pp. 209-216, March, 2008.
  18. Face_alignment, Available: http://dlib.net/face_ alignment .py.html.
  19. I. Perrett , K. A. May and S. Yoshikawa, "Facial shape and judgements of female attractiveness," Nature, vol. 368, no. 6468, pp. 239-242, 1994.
  20. O'Toole, T. Price, T. Vetter, J. Bartlett and V. Blanz, "3D shape and 2D surface textures of human faces: the role of “averages” in attractiveness and age," Image Vis Comput, vol. 18, no. 1, pp. 9-19, 1999.
  21. Fink , K. Grammer and R. Thornhill, "Human (Homo sapiens) facial attractiveness in relation to skin texture and color," J. Comp. Psychol., vol. 115, no. 1, pp. 92-99, 2001.
  22. C. Jones, A. C. Little, D. M. Burt and D. I. Perrett, "When facial attractiveness is only skin deep," Perception, vol. 33, no. 5, pp. 569-576, 2004.
  23. Gunes and M. Piccardi, "Assessing facial beauty through proportion analysis by image processing and supervised learning," Int. J. Hum. Comput., vol. 64, no. 12, pp. 1184-1199, 2006.
  24. Eisenthal, G. Dror and E. Ruppin, "Facial attractiveness: beauty and the machine," Neural Comput., vol. 18, no. 1, pp. 119-142, 2006.
  25. Fink, K. Grammer and P. J. Matts, "Visible skin color distribution plays a role in the perception of age, attractiveness, and health in female faces," Evol. Hum. Behav., vol. 27, no. 6, pp. 433-442, 2006.
  26. Schmid, D. Marx and A. Samal , "Computation of a Face Attractiveness Index Based on Neoclassical Canons, Symmetry, and Golden Ratios," Pattern Recognit, vol. 41, pp. 2710-2717, 2008.
  27. M. Bronstad , J. H. Langlois and R. Russell, "Computational models of facial attractiveness judgments," Perception, vol. 37, no. 1, pp. 126-142, 2008.
  28. Fan, K. Chau, X. Wan, L. Zhai and E. Lau, "Prediction of facial attractiveness from facial proportions," Pattern Recognit, vol. 45, no. 6, pp. 2326-2334, 2012.
  29. Fink , P. J. Matts, D. D'Emiliano, L. Bunse, B. Weege and S. Röder, "Colour homogeneity and visual perception of age, health and attractiveness of male facial skin," J. Eur. Acad. Dermatol. Venereol., vol. 26, no. 12, pp. 1486-1492, 2012.
  30. Liao , X. Jin and W. Zeng, "Enhancing the symmetry and proportion of 3D face geometry," IEEE Trans Vis Comput Graph, vol. 18, no. 10, pp. 1704-1716, 2012.
  31. Milutinovic, K. Zelic and N. Nedeljkovic, "Evaluation of facial beauty using anthropometric proportions," Sci. World J., vol. 20, pp. 1-8, 2014.
  32. d. L. Carrito, I. M. B. d. Santos, C. E. Lefevre, R. D. Whitehead, C. F. d. Silva and D. I. Perrett, "The role of sexually dimorphic skin colour and shape in attractiveness of male faces," Evol. Hum. Behav., vol. 37, no. 2, pp. 125-133, 2016.
  33. Lei, Z. David, S. Ming-Ming and C. Fang-Mei, "Facial beauty analysis based on geometric feature: Toward attractiveness assessment application," Expert Syst. Appl., vol. 82, pp. 252-265, October, 2017.
  34. -J. Hong , G. P. Nam , H. Choi , J. Cho and . I.-J. Kim, "A Novel Framework for Assessing Facial Attractiveness Based on Facial Proportions," Symmetry, vol. 9, no. 12, 2017.
  35. L. Jones, "The influence of shape and colour cue classes on facial health perception," Evol. Hum. Behav., vol. 39, no. 1, pp. 19-29, 2018.
  36. Przylipiak , J. Przylipiak, R. Terlikowski , E. Lubowicka , L. Chrostek and A. Przylipiak, "Impact of face proportions on face attractiveness," J Cosmet Dermatol, vol. 17, no. 6, pp. 954-959, 218.
  37. Zhao , M. Zhang, C. He and K. Zuo, "Data-Driven Research on the Matching Degree of Eyes, Eyebrows and Face Shapes," Front. Psychol., vol. 10, 2019.
  38. Xiao, Y. Wu, D. Wang, Y.-L. Yang and X. Jin, "Beauty3DFaceNet: Deep geometry and texture fusion for 3D facial attractiveness prediction," Comput Graph, vol. 98, pp. 11-18, 2021.
  39. Chen, W. Li, X. Gao and B. Xiao, "Novel Multi-feature Fusion Facial Aesthetic Analysis Framework," IEEE Trans. Big Data, vol. 9, no. 5, pp. 1302 - 1320, March, 2023.
  40. Żejmo, M. Gielert, M. Grabski and B. Kostek, "Assessing the attractiveness of human face based on machine learning," Procedia Comput. Sci, vol. 225, pp. 1019-1027, 2023.
  41. Sano and H. Kawabata, "A computational approach to investigating facial attractiveness factors using geometric morphometric analysis and deep learning," Sci. Rep, vol.13, 2023.
  42. Okur , I. Yildirim, B. Aydogan and M. A. Kilic, "Outcome of Surgery for Crooked Nose: An Objective Method of Evaluation," Aesthetic Plast Surg, vol. 28, no. 4, pp. 203-207, July, 2004.
  43. Daelemans, T. Leloup, C. Decaesteker and A. D. Mey, "New digital method for quantitative assessment of nasal morphology," Scand J Plast Reconstr Surg Hand Surg Suppl, vol. 40, no. 6, pp. 335-344, 2006.
  44. NAGY and M. Y. MOMMAERTS, "Analysis of the cleft-lip nose in submental-vertical view, Part I – Reliability of a new measurement instrument," J Craniomaxillofac Surg, vol. 35, no. 6-7, pp. 265-277, September–October, 2007.
  45. Terajima , Y. Furuichi, Y. Aoki, T. K. Goto, K. Tokumori and A. Nakasima, "A 3-dimensional method for analyzing facial soft-tissue morphology of patients with jaw deformities," Am J Orthod Dentofacial Orthop, vol. 135, no. 6, pp. 715-722, 2009.
  46. Rosati, C. Dellavia, A. Colombo, M. D. Menezes and C. Sforza, "Nasal base symmetry: a three dimensional anthropometric study," Minerva Stomatol, vol. 58, no. 7-8, pp. 347-357, July-August, 2009.
  47. Roxbury , M. Ishii, A. Godoy, I. Papel, P. J. Byrne, K. D. O. Boahene and L. E. Ishii, "Impact of Crooked Nose Rhinoplasty on Observer Perceptions Of Attractiveness," Laryngoscope, vol. 122, no. 4, pp. 773-778, 2012, April.
  48. Karube , H. Sasaki, S. Togashi, T. Yanagawa, S. Nakane, N. Ishibashi, K. Yamagata, K. Onizawa, K. Adachi, K. Tabuchi, M. Sekido and H. Bukawa, "A novel method for evaluating postsurgical results of unilateral cleft lip and palate with the use of Hausdorff distance: presurgical orthopedic treatment improves nasal symmetry after primary cheiloplasty," Oral Surg Oral Med Oral Pathol Oral Radiol, vol. 114, no. 6, pp. 704-711, December, 2012.
  49. J. Desmedt , T. J. Maal, M. A. Kuijpers , E. M. Bronkhorst, A. M. Kuijpers-Jagtman and P. S. Fudalej, "Nasolabial symmetry and esthetics in cleft lip and palate: analysis of 3D facial images," Clin Oral Investig, vol. 19, no. 8, pp. 1833-1842, November, 2015.
  50. Mosmuller, R. Tan, F. Mulder, Y. Bachour, H. d. Vet and P. D. Griot, "The use and reliability of SymNose for quantitative measurement of the nose and lip in unilateral cleft lip and palate patients," J Craniomaxillofac Surg, vol. 44, no. 10, pp. 1515-1521, October, 2016.
  51. L. Abbas , A. Kurkcuoglu , C. D. Aytop , C. Uysal and C. Pelin, "Perception of Symmetry in Aesthetic Rhinoplasty Patients: Anthropometric, Demographic, and Psychological Analysis," Perception, vol. 46, no. 10, pp. 1151-1170, October, 2017.
  52. Ç. Özkan, "Rhinoplasty Setsquare Device: A Novel Instrument to Verify Columellar and Nasal Dorsal Position at the Midsagittal Line," Plast Reconstr Surg Glob Open, vol. 5, no. 7, p. e1399, July, 2017.
  53. Pietruski, M. Majak, T. Debski and B. Antoszewski, "A novel computer system for the evaluation of nasolabial morphology, symmetry and aesthetics after cleft lip and palate treatment. Part 1: General concept and validation," J Craniomaxillofac Surg, vol. 45, no. 4, pp. 491-504, April, 2017.
  54. Pietruski , M. Majak, E. Pawlowska, A. Skiba and B. Antoszewski, "A novel computer system for the evaluation of nasolabial morphology, symmetry and aesthetics after cleft lip and palate treatment. Part 2: Comparative anthropometric analysis of patients with repaired unilateral complete cleft lip and palate and healthy individual," J Craniomaxillofac Surg, vol. 45, no. 4, pp. 505-514, April, 2017.
  55. E. Linden, H. O. Taylor, S. Vasudavan, M. E. Byrne, C. K. Deutsch, J. B. Mulliken and S. R. Sullivan, "Three-Dimensional Analysis of Nasal Symmetry Following Primary Correction of Unilateral Cleft Lip Nasal Deformity,"  Cleft Palate Craniofac, vol. 54, no. 6, pp. 715-719, November, 2017.
  56. Al-Rudainy, X. Ju, S. Stanton, F. V. Mehendale and A. Ayoub, "Assessment of Regional Asymmetry of the Face Before and After Surgical Correction of Unilateral Cleft Lip," J Craniomaxillofac Surg, vol. 46, no. 6, pp. 974-978, June, 2018.
  57. Liang, J. Yao, P. K. T. Chen, C. Zheng and J. Yang, "Effect of Presurgical Nasoalveolar Molding on Nasal Symmetry in Unilateral Complete Cleft Lip/Palate Patients after Primary Cheiloplasty without Concomitant Nasal Cartilage Dissection: Early Childhood Evaluation," Cleft Palate Craniofac, vol. 55, no. 7, pp. 935-940, August, 2018.
  58. Mercan , C. S. Morrison , E. Stuhaug, L. G. Shapiro and R. W. Tse, "Novel Computer Vision Analysis of Nasal Shape in Children with Unilateral Cleft Lip," J Craniomaxillofac Surg, vol. 46, no. 1, pp. 34-43, January, 2018.
  59. G. Maliha, J. Bekisz, H. A. Liss, L. Witek, P. Coelho and R. L. Flores, "Precision Rhinoplasty Using Virtual Surgical Planning and Departmentally-Manufactured, 3D-Printed, Sterilizable, Patient-Specific Anatomic Models," Plast Reconstr Surg Glob Open, vol. 6, no. 9, pp. 87-88, September, 2018.
  60. Ekrami, P. Claes, A. A. Zaidi, M. D. Shriver and S. V. Dongen, "Measuring asymmetry from high-density 3D surface scans: An application to human faces," PLoS One, vol. 13, no. 12, 2018.
  61. Denadai , C. A. Raposo-Amaral, C. L. Buzzo and C. E. Raposo-Amaral, "Paranasal Fat Grafting Improves the Nasal Symmetry in Patients With Parry-Romberg Syndrome," J. Craniofac. Surg., vol. 30, no. 3, pp. 658-960, May, 2019.
  62. Cheng-I, J. A. Zelken, C. Chun-Shin, L. Lun-Jou, Y. Jui-Yung, C. Shiow-Shuh, C. A. Araniego and H. Yen-Chang, "Computer-aided design and three-dimensional printing improves symmetry in heminasal reconstruction outcomes," J. Plast. Reconstr. Aesthet. Surg., vol. 72, no. 7, pp. 1198-1206, July, 2019.
  63. Mancini , T. L. Gibson , B. H. Grayson, . R. L. Flores , D. Staffenberg and P. R. Shetye, "Three-Dimensional Soft Tissue Nasal Changes After Nasoalveolar Molding and Primary Cheilorhinoplasty in Infants With Unilateral Cleft Lip and Palate," Cleft Palate Craniofac, vol. 56, no. 1, pp. 31-38, Januray, 2019.
  64. Kimura, E. Nozoe, T. Okawachi, K. Ishihata, T. Fuchigami and N. Nakamura, "Three-dimensional analyses of nasolabial forms and upper lip surface symmetry after primary lip repair in patients with complete unilateral cleft lip and palate," J Craniomaxillofac Surg, vol. 47, no. 2, pp. 245-254, February, 2019.
  65. Fishman, C. Whyne, A. Hope and J. Fialkov, "Modeling and measuring average nasal asymmetry by dorsum midline and nose tip lateral deviation," J Plast Reconstr Aesthet Surg, vol. 74, no. 4, pp. 857-865, April, 2021.
  66. C. Cho, J. W. Lee, J. S. Lee, J. H. Lee, J. Y. Ryu, L. Tian, K. Y. Choi, J. D. Yang and H. Y. Chung, "Correction of Secondary Unilateral Cleft Lip Nasal Deformity in Adults Using Lower Lateral Cartilage Repositioning, Columellar Strut, and Onlay Cartilage Graft on the Nasal Tip with Open Rhinoplasty Combined with Reverse-U Incision,"  J Plast Reconstr Aesthet Surg, vol. 74, no. 5, pp. 1077-1086, May, 2021.
  67. J. Kim, H. Lim and D. H. Park, "Current strategies for aesthetic soft tissue refinement in nasal reconstruction," Arch Craniofac Surg, vol. 23, no. 3, pp. 95-102, Jun, 2022.
  68. Li, F. Shu, Y. Zhen, Z. Song, Y. An and Y. Jiang, "Artificial Intelligence for Rhinoplasty Design in Asian Patients," Aesthetic Plast Surg, August, 2023.
  69. T. Bhowmik, C. K. Kandathil and S. P. Most, "Automating the Standardized Cosmesis and Health Nasal Outcomes Survey Classification with Convolutional Neural Networks," Facial Plast Surg Aesthet Med, vol. 25, no. 6, pp. 487-493, November-December, 2023.
  70. M. A. Kiekens, A. M. Kuijpers-Jagtman, M. A. v. Hof, B. E. v. Hof, H. Straatman and J. C. Maltha, "Facial esthetics in adolescents and its relationship to “ideal” ratios and angles," Am J Orthod Dentofacial Orthop, vol. 133, no. 2, pp. e1-8, 2008.
  71. A. Lines, R. R. Lines and C. A. Lines, "Profilemetrics and facial esthetics," Am. J. Orthod., vol. 36, no. 7, pp. 648-657, 1978.
  72. J. Bailey, Atlas of Head & Neck Surgery-otolaryngology, Lippincott Williams & Wilkins, 2001, pp. 458-459.
  73. Jacques, Nasenplastik und sonstige Gesichtsplastik nebst einem Anhang über Mammaplastik und einigen weiteren Operationen aus dem Gebiete der äusseren Körperplastik. Ein Atlas und Lehrbuch, Germany, 1931.
  74. Niechajev, "Prominent nose, its modalities and their treatment," Eur J Plast Surg, vol. 41, pp. 543-556, 2018.
  75. Yang, X. Tan and J. Fang, "Assessment of nasal base morphology using new proportion indices in Chinese," Springer plus, vol. 5, no. 1, p. 1275, 2016.
  76. Dlib C++ Library, Available: http://dlib.net/.
  77. K. Bansode and P. K. Sinha, "Face Shape Classification Based on Region Similarity, Correlation and Fractal Dimensions," Int. J. Comput. Sci., vol. 13, no. 1, pp. 24-31, January, 2016.
  78. Sunhem and K. Pasupa, "An approach to face shape classification for hairstyle recommendation," in Eighth International Conference on Advanced Computational Intelligence, Chiang Mai, Thailand, ICACI , February, 2016.
  79. M. Haralick, K. Shanmugam and I. Dinstein, "Textural Features for Image Classification," IEEE Trans. Syst., Vols. SMC-3, no. 6, pp. 610 - 621, November, 1973.
  80. Hall-Beyer, "GLCM Texture: A Tutorial v. 3.0," 2017.
  81. Naouar, E. Vidal-Salle, J. Schneider, E. Maire and P. Boisse, "3D composite reinforcement meso F.E. analyses based on X-ray computed tomography," Compos Struct, vol. 132, pp. 1094-1104, November, 2015.
  82. Jia, H. Li, X. Wu and Q. Li, "Laboratory-based hyperspectral image analysis for the classification of soil texture," J Appl Remote Sens, vol. 13, no. 4, pp. 1-11, November, 2019.
  83. R. Vimina and K. P. Jacob, "Content Based Image Retrieval Using Low Level Features of Automatically Extracted Regions of Interest," J. Image Graph, vol. 1, no. 1, pp. 7-11, 2013.
  84. M. Mastelini, V. G. T. d. Costa, E. J. Santana, F. K. Nakano, R. C. Guido, R. Cerr and S. B. Jr, "Multi-Output Tree Chaining: An Interpretative Modelling and Lightweight Multi-Target Approach," J Signal Process Syst, vol. 91, pp. 191-215, 2019.
  85. scikit-learn: machine learning in Python, Available: https://scikit-learn.org/stable/.
  86. Berrar, "Cross-Validation," in Encyclopedia of Bioinformatics and Computational Biology, 2019.
Iranian Journal of Biomedical Engineering (IJBME)
Volume 17, Issue 2
Summer 2023
Pages 137-151

  • Receive Date 02 November 2023
  • Revise Date 30 December 2023
  • Accept Date 31 January 2024

Home

Contact Us