Background

Current embryo assessment methods for in vitro fertilization (IVF) depend on subjective morphological assessments. Recently, artificial intelligence (AI) has emerged as a promising tool for embryo assessment; however, its clinical efficacy and trustworthiness remain unproven. Simulation studies may provide additional evidence, provided that they are meticulously designed to mitigate bias and variance.

Objective

The primary objective of this study was to evaluate the benefits of an AI model for predicting clinical pregnancy through well-designed simulations. The secondary objective was to identify the characteristics of and potential bias in the subgroups of embryologists with varying degrees of experience.

Methods

This simulation study involved a questionnaire-based survey conducted on 61 embryologists with varying levels of experience from twelve IVF clinics. Inter- and intra-observer assessments and the accuracy of embryo selection from 360 day 5 embryos before and after AI guidance were analyzed for all embryologists and subgroups of senior and junior embryologists.

Results

With AI guidance, the inter-observer agreement increased from 0.355 to 0.527 and from 0.440 to 0.524 for junior and senior embryologists, respectively, thus reaching similar levels of agreement. The overall accuracies of the embryologists only, embryologists with AI guidance, and AI only were 37.7%, 50%, and 65.5%, respectively. Without AI, the average accuracy of the junior group was 33.516 (37.2%), while that of the senior group was 35.967 (40.0%). With AI’s guidance, the junior group’s accuracy improved to 46.581 (51.8%), reaching a level similar to that of the senior embryologists, 44.833 (49.8%). The junior embryologists had a higher level of trust in the AI score.

Conclusion

This study demonstrates the potential benefits of AI in selecting embryos with high chances of pregnancy, particularly for embryologists with less than or equal to 5 years of experience, possibly due to their trust in AI. Thus, using AI as an auxiliary tool in clinical practice has the potential to improve embryo assessment and increase the probability of a successful pregnancy.