Abstract

Background

To evaluate the quality and content of websites related to nasoalveolar molding (NAM).

Methods

A systematic Internet search was conducted using the keywords 鈥渘asoalveolar molding鈥� and 鈥減resurgical nasoalveolar molding鈥� determined through Google Trends and popular search engines: GoogleTM, YAHOO!^庐 and Yandex^庐. The top 50 websites were evaluated for each keyword and search engine. Repetitive websites, advertisements, scientific articles, and irrelevant websites were excluded from the study. The remaining 40 websites were evaluated using JAMA benchmarks, HON criteria, DISCERN, and EQIP tools.

Results

75% of websites were created for informational purposes, and the remaining websites were created for clinical promotion and advertising. According to DISCERN, only 2.5% of the sites were classified as excellent (63 points): 7.5% very weak (16鈥�26 points), 22.5% weak (27鈥�38 points), 40% moderate (39鈥�50 points), and 27.5% good (51 points-62 points).

Conclusions

Internet information on NAM was generally at a moderate to poor level in terms of quality and content. In this regard, patients should be informed about the limits of their Internet information, and the Internet content that patients can obtain information should be increased.

Introduction

The nasoalveolar molding (NAM) technique, first described by Grayson et al. [1, 2], is a presurgical orthopedic intervention designed to align the nose, lip, and alveolar segments in newborns with cleft lip and/or palate (CLP). NAM reduces both the nasal deformity and the cleft amount in the alveolar bone. This technique has been shown to reduce the severity of the anomaly and minimize surgical intervention, providing convenience to both the patient and the surgeon [3]. Less scar tissue is formed during surgery and the number of interventions is reduced [4,5,6]. Despite these advantages, there is not a consensus among all physicians on the use of NAM [7, 8]. Some studies have reported negative effects on long-term maxilla growth, high costs and burden of care, and complications such as mucosal ulcerations, fungal tissue infections, and tissue irritation [8, 9]. The information conveyed to parents on this subject via Internet information websites may not always be accurate or of high quality [10]. However, these issues remain controversial.

Websites can be utilized to acquire information regarding health, to inspire patients, to communicate with healthcare professionals and other individuals affected by these conditions [11]. A significant proportion of Internet users, approximately two-thirds, report that the Internet influences their treatment decisions [12,13,14]. Despite the benefits of virtual health resources, concerns exist regarding the potential for inaccurate information to negatively impact users鈥� health [15, 16].

Low-quality online health information can significantly impact the treatment decision-making process for parents and caregivers, often leading to suboptimal outcomes. Misleading or incomplete data may result in unrealistic expectations regarding treatment effectiveness, delays in seeking professional care, or the selection of inappropriate interventions. For example, caregivers may overestimate the benefits of a particular treatment while underestimating its risks, potentially compromising the child鈥檚 health outcomes. Such consequences underline the importance of providing accurate, comprehensive, and reliable online resources to empower parents with the knowledge needed for informed decision-making. Therefore, it is crucial to evaluate the credibility, quality, and content of Internet websites to ensure users have access to reliable information [13, 15]. Various assessment indices have been created to fulfill this purpose [17,18,19,20].

Main purpose of this research was to analyze the quality and reliability of information and assess the substance of websites that present data on NAM. The JAMA benchmarks were employed to determine if a website met the minimum requirements, the HONCode criteria to evaluate ethical standards, the DISCERN tool to assess quality and reliability, and the EQIP tool to evaluate the quality of written health information, focusing on its clarity, structure, and relevance to ensure usability for patients [17,18,19,20].

The null hypothesis is that there is no significant difference in the quality and content of websites providing information about NAM when assessed using established evaluation tools.

Materials and methods

A systematic Internet search was conducted on 20 August 2023 using the keywords 鈥渘asoalveolar molding鈥� and 鈥減resurgical nasoalveolar molding鈥� which were the most searched phrases determined through Google Trends and three popular search engines: GoogleTM, YAHOO!^庐 and Yandex^庐 [14, 15, 21]. The top 50 websites were evaluated for each keyword and search engine.

The searches were conducted using a virtual private network (VPN) to simulate a neutral browsing environment and minimize local browsing biases. No specific geographic preferences were applied, and the search results aimed to reflect what is accessible to the general public. However, we recognize that search results may still vary based on the geographic location of the search, as reported in previous studies [22], and this remains a limitation of our study.

Repetitive websites, advertisements and sites that were not relevant to the subject were identified and excluded from the evaluation. Video content and academic research not intended for patients were excluded from the evaluation, as the tools employed in this study were specifically designed to assess the quality of written health information aimed at patients and the general public. Websites with identical content, regardless of different extensions, were excluded from the analysis, and only the most specific and relevant version was retained. The remaining 40 websites were scored by two examiners (M.N.E. and C.A.), each with over 10 years of experience in cleft lip and palate treatment, according to the JAMA benchmarks, HON criteria, DISCERN, and EQIP tools. The website type, author鈥檚 profession, and target group were also recorded. The study focused on evaluating websites intended for patients and the general public. Websites explicitly targeting healthcare professionals were excluded unless their content overlapped significantly with patient-oriented information and was deemed relevant for the general public. This inclusion ensured the analysis captured all valuable resources accessible to patients. Due to the non-clinical structure of this study, ethical approval was not applicable.

JAMA benchmarks

The benchmarks for assessing the quality standards of Internet-based healthcare information were published in the Journal of the American Medical Association. Four essential features were evaluated, including authorship and attribution (e.g. name and affiliation, references, copyright information), currency (e.g. uploading date, update date), and disclosure (e.g. conflicts of interest, sponsorship, advertising, and commercial financing). A score of 1 point (1) was assigned for each question if these features were presented clearly; otherwise, 0 points (0) were assigned. The total score ranges from 0 to 4 [15]. Web sites about medical information that fail to meet at least these basic standards of JAMA should be considered doubtful. The benchmarks were published in the Journal of the Websites that fail to meet the basic standards of JAMA for medical information should be viewed with skepticism [19].

HON (health on the net) criteria

The Health on the Net Foundation, a non-profit organization based in Switzerland, developed the HONcode to address ethical issues in the Internet field. The HONcode Principles consist of eight key aspects that ensure the website provides reliable and high-quality information about health in accordance with ethical standards. These aspects include the authority of the author in discussing the topic, the relationship between the doctor and patient, confidentiality of medical data, clear references to source data, evidence-based support, contact information for visitors, funding sources, and advertising policies [23, 24].

DISCERN tool (quality criteria for consumer health information)

The DISCERN tool was developed to evaluate the quality of written information on health-related websites, specifically regarding treatment choice. The tool includes three sections, and a validated 16-item questionnaire is scored at least 1 with a maximum of 5 points. The first section is used to assess the reliability of the publication with eight questions, and the second section is used to evaluate the quality of the treatment options with seven questions. Finally, Sect.听3 presents a separate question for evaluating the overall rating of the website. The DISCERN tool scores websites on a scale of 16 to 80 points. These scores are categorized as very poor (16鈥�26; 20鈥�33%), poor (27鈥�38; 34鈥�48%), fair (39鈥�50; 49鈥�63%), good (51鈥�62; 64鈥�78%), and excellent (63鈥�80; 79鈥�100%) based on the percentage of the maximum possible score [25].

EQIP tool (ensuring quality information for patients)

The EQIP is a 20-item tool which was developed by a patient information group in London to evaluate the reliability, validity, and utility of health-related websites. Various aspects were assessed, such as clarity of information, quality of written work, language, and website design. The EQIP tool is the only system designed to evaluate quality in all types of written healthcare information, not only for treatment alternatives such as DISCERN, or readability scoring systems such as the SMOG index. Previous scales were not large enough to reflect patient satisfaction [26].

The EQIP evaluates various aspects of the information presented on the website, including the aim of the information, the accuracy and timeliness of the data, therapy options and their impact on quality of life, the advantages and disadvantages of therapies, and the language and presentation of the website. The evaluation is conducted using a 4-level scoring system, where a score of 1 is given for fully fulfilled criteria, 0.5 for partly fulfilled criteria, and 0 for not fulfilled or not applicable criteria. The total score is obtained by adding the scores obtained from each criteria [20, 27].

If the total score is less than 25, the data must be updated immediately. Scores between 26 and 50 points require updating in the following 6 months, while scores between 51 and 75 points require updating in 1鈥�2 years. If the score is more than 75 points, the updating process can be done in 2鈥�3 years. The EQIP鈥檚 superiority lies in its assessment of the timeliness of the data [20, 27].

The original EQIP tool was employed in this study due to its established reliability and widespread use in evaluating written healthcare information. This decision also aligns with previous studies, ensuring comparability of results across similar research [24, 26]. However, the use of the original EQIP rather than its expanded version is acknowledged as a limitation.

Statistical analysis

All assessments were performed simultaneously and unannounced by two experienced orthodontists (M.N.E. and C.A.), and inter-class correlation coefficients were calculated to assess inter-observer reliability. Two weeks after the first evaluation, the websites were re-evaluated by both researchers to determine the intra-observer reliability, and the intraclass correlation coefficient was calculated. Statistical analyses were performed using SPSS version 25 (IBM Corp., Armonk, New York).

Results

Intraclass correlation coefficient (ICC) values were found to be high, ranging between 0.96 and 1 for intra-rater reliability, and from 0.95 to 0.98 for inter-rater reliability in terms of JAMA benchmarks, HON criteria, DISCERN and EQIP tools. Although both researchers independently assessed the websites, the data from one researcher were used for the final analysis due to the high inter-rater reliability (ICC: 0.95鈥�0.98), demonstrating consistent agreement between evaluators. This approach streamlined the analysis and ensured clarity in data presentation.

Initially, 300 websites were assessed. However, after evaluating the websites, it was found that 260 of them were excluded due to various reasons, such as 80 duplicated websites and 180 irrelevant websites, like articles and videos. Eventually, 40 websites were analyzed (Fig.听1.). The website type was analyzed, and it was found that 75% (n鈥�=鈥�30) of the 40 websites evaluated were created for informational purposes, whereas 25% (n鈥�=鈥�10) were created for clinical promotion and advertising. The authorship and ownership of the websites were also analyzed, and it was found that 70% (n鈥�=鈥�28) of the websites were unknown, while 17.5% (n鈥�=鈥�7) were written by orthodontists, 7.5% (n鈥�=鈥�3) by plastic surgeons, and 5% (n鈥�=鈥�2) by the parents of the patients. It was observed that 92.5% (n鈥�=鈥�37) of the websites were intended for patients, while 7.5% (n鈥�=鈥�3) were intended for physicians. Furthermore, only 24% (n鈥�=鈥�9) of the websites belonged to professional organizations, including orthodontic societies and general online health information websites (Table听1).

Fig. 1

Flow chart diagram of the Internet research

Table 1 Categorisation of websites based on Affiliation, Specialisation and Content Type

It was observed that 80% (n鈥�=鈥�32) of the sites did not specify the resources used during the preparation phase and 37.5% (n鈥�=鈥�15) did not specify the date of creation or information updating of the website. While 95% (n鈥�=鈥�38) of the websites provided information about the advantages of treatment, 32.5% (n鈥�=鈥�13) mentioned risks and side effects. However, only 17.5% (n鈥�=鈥�7) of the Internet content mentioned the possibility of treatment failure. Other treatment options are not mentioned on 80% (n鈥�=鈥�32) of the websites. 47.5% (n鈥�=鈥�19) of Internet content did not mention the effect of NAM treatment on quality of life. 77.5% (n鈥�=鈥�31) of the sites included in the study described the treatment mechanism.

Table听2 presents the outcomes according to the JAMA benchmark standards. The criterion most frequently achieved was the display of currency (32.5%, n鈥�=鈥�13), followed by the display of the author (30%, n鈥�=鈥�12), and the display of disclosure (30%, n鈥�=鈥�12). None of the websites met all four criteria.

Table 2 Content of websites based on JAMA benchmarks

Table听3 presents the outcomes according to the HON code criteria. No website was discovered that met all the criteria. The mean HONcode rating was 3.4 points. The author鈥檚 expertise in the topic was demonstrated on 90% (n鈥�=鈥�36) of the sites, which was the highest score in the HON code evaluation. However, only 2.5% (n鈥�=鈥�1) of the websites displayed appropriate and balanced evidence, which is the lowest score.

Table 3 HONcode Criteria scores of websites

As Table听4 indicates, only 2.5% (n鈥�=鈥�1) of the websites were classified as excellent, while 7.5% (n鈥�=鈥�3) was deemed very weak, 22.5% (n鈥�=鈥�9) were weak, 40% (n鈥�=鈥�16) were moderate, and 27.5% (n鈥�=鈥�11) were good. The mean DISCERN score was found to be 46.2 points (Fig.听2.). Upon further examination of the content, it was determined that providing excellent information about the benefits of treatment (67.5%; n鈥�=鈥�27) and shared decision-making (68%; n鈥�=鈥�27) were the strongest elements of all websites. However, the presentation of the areas of uncertainty was found to be the weakest, with 85% (n鈥�=鈥�34) of websites falling short in this regard.

Table 4 Content of websites based on DISCERN Tool

Fig. 2

The donut chart illustrating the distribution of DISCERN scores

The results of the EQIP scale investigation revealed that the top items which were addressed included 鈥渦se of everyday language鈥�, 鈥渞espectful tone鈥�, 鈥渟atisfactory design and layout鈥�, and 鈥渋nformation in logical order鈥� with a 100% response rate (n鈥�=鈥�40). However, 鈥渦se of references or source of the information鈥�, 鈥渘ame or affiliations of the author鈥�, and 鈥渄escription of risks and side effects鈥� were found to be rarely addressed (Table听5).

Table 5 Content of websites based on EQIP Tool

Furthermore, the EQIP scale indicated that no websites required immediate updating of their information. However, 12.5% (n鈥�=鈥�5) of them should update within 6 months, 70% (n鈥�=鈥�28) within 1鈥�2 years, and 17.5% (n鈥�=鈥�7) within 2鈥�3 years.

Discussion

One of the most prevalent congenital craniofacial defects in developing nations is cleft lip and palate [4]. The deformity can have a significant impact on a patient鈥檚 entire life, including their appearance, their ability to eat, speak, and hear, as well as their dental health. CLP treatment begins at birth and continues through adulthood as a lifelong multidisciplinary care journey, with NAM being one of the major steps [3, 5]. Based on Matsuo鈥檚 study, the hypothesis behind NAM treatment holds that the nasal cartilage is still forming and susceptible to repositioning due to nasal flexibility within the first six weeks of birth [28]. The informational value of NAM-related websites is crucial given that caregivers of patients commonly use Internet as a source of information. In order for this therapy procedure to be successful, there must be strong caregiver collaboration. However, caregivers do not always have a thorough understanding of NAM. Research demonstrates that at the start of NAM, caregivers typically experience anxiety about the procedure and their responsibilities [29]. The informational value of NAM-related websites is crucial, given that caregivers of patients commonly use the Internet as a source of information. Inaccurate or incomplete information can result in delays in seeking appropriate care, poor treatment adherence, or unrealistic expectations regarding outcomes. For parents of children undergoing NAM treatment, misinformation can exacerbate anxiety, as caregivers may struggle to understand the procedure or feel unprepared to manage the responsibilities involved. This heightened anxiety can further impede their ability to provide effective care and adhere to treatment protocols.

Google currently holds the majority of the desktop market share in the search engine industry, with a percentage of 83.49%. Bing and Yahoo follow behind with market shares of 9.19% and 2.72%, respectively [30].These three search engines were chosen for this study based on their combined market share of 95.71%. Google Trends was utilized to identify the most popular search terms related to NAM, resulting in the establishment of two search terms. Each search engine analyzed the top 50 websites for each query, which was comparable to the online search habits of a potential patient. However, the number of websites analyzed was low in comparison to other studies [31]. Considering the limited amount of online information available about NAM, this outcome was expected. Our search methodology was similar to previous research, including the exclusion criteria [32, 33].

Information about NAM was generally at a moderate-poor level in terms of quality and content. These results are similar to the findings of a limited number of previous studies on CLP and NAM. Karamitros et al. [34] revealed that the overall quality of Internet-based patient information on orofacial clefts is low. Moreover, a study conducted by Korkmaz et al. [35] demonstrated that YouTube鈩� could still not be fully regarded as a reliable source of information for patients on treatment for CLP, even though the majority of the videos were rated as moderately sufficient in terms of the information they provided. Similarly, Arslan et al. [36] concluded that the majority of YouTube鈩� videos on NAM lacked sufficient information and were of poor quality. Obviously, patients should be made aware of the limitations of their Internet access in this regard, and there should be more Internet content available about NAM.

Current study concluded that there is no NAM related website that meets all of the JAMA benchmarks. The main reason for this is the low probability of attribution (20%, n鈥�=鈥�8). Overall, very few websites meet al.l of the JAMA criteria in most studies: McMorrow and Millett [17] found 4 websites, Jo et al. [24] showed 3 websites and Alpayd谋n et al. [31] reported that only 6 of 111 websites met al.l the criteria [17, 24, 31].

In their study about clear aligners, Alpayd谋n et al. [31] found that disclosure display had the highest JAMA benchmark, and they argued that aligner companies do not show conflict of interest, indicating a positive marketing strategy [31]. On the contrary, in the adult orthodontic study conducted by McMorrow and Millett [17]. and in the halitosis study conducted by Jo et al. [24], the disclosure display was found to be the lowest value (less than 30%), and the authors stated that most websites ignore the conflict of interest and hide the financing, for their own benefits. They reported that websites avoided giving accurate information [17, 24]. Similarly, in our study, disclosure display was found to be around 30%. The fact that approximately 30% of the sites that provide information about NAM are official health sites, orthodontic societies, foundations, universities or educational programs, which do not take any interest, supports this result.

HONcode seal states that the website contains ethically reliable information about health. While Alpayd谋n et al. [31] found 4 websites belonged to professional organizations had the HONcode seal, in our study none of the websites about NAM had HONcode seal. Similar to our results, McMorrow and Millett [17] found only 1 website about adult orthodontics with the HONcode seal. The reason of the varying results might be about mandatory renewal fee for the license of HONcode certification applied since 2014 [17, 31]. In addition, although there are eight main aspects in HONcode score calculation, there is no website that completes all these aspects in NAM-related websites. Only one website was found to mention at most 6 of them. The mean HONcode aspects of the websites related to NAM was found to be 3.4 in our study, which is another indicator that explains why these websites do not have a HONcode seal. This means that the websites that provide information about NAM do not contain enough ethically reliable information.

Some previous studies showed that the professional organization and nonprofit websites had good performance in DISCERN, however, regardless of authorship and ownership, they were generally very poor or poor in quality and needed improvements [31]. In an Internet study evaluating the quality of temporomandibular joint disorders, the score was 38.9 and Internet information was reported to be insufficient [25]. In studies of the medical field, where Internet information is said to be sufficient and of high quality, this score reaches only around 67鈥�73 [37]. A study evaluating high blood pressure reported that the overall mean DISCERN score of all studied websites was 48.1 (60.1%), which is below the norm for a highly common and important disease as hypertension [38]. Similarly, in our study, the overall score was found to be 46.2, and NAM-related websites were found to be moderate-poor. These differences may be due to several factors such as the searching terminology, date, language or region of interest, and differences between examiners.

Most of the information provided on the websites included of short summaries of the CLP syndrome followed by advantages of the NAM. While there were a vast number of advantages given in the websites (95%), the rate of mentioning complications and risks of the NAM (32,5%) was almost only one-third of all. This information bias can cause patients to have unrealistic expectations of treatment outcomes and choose treatment options without awareness of their side effects and complications [17, 25]. One of the reasons why the side effects and risks are less mentioned may be that the websites provide information for commercial purposes [38]. Since most of these websites belong to a private practice physician, hospital or clinic (61%) advertising purpose might become an issue. Ethical concerns and conflicts of interest significantly impact the quality of online health information. Private practices often emphasize marketing over education, leading to biased or incomplete content, which can misguide patients into making decisions based on commercial interests rather than evidence-based recommendations [17, 24]. Ensuring transparency and prioritizing unbiased information is essential for improving patient education and outcomes.

Evaluations showed that the rate of mentioning other treatment options was low. This may be because alternative treatment options are of interest to physicians rather than patients. However,

it was seen that 92.5% of the websites related to NAM were directed to the patient and the patient鈥檚 families, and 7.5% to the physicians. Although the primary focus of this study is on patient-directed information, it is worth noting that the presentation of alternative treatments can also influence clinician decision-making. Studies suggest that biased or incomplete information on treatment options can lead clinicians to favor specific interventions, potentially limiting shared decision-making and comprehensive patient care [39, 40]. This highlights the importance of ensuring that all health information, whether for patients or clinicians, is transparent, balanced, and evidence-based.

According to the results of the DISCERN, 鈥渟hared decision making鈥� (68%; n鈥�=鈥�27) about NAM was one of the strongest elements of all websites, however, the weakest was the presentation of the areas of uncertainty (85%; n鈥�=鈥�34). It is thought that websites could transform the doctor鈥損atient relationship allowing for greater shared decision-making, encouraging patients who are interested and informed to assume an active role in the management of their health [41]. Mullen [41] concludes that a well-informed patient is more likely to have improved treatment compliance and better outcomes. Nevertheless, without an accurate interpretation of health related Internet information, a patient may fail to recognize the crucial information and focus on the non-evidence-based material instead [41]. These concerns regarding biased content and poorly referenced non-evidence based material is well founded, based upon the poor scores in the DISCERN questions [42].

Interestingly, in 47.5% of all NAM related websites, the question yielding the lowest DISCERN score is related to quality of life. Over the last number of years, the impact of disease on a patient鈥檚 quality of life has become a crucial point in the management of the disorders. The quality of life is not only for patients, but also to the caregivers [42]. Similarly, in the Internet information study about head and neck cancer conducted by Riordain et al. [42], the quality of life response score was found to be quite low. The fact that cancer patients experience very serious problems related to pain and nutrition, which significantly affect their daily lives, was not explained in these websites [42]. Babies with CLP may also encounter soft tissue irritations during NAM treatment, and this is adequately stated on only 15鈥�20% of websites. Furthermore, the fact that patients visit the clinic every week during NAM treatment affects the quality of life of their families and creates an additional burden on them. By letting patients know the frequency of visits on these websites, patients, particularly those who are far away, can be aware and make informed choices and plans. However, there is not enough information about it on their website.

Top items of EQIP scale addressed and 鈥渦se of everyday language鈥�, 鈥渞espectful tone鈥�, 鈥渟atisfactory design and layout鈥�, 鈥渋nformation in a logical order鈥� were found 100%. 鈥淯se of references or source of the information (20%)鈥�, 鈥渘ame or affiliations of the author (30%)鈥� and 鈥渄escription of risks and side effects (30%)鈥� were rarely addressed.

The fact that the given information is trustworthy does not certainly guarantee it is scientifically correct, websites must give references [25]. Only 20% of the websites about NAM accomplished to do so. The study on high blood pressure by Tahir et al. [38] revealed that physician-provided web sites had higher quality scores, but lower readability. In other words, the comparison of the website to the reader and the fact that it is understandable for the reader does not always indicate that the content is of high quality.

Moreover, EQIP scale gives information about whether a website needs to be updated or completely renewed. A study conducted by Jo et al. [24] about halitosis related websites showed that they needed to be reviewed and updated within a year. No website about NAM founded that need to update their information immediately. However, 70% need to update within 1鈥�2 years. This may be due to the fact that the scientific information about NAM is more 鈥渟table鈥� and have a lesser tendency to change.

Finally, the language which is used to conduct the research is English, which is one of the limitations of the study. It should be noted that the quality of NAM-related websites may differ from studies in other languages. However, this limitation exists in all studies that measure website quality. English is one of the most used languages around the world, used by billions of people as a first or second language [43]. Therefore, it is possible to say that the results of these studies reflect the vast majority of websites.

This study focused exclusively on written online content due to the availability of validated tools such as DISCERN, JAMA, HONcode, and EQIP for evaluating written information. Other popular sources of health information, including video content, social media platforms, and AI-generated language models, were not assessed as they fall outside the scope of these tools. The absence of standardized evaluation methods for these mediums represents a limitation and an area for future research.

Overall, it is vital to inform patients about the constraints of relying solely on online resources for medical information, and to enhance the availability of trustworthy and comprehensive content on the web.

Conclusion

In this research, for all that Websites about nasoalveolar molding contains understandable and relatively up-to-date information for the patient, the quality and quantity of Internet-sourced information regarding nasoalveolar molding was found to be moderately poor. Inadequate explanation of the drawbacks of nasoalveolar molding can cause patients to have unrealistic expectations of treatment outcomes. With this study, it is recommended that the website contents be edited and improved by professionals interested in nasoalveolar molding.