Development of consensus-based rules for AI-assisted Clinical Trial Agreement review: a Delphi study
Highlight box
Key findings
• A three-round Delphi study with 23 Chinese experts developed a 76-item consensus-based, large language models (LLM)-ready Clinical Trial Agreement (CTA) review rule set covering legal, financial, operational and quality management domains. The process achieved high expert engagement (≥95.65%), authority (composite reliability =0.89) and consensus (Kendall’s W=0.87–0.91, P<0.001), with all rules standardized for LLM-utilizable.
What is known and what is new?
• CTA review is a trial initiation bottleneck; artificial intelligence (AI) can boost efficiency but lacks standardized expert-endorsed rules.
• Prior work has non-AI CTA checklists, while this study is the Delphi-consensus LLM-ready rule set tailored to China’s regulatory context, structuring unstructured expert CTA knowledge for subsequent AI integration.
What is the implication, and what should change now?
• This rule set provides a foundational framework for LLM-assisted CTA review tools to improve review efficiency/consistency. Future research should validate the rules in LLM-based AI systems, expand the expert panel for multi-disciplinary insights, and update rules for emerging trial designs to enhance real-world applicability.
Introduction
As the legal foundation of research operations, Clinical Trial Agreements (CTAs) must ensure compliance with regulatory mandates, ethical guidelines, and institutional policies. However, the review and negotiation of these agreements frequently represent critical bottlenecks, significantly delaying trial initiation (1). Such delays are further exacerbated by the increasing complexity of multi-center trials and the diverse legal frameworks across jurisdictions (2). Published estimates show that the average person-hours for a single CTA review (including negotiation and revision) is 80–120 hours for a single-center trial and 200–300 hours for a multi-center trial (3), with contract negotiation accounting for an average of 4–6 months of trial initiation delay (4). Although previous research has demonstrated that standardized clinical trial protocol templates can improve operational efficiency (5), traditional manual review processes remain heavily dependent on individual expertise, resulting in inconsistencies, elevated labor costs, and procedural inefficiencies, particularly in institutions managing high volumes of clinical research (3). Consequently, there is a pressing need for standardized, scalable tools to improve both the accuracy and efficiency of contract review workflows.
Prior studies and institutional audits (2,6) have quantified the most common CTA negotiation points, with the top five being: indemnification clauses (negotiation frequency: 92%), intellectual property ownership (88%), budget and payment terms (85%), publication rights (80%), and data ownership/storage (78%). These negotiation points are the primary sources of CTA review delays (accounting for about 70% of total negotiation time), which informs the domain focus of our rule set development.
The advent of artificial intelligence (AI) and large language models (LLMs) offers promising solutions for enhancing expertise and boosting productivity for contract management. AI-driven contract analysis tools have shown considerable potential to streamline review processes and improve risk management strategies (7). Nevertheless, despite recent technological advancements, general-purpose LLMs encounter notable limitations in the precision-driven context of contract review. These models often lack domain-specific knowledge, resulting in an inability to grasp nuanced legal meanings, omission of critical interdependencies between clauses, and ultimately compromising the reliability of contract interpretations (8). Existing AI methodologies frequently rely on superficial semantic or lexical similarity measures, which are insufficient for navigating the intricate, context-dependent nature of legal and technical documents. Moreover, the unstructured and voluminous format of contractual materials presents additional challenges for effective information retrieval, further diminishing system efficacy. Addressing these challenges necessitates a specialized approach that systematically integrates expert domain knowledge into the LLM or retrieval-augmented generation (RAG) framework.
Although previous studies have highlighted critical risk domains and essential contract language in CTAs, a fully integrated framework that draws on multidisciplinary expertise is still lacking. More importantly, there is no established methodology for converting the extensive administrative knowledge in contract management into structured, machine-interpretable formats that AI systems can effectively utilize. This study addresses this core gap by developing a consensus-based, LLM-ready rule set tailored for CTA review—representing a foundational step in bridging unstructured expert experience with AI operationalization.
The Delphi method is a structured communication and consensus-building approach and has been widely validated in healthcare research. It provides a robust mechanism for synthesizing expert insights (9). Its proven use in clinical trial management—such as creating quality assessment frameworks (10) and risk mitigation protocols (11)—demonstrates its capacity to balance scientific rigor with practical feasibility. Building on this foundation, our study applies the Delphi method to develop a consensus-driven, expert-validated rule set that is explicitly designed for subsequent AI integration.
This study aimed to establish a comprehensive set of LLM-ready rules to guide LLM -assisted CTA review process, covering legal, financial, and operational aspects, tailored specifically to China’s regulatory environment. We hypothesized that leveraging expert insights from diverse Chinese institutions would enable the creation of a rule set that is both legally aligned with Chinese regulatory requirements and operationally practical—serving as a critical precursor for future AI system development and validation.
Methods
Study design
This Delphi process was conducted in three rounds, following the guidelines outlined in the Conducting and Reporting of Delphi Studies (CREDES) and reporting quality indicators (12). The study was conducted from March 2025 to May 2025.
This study focuses on analyzing the legal validity of clinical trial contract clauses, optimizing contract review workflows, and comparing differences across industry-standard contract templates. Its research objects are contract texts, commercial provisions, and legal provisions themselves—not human subjects or sensitive personal data (e.g., health information, biological samples, personal behavioral data). No collection, use, or disclosure of individual-level sensitive information is involved; the Delphi survey only solicits experts’ professional judgments on regulatory rules and contract clauses, with all responses anonymized and unrelated to personal privacy.
Development of the preliminary rules
A preliminary set of contract review rules was created through a structured three-step approach. First, an extensive and systematic literature review was undertaken in PubMed, CNKI, WanFang, and the ICH official database for the period January 2020 to February 2025, with core keywords including “Clinical Trial Agreement review”, “ICH-GCP”, “Chinese GCP”, “contract review rules”, and “AI-assisted contract analysis”. The review was based on the major regulatory frameworks including the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use-Good Clinical Practice (ICH-GCP) (13) and the Good Clinical Practice (GCP) (14) guidelines issued by China’s National Medical Products Administration (NMPA). Additionally, institutional Standard Operating Procedures (SOPs) sourced from 12 NMPA-compliant tertiary general and oncology specialty institutions with ≥5 years of CTA review experience were examined, and only formally issued and implemented SOPs were included to determine the key contractual components for legal and operational compliance. Second, inputs were provided by five clinical trial administrators with over 2 years of CTA review experience, who were recruited from the above-mentioned tertiary institutions across different cities. The five clinical trial administrators were from 5 different tertiary institutions—no single-institution bias and they represent diverse trial types and key therapeutic areas, ensuring their inputs cover the practical pain points of CTA review across different trial types and therapeutic areas.
Finally, the study team adopted thematic synthesis for consolidation: regulatory requirements and practical insights were coded into pre-defined themes, with overlapping rules merged and ambiguous content refined via team unanimous agreement (discrepancies resolved by an independent senior researcher with over 15 years of clinical trial contract and regulatory compliance experience). The consolidated preliminary rules were standardized with three core elements—clear review criterion, definite non-compliance consequence, and designated applicable party—and organized into formal and substantive review categories, covering legal/regulatory compliance, financial provisions, risk management, intellectual property, and quality assurance. Most rules align with established standards, while some are tailored to institutional needs and operational priorities.
Participants enrollment criteria
The whole participants represented a diverse group of professionals, including legal advisors and contract managers from sponsoring organizations, seasoned lawyers specializing in CTA law, and clinical research managers and investigators from tertiary general hospitals and oncology specialty institutions across China. To qualify, the participants should have a minimum of three years of experience in CTA review or management and had to provide voluntary consent. Individuals with potential conflicts of interest—such as consultants for AI-based contract review platforms—were excluded to ensure objectivity and credibility. Each participant should have extensive experience in contract review, ethical compliance, and GCP standards. To contextualize their contributions, demographic and professional details, such as roles, institutional affiliations, and years of relevant experience, were documented. Participants were recruited from different regions across China, including cities such as Guangzhou, Shanghai, Beijing, Kunming and Fuzhou, to ensure the geographic diversity. Consistent with the established recommendations for Delphi studies, which advise 15 to 50 participants (15), we aimed to recruit at least 20 participants each round to ensure heterogeneity and comprehensive domain insight.
Delphi process
The Delphi method collects expert feedback through questionnaires instead of face-to-face meetings, preserving anonymity and minimizing bias from dominant voices. In the first round, 23 participants received an email containing a three-part questionnaire: (I) an overview of the survey, (II) a table that listed the 74 initial rules alongside columns for the Likert scale, importance classification, and free-text comments to capture their specific recommendations or rationale for change, (III) a section for personal information and an optional section for additional suggestions, where participants could provide any extra rules or feedback at their discretion.
Participants assessed each rule using a five-point Likert scale from 1 (strongly disagree) to 5 (strongly agree) and classified its importance as “critical”, “important” or “moderate”. A free-text comment was mandatory to justify any rating of 3 or below. Additionally, participants rated their familiarity with the content on a five-level scale (5= familiar, 1= not familiar) and indicated the basis of their judgement, by selecting from 4 categories: theoretical analysis, practical experience, reference to relevant literatures or intuitive judgment. To maintain the quality and timeliness, participants were asked to complete the questionnaire within ten days, with reminders sent by email or telephone if necessary.
There is no universally accepted definition of consensus in Delphi studies (16), and approaches vary considerably across literature. Commonly used methods include percentage agreement, measures of central tendency, and dispersion metrics. To address the limitations of single metrics—such as the sensitivity of the mean to outliers or the potential for the mode to mask bimodal distributions—this study applied a comprehensive, pre-defined set of criteria for evaluating consensus. Rules were categorized as low consensus if they had a coefficient of variation (CV) >25%, a mean score <3.5, or a standard deviation (SD) >1.2. Conversely, rules with a mean score >4.5 and an SD <0.8 were classified as high consensus. Remaining rules were classified as moderate consensus.
The ≥85% threshold for “critical/important” ratings is based on recommended standards for Delphi studies in healthcare research (16,17) and is widely used in consensus development for clinical practice guidelines and regulatory rules. This threshold ensures that the rules have strong expert endorsement and avoid the inclusion of rules with significant expert disagreement, which is critical for the reliability of AI-assisted review rules that require high consistency. A post-hoc sensitivity analysis was conducted: (I) if the threshold was lowered to 75%, 8 additional rules would be included (all with medium consensus), but these rules had >15% expert ratings of “moderate” importance, indicating insufficient practical necessity; (II) if the threshold was raised to 95%, 6 rules with 85–94% “critical/important” ratings would be excluded, most of which were operational details (e.g., bilingual contract execution), leading to an over-restrictive rule set. Thus, the 85% threshold balances expert consensus and practical comprehensiveness.
Regarding CV threshold sensitivity: (I) CV >20% (stricter): 12 additional rules classified as low consensus (excluded); (II) CV >30% (looser): 8 fewer rules classified as low consensus (included). The chosen CV >25% threshold balances stringency and comprehensiveness.
Rules in the low-consensus category were retained, revised or removed based on the research team’s decisions, which were informed by feedback from the expert panel. Meanwhile, rules in the moderate or high-consensus categories, which fewer than 85% of participants rated as ‘critical’ or ‘important’, were also modified based on qualitative feedback. New rules suggested by over two participants would be adopted by the research team and incorporated into the updated set. Subsequent Delphi rounds re-evaluated these revised and newly-added rules using the same scoring criteria. After each round, participants received a summary of prior results to encourage reflection and support consensus development. This iterative process continued until all rules simultaneously meet the following two conditions: (I) high or medium consensus; (II) cumulative proportion of importance rated as “important” and “critical” ≥85%.
The Delphi survey was conducted using an online platform (www.wjx.cn) in mainland China, which enables real-time data collection and analysis, while maintaining anonymity and reducing potential bias.
Quality control
All returned questionnaires were carefully reviewed for completeness and accuracy. Incomplete responses were excluded from further analysis. Upon collection of all completed forms, the open-ended comments were independently reviewed by the study team. The study team then systematically reviewed and analyzed these qualitative inputs to guide the modification, consolidation, or elimination of items, ensuring the alignment with the study’s objectives and supporting literature.
Statistical analysis
Data were analyzed using SPSS (version 26.0; IBM Corp., Armonk, NY, USA) for analysis. Descriptive statistics included frequency, proportion, mean, SD and CV. Participant authority was quantified by the composite reliability (Cr), calculated as Cr = (Cs + Ca)/2, where Cs, represents the Familiarity Score and Ca the Judgment Basis Score. Cs was assessed via a 5-point Likert scale from 1 (“not familiar”) to 5 (“familiar”) and normalized to a 0–1 range. Ca reflected the primary basis for expert judgment, weighted as follows: theoretical analysis (0.3), practical experience (0.5), peer recommendations (0.1), and intuitive judgment (0.1). Both Ca and Cs were self-reported by the participants. A Cr value ≥0.70 indicated the high authority (12). Participant engagement was expressed through questionnaire recovery rates, while consensus consistency was measured using Kendall’s W, ranging from 0 to 1, with higher values signifying stronger agreement.
Ethical consideration
The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. No IRB approval and consent forms are needed as this study didn’t involving human biomedical research or sensitive personal information processing.
Results
Demographics of participants
Twenty-three participants were enrolled in the Delphi Process. Table 1 presents the expert demographic data. The sample comprised one participant with Ph.D. degree, 13 with master’s, and 9 with bachelor’s degrees. In terms of composition, professional experience, and areas of specialization, the participants met the principles of diversity and representativeness, providing constructive feedback and valuable suggestions for the study.
Table 1
| Demographic characteristics | Frequency | Percentage |
|---|---|---|
| Gender | ||
| Male | 7 | 30.43% |
| Female | 16 | 69.57% |
| Age (years) | ||
| 18–30 | 4 | 17.39% |
| 31–40 | 12 | 52.17% |
| 41–50 | 5 | 21.74% |
| 51–60 | 2 | 8.70% |
| Educational level | ||
| PhD | 1 | 4.35% |
| MSc | 13 | 56.52% |
| BSc | 9 | 39.13% |
| Years of experience | ||
| 3–5 years | 5 | 21.74% |
| 6-10 years | 9 | 39.13% |
| 11–20 years | 7 | 30.43% |
| >20 years | 2 | 8.70% |
| Specialization | ||
| Clinical Research Management | 17 | 73.91% |
| Contract management of the sponsor or CRO | 3 | 13.04% |
| Legal compliance | 1 | 4.35% |
| Medical ethics | 2 | 8.70% |
CRO, Contract Research Organization.
Positivity, authority, and coordination of experts
The Delphi survey achieved reconciliation rates of 100% in the first two rounds (23/23) and 95.65% in the third (22/23), reflecting strong engagement and interest among experts. The mean Cr across all rounds was 0.89, indicating a high level of authority on the subject matter. After three rounds, all items demonstrated a CV ≤0.25 and Kendall’s W ranging from 0.87 to 0.91 (P<0.001), confirming a high degree of consensus among the participants.
Delphi consensus outcomes
Figure 1 illustrated the Delphi survey outcome. In Round 1, all 23 participants evaluated the initial set of 74 contract review rules, with 78.3% (18/23) suggesting revisions, generating 155 actionable comments. Feedback primarily addressed linguistic precision (52.9%), structural optimization (28.4%), and content expansion (18.7%). Based on predefined consensus criteria, 14 initial rules required reassessment—nine with medium consensus and five with low consensus—while 60 rules achieved immediate high consensus. The study team incorporated some participants’ recommendations without removing any rules. As shown in Table S1, the study team revised and refined specific rules, which included: adding the description “if the required number of participants is reached and additional participants need to be enrolled, a supplementary agreement to increase the number of participants must be signed” to Rule X16; revising the wording in Rule X46 from “if the other party discovers any new uses of investigational products eligible for patenting shall be co-owned” to “if the other party discovers any technical features or design that meet the criteria for patent application, the patent application rights and patent rights shall be negotiated separately by both parties” in order to standardize terminology and align with patent law provisions; modifying the time frame in Rule X48 from “ten working days” to “within the agreed time period” to provide greater flexibility; deleting the specific requirement “the monitoring frequency should be ≥1 time/month” from Rule X67; adding more detailed scenario descriptions to Rule X72; and introducing two new rules X75 (about liability: “Explicit penalty clauses for sponsor protocol deviations”) and X76 (about administrative: “Bilingual (English/Chinese) contract execution”).
In Round 2, the updated set of 16 rules (14 initial +2 new rules) were re-assessed by the original 23 participants, through a blinded review to minimize bias. Participants also received the scoring statistics for the 14 initial rules from the first round of the survey, along with a comparative indication of any modifications made to the rules or notes stating that no modifications were made, to be used as reference during the second round of evaluation. Overall, 60.87% (14/23) of participants submitted 36 substantive comments. Of these, 86.1% (31 comments) supported the updated set of 16 rules, while 13.9% (5 comments) raised concerns about two specific rules, X4 and X15. Based on predefined consensus criteria, these two rules required reassessment. Rule X4, addressing tax refund procedures, was initially stated as “Sponsor shall bear non-refundable taxes during trial fund reimbursement”. Participants noted this conflicted with national tax policies, particularly value-added tax (VAT) refund eligibility under China’s Tax Collection Law. The rule was revised to “Upon sponsor’s application to tax authorities, the institution may refund taxes after receiving official approval”. For Rule X15, which involving protocol numbers in contracts, the study team believed that it is necessary to retain the original wording, as including protocol numbers could risk confidentiality breaches according to ICH GCP E6(R3) (13).
In Round 3, all 22 participants evaluated the two disputed rules, and consensus was achieved for both. Rule X4 received a mean score of 4.2 (SD =0.6, CV =14.3%), while Rule X15 scored 4.5 (SD =0.5, CV =11.1%).
The list of final rules, as shown in Table 2 and Table S1, consists of 76 validated items after three rounds of Delphi, representing a comprehensive, standardized approach to contract review. Table 2 presents the core final rule set organized by rule category, applicable party, and specific final version rules. Table S1 provides the full revision history. The statistical results of expert evaluations of the rules during the three rounds of Delphi process are shown in Table S2. The inclusion of expert-driven revisions ensures the rules are practical, legally aligned with Chinese regulatory requirements, and adaptable to evolving clinical trial environments.
Table 2
| Final established review rules | Rule category | Applicable party under the rules |
|---|---|---|
| X1. If the contract does not contain a section for the signature of the legal representative of Party A, it is deemed non-compliant with the review requirements | Contract format | Party A, the sponsor |
| X2. If the contract text includes a clinical trial budget table, it will be deemed non-compliant with the review requirements | Contract format | Party A, the sponsor |
| X3. If the clause “After unblinding a blinded trial, the sponsor should promptly inform the researcher in writing about the trial medication status of the subjects” is not included in the contract, it is deemed non-compliant with the review requirements | Responsibility for blinded trials | Party A, the sponsor |
| X4. If the clause “at the end of the project, if a refund is required, Party A (or the sponsor) can initiate a red-letter invoice application in the tax bureau system, and after Party B (or the hospital) receives the application, they should issue a red-letter invoice and simultaneously refund the remaining project funds” is not included in the contract, it is deemed non-compliant with the review requirements | Financial related terms | Party A, the sponsor |
| X5. If the clause “the research institution is responsible for storing the relevant clinical trial data free of charge for 5 years after the termination of the clinical trial in accordance with current laws and regulations, and 3 months before the expiration of the 5-year period, Party A should actively contact Party B to discuss subsequent storage issues. If the sponsor needs the institution to continue storing the data, the applicable fees will be charged according to the institution’s current fee standards (currently 6000 yuan/year). If Party A does not contact Party B and more than 1 year passes, the data can be retrieved after paying the annual storage fee; otherwise, it will be considered that the institution has the right to handle the documents on its own.” is not included in the contract, it is deemed non-compliant with the review requirements | Archive retention clause | Both parties, the sponsor and the Clinical Trial Institution |
| X6. If the names of the parties on the front page of the contract and the signature and seal page do not match, it will be deemed non-compliant with the review requirements | Contract format | Both parties, the sponsor and the Clinical Trial Institution |
| X7. If the contract does not specify that Party B or Sun Yat-sen University Cancer Center must hold at least two copies, it will be deemed non-compliant with the review requirements | Contract format | Both parties, the sponsor and the Clinical Trial Institution |
| X8. If the contract does not display page numbers in the header or footer, or if the page numbers are incorrect, or if there are blank pages, it will be deemed non-compliant with the review requirements | Contract format | Both parties, the sponsor and the Clinical Trial Institution |
| X9. If the bank account name of the hospital (Party B) in the contract does not match the designated name, it will be deemed non-compliant with the review requirements | Financial related terms | Party B, the Clinical Trial Institution |
| X10. If the contract does not have a place for the [legal representative’s signature] for Party B, it will be deemed non-compliant with the review requirements | Contract format | Party B, the Clinical Trial Institution |
| X11. If the contract does not have a place for the [principal investigator’s signature] for Party B, it will be deemed non-compliant with the review requirements | Contract format | Party B, the Clinical Trial Institution |
| X12. If the address of Party B in the contract does not match the publicly announced designated address, it will be deemed non-compliant with the review requirements | – | – |
| X13. If the contract does not include the statement: ‘The sponsor/CRO should provide the company’s business license to prove its qualification type when submitting the project for approval. If the company’s qualification type changes during the contract period, it should be reported to the institution in a timely manner and continue after approval according to regulations. The sponsor/CRO will bear all responsibility for any adverse consequences caused by violating the national human genetic resources management laws and requirements,’ it will be deemed non-compliant with the review requirements | Human genetic resources management policy | Party A, the sponsor |
| X14. If the contract does not include the statement: ‘The sponsor is primarily responsible for the quality of the clinical trial. (I) Before starting the project or recruiting subjects, the sponsor should organize a project initiation meeting to explain and train on the protocol and implementation details. If a CRO is commissioned, the sponsor should ensure that relevant personnel from both the sponsor and the research team attend the meeting. (II) After enrolling 3 subjects in the project, the sponsor should organize a feedback meeting with the research team to address issues found during prior monitoring, improve and optimize the execution of the project, and keep records of relevant meetings or corrective actions. Projects that do not hold feedback meetings are generally not allowed to continue recruitment. (III) During the trial, if the number of enrolled cases is ≥20 or there are significant/multiple issues, the sponsor should promptly organize inspections. (IV) At the conclusion of the project, before submitting the center’s summary or final report to the institution’s office for stamping, the sponsor should conduct a self-inspection as required by the regulatory authorities and submit the self-inspection report to the institution’s office for review as part of the project conclusion process,’ it will be deemed non-compliant with the review requirements | Clinical trial quality management measures | Party A, the sponsor |
| X15. If the research content section of the contract includes the version number of the protocol, it will be deemed non-compliant with the review requirements | Contract format | Party A, the sponsor |
| X16. If the contract does not include the clauses ‘Research funding will be paid based on the actual number of enrolled participants’ or ‘if the required number of participants is reached and additional participants need to be enrolled, a supplementary agreement to increase the number of participants must be signed,’ it will be deemed non-compliant with the review requirements | Financial related terms | Party B, the Clinical Trial Institution |
| X17. If the contract does not clearly specify that the “sponsor” is the “Party A” of the contract, it will be deemed non-compliant with the review requirements | Contract format | Party A, the sponsor |
| X18. If the contract does not clearly specify that the “research institution” is the “Party B” of the contract, it will be deemed non-compliant with the review requirements | Contract format | Party B, the Clinical Trial Institution |
| X19. If the contract does not include a commitment to immediately destroy all remaining biological samples after testing all blood/urine samples, ensuring they will not be used for any purpose other than this trial, and that Party A is responsible for the proper destruction, it will be deemed non-compliant with the review requirements | Biological sample management measures | Party A, the sponsor |
| X20. If the responsibility of Party A to cooperate with the drug regulatory authorities’ inspection is not clearly stated, it will be deemed non-compliant with the review requirements | Obligation to cooperate with regulatory authorities’ inspection | Party A, the sponsor |
| X21. If there is no statement such as ‘Researchers and clinical trial institutions must allow monitors, auditors, ethics committee reviewers, and inspection personnel from the drug regulatory authorities to directly access the source data and source documents related to the clinical trial,’ it will be deemed non-compliant with the review requirements | The sponsor’s representative’s right to directly access the source data and source documents | Party B, the Clinical Trial Institution |
| X22. If there is no statement such as ‘The monitors assigned by the sponsor should have received appropriate training and possess the knowledge required for clinical trial monitoring in fields such as medicine and pharmacy, and be able to effectively perform monitoring duties,’ it will be deemed non-compliant with the review requirements | Monitoring responsibilities | Party A, the sponsor |
| X23. The sponsor may delegate part or all of the work and tasks of its clinical trial to a contract research organization, but the sponsor remains the ultimate responsible party for the quality and reliability of clinical trial data, and should supervise the work undertaken by the contract research organization | Delegated responsibilities to the CRO | Party A, the sponsor |
| X24. If there is no statement such as ‘The implementation of biological sample testing (e.g., genetics) unrelated to the trial protocol approved by the ethics committee is prohibited,’ it will be deemed non-compliant with the review requirements | Biological sample management measures | Party A, the sponsor |
| X25. If there is no statement such as ‘After the clinical trial ends, the continued storage or potential future use of remaining specimens should be covered by an informed consent form signed by the participants, which should explain the duration of storage, confidentiality of the data, and under what circumstances the data and samples may be shared with other researchers,’ it will be deemed non-compliant with the review requirements | Biological sample management measures | Party A, the sponsor |
| X26. If there is no statement such as ‘The preparation of investigational drugs must comply with the relevant requirements of clinical trial drug production quality management; the packaging labels of investigational drugs must indicate that they are for clinical trial use only, along with clinical trial information and investigational drug information; and the blinding procedure must be maintained in blinded trials,’ it will be judged as non-compliant with review requirements | Responsibility for providing investigational drugs | Party A, the sponsor |
| X27. If there is no statement such as ‘The sponsor should clearly specify the storage temperature, transportation conditions (whether light protection is required), storage duration, preparation methods and processes for drug solutions, and requirements for drug infusion devices of investigational drugs. The method of use for investigational drugs should be communicated to all relevant personnel in the trial, including monitors, researchers, pharmacists, and drug storage personnel,’ it will be judged as non-compliant with review requirements | Responsibility for providing investigational drugs | Party A, the sponsor |
| X28. If there is no statement such as ‘The packaging of investigational drugs must ensure that the drugs are not contaminated or deteriorated during transportation and storage,’ it will be judged as non-compliant with review requirements | Responsibility for providing investigational drugs | Party A, the sponsor |
| X29. If there is no statement such as ‘In blinded trials, the coding system for investigational drugs must include an emergency unblinding procedure, so that in case of an emergency medical condition, the investigational drug can be quickly identified without compromising the blinding of the clinical trial,’ it will be judged as non-compliant with review requirements | Responsibility for providing investigational drugs | Party A, the sponsor |
| X30. If there is no statement such as ‘When it is found that the researchers, clinical trial institutions, or personnel of the sponsor do not comply with the protocol, standard operating procedures, this guideline, or relevant laws and regulations during the clinical trial, the sponsor must take immediate corrective actions to ensure good compliance with the clinical trial,’ it will be judged as non-compliant with review requirements | Clinical trial quality management measures | Party A, the sponsor |
| X31. If there is no statement such as ‘The sponsor must provide the researchers and clinical trial institutions with legal and financial insurance or guarantees related to the clinical trial, which are appropriate to the nature and degree of the risks involved in the clinical trial. However, this does not include damages caused by the negligence of the researchers or clinical trial institutions,’ it will be judged as non-compliant with review requirements | Responsibility for compensating or reimbursing damages to subjects or researchers | Party A, the sponsor |
| X32.If there is no statement such as ‘The sponsor must bear the diagnostic and treatment costs related to damages or death of the subjects in the clinical trial, as well as the corresponding compensation. The sponsor and researchers must promptly provide the compensation or reimbursement to the subjects,’ it will be judged as non-compliant with review requirements | Responsibility for compensating or reimbursing damages to subjects or researchers | Party A, the sponsor |
| X33. If there is no statement such as ‘Party A is responsible for providing legal and financial guarantees to Party B’s medical institution and researchers. This includes damages related to the trial (including damages to the subjects, Party B’s medical institution, and researchers) and any disputes arising from the trial, ensuring that Party B is indemnified and compensated (including reasonable legal fees, litigation costs, and expenses, collectively referred to as ‘losses’).’ it will be judged as non-compliant with review requirements | Responsibility for compensating or reimbursing damages to subjects or researchers | Party A, the sponsor |
| X34. If there is no statement such as ‘The sponsor must provide the investigational drugs to the subjects free of charge and cover the medical testing costs related to the clinical trial,’ it will be judged as non-compliant with review requirements | Responsibility for providing investigational drugs and medical testing costs | Party A, the sponsor |
| X35. If there is no statement such as ‘If the sponsor suspends the clinical trial, they must immediately inform the researchers, clinical trial institutions, and the drug regulatory authority, and provide the reasons,’ it will be judged as non-compliant with review requirements | Responsibility to notify in the event of premature termination or suspension of the trial | Party A, the sponsor |
| X36. If there is no statement such as ‘If the sponsor terminates the clinical trial prematurely, they must immediately inform the researchers, clinical trial institutions, and the drug regulatory authority, and provide the reasons,’ it will be judged as non-compliant with review requirements | Responsibility to notify in the event of premature termination or suspension of the trial | Party A, the sponsor |
| X37. If there is no statement such as ‘The sponsor must promptly inform the researchers, clinical trial institutions, and the drug regulatory authority of any issues discovered during the clinical trial that may affect the safety of the subjects, may affect the implementation of the clinical trial, or may change the ethical committee’s approval,’ it will be judged as non-compliant with review requirements | Responsibility to inform about safety information | Party A, the sponsor |
| X38. If there is no statement such as ‘The sponsor must rapidly report any suspected and unexpected serious adverse reactions to all researchers and clinical trial institutions participating in the clinical trial, as well as the ethical committee,’ it will be judged as non-compliant with review requirements | Responsibility to inform about safety information | Party A, the sponsor |
| X39. If there is no statement such as ‘The sponsor’s safety update report during the drug development period must include an assessment of the clinical trial’s risks and benefits, and the relevant information must be communicated to all researchers, clinical trial institutions, and the ethical committee participating in the clinical trial,’ it will be judged as non-compliant with review requirements | Responsibility to inform about safety information | Party A, the sponsor |
| X40. If there is no statement such as ‘The electronic data management system provided by the sponsor for the researchers should undergo reliable system validation, meet the preset technical performance, ensure the integrity, accuracy, and reliability of trial data, and ensure that the system remains in a validated state throughout the entire trial process,’ it will be judged as non-compliant with review requirements | Responsibility to provide an electronic data management system that complies with regulatory requirements | Party A, the sponsor |
| X41. If there is no statement such as ‘The sponsor must appoint competent medical experts to provide timely consultation on relevant medical issues in the clinical trial,’ it will be judged as non-compliant with review requirements | Responsibility to provide professional medical issue consultation | Party A, the sponsor |
| X42. If there is no statement such as ‘After receiving the trial medication treatment, and for any injuries caused by following the prescribed trial process (even though such injuries may also occur in normal treatment), the subjects will receive appropriate medical treatment, and the sponsor will bear the medical treatment costs for adverse events related to this trial and provide compensation according to Chinese laws and regulations based on specific circumstances,’ it will be judged as non-compliant with review requirements | Liability for damages and compensation terms | Party A, the sponsor |
| X43. If there is no statement such as ‘The scope of confidentiality, the duration, and breach of contract liabilities,’ it will be judged as non-compliant with review requirements | Confidentiality responsibility | Both parties, the sponsor and the Clinical Trial Institution |
| X44. If there is no statement such as ‘The recording, processing, and storage of clinical trial data must ensure the confidentiality of records and subject information,’ it will be judged as non-compliant with review requirements | Confidentiality responsibility | Both parties, the sponsor and the Clinical Trial Institution |
| X45. If there is no statement such as ‘All parties involved in the research must sign a conflict of interest declaration and an anti-bribery commitment,’ it will be judged as non-compliant with review requirements | Anti-commercial bribery declaration | Both parties, the sponsor and the Clinical Trial Institution |
| X46. “If there is no statement such as ‘During the trial process, if the other party discovers any technical features or design that meet the criteria for patent application, the patent application rights and patent rights shall be negotiated separately by both parties,’ it will be judged as non-compliant with review requirements | Intellectual property terms | Both parties, the sponsor and the Clinical Trial Institution |
| X47. If there is no statement such as ‘After the research observation ends, if the sponsor publishes an academic paper related to the research in a public setting, the other party is entitled to authorship of the paper,’ it will be judged as non-compliant with review requirements | Intellectual property terms | Both parties, the sponsor and the Clinical Trial Institution |
| X48. If there is no statement like “After friendly negotiation, the main researchers and researchers of Party B may publish and disclose the trial results of this center for non-commercial purposes, including publishing papers related to this clinical trial in journals or presenting the research at professional or other conferences. Before publishing any research findings, other than those regarding serious adverse events or defects of the trial drug/product, Party B shall inform Party A of the content to be published and obtain Party A’s consent. If Party A does not respond within the agreed time period after the submission or fails to provide specific reasons for disagreement or modification suggestions, it will be deemed as Party A’s consent,” then it will be deemed as not meeting the review requirements | Intellectual property clause | Both parties, the sponsor and the Clinical Trial Institution |
| X49. If there is no statement like “Party B shall arrange to retain the subject documents, other original data, and all documents related to this clinical trial, and store them for at least 5 years after the completion of the clinical trial or 5 years after the registration and marketing of the product, or according to the retention requirements specified in the clinical trial protocol, whichever is longer,” then it will be deemed as not meeting the review requirements | Archives retention clause | Party B, the Clinical Trial Institution |
| X50.If there is no statement like “During inspections by the drug regulatory authority, both the research and management teams must send personnel to participate,” then it will be deemed as not meeting the review requirements | Obligation to cooperate with third-party inspections | Party B, the Clinical Trial Institution |
| X51. If there is no statement like “In a blinded trial, researchers must implement the unblinding according to the trial protocol. If unblinding occurs unexpectedly or due to an emergency unblinding due to serious adverse events, the researcher must provide a written explanation to the sponsor,” then it will be deemed as not meeting the review requirements | Blinding and unblinding procedures and reporting responsibilities | Party B, the Clinical Trial Institution |
| X52. If there is no statement like “Except for serious adverse events that do not require immediate reporting as stipulated in the trial protocol or other documents (such as the investigator’s manual), researchers must immediately report all serious adverse events in writing to the sponsor, and then provide detailed, written follow-up reports in a timely manner,” then it will be deemed as not meeting the review requirements | Investigator’s responsibility for reporting safety information | Party B, the Clinical Trial Institution |
| X53. If there is no statement like “Researchers must promptly acknowledge and review the safety information provided by the sponsor, consider whether adjustments are needed in the treatment of subjects, and if necessary, communicate with the subjects as soon as possible. They must also report to the ethics committee any suspicious and unexpected serious adverse reactions provided by the sponsor,” then it will be deemed as not meeting the review requirements | Investigator’s responsibility for reporting safety information | Party B, the Clinical Trial Institution |
| X54. If there is no statement like “Researchers must submit an annual report of the clinical trial to the ethics committee, or provide progress reports as required by the ethics committee,” then it will be deemed as not meeting the review requirements | Investigator’s reporting responsibilities | Party B, the Clinical Trial Institution |
| X55. If there is no statement like “If any situation arises that may significantly affect the implementation of the clinical trial or increase the risk to the subjects, the researcher must promptly provide a written report to the sponsor, ethics committee, and clinical trial institution,” then it will be deemed as not meeting the review requirements | Investigator’s reporting responsibilities | Party B, the Clinical Trial Institution |
| X56. If there is no statement such as ‘After the clinical trial is completed, the researcher should report to the clinical trial institution; the researcher should provide a summary of the clinical trial results to the ethics committee, and provide the clinical trial-related reports required by the drug supervision and management department to the sponsor,’ it will be deemed non-compliant with review requirements | Investigator’s reporting responsibilities | Party B, the Clinical Trial Institution |
| X57. If there is no statement such as ‘The researcher and the clinical trial institution should appoint qualified pharmacists or other personnel to manage the trial medication,’ it will be deemed non-compliant with review requirements | Responsibility for the management of investigational medicinal products | Party B, the Clinical Trial Institution |
| X58. If there is no statement such as ‘The researcher should ensure that all clinical trial data is obtained from the source documents and trial records of the clinical trial, and that it is accurate, complete, readable, and timely. The source data should have traceability, readability, contemporaneity, originality, accuracy, completeness, consistency, and durability. Modifications to the source data should leave a trace, not conceal the original data, and the reason for the modification should be recorded,’ it will be deemed non-compliant with review requirements | Investigator’s responsibility for record keeping | Party B, the Clinical Trial Institution |
| X59. If there is no statement such as ‘The researcher should fill in and modify the case report form according to the guidance provided by the sponsor, ensuring that the data in the various case report forms and other reports are accurate, complete, clear, and timely,’ it will be deemed non-compliant with review requirements | Investigator’s responsibility for completing case report forms | Party B, the Clinical Trial Institution |
| X60. If there is no statement such as ‘The researcher and the clinical trial institution must obtain the sponsor’s consent if they authorize an entity outside the clinical trial institution to undertake trial-related duties and functions,’ it will be deemed non-compliant with review requirements | Investigator’s obligation to inform and delegate authorization | Party B, the Clinical Trial Institution |
| X61. If there is no statement such as ‘The researcher, as a clinical doctor or an authorized clinical doctor, must bear all medical decision-making responsibilities related to the clinical trial,’ it will be deemed non-compliant with review requirements | Investigator’s responsibility for medical treatment | Party B, the Clinical Trial Institution |
| X62. If there is no statement such as ‘The researcher and the clinical trial institution must accept the monitoring and inspection organized by the sponsor and the inspections by the drug supervision and management department,’ it will be deemed non-compliant with review requirements | Investigator’s responsibility | Party B, the Clinical Trial Institution |
| X63. If there is no statement such as ‘The researcher should use the latest version of the informed consent form and other information provided to the participants, as approved by the ethics committee. If necessary, the participants in the clinical trial should sign the informed consent form again during the process of the trial,’ it will be deemed non-compliant with review requirements | Investigator’s responsibility for informed consent | Party B, the Clinical Trial Institution |
| X64. If there is no statement such as ‘The researcher from Party B must have the practice qualification at the clinical trial institution; possess the professional knowledge, training experience, and capability required for clinical trials; and be able to provide the latest work history and relevant qualification documents according to the sponsor, ethics committee, and drug supervision and management department’s requirements,’ it will be deemed non-compliant with review requirements | Investigator’s qualifications | Party B, the Clinical Trial Institution |
| X65. If there is no statement such as ‘The researcher must implement the clinical trial according to the trial protocol approved by the ethics committee,’ it will be deemed non-compliant with review requirements | Investigator’s responsibility to follow the protocol | Party B, the Clinical Trial Institution |
| X66. If there is no statement such as ‘Either party can file a lawsuit in the court where the research institution is located,’ it will be deemed non-compliant with review requirements | Dispute resolution clause | Party A, the sponsor |
| X67. If there is no statement such as ‘The monitoring frequency should be coordinated with the enrollment progress’ it will be deemed non-compliant with review requirements | Sponsor’s monitoring responsibility | Party A, the sponsor |
| X68. If there is no statement such as ‘Party B is responsible for establishing the management system and record system for trial medication. Provide a storage place for clinical trial drugs that meets the clinical trial protocol’s requirements, with dedicated personnel responsible for storage and distribution, and provide drug return records. Unused clinical trial drugs should be returned to Party A after verifying with Party A after the trial is completed,’ it will be deemed non-compliant with review requirements | Responsibility for the management of investigational medicinal products | Party B, the Clinical Trial Institution |
| X69. If there is no statement such as ‘Party B has the right to propose the suspension of the clinical trial due to the frequency and severity of SAE, and should promptly notify Party A, the participants, the ethics committee, and the drug supervision and management department, explaining the reason. It should also be reported to the institution’s office,’ it will be deemed non-compliant with review requirements | Investigator’s right to terminate the trial | Party B, the Clinical Trial Institution |
| X70. If there is no statement such as ‘Party A promises that even if the insurance policy payout is insufficient to cover the sponsor’s compensation responsibilities, the remaining amount will still be paid by Party A. Party A should not use the insurance payout amount or payout time as a reference for compensating the participants. The sponsor should advance the participants’ related treatment expenses based on the advance payment principle to ensure their safety and rights. After responsibility is determined, the responsible party shall bear the expenses in accordance with the law,’ it will be deemed non-compliant with review requirements. | Sponsor’s compensation responsibility | Party A, the sponsor |
| X71. “If the statement ‘In the event of a compensation dispute or litigation concerning damage to the subject or researcher, the researcher must immediately notify Party A, who must immediately assign a specialist (lawyer or its staff) to fully handle the claim, compensation, or litigation matters, and the research institution or researcher agrees to provide relevant assistance to Party A. Legal liability will be undertaken by both parties according to the determination of responsibility by the medical association and relevant judicial appraisal centers’ is not included, it will be deemed non-compliant with the review requirements | Research institution’s responsibility to inform the sponsor | Party B, the Clinical Trial Institution |
| X72. If the statement ‘Notwithstanding the above provisions, the sponsor will not provide compensation, indemnification, or assume any responsibility for the following situations: (I) the research institution, researchers, or other research staff fail to comply with this agreement, study protocol, the sponsor’s written instructions for this study, or applicable laws; (II) the research institution, researchers, or other research staff make unauthorized guarantees or engage in negligent or intentional misconduct (including but not limited to failure to strictly adhere to the study protocol, violation of standard procedures, or improper medical equipment operation); (III) the subject fails to undergo diagnosis and treatment according to the study protocol, fails to comply with the reasonable instructions of the investigator or any research staff, or engages in negligence or intentional misconduct; (IV) the natural course or complications of the subject’s primary disease, or any other disease or injury the subject may suffer during the study, unless such disease or injury is related to the study (i.e., caused by the activities described in the study protocol, which are different from the treatment the subject would have received had they not participated in the study); (V) medical accidents; (VI) injury caused by using any drug (including biological materials) or device supplied by a third party’ is not included, it will be deemed non-compliant with the review requirements | No retroactive compensation obligation or liability clause | Party B, the Clinical Trial Institution |
| X73. If the statement ‘Party B and the researcher are responsible for keeping confidential all contents related to the clinical trial, such as the study protocol, case report forms, investigator manual, study progress, and summary reports, and must not disclose them to any third party; all parties must not disclose the personal privacy of the subjects. The confidentiality obligation does not terminate due to the invalidity or termination of this contract, and the confidentiality period is for the duration of the clinical trial and for 5 years after its termination. The confidentiality obligation does not apply to the following information: (I) Information that has entered the public domain; (II) information that entered the public domain not due to Party B’s fault; (III) information that Party B is required by law to disclose or that Party A has consented to disclose in writing. In any of these cases, the receiving party must notify the disclosing party in writing and obtain its consent before the disclosure applies. If any party has additional confidentiality requirements, the parties may sign a separate confidentiality agreement’ is not included, it will be deemed non-compliant with the review requirements | Research institution’s confidentiality responsibility | Party B, the Clinical Trial Institution |
| X74. If the statement ‘[Arbitration should be conducted in English as far as possible]’ is included, it will be deemed non-compliant with the review requirements | Arbitration method | Party A, the sponsor |
| X75. If the statement ‘If either Party A or Party B fails to perform according to the relevant provisions of this agreement, it will be considered a breach of contract. The breaching party must compensate the other party for the losses. Moreover, if Party A breaches the contract, Party B has the right to require Party A to pay the clinical trial fees incurred; if Party B breaches, Party A has the right to recover the clinical trial fees already paid to Party B for the breaching portion and has the right to terminate this agreement and refuse to pay any fees not yet paid to Party B’ is not included, it will be deemed non-compliant with the review requirements | Breach of contract liability clause | Both parties, the sponsor and the Clinical Trial Institution |
| X76. If the statement ‘If the sponsor, research institution, principal investigator, or CRO is delayed or unable to perform obligations under this agreement due to circumstances beyond their reasonable control (including but not limited to natural disasters, government actions, accidents, strikes, terrorist attacks, biological terrorism, shutdowns, or other forms of industry action) (“force majeure”), and has provided timely notice of the delay or non-performance to the other party, the sponsor, research institution, principal investigator, or CRO will not be held liable for such delay or non-performance. Any force majeure event does not constitute a breach of this agreement, and the performance period will be postponed accordingly; however, if the force majeure event lasts more than thirty [30] days, the parties may discuss how to mitigate the impact of the force majeure event, and if possible, reasonable alternative arrangements may be agreed upon in all circumstances’ is not included, it will be deemed non-compliant with the review requirements. | Force majeure clause | Both parties, the sponsor and the Clinical Trial Institution |
Xn, number of rule items; CRO, Contract Research Organization; SAE, serious adverse event.
Discussion
This Delphi study tackles a pivotal challenge in clinical trial management: the absence of standardized, scalable approaches for contract review. To address this, it systematically converts expert administrative knowledge into a structured rule set tailored for LLM-assisted implementation. Unlike general-purpose AI contract review tools (7), which often operate as non-transparent “black boxes” and fail to capture domain-specific legal nuances, our consensus-driven framework delivers transparent, regulatory-aligned rules that serve as a reliable knowledge base for LLM systems.
A key distinction from existing work is as follows: most existing CTA templates/checklists (4) are unstructured or semi-structured and designed for human review; our work is the first to develop a structured, LLM-ready rule set with standardized terminology and logical structures that LLM can parse and execute. Additionally, our rule set is consensus-based (23 experts, 3-round Delphi) and tailored to China’s regulatory context. Compared to prior AI-assisted contract review systems (7,8), which use end-to-end LLM reasoning and lack domain-specific CTA knowledge, our system combines a consensus-based rule set with LLM reasoning, improving review accuracy and consistency for CTAs. Unlike regulatory automation frameworks that focus on single domains (e.g., data management), our rule set covers all core CTA review domains (legal, financial, operational, quality management).
One of the most significant findings was the strong emphasis of participants on linguistic precision in contract drafting. More than half of the feedback (52.9%) focused on improving terminological clarity and ensuring alignment with relevant legal frameworks, such as patent law. For example, revising the rule on “new uses of investigational products eligible for patenting” to conform with patent law definitions highlights the importance of legal and technical accuracy in contract language. This attention to precision is critical for minimizing ambiguities and the enforceability of CTAs.
The inclusion of penalty clauses for sponsor protocol deviations and bilingual contract execution responds to two key challenges in clinical trial management: the growing complexity of international trials and the need for clear, accessible legal language for all stakeholders (5). The inclusion of these rules demonstrates the ability of the Delphi method to identify and address practical issues that may not be fully captured in existing contract review guidelines.
The study highlights the effectiveness of the Delphi method in establishing consensus on rules that harmonize scientific rigor with practical feasibility. Such balance ensures that the final rule set is not only legally aligned with Chinese regulatory requirements but also adaptable to the complex, evolving landscape of clinical trials. The high level of consensus and strong endorsement of rule importance indicate that the resulting framework aligns well with expert expectations and provides a reliable foundation for LLM-assisted contract review agents.
A major conceptual consideration is the role of fixed rule sets in the era of LLMs and generative AI. The rule set is not a rigid replacement for LLM contextual reasoning but a high-fidelity knowledge base to constrain and guide LLM reasoning. LLMs have strong contextual understanding but suffer from hallucinations and inconsistent judgment in domain-specific tasks (e.g., CTA review); the rule set provides standardized, regulatory-aligned constraints to reduce hallucinations and ensure AI review results are consistent with expert and regulatory requirements.
The rule set offers several advantages over end-to-end LLM reasoning, fine-tuning, or reinforcement learning from human feedback (RLHF): (I) transparency: rule-based review is interpretable (AI can clearly flag which rule a CTA violates), whereas end-to-end LLM reasoning is a “black box”; (II) regulatory compliance: clinical trial regulation requires traceable review decisions, which rule-based sets provide; (III) low cost of update: The rule set can be modularly updated for new regulatory requirements, whereas LLM fine-tuning/RLHF requires large amounts of labeled data and computational resources.
Rules may become obsolete, brittle, or counterproductive under the following conditions: (I) rapid regulatory change without corresponding rule updates; (II) emerging trial designs with no corresponding rule adaptations (e.g., AI-driven adaptive trials); (III) over-reliance on rules without LLM contextual reasoning (e.g., rigid application to non-standard CTAs with unique clauses). To avoid this, we propose a human-in-the-loop + modular rule update mechanism to combine rule-based constraints with LLM flexibility.
The expert consensus achieved in this Delphi study provides a robust set of natural language rules, raising an important question: how can these rules be operationalized for an AI-assisted review process? While the rules are not AI-executable, they are LLM-utilizable-designed to be AI-ready, serving as a validated, high-fidelity knowledge base for building reliable agents. Simple checks—such as verifying the presence of key clauses—can be handled through pattern-matching techniques like regular expressions. More complex rules, involving logical inference (e.g., cross-verifying protocol and budget consistency or assessing regulatory compliance), would be encoded within a rule engine or integrated into structured prompts for LLMs using a RAG framework. This enables a human-in-the-loop AI agent that flags potential violations, highlights associated risks, and provides expert rationale without autonomously approving or rejecting contracts. Ultimately, this study lays the foundation for developing transparent, efficient AI-assisted review agent that improve consistency and speed while preserving essential human oversight.
Although this study established a foundational set of LLM-utilizable rules for CTA review, we acknowledge that formal, large-scale AI validation and comparative testing have not been fully completed at this stage. However, we have preliminarily input the 76 consensus rules into a LLM and conducted initial AI-assisted review on real CTA documents. We plan to further verify each rule manually, compare LLM performance with expert review in terms of issue identification efficiency and time consumption, and will report these validation results in detail in subsequent studies.
Regarding the applicability to diverse clinical trial designs: the rule set is primarily design-agnostic and based on core regulatory requirements (ICH-GCP, Chinese GCP) that apply to all clinical trial types. The core domains (legal compliance, financial provisions, IP ownership, data management, safety reporting) are universal for all trial designs, and the rules in these domains are fully applicable to adaptive designs, decentralized/hybrid trials, and platform studies.
This study has several limitations. The expert panel was composed primarily of professionals from China [73.91% clinical research management, 13.04% contract management of sponsor or Contract Research Organization (CRO), 4.35% legal compliance, 8.70% medical ethics], which may limit the generalizability of the developed rules to other regulatory contexts. While the proportional representation of specialized legal and ethics experts is relatively low, it is important to note that all 23 participating experts possess ≥3 years of practical CTA review experience and have received systematic training or hold educational backgrounds in clinical trial-related legal compliance. Specifically, all experts completed mandatory training on core legal frameworks including China’s Patent Law, Medical Administration Law, and ICH-GCP/Chinese GCP regulatory requirements—key foundations for evaluating legal clauses in CTAs. Their day-to-day CTA review work involves frequent assessment of legal-critical provisions (e.g., indemnification, intellectual property, tax compliance), equipping them with practical legal insight aligned with the operational context of CTAs. To mitigate potential gaps in legal domain depth (e.g., patent law alignment for intellectual property clauses, tax policy compliance for financial clauses), we supplemented the expert consensus with targeted measures: (I) all legal-related rules were cross-verified against the latest Chinese laws and regulations to ensure regulatory alignment; (II) two external legal experts specializing in pharmaceutical clinical trials were consulted to validate the legal precision of core clauses. Future research will expand the expert panel to include more legal, ethics, and CRO professionals (targeting 20–30% of the panel) to further enhance the depth of legal domain coverage and enrich multi-perspective insights for rule refinement.
Conclusions
This study represents the first effort to combine Delphi-derived expert consensus with AI operationalization for CTAs in this context. The resulting framework aligns with international standards and local regulatory requirements, offering a relevant and scalable solution for CTA management in China. The Delphi methodology offers a transferable blueprint for developing similar LLM-assisted agents in other jurisdictions.
Acknowledgments
We would like to express our gratitude to our expert colleagues who provided professional opinions in the Delphi survey.
Footnote
Data Sharing Statement: Available at https://jmai.amegroups.com/article/view/10.21037/jmai-2025-1-255/dss
Peer Review File: Available at https://jmai.amegroups.com/article/view/10.21037/jmai-2025-1-255/prf
Funding: None.
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jmai.amegroups.com/article/view/10.21037/jmai-2025-1-255/coif). The authors have no conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. No IRB approval and consent forms are needed as this study didn’t involving human biomedical research or sensitive personal information processing.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
References
- Dilts DM, Sandler AB. Invisible barriers to clinical trials: the impact of structural, infrastructural, and procedural barriers to opening oncology clinical trials. J Clin Oncol 2006;24:4545-52. [Crossref] [PubMed]
- Thompson MA, Hurley PA, Faller B, et al. Challenges With Research Contract Negotiations in Community-Based Cancer Research. J Oncol Pract 2016;12:e626-32. [Crossref] [PubMed]
- Pfeiffer JP, Richmond FJ. Management of Clinical Trial Agreements: Current Practices of Investigators in the United States. Ther Innov Regul Sci 2015;49:139-45. [Crossref] [PubMed]
- Lai J, Forney L, Brinton DL, et al. Drivers of Start-Up Delays in Global Randomized Clinical Trials. Ther Innov Regul Sci 2021;55:212-27. [Crossref] [PubMed]
- Lawrence CE, Bruce VNM, Salberg LD, et al. Quantitative assessment of the impact of standard agreement templates on multisite clinical trial start up time. J Clin Transl Sci 2023;7:e204. [Crossref] [PubMed]
- Pfeiffer JP. Clinical trial contracts: Language and limitations. 2017;18. Available online: https://www.pharmoutsourcing.com/Featured-Articles/336113-Clinical-Trial-Contracts-Language-and-Limitations/
- Giampieri P. AI-Powered Contracts: a Critical Analysis. Int J Semiot Law 2025;38:403-20. Available online: https://doi.org/
10.1007/s11196-024-10137-z - Kim EW, Shin YJ, Kim KJ, et al. Development of an Automated Construction Contract Review Framework Using Large Language Model and Domain Knowledge. Buildings 2025;15:923. Available online: https://doi.org/
10.3390/buildings15060923 - Boulkedid R, Abdoul H, Loustau M, et al. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One 2011;6:e20476. [Crossref] [PubMed]
- Elwyn G, O’Connor A, Stacey D, et al. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ 2006;333:417. [Crossref] [PubMed]
- Hafeez S, Shakil-Ur-Rehman S, Riaz S, et al. Consensus-Driven Development of an Exercise Base Manual Programme for Prediabetic Patients: A Delphi Study. J Multidiscip Healthc 2025;18:2461-76. [Crossref] [PubMed]
- Dai F, Wei K, Chen Y, et al. Construction of an index system for qualitative evaluation of undergraduate nursing students innovative ability: A Delphi study. J Clin Nurs 2019;28:4379-88. [Crossref] [PubMed]
- International Council for Harmonisation. ICH E6(R3): Guideline for Good Clinical Practice. 2025. [Last accessed on 2025 Feb 09]. Available online: https://database.ich.org/sites/default/files/ICH_E6%28R3%29_Step4_FinalGuideline_2025_0106.pdf
- Yang F, Heng J, Li K, et al. The new Good Clinical Practice-2020 in China: Views from ethical perspective. Lancet Reg Health West Pac 2021;8:100117. [Crossref] [PubMed]
- McPherson S, Reese C, Wendler MC. Methodology Update: Delphi Studies. Nurs Res 2018;67:404-10. [Crossref] [PubMed]
- Shang Z. Use of Delphi in health sciences research: A narrative review. Medicine (Baltimore) 2023;102:e32829. [Crossref] [PubMed]
- Nasa P, Jain R, Juneja D. Delphi methodology in healthcare research: How to decide its appropriateness. World J Methodol 2021;11:116-29. [Crossref] [PubMed]
Cite this article as: Li Y, Wang J, Guo Q, He L, Zhao B, Zhang Y, Guan G, Xu R, Liang J, Cao Y. Development of consensus-based rules for AI-assisted Clinical Trial Agreement review: a Delphi study. J Med Artif Intell 2026;9:59.


