Exploring the Potential Application of Artificial Intelligence Tools in Preparing For ABET Accreditation

1. Introduction

1.1 Challenges in ABET Accreditation

Figure 1 shows the ABET accreditation process and its structured timeline [1]. In January to July, an institution submits a Self-Study report to initiate the accreditation request for an engineering or technology program. From May to July, ABET determines the visit date for the campus evaluation. The campus visit takes place between September and December, during which ABET prepares a draft statement based on the institution’s 7-day response immediately after the visit. In February to April, the institution provides a 30-day response (Due Process) to the draft statement. Finally, in August, ABET notifies the institution of the accreditation decision. This process ensures a thorough evaluation and decision-making process to uphold the quality and standards of engineering and technology programs [2-6].

Figure 1: Illustration of the timeline depicting the development of different architectures and methodologies in the field of image captioning.

Meeting ABET criteria requires significant effort and resources to collect and compile data about an engineering or technology program, including curriculum development, faculty qualifications, assessment processes, and continuous improvement initiatives [7,8]. The accreditation process involves extensive documentation, including self-study reports, curriculum maps, assessment data, faculty qualifications, and more [9]. The major challenges in preparing for ABET accreditation include [10-14].

• Compliance with ABET Criteria: Meeting the specific criteria and requirements set by ABET can be challenging, particularly in ensuring alignment of curriculum, program outcomes, and assessment methods.
• Continuous Improvement: Establishing a culture of continuous improvement that emphasizes ongoing assessment, evaluation, and feedback to enhance program quality and student learning outcomes. This is particularly difficult for programs that historically lacked such cultures.
• Documentation and Recordkeeping: Collecting and organizing comprehensive documentation and evidence to demonstrate compliance with ABET criteria, including assessment data, student work samples, and program outcomes. Maintaining a consistent and disciplined routine is crucial to ensure the thoroughness and accuracy of data collection.
• Faculty Engagement and Training: Engaging faculty members in the accreditation process, providing them with necessary training and support, and ensuring their active participation in curriculum development and assessment activities.

The two authors of this manuscript both have almost twenty years of experience in engineering education. The first author is a member of the Committee Engineering Accreditation (CEA) and the Committee of Engineering Technology Accreditaon (CETA) of ASME (American Society of Mechanical Engineers). He has been an ABET program evaluator since 2015 and accredited Mechanical Engineering Technology program at six institutions. The second author has been a founding faculty member for two engineering programs (in public and private universities), where he has gone through the complete ABET accreditation cycles in charge of various aspects. Recently, he has led the institutional accreditation efforts and helped their engineering, computer science, occupational safety, and construction management programs complete the ABET and ACCE (Accreditation for Construction Education) accreditation processes successfully. The authors believe the following six approaches are critical to enhance the institutional capabilities in meeting the above challenges:

• Enhance understanding of ABET.
• Improve documentation quality.
• Adopt proactive approach to identify potential shortcomings.
• Conduct diagnostic review on the due process response.
• Establish action plans to address shortcomings identified in ABET accreditation.
• Develop faculty-training plans.

1.2 The Benefits of Using Chat GPT in Engineering Education

ChatGPT, a language model developed, has triggered a new wave of AI development [15]. It has garnered significant attention due to its ability to effectively answer a broad range of general and specific inquiries with fluent and comprehensive answers [16,17]. It has surpassed expectations with its remarkable capabilities and holds the potential for significant benefits to the educational system [18]. Its impacts in education are reflected in several aspects [19-24].

• Conceptual Understanding: ChatGPT can provide explanations, examples, and clarifications to enhance students' understanding of complex topics.
• Problem Solving: Students can seek guidance on solving engineering problems and receive step-bystep solutions or approaches.
• Design Assistance: ChatGPT can offer insights and suggestions during the engineering design process. It can help students brainstorm ideas, evaluate design alternatives, and provide recommendations based on engineering principles and best practices.
• Virtual Tutoring: ChatGPT can serve as a virtual tutor, offering personalized assistance and guidance to students.
• Learning Resources: ChatGPT can generate supplementary learning materials such as practice problems, examples, and case studies.
• Language Support: ChatGPT can help with technical writing, grammar, vocabulary, and clarity of communication.

Given ChatGPT’s potential to significantly aid in engineering education, it raises the question of whether this tool can be utilized to enhance institutional capabilities in preparing for ABET accreditation. Considering the six approaches for institutional capability enhancement, the authors undertook a pilot experiment utilizing ChatGPT to explore the potential application of this AI tool. The subsequent sections will present the experimental findings.

2. Methodology of Experiment

In this manuscript, ChatGPT was assessed by presenting questions across several areas and evaluating the helpfulness of the responses in each area against established rubrics. These areas include:

• Offering guidance to gain a comprehensive understanding of ABET.
• Assisting in editorial tasks for ABET self-study reports.
• Providing proactive advice on potential areas of improvement.
• Conducting diagnostic evaluations of the due process response(30-Day Response).
• Developing action plans to address identified shortcomings.
• Generating faculty training plans to enhance ABET criteria compliance.

To collect information, the authors utilized publicly available Self-Study reports and 30-Day Responses as their source materials. Excerpts from these documents were input into ChatGPT along with specific questions posed by the authors. The answers obtained were slightly reformatted by the authors and italicized when included in this section of the manuscript. Furthermore, drawing upon their prior experience in ABET accreditation, the authors formulated additional pertinent questions. These questions were subsequently presented to ChatGPT, yielding results that were analyzed by the authors. To evaluate the usefulness of the answers provided by ChatGPT,the authors leveraged their experience in ABET accreditation and engineering education. Each answer was assigned a general score ranging from 1 to 4, following the rubrics outlined in Table I below.

Rating	Rubrics
1	Response is not useful at all or irrelevant
2	Response is relevant but lacks specific executable details
3	Response provides general guidance and some executable details
4	Response provides guidance with enough executable details

Table 1: Rubrics to Evaluate the Chatgpt Answers.

A. Offering Guidance to Gain a Comprehensive Understanding of ABET For individuals who are new to the ABET accreditation process, receiving assistance in swiftly comprehending the general process and timeline of ABET accreditation is invaluable.

Authors: Please give a timeline for ABET accreditation preparation.

ChatGPT generated the answer below: “The timeline for ABET accreditation preparation can vary depending on the specific circumstances of the program and institution. However, here's a general timeline that can serve as a starting point for planning.

1-2 Years before the Accreditation Visit

•Research ABET requirements and eligibility criteria.
• Determine program readiness and identify areas for improvement.
• Form a self-study team and designate a team leader.
• Develop a self-study plan and establish milestones.
• Begin collecting data and evidence for the self-study process.
• Conduct an initial analysis of the program's strengths and weaknesses.

12-18 Months before the Accreditation Visit

•Continue gathering data and evidence.
• Conduct a comprehensive analysis of the program and identify improvement areas.
• Develop action plans to address the identified deficiencies.
• Make necessary changes to the program, curriculum, assessment methods, or faculty qualifications.
• Implement continuous improvement processes and collect assessment data.
• Create a draft of the self-study report.

6-12 Months before the Accreditation Visit

•Finalize the self-study report and ensure it meets ABET's requirements.
• Conduct a mock visit or internal review process to simulate the accreditation visit.
• Address any remaining areas for improvement identified during the mock visit.
• Refine the self-study report based on the feedback from the mock visit.
• Review and analyze assessment data and make adjustments as needed.
• Prepare documentation and evidence for the accreditation visit.

3-6 Months before the Accreditation Visit
 

•Submit the application for accreditation and required documentation.
• Review the final version of the self-study report and ensure its completeness and accuracy.
• Coordinate logistics for the accreditation visit, including scheduling and facilities.
• Prepare faculty, staff, and students for the accreditation visit.
• Conduct any additional training or workshops to address specific accreditation requirements.

Accreditation Visit

•Host the accreditation visit by ABET evaluators.
•Provide access to program information, facilities, faculty,students, and documentation for evaluation
• Address any questions or concerns raised by the evaluators during the visit.
• Share additional evidence or information requested by the evaluators.

After The Accreditation Visit

•Review the evaluator's report and address any recommendations or concerns.
•Make necessary adjustments to the program based on the feedback received.
•Provide any additional documentation or clarification requested by ABET.

•Await the final decision from ABET regarding your program's accreditation status.

Please note that this timeline is a general guideline, and the actual timeline can vary based on the specific circumstances of your program and institution. It's important to refer to ABET's guidelines and consult with ABET directly for detailed instructions and timelines tailored to your specific situation.”

Evaluation

ChatGPT demonstrates proficiency in summarizing large quantities of data and provides concise and valuable guidance to assessment preparers. The response offers a general framework for preparing ABET accreditation. The suggestions for SelfStudy Report preparation, including forming a Self-Study team, developing a Self-Study plan, creating a draft report, finalizing the report, and reviewing the final version, align coherently with the general timelines specified by ABET. However, the authors acknowledge that the ChatGPT response overlooks important logistical aspects, such as application fee payment and scheduling visit times with the ABET evaluation team. Additionally, crucial milestones in the accreditation process, namely the 7-Day Response and the 30-Day Response, were not mentioned. As a result, the authors assigned a score of "3" to ChatGPT’s response.

Assisting in Editorial Tasks for Writing ABET Self-Study Reports

When preparing an ABET Self-Study report, much time must be spent on making editorial changes to the documents after raw materials and data have been collected. Editorial assistance, if ChatGPT can offer, will save a great deal of time during the Self-Study writing phase. To test the effectiveness of ChatGPT editorial assistance, the authors found an overview of advisement for a student advising program from a Self-Study selected and itemized key phrases that they believed substantial, and arbitrarily changed the order of these words as listed below [25].

•Advisement uses a two-tier system.
•The academic advisors SEAS Office of UndergraduateEducation (SEAS OUE), mainly in the first two year
•Program advisement, mainly in the junior and senior years.
•Early in their academic career, students are not yet integrated in their major (or perhaps have not even selected a major).
•Academic advisement is best delivered through SEAS OUE professional academic advisors.
•In the junior and senior years, students need advice on technical electives and career paths and this advice is best delivered through their programs.
•Students are free to seek advice from the SEAS OUE advisors at any time they wish.
•Students are encouraged to meet with an advisor each semester.

The authors input these fragmented phrases into ChatGPT, resulting in an AI-generated version in seconds. Both versions are listed below:

AI-Generated Version

“Advisement within the program follows a two-tier system to cater to the specific needs of students. During the initial two years, academic advisors from the SEAS Office of Undergraduate Education (SEAS OUE) take the lead in providing guidance. This is particularly important as students are still in the early stages of their academic journey, often without a declared major or full integration into a specific program. The expertise of SEAS OUE professional academic advisors ensures that academic advisement is effectively delivered during this phase. As students progress into their junior and senior years, program advisement becomes the focal point. At this stage, students require advice regarding technical electives and career paths, and their respective programs are best equipped to provide such guidance. However, it is important to note that students are always welcome to seek advice from the SEAS OUE advisors at any time they wish. To ensure continuous support, students are strongly encouraged to meet with an advisor each semester, facilitating a comprehensive and personalized academic experience.”

Original Version from Self-Study

“Advisement of SEAS students is conducted using a twotier system consisting of the SEAS Office of Undergraduate Education (SEAS OUE) academic advisors, mainly in the first two years, and program advisement, mainly in the junior and senior years. The concept is that early in their academic career, students are not yet integrated in their major (or perhaps have not even selected a major) and academic advisement is best delivered through SEAS OUE professional academic advisors. In the junior and senior years, students need advice on technical electives and career paths and this advice is best delivered through their programs. This demarcation is not absolute and students are free to seek advice from the SEAS OUE advisors at any time they wish. Students are encouraged to meet with an advisor each semester.”

Evaluation

Upon initial assessment, the authors observed a striking resemblance between the original version and the version generated by ChatGPT. However, upon closer examination, they discovered something even more intriguing. The original 131-word paragraph elucidates the SEAS two-tier advising structure and its justification in a dense manner, which may pose comprehension challenges for some readers. On the other hand, the version generated by ChatGPT, despite using more words (168), presents the information in a manner that is easier to read and follow.

ChatGPT’s strong editing capability can conveniently provide drafts for various parts of a program’s Self-Study report that the team can later revise and improve, saving tremendous resources that are required for institutions when preparing their ABET Self-Study writing. The authors rated a score ChatGPT’s response.

Providing Proactive Advice on Potential Areas of Improvement

Shortcomings in program development are not uncommon. The ability for the faculty and administration to understand the seriousness of a shortcoming and take preventive measures against it will not only increase the likelihood for the program to pass the rigorous accreditation process, more importantly, it ultimately enhances the quality and solidness of the program. The authors tested ChatGPT with two questions.

Authors: One student did not take the senior project, but he graduated from mechanical engineering program. Will this be a deficiency in ABET accreditation?

ChatGPT

“If a student did not complete the senior project requirement but still graduated from the mechanical engineering program, it could potentially be considered a deficiency in the context of ABET accreditation. ABET has specific criteria and standards that programs must meet, and the completion of senior projects is often a requirement or an important component of those criteria.

The senior project is typically designed to provide students with a culminating experience that integrates the knowledge and skills they have acquired throughout their program. It allows students to apply engineering principles and problem-solving techniques to a real-world project. The absence of a senior project may indicate a gap in the program's ability to provide students with this valuable experiential learning opportunity.

However, it's important to note that the impact on ABET accreditation will depend on various factors, such as the specific requirements of the accrediting body, the overall strength of the program, and any alternative mechanisms in place to fulfill the experiential learning objectives. Programs that encounter such situations are typically required to provide a justification or an alternative approach to meeting the intended learning outcomes.

It is recommended that the institution consult with ABET directly and review the accreditation criteria to determine the specific implications of a missing senior project and work towards addressing any deficiencies in the program.”

Authors: Prerequisite override sometimes is not given before the student takes a class. What shortcoming might it be in ABET accreditation?

Instead of indicating the severity directly, ChatGPT provided multiple perspectives to evaluate the extent of noncompliance.

ChatGPT

“The lack of granting prerequisite overrides to students before they take a class can potentially have implications for ABET accreditation. Accreditation standards often emphasize the importance of ensuring that students have the necessary knowledge and skills before progressing to higher-level courses. Prerequisites serve as a foundation for subsequent coursework and help maintain the integrity and rigor of the program.

Here are some potential shortcomings that may arise from not granting prerequisite overrides:

Curriculum Alignment:Accreditation standards typically require programs to demonstrate a coherent and aligned curriculum that builds upon foundational knowledge. By not enforcing prerequisite requirements, there is a risk that students may lack the necessary foundational knowledge, which can hinder their understanding and success in advanced courses.

Learning Outcomes:Accreditation often requires programs to define and assess specific learning outcomes for each course. Not enforcing prerequisite requirements can compromise the achievement of those learning outcomes if students are illprepared or lack the necessary background knowledge.

Student Progression:Prerequisites are designed to ensure that student’s progress through the curriculum in a logical and sequential manner. By not granting prerequisite overrides, students may enroll in courses for which they are unprepared, leading to potential difficulties, lower success rates, and delays in their academic progression.

Academic Quality:Accreditation standards prioritize maintaining high academic standards and ensuring the quality of education. Not enforcing prerequisite requirements may raise concerns about the program's commitment to upholding these standards and ensuring that students receive a rigorous and comprehensive education.

It is important for programs seeking ABET accreditation to have appropriate policies and procedures in place to enforce prerequisite requirements and ensure that students are adequately prepared for advanced coursework. This includes clear communication of prerequisite requirements, efficient processes for prerequisite overrides, and monitoring of student progress.

It is recommended to consult with ABET directly to understand the specific implications and expectations regarding prerequisite enforcement for accreditation purposes. They can provide guidance on addressing any shortcomings and establishing effective policies and procedures to meet accreditation standards.”

Evaluation

Although ChatGPT may not provide a direct and conclusive answer using the ABET terms (i.e., deficiency, weakness, concern) regarding the severity of the shortcomings, the authors found that not only the answers are in alignment with ABET accreditation standards and best practices, ChatGPT intends to provide complete analysis to a YES/NO question such as the first presented question. The broader perspectives provided in the elaborated answers are often insightful and can possibly prevent oversights and noncompliance associated with these shortcomings. The authors gave a score of “4” to ChatGPT.

Conducting Diagnostic Evaluations of the Due Process Response (30-Day Response)

The authors used the 30-Day Response of a Mechanical Engineering program which after the Due Process, to examine if ChatGPT is able to evaluate the effectiveness of the actions the university took. The authors input the following into ChatGPT [26].

Authors: ABET found such weakness in Criterion 5 Curriculum as “This criterion requires that the curriculum include a culminating major engineering design experience that incorporates appropriate engineering standards and multiple constraints. The syllabi and other project documents for the senior project course sequence showed minimal evidence of the use of appropriate engineering standards and multiple constraints. Only one of the two project reports reviewed incorporated constraints and one project assignment included a reference to an ASTM test standard. Without a major design experience that incorporates multiple constraints and appropriate engineering standards, student may not be adequately prepared for professional practice. Thus, strength of compliance with this criterion is lacking.” We took Corrective Action Implemented by saying that “The syllabi for the capstone senior design I and senior design II have been modified to include appropriate engineering standards and constraints. These updates in the syllabi went into effect for the current 2020 – 2021 academic year. Documentation of both syllabi’s can be seen in the Appendix along with its implementation into the ME 2020-2021 senior project.” Did their response address ABET weakness? ChatGPT answered the question as follows and indicated that the response addressed the weakness.

ChatGPT

“Based on the information provided, it appears that your response addressed the weakness identified by ABET regarding the lack of incorporation of appropriate engineering standards and multiple constraints in the senior design experience. By modifying the syllabi for the capstone senior design courses to include these elements, you have taken corrective action to address the identified deficiency. Including documentation of the modified syllabi in the Appendix and noting their implementation for the current academic year further strengthens your response and demonstrates your commitment to compliance with ABET criteria. To ensure the effectiveness of your response, it is important to ensure that the modifications are effectively implemented and that students are indeed exposed to appropriate engineering standards and multiple constraints in their senior project experiences. Ongoing monitoring and assessment of the updated curriculum can help verify compliance and ensure that students are adequately prepared for professional practice. It's important to consult with your institution's ABET accreditation coordinator or evaluator to confirm whether your response adequately addresses the weakness and meets the specific requirements outlined by ABET.”

Evaluation

The authors put another 30-day response for ChatGPT to review. In both cases, ChatGPT reaffirmed that the universities had taken appropriate actions to ensure successful re-accreditation [27]. Again, the question (“Did their response address ABET weakness?”) presented by the authors were a YES/NO question. ChatGPT not only responded with the correct (affirming) statement, but also offered additional (and meaningful) suggestions (implementation of the modifications, ongoing monitoring and assessment, and maintaining dialogs with the accreditation team, etc.). The authors acknowledged the effectiveness of ChatGPT in providing valuable insights and gave a score of “4” to ChatGPT’s response.

Developing Action Plans to Address Identified Shortcomings

Immediate action is crucial in addressing any shortcomings identified during and after an ABET accreditation visit, as required by the 7-Day Response and 30-Day Response (Due Process). For instance, the ABET visit identified the following weakness in an Electrical Engineering program [28].

“While the program did provide some measures and data specific to the electrical engineering cohort, the process does not guarantee disaggregation of assessment data across programs for evaluation of outcome attainment. Therefore, the program cannot determine the degree to which some of the program outcomes are attained.”

Actions were taken by the university to address the weakness and the program weakness had been resolved as a result. These actions included.

• “Retrospectively disaggregating assessment data from spring 2017 to spring 2018 to obtain programspecific data.
• Formally revising assessment methods to ensure the separation of assessment data, starting from select courses taught in fall 2018 and continuing from spring 2019.
• Revising the assessment of ABET student outcomes (a) to (k) using the newly implemented assessment methods.”
• The authors sought an action plan from ChatGPT to mitigate this weakness, and the provided response included corresponding actions that were highlighted in the AIgenerated plan:
• “Based on the finding from the ABET accreditation, here are some suggested action plans to address the issue and improve the program's ability to evaluate outcome attainment across programs:
• Review and Revise Assessment Process: Evaluate the current assessment process and identify areas where improvements can be made to ensure the disaggregation of assessment data across programs. This may involve reviewing assessment methods, data collection techniques, and analysis procedures to capture program-specific data.
• Establish Clear Outcome Metrics: Define clear and measurable outcome metrics for each program. Ensure that the program outcomes are well defined, aligned with accreditation criteria, and can be effectively evaluated and measured.
• Develop a Data Collection and Reporting System: Implement a systematic data collection and reporting system that allows for the disaggregation of assessment data across programs. This may involve developing standardized templates, tools, and processes for collecting, organizing, and analyzing data to evaluate outcome attainment.
• Train Faculty and Staff: Provide training and professional development opportunities for faculty and staff involved in the assessment process. This training should focus on understanding the importance of disaggregating assessment data, using appropriate assessment techniques, data analysis methods, and reporting procedures.
• Implement Regular Program Evaluation: Establish a regular program evaluation cycle to assess the attainment of program outcomes across programs. This evaluation should include the analysis of disaggregated assessment data and provide insights into areas where improvements are needed.
• Utilize Benchmarking and Best Practices: Explore benchmarking and best practices in assessment and evaluation from other accredited programs or institutions. Learn from their approaches to disaggregating assessment data and adapt relevant strategies to suit your program's needs.
• Enhance Communication and Collaboration: Foster a culture of collaboration among faculty, staff, and program stakeholders. Encourage discussions and sharing of assessment data and findings across programs to facilitate a comprehensive understanding of outcome attainment and identify areas for improvement.
• Document and Monitor Progress: Keep detailed documentation of the actions taken to address the finding and monitor progress over time. Regularly review and update the documentation to reflect any changes made to the assessment process or data disaggregation procedures.

It is important to consult with ABET and accreditation evaluators to seek their guidance on specific expectations and requirements related to disaggregating assessment data across programs. They can provide further insights and clarification to ensure the proposed action plans align with accreditation standards and expectations.”

Evaluation

ChatGPT provided a more comprehensive approach compared to the actual (and adequate) plan implemented by the university. The authors acknowledged ChatGPT’s capability in providing advisory assistance and rated it with a score of “4”.

Generating Faculty Training Plans to Enhance ABET Criteria Compliance

Preparing for an engineering/technology ABET accreditation requires broad collaboration from the entire faculty associated with the program. Faculty training is key, yet the most challenging to ensure the success of accreditation efforts. Here is an exemplary request the authors put forward to test how ChatGPT will help generate faculty training plans with certain constraints provided.

Authors: Please generate a faculty-training plan for ABET accreditation. We are a small department that can’t commit many resources to ABET accreditation. Faculty has limited knowledge in ABET as well. ChatGPT proposed the following plan:

Faculty Training Plan for ABET Accreditation

Objective: To provide faculty members with the necessary knowledge and skills to actively contribute to the ABET accreditation process, despite limited resources and prior knowledge.

Introduction to ABET Accreditation (1 hour)

• Overview of ABET accreditation: Purpose, benefits, and process.
• Understanding the ABET criteria and standards.
• Importance of faculty involvement in accreditation.

ABET Self-Study Process (2 hours)

• Overview of the Self-Study report and its components.
• Steps and timeline for completing the Self-Study.
• Identifying the required documentation and evidence.Understanding ABET Criteria (2 hours).
• In-depth exploration of each ABET criterion relevant to the department.
• Discussion of the expectations and evidence required for compliance.
• Reviewing examples of successful implementation in other programs.

Data Collection and Assessment (2 hours)

• Introduction to effective data collection methods for outcome assessment.
• Implementing assessment strategies to measure studentlearning outcomes.
• Analyzing assessment data and using it for continuous improvement.

Documentation and Report Writing (2 hours)

• Guidelines for documenting evidence and preparing the Self-Study report.
• Techniques for effective report writing, including clarity, organization, and coherence.
• Reviewing samples of well-written Self-Study reports from other institutions.

Addressing Deficiencies and Improvement Plans (2 hours)

• Understanding common deficiencies and challenges in ABET accreditation.
• Developing action plans to address identified shortcomings.
• Implementing improvement strategies and monitoring progress.

Faculty Roles and Responsibilities (1 hour)

• Clarifying faculty roles and responsibilities in the accreditation process.
• Encouraging collaboration and teamwork among faculty members.
• Ensuring ongoing commitment to quality improvement.

Resource Maximization Strategies (1 hour)

• Exploring ways to maximize limited resources for ABET accreditation.
• Leveraging existing initiatives and projects for accreditation purposes.
• Identifying external resources and professional development opportunities.

Note: The training plan should be flexible and adapted to the specific needs and resources of the department. It is recommended to allocate dedicated time for faculty discussions, sharing experiences, and addressing department-specific concerns throughout the training sessions. Regular follow-up and support should be provided to ensure faculty members are well-prepared and engaged in the ABET accreditation process.”

Evaluation

ChatGPT is powerful in providing general training plans for ABET accreditation preparation. It not only successfully identifies important areas that are truly relevant and outlines them with time required for each topic. Although some finetuning and customization is required to accommodate specific needs of faculty in a particular program, the authors were amazed that the proposed time (number of hours) for the topical areas generally makes sense. Those challenges identified earlier in Section I was given more training time compared to the other areas. The authors gave ChatGPT a score of “4” for its capability in generating faculty ABET training plan.

Additional Questions Asked

To obtain a more comprehensive understanding of how ChatGPT would respond to frequently asked questions by faculty and administrators, the authors conducted more inquiries spanning across five of the six categories mentioned earlier. They excluded the category of editorial assistance since it is not as specific to the context of ABET accreditation as the other five categories. ChatGPT provided satisfactory responses to certain questions, scoring “3” or higher, as shown in Table II..

 

Category	Questions asked
General guidance	•Do we have to have a graduate before requesting ABET accreditation? • How much does it cost before obtaining ABET accreditation? • Does ABET accreditation apply to master’s program? • What is the difference between 7-Day and 30-Day Responses on ABET accreditation? • What should be included in the 7-Day Response? • Can we give gifts to ABET PEV?
Shortcoming analysis	•Due to leadership change, we did not collect data for assessment for two years. What shortcoming might it be in ABET accreditation? • We do not have a documented procedure to grant credits based on past work experience. What risk might that be involved?. • We have just one faculty member in our program. But the administration has started the hiring process. Will this be a deficiency?. •Some students transferred over their lower-level courses as upper-level ones to our institution. Will that be a problem for ABET accreditation?. • Students used their internship to count for their required academic credits. Could that be an issue for ABET accreditation?.
Response evaluation	•Students needed to graduate soon as they had found a job. They had to take a couple of classes out of sequence. This has been identified as a weakness in our program. We will override the pre-requisite in the future to accommodate those students only. Does this action address weaknesses?. • Students used their internship to count for their required academic credits. Could that be an issue for ABET accreditation?
Addressing shortcomings	•Our mechanical vibration lab is basically using simulation methods not physical equipment. Do you have any suggestions on how to align this lab with ABET requirements on hands-on learning?. • Do you have any suggestion on how to solve ABET finding “However, provided records had extensive detail, were not organized and information was difficult to follow. It was not clear from the provided information if student outcome “1a” was assessed or evaluated.” • ABET has the finding that “While the program has documented processes for assessment of outcomes, methods used to assess outcome attainment do not segregate by specific outcome. In a number of situations, a single assessment instrument is used for multiple outcomes. For example, the score on a course final exam is used to assess multiple outcomes addressed in that course without any disaggregation by outcome.” How do we address this weakness?.
Faculty training	•Generate a training seminar plan to introduce ABET at an engineering college in a foreign country. •Generate a training workshop plan on how to introduce university facilities to ABET. •Generate a 2-hour case study session about ABET accreditation.

Table 2: List of Questions that Chatgpt Scored “3” or Above.

3. Results and Discussions

The rated helpfulness scores of ChatGPT’s responses to the six areas, including those additional questions, are summarized below in Table III.

Category	Score
General guidance	3.5
Thought compiling	4.0
Shortcoming advice	4.0
Response evaluation	4.0
Addressing shortcomings	4.0
Faculty training	3.8

Table 3: Overall Helpfulness of Chatgpt in the Examined Areas.

Based on the above table, most categories receive high ratings except for general guidance, which has a slightly lower rating of 3.5. This indicates that there may be some areas where the general guidance provided could be improved. However, the overall performance is positive, with effective thought compiling, shortcoming advice, response evaluation, addressing shortcomings, and faculty training.

ChatGPT consistently provides relevant responses to questions pertaining to all stages of ABET accreditation, effectively directing the attention of the preparation team to the appropriate areas. When presented with simple YES/NO questions, ChatGPT offers multiple perspectives that are pertinent to the inquiries, which can facilitate fruitful discussions among faculty members to reach a consensus. It was pleasantly surprising to discover that, in addition to its relevant and insightful perspectives in evaluating ABET matters (i.e., analytical ability), ChatGPT often generates actionable plans (i.e., synergistic ability) for addressing shortcomings and designing faculty training initiatives.

This study also acknowledged the limitations of ChatGPT in providing answers to specific questions that require real-time data, which are defined as data that is current and reflects the most recent state or status of a particular subject or event. These questions were excluded from the evaluation of ChatGPT’s capabilities. Some of the exemplary questions are:

•How many electrical engineering programs in the US have ABET accreditation at present?
• Which university in Latin America (or any other foreign country) has been accredited by ABET?
• How much monetary amount is the ABET accreditation application fee?
• Give a name of ABET PEV who accredits Electrical Engineering Program.
• How many ABET PEV’s in North Carolina who accredit Mechanical Engineering program?
4. Conclusions and Future Research

Using AI tools, such as ChatGPT, can enhance efficiency and quality in preparing for ABET accreditation. They can provide comprehensive assistance in various areas, including:

•Answering questions and offering guidance on accreditation requirements, documentation, steps, and best practices.
• Which university in Latin America (or any other foreign country) has been accredited by ABET?
• Reviewing documentation, providing feedback on clarity, organization, and compliance with ABET criteria, and suggesting improvements.
• Providing training plans and resources on topics relevant to ABET accreditation, such as designing effective assessment processes and developing program outcomes.
• Facilitating collaboration among team members, allowing for brainstorming, discussion of challenges, and exploration of innovative approaches to meet accreditation requirements.
•However, it’s important for ABET accreditation preparers to approach AI tools with an open mind while keeping certain considerations in mind:
• WStay updated on the latest changes in ABET accreditation policies, rules, and procedures.

• Maintain a critical thinking approach when evaluating information provided by AI.
• Avoid over-reliance on AI tools and maintain control as the experts in using AI technology.
• Ensure data security by implementing proper supervision and authentication of data management when interacting with AI tools. Caution should be taken to protect the confidentiality of any classified or unclassified information related to the institution undergoing accreditation.

By leveraging the power of AI tools in a strategic manner and keeping the above considerations in mind, ABET accreditation preparers can maximize the benefits of technology while upholding control and ensuring the integrity of their data.

Moving forward, the authors have outlined their plans to further investigate the potential of ChatGPT in analyzing assessment data, student outcomes, and other quantitative information pertinent to accreditation. This future research will delve into utilizing ChatGPT for conducting statistical analyses on the gathered assessment data, as well as exploring how this data can be harnessed to enrich evidence-based decision-making processes for continuous improvement. By incorporating ChatGPT into these analytical endeavors, the authors aim to unlock new insights and possibilities for leveraging data-driven approaches in the pursuit of enhanced educational quality and accreditation practices.

References

1. ABET, “Accreditation Step by Step.”
2. Heydt, G. T., & Bohmann, L. J. (2020). The Impact of ABET Accreditation on Power Engineering Educational Programs. In 2020 52nd North American Power Symposium (NAPS) (pp. 1-5). IEEE.
3. McPherson, P., Phillips, M., & Reiter, K. (2019, February). Integrating Technical Standards into ET Curricula to Meet ABET standards and Industry Needs. In 2019 CIEC.
4. Grinberg, I. Y., & Singer, J. (2021). ETAC-ABET and EvaluateUR-CURE: Findings from Combining Two Assessment Approaches as Indicators of Student-learning Outcomes. In 2021 ASEE Virtual Annual Conference Content Access.
5. Lutz, A. (2022, August). A Rankine Cycle Design Project for Assessment of ABET Student Outcome# 1. In 2022 ASEE Annual Conference & Exposition.
6. Minaie, A., & Sanati-Mehrizy, R. (2022, August). Capstone Courses in a New ABET Accredited Electrical Engineering Program. In 2022 ASEE Annual Conference & Exposition.
7. Baine, N., Brakora, K., & Pung, C. (2022, August). Evaluating ABET Student Outcome (2) in a Multidisciplinary Capstone Project Sequence. In 2022 ASEE Annual Conference & Exposition.
8. Zaid Abualkishik, A., Atassi, R., Singh, A., Elhoseny, M., Alwan, A. A., Iqbal, R., & Khelifi, A. (2022). Outcomesbased assessment and lessons learned in ABET-CAC accreditation: A case study of the American University in the Emirates. Mobile Information Systems, 2022.
9. Cong, B., Ryu, C., & Unnikrishnan, R. M. (2020). Make Your Data Work: Infusing CMMI Culture in Data Analysis for ABET Accreditation. In 2020 ASEE Virtual Annual Conference Content Access.
10. Bachnak, R., Marikunte, S. S., & Shafaye, A. B. (2019). Fundamentals of ABET accreditation with the newly approved changes. In 2019 ASEE Annual Conference & Exposition.
11. Hall, T. M., Danielson, S., & Cheung, A. C. (2020). Streamlining continuous improvement: Efficiently creating value while satisfying ABET criterion 4. In 2020 ASEE.
12. Carelli, J. (2020). Achieving ABET Accreditation: An Outcome Assessment Case Study. In 2020 International Conference on Computational Science and Computational Intelligence (CSCI) (pp. 959-963). IEE
13. McCullough, C. L. (2020). Can ABET Assessment Really Be This Simple? In 2020 ASEE Virtual Annual Conference Content Access.
14. Battistini, A., & Kitch, W. A. (2021). Make Assessment Straightforward: A Case Study on the Successful Implementation of ABET Student Outcomes 1-7. In 2021 ASEE Virtual Annual Conference Content Access.
15. He, H. M. (2023). Robotgpt: From chatgpt to robot intelligence.
16. NBorji, A. (2023). A categorical archive of chatgpt failures.
17. Grbic, D. V., & Dujlovic, I. (2023, March). Social engineering with ChatGPT. In 2023 22nd International Symposium INFOTEH-JAHORINA (INFOTEH) (pp. 1-5). IEEE.
18. Malinka, K., Peresíni, M., Firc, A., Hujnák, O., & Janus, F. (2023). On the educational impact of chatgpt: Is artificial intelligence ready to obtain a university degree? In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (pp. 47- 53).
19. Shoufan, A. (2023). Exploring Students’ Perceptions of CHATGPT: Thematic Analysis and Follow-Up Survey. IEEE Access.
20. Bahrini, A., Khamoshifar, M., Abbasimehr, H., Riggs, R. J., Esmaeili, M., Majdabadkohne, R. M., & Pasehvar, M. (2023). ChatGPT: Applications, opportunities, and threats. In 2023 Systems and Information Engineering Design Symposium (SIEDS) (pp. 274-279). IEEE.
21. Kashefik and T. Mukerji, “ChatGPT for Programming Numerical Methods.”
22. Banerjee, P., Srivastava, A., Adjeroh, D., Reddy, Y. R., & Karimian, N. (2023). Understanding ChatGPT: Impact Analysis and Path Forward for Teaching Computer Science and Engineering.
23. Pardos, Z. A., & Bhandari, S. (2023). Learning gain differences between ChatGPT and human tutor generated algebra hints.
24. Qadir, J. (2023). Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education. In 2023 IEEE Global Engineering Education Conference (EDUCON) (pp. 1-9). IEEE.
25. Granfield, R., & Mangan, L. (2012). University at Buffalo, The State University of New York. Journal of Higher Education Outreach and Engagement, 16(4), 205-206”.
26. Weber State University, “Thirty Day Due Process Response for the Mechanical Engineering Program.”
27. Central Washington University, “The 30-Day Due-Process Response to the ETAC-ABET Draft Statement for the Bachelor of Science in Mechanical Engineering Technology Program.”
28. ABET, “Summary of Accreditation Actions.”

Copyright:

Copyright: ©2023 Jianchu Yao, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License,which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.