With the widespread availability of Artificial Intelligence (AI) through tools like ChatGPT, there is an equally widespread availability of AI checkers. It can be difficult to sift through what to do with such a proliferation of new technology, especially when it has such direct implications for higher education. This feature gives an overview of AI generator technology and briefly investigates the reliability of current AI generation detection tools.
Conversations surrounding AI generators
The recent proliferation of artificially intelligent chatbots that enable users to produce text based on prompts that dictate length, format, style, level of detail, and language have sparked many recent conversations in higher education circles. Here at Ohio State, many of our partners in teaching and learning have already released formal statements and assistive content on the subject. If you are not familiar with these items, we encourage you to explore these statements and resources using the links below:
Prepared Statement
Teaching and Learning Guides
Many of the conversations pertaining to AI generators surround what can be done to reliably detect when these tools have been or are being used. ASC ODE has explored some considerations surrounding these questions to help faculty imagine the future of these tools for teaching and understand their current limitations, the findings of which are shared below.
Detecting AI generated text
One of the most pressing concerns that people have raised regarding AI-generated texts in the context of higher education is the implications it poses for academic integrity. Because widespread use of AI-generated content is an emergent issue, AI detection software is popping up as quickly as AI-generating software and both are, ironically, heavily reliant on AI programming to learn and process text and commands. Thus, it is difficult to trust the reliability of some of these tools and know with confidence if something is AI-generated.
As a part of our investigation, ASC ODE experimented with AI detectors and ran the same text through three different free programs. The text we used was AI generated using the OpenAI chatbot, ChatGPT, based on the prompt, “describe in detail the challenges and benefits of using AI in higher education.” The text that ChatGPT produced was as follows, with only the formatting altered to increase accessibility:
Using artificial intelligence (AI) in higher education presents both challenges and benefits. Let's explore them in detail:
Challenges:
- Data privacy and security: Implementing AI requires collecting and analyzing large amounts of data, including personal information about students and faculty. Ensuring the privacy and security of this data can be a significant challenge, as any data breaches could lead to serious consequences.
- Bias and fairness: AI systems heavily rely on data for training, and if the data used to develop AI algorithms contain biases, it can result in biased outcomes. This can perpetuate existing inequalities and discrimination in higher education, such as biased admissions or grading processes. Ensuring fairness and mitigating bias in AI systems is an ongoing challenge.
- Ethical considerations: AI raises ethical questions, such as the appropriate use of student data and the responsibility of AI systems in decision-making processes. Institutions must establish clear ethical guidelines to address these concerns and ensure that AI technologies are used responsibly.
- Adaptation and acceptance: Implementing AI in higher education requires significant adaptation and acceptance from faculty, staff, and students. There may be resistance to change, fear of job displacement, or concerns about the accuracy and reliability of AI systems. Overcoming these challenges requires effective communication, training, and support for all stakeholders.
Benefits:
- Personalized learning experiences: AI can analyze vast amounts of data to understand individual student needs, learning styles, and progress. This enables the creation of personalized learning paths, adaptive assessments, and tailored feedback, promoting student engagement and improving learning outcomes.
- Intelligent tutoring and support: AI-powered virtual tutors or chatbots can provide personalized assistance to students, answering their questions, offering guidance, and providing immediate feedback. This can enhance student support services, alleviate the burden on faculty, and enable round-the-clock availability of resources.
- Enhanced administrative processes: AI can streamline administrative tasks, such as admissions, enrollment, and course scheduling. This frees up valuable time for faculty and staff to focus on more complex and meaningful activities, such as curriculum development, research, and student mentorship.
- Data-driven decision making: AI can analyze large datasets to generate insights and predictive models, helping institutions make informed decisions. This includes identifying at-risk students, predicting enrollment trends, and optimizing resource allocation, leading to improved institutional efficiency and strategic planning.
- Research advancements: AI can facilitate data analysis, pattern recognition, and simulations, accelerating scientific research in various disciplines. It enables researchers to process vast amounts of data quickly, discover new patterns, and generate novel insights, contributing to breakthroughs and innovation.
- Lifelong learning and upskilling: AI technologies can support continuous learning and professional development by providing personalized recommendations for online courses, suggesting relevant resources, and delivering targeted skill-building opportunities. This promotes lifelong learning and helps individuals adapt to evolving workplace requirements.
In conclusion, while there are challenges associated with implementing AI in higher education, the benefits it brings, such as personalized learning, administrative efficiency, and research advancements, hold significant potential for transforming education and improving student outcomes. It is crucial to address the challenges through responsible implementation, ethical guidelines, and ongoing monitoring to maximize the benefits and ensure equitable and inclusive AI adoption in higher education.
The results of this quick test produced a high degree of variability:
- Copyleaks.com: reported an “75.3% probability for AI.”
- Zerogpt.com: reported “43.08% AI GPT, suspected to be most likely generated by AI.”
- SEO.ai: reported an “100% probability for AI,” with 100% prediction, entropy, and correlation, and 99% perplexity as measures cited.
Based on our findings, one of the most immediately apparent issues is the inconsistency of parameters used for identifying supposed AI-generated text. For example, the language that these free AI generator-detectors use to discuss their results can be confusing. For example, in two of the above cases, the word “probability” suggests that there is a high likelihood of AI generation, but it does not give any relevant linked information or cite any mechanisms used to reach this probability. Perhaps the paid versions of these tools would do this, but for a quick look, the grounds on which one might determine academic integrity violations in this case are fraught. Further, much of the programming that developers for these programs and others like them are using is proprietary. Thus, even attempting to use such results to reinforce and/or inform each other is unreliable.
Because the use of AI and AI detectors is rapidly developing, it is worth mentioning that research in this area is working in parallel. An early, pre-publication study that goes into great detail in an effort to answer this question can be found below:
What can be done?
Right now, it is apparent that AI presents a moving target. It brings with it both affordances and limitations that warrant further investigation. In the College of Arts and Sciences at Ohio State, broadly speaking, AI remains an important topic of conversation and exploration, particularly for teaching and learning.
So, what can be done at present to approach some of the challenges of AI generators and checkers? The article from the TLRC linked above includes several examples of how you can create or adapt assignments to promote academic integrity in the face of some of the challenges posed by AI generators. Some of these strategies that ASC ODE would emphasize are:
- Include assessments and activities in your teaching that highlight student investment in processes rather than outputs. One way to do this is through scaffolded activities that breakdown parts of bigger assignments and allow for targeted review along the way. An example of this might be a research paper that is broken down into proposal, outline, and draft rounds where specific and measurable improvements are marked that AI might have a harder time anticipating.
- Highlight and emphasize personal or group reflection that is related to specific learning happening in your classrooms. It is much more difficult for AI to interpret inputs for personal reflection, and certainly easier to spot AI outputs of this kind, from an empirical perspective. For example, the AI chatbot would need multiple examples of a writer’s personal style to produce a reflection that is in the voice of that individual. This is a moment when an instructor might note style and voice that seem incongruous with a student’s ordinary bearing and seek to further investigate.
- Focus on emerging, local, or immediately current events. If applicable to your course context, attention to topics that AI generators cannot pull large amounts of information about means that their success rate in generating reliable outputs is limited. Currently, AI chatbots have programming limitations that result in them being slower in accessing very recent and localized web data. This may not always be the case, but could be a good temporary consideration.
- The rise of AI can provide opportunities and affordances for higher education. For example, designing a prompt for AI chatbots can demonstrate sophisticated critical thinking, as the specificity and relevance of the output is directly tied to the detail of the input. In this way, one might also think of ways to creatively leverage the potential of AI-generated text and information in your classroom. Consider how AI might be used as a complement to your students’ learning to enhance it and their digital literacy skills.
We appreciate that it can be difficult to navigate this landscape. ASC ODE is here to serve as partners for instructors looking to have conversations about the future of AI in teaching and learning and how to teach effectively within these contexts. If you are interested in brainstorming with a partner about potential challenges or usages of AI or want help create assignments that alleviate your concerns surrounding AI, schedule a consultation with one of ODE’s instructional designers using this link.
