Skip to Main Content

Generative AI

Evaluating GenAI Output

Any content you create using GenAI will need to be verified carefully. GenAI systems are known to generate content that is false and inaccurate, referred to as hallucinations.

Using elements of the CRAAP test you can carefully evaluate and verify the information generated. Follow the tabs here to see how to apply the CRAAP test to AI generated content.

The use of GenAI and the validity of the content generated will depend widely on the context of how you are using it. Our guidance is to use it merely as a tool to assist in your research and writing practices, but not to rely on it to do the work for you.

Currency

GenAI tools are limited to the information within their datasets. This may mean you are not including the most up to date current information in your work.

Check the currency of the GenAI tool by asking "how recent is your training data", to get a better understanding of how recent the material is.

And then ask yourself:

  • Does this meet your assignment requirements?
  • Does your topic require more current information?

Relevance

GenAI has a tendency of providing general information on most topics, so it is important to determine if the information provided is appropriate for inclusion in your assignment or if it should be used simply as a step in your process to find more information.

Ask yourself,

  • Is this information important to the assignment?
  • Does the information add insight or does it merely assist in guiding you on what to look for elsewhere?
  • How does this compare to other source materials you will use?

Authority

Author:

Where your GenAI output has provided you with sources or citations, who is the author? What is there expertise? Is there a potential bias with what they have written? 

Developer:

With GenAI you should also consider the tool itself and who it has been developed by? Does this present a potential bias in the information the GenAI has been trained on? Does the tool have a statement on whether they take responsibility for the output of the tool?

 

Accuracy

Some GenAI tools may provide links to sources that you can use to verify the information it has generated. 

  • Read the source material (where available).
  • If you can not verify the information as accurate look for alternate sources to support claims.

Checking citations

GenAI is prone to generating references when requested to include them, but they are not always correct or real. Presenting material with false citations may be deemed an academic integrity violation.

GenAI citations may use some "real" information in these hallucinations, so it is important to verify all of the information the system provides you. Here are the steps in verifying citations:

  • Search for the title of the publication (book title or article title) in Page1+ or a search engine.
  • Search for the journal and review the date and issue information for accuracy.
  • If you are unable to find the publication, prompt the GenAI tool for additional information (ISBN, ISSN or DOI) for the publication.
  • If the publication exists, verify the information provided is accurate. If the publication does not exist do not include in your work.

Purpose

Challenge yourself to think of purpose both of the output and the GenAI tool.

  • What is the purpose of the content? 
  • What is the purpose of the GenAI tool, are you using it appropriately?
  • Does using the GenAI tool enhance your learning or replace it? 
  • Does the content meet the purpose you intended?

 

chat loading...