CS373 - Effective Reviews

Introduction to Writing Effective (AI-assisted) Article Reviews

What is an Article Review?

In your professional career you may be asked by a colleague to read their technical article and give feedback, or you may be part of a conference committee or journal editorial board with the responsibility to review submissions that are being considered for publication. Your job is to provide a review that helps the author to improve their article prior to distribution or publication.

Structure for a Good Review

A good review has three basic elements:

1. A 1 - 2 paragraph article summary: It is important to summarize your understanding of the article. This important for two reasons: (i) the summary shows you read the article and (ii) if you misunderstood the article, your summary will make that clear.
2. A description of article strengths: What did you like about the article? Was it technically solid with sufficient breadth and depth? Did it have a good story line? Was the author efficient and effective with their words?
3. A description of opportunities to improve the article: How could the article be improved? Be constructive in your language. Transform your observation of a weakness into a positive statement of what could be done to improve it.

AI-assisted Reviewing

Generative AI tools are exceptionally good at summarizing content, if there is a sufficient amount of similar content it has been trained on. For our purposes, we can generally find an AI summary useful as a starting point for our own reviews, including self review of our written work.

We will use AI tools in our review process in the following ways:

1. We will use the following rubrik:
   • Breadth of topic exploration: Is there something missing in the discussion of the topic?
   • Depth of topic exploration: Demonstrates knowledge about the topic?
   • Enough words, no more: Prose expressive and efficient, free of vague, imprecise terms and jargon?
   • Story integrity: Good beginning, middle and end? Conclusions synthesize points?
2. As an author:
   • Ask an AI tool to find strengths and opportunities for improvement in your content and story telling using the rubrik.
   • Iterate on your content to address issues the tool found and request an updated AI review.
3. As a reviewer:
   • Start your review process by asking the AI tool to:
     • Summarize the article in 1 - 2 paragraphs.
     • Provide a description of the article strengths and opportunities for improvement using the above rubrik.
   • Assess the quality of the AI review.
   • Add, subtract, and correct anything you find to be insufficient.
   • Include the AI content, your assessment, and the modified review as separate parts of your submission.