For a small batch, paste permitted review text with stable IDs. You can ask for a table in an ordinary conversation; a spreadsheet or an integration is unnecessary for the exercise here. OpenAI's product documentation describes analyzing supplied material and checking the result against its sources. The labels still need a human decision where the language is unclear.
Define the labels before reading a percentage
Use positive for an expressed favorable opinion, negative for an unfavorable one, and mixed when both occur about the same aspect. Reserve neutral for a statement about the aspect without an expressed favorable or unfavorable opinion. Keep unmentioned and uncertain separate from neutral.
These are proposed rules for this exercise, not a universal sentiment standard. Product covers the case's physical qualities; delivery covers arrival and fulfillment. A workplace may use another annotation scheme. Record the chosen definitions with the results so two reviewers can apply the same rule.
The following four reviews are original fictional teaching material. The suggested annotations are authored examples, not a recorded ChatGPT run.
| ID and unchanged review | Product annotation | Delivery annotation |
|---|---|---|
| R1: The case is sturdy, but delivery was a mess. | Positive: “sturdy” | Negative: “a mess” |
| R2: The case arrived yesterday. | Unmentioned: no opinion or property of the case itself | Neutral: arrival timing stated without evaluation |
| R3: The case isn't flimsy. | Positive under our rule: negated criticism of sturdiness | Unmentioned |
| R4: Great, another delivery to the wrong building. | Unmentioned | Negative: wrong-building complaint; “Great” is not praise here |
R2 does not say that the product is satisfactory. R3 supports a favorable reading about sturdiness, but it does not establish an enthusiastic overall recommendation. For R4, the whole sentence supplies the context for the apparent praise. A bare positive-word count would miss the complaint.
Inspect disagreement instead of averaging it away
Suppose one proposed annotation marks R3 neutral and another marks it positive. Return to the definition: does your scheme treat a negated unfavorable property as favorable, or does it require an explicit positive adjective? Choose and document the rule, then revisit similar records. Do not average two label names into a confidence score.
If a comment says only “Great” with no context, uncertain may be more defensible than a confident sarcasm judgment. Keep the original text and the reviewer's reason. A model's numerical confidence, if requested, is not a measured probability that the annotation is correct.
Sentiment is also different from topic. “Delivery” tells you what the review discusses; “negative” describes the evaluation expressed about it. The survey-analysis guide covers grouping themes and counting respondents. Use that process when your question is what people mention rather than how they evaluate it.
Copy an annotation prompt
Propose sentiment annotations for the permitted reviews below.
Review IDs and unchanged text: [PASTE]
Aspects to label and what each includes: [ENTER]
Our definitions of positive, negative, mixed, neutral, unmentioned,
and uncertain: [PASTE]
Return one record per review and aspect. Include the proposed label,
an exact supporting phrase, and a short reason. Use the whole sentence
when negation or sarcasm changes a phrase's meaning. Do not infer a
writer's identity, motives, purchase history, or unstated opinion.
Keep unmentioned separate from neutral. Flag ambiguous cases for
my decision instead of forcing a positive or negative label.
Do not invent quotations, scores, or confidence probabilities.
Wait for my review before counting or summarizing the annotations.
Try a small sample containing the cases you expect to encounter. Inspect every proposed label in that sample, including apparently easy cases. Once you revise a definition, recheck the affected records before applying it to more text.
Count the reviewed labels with the coverage visible
For the delivery aspect in this fixture, the four reviewed records contain two negative annotations, one neutral and one unmentioned. Negative delivery sentiment is two of three reviews that mention delivery, or two of all four supplied reviews. Those are different denominators; state which you report.
Neither fraction establishes the opinion of every customer. The four examples were invented to demonstrate language cases, and a real collection may also have selection and duplication problems. Keep the annotation table, its rule version, and unresolved records beside any summary. Resolve the uncertain cases first, then publish only the aggregate your reviewed data supports.