ChatGPT survey analysis: trace each theme back to a response

One comment says the workshop was useful but moved too quickly. A single positive or negative label loses half the answer. Ask ChatGPT to propose separate themes, retain the original response beside every proposed label, and count people mentioning each theme only after you review the coding.

ChatGPT can help organize supplied text and analyze files. The official OpenAI documentation describes those capabilities; available tools depend on the account and workspace. You can begin with a small permitted, de-identified text sample without connecting a survey account. The coding exercise below is an original fictional example.

Preserve what each person actually wrote

Keep a response ID and the unchanged answer. Remove identifying material before supplying it under your organization's policy. A replacement ID does not make a revealing comment anonymous: a person's project, dates, or unusual circumstances can still identify them.

This example asks, “What would you change about the practice session?” Six people submitted records; R6 left this question blank.

ID Original answer Proposed coding
R1 The example helped, but the practice moved too fast. Useful example; pace too fast
R2 More time to try the steps myself. Practice time
R3 I could not find the exercise file. Material access
R4 Nothing to change. No change requested
R5 More practice time, though the worked example was useful. Practice time; useful example
R6 [Blank answer] Missing response

R1 mentions speed, while R2 requests time for independent practice. You might eventually combine them under a broader timing theme. Do not let ChatGPT make that decision silently: the two findings could lead to different changes, slower explanation versus a longer exercise.

Agree on a small codebook before counting

Give each theme an inclusion rule and an exclusion. “Practice time” includes an explicit request for more time doing the task; it excludes a general comment that the explanation was rushed unless that rule is deliberately broadened. “Material access” covers finding or opening supplied materials, rather than the difficulty of the exercise itself.

Keep “No change requested,” “Missing response,” and “Unclear” separate. R4 answered the question. R6 did not. Neither should become evidence that the person found the session useful.

Review the first proposed labels against the original wording. If you change a definition, revisit every response it might affect. Saving the codebook version with the coded table makes a later comparison possible.

Count respondents, with the denominator visible

Under these proposed definitions, useful example appears in R1 and R5: two of five nonblank answers, or 40%. Practice time appears in R2 and R5: also two of five, or 40%. Pace too fast, material access, and no change requested each appear once: one of five, or 20%.

Those shares total 140% because one person may mention multiple themes. That is expected under this coding rule. They are shares of the five people who answered this question, not mutually exclusive shares of all six survey records. If you also report question completion, it is five of six records; the number originally invited is not supplied, so a survey response rate cannot be calculated.

Count each response ID once within a theme even if its comment repeats the idea. Keep the source IDs behind the summary so a colleague can inspect why a response was included. Small counts describe this sample; they do not establish what an entire workforce wants.

Copy a review-first analysis prompt

Help analyze these open-ended survey answers.
Question and decision the answers should inform: [PASTE]
Approved de-identified rows with stable response IDs: [PASTE]
Missing-answer markers and known questionnaire routing: [PASTE]
Existing theme definitions, if any: [PASTE OR NONE]

Propose a small codebook with inclusion/exclusion rules. Return each
unchanged answer beside proposed themes and the exact supporting phrase.
Allow multiple themes. Keep missing, unclear, and no-change responses
separate. Do not invent motives, sentiment, quotations, or respondents.
Flag uncertain coding for my decision. Wait for approval of definitions.

After approval, count distinct response IDs per theme and show those IDs.
State the denominator and exclusions. Explain that multi-theme shares
may exceed 100% in total. Preserve the originals and codebook version.
Return a reviewable table and a short finding limited to this sample.
Do not publish, contact respondents, or overwrite source records.

Turn a checked finding into a bounded next step

For this fixture, a reasonable proposal is to inspect the exercise-file instructions and discuss independent practice time. The comments do not prove that adding ten minutes will improve learning, and they do not tell you which file-access step failed. Keep those as follow-up questions.

When the coding is agreed, the survey results report guide can help present the approved findings. Retain the coded table privately in the authorized research location. A reader-facing summary should include only the counts and excerpts approved for that audience.

Keep reading