At The Nature Conservancy I served on the executive team overseeing the organization’s AI deployment, and I used AI to analyze employee comments, engagement trends and workforce data. It saved a great deal of time. It also taught me that the questions you settle before you start matter more than the tool you pick.
Be clear about what employees were promised
Read the survey’s confidentiality statement before anything else. If employees were told only a small team would read their comments, sending those comments to an outside AI service may break that promise even if it is technically allowed. Use tools your organization has approved for confidential data, and if the promise needs to change, change it before the next survey, not after.
Remove what does not need to be there
Names, email addresses and very specific details can identify people, especially in small teams. Strip them out first. The analysis rarely needs them.
Ask a specific question
“Summarize these comments” produces a bland summary. Better questions get better answers: What are the most common concerns about managers? How do comments from employees in their first two years differ from everyone else’s? Which themes appear in comments that also mention leaving?
Keep a human in the loop
AI is good at grouping themes and bad at knowing what matters to your organization. Have someone who knows the organization read a sample of the original comments behind every major theme. If the examples do not support the theme, the theme is wrong.
Watch for the loudest voice
A theme can look large because a few people wrote at length. Check how many different people raised it, not just how many words were written about it.
Write down how you did it
Record which tool you used, what you removed, what you asked and who reviewed the results. When a leader asks “how do we know this?”, you will have an answer. When you run the survey again, you will be able to compare like with like.
Use the time you saved
The real benefit of AI here is not the summary. It is the weeks it gives back. Spend them on the part no tool can do: talking with employees about what you found and deciding, with leaders, what will change.