Question Insights and Natural Language Generation

Question Insights are differences in traits, attitudes, and behaviors associated with the answers to your question. Insights are ranked by strength, a measure which combines both effect size and confidence level of the association.

Question Insights are a quick way to discover insights about how a question was answered by different respondents. When Insights are calculated, InsightStoreTM automatically looks for statistically meaningful differences on how pre-selected segments of respondents answered the question, and presents these differences to the user. By having it be automatically calculated and summarized for the user, it makes it much easier and quicker to find relevant information about the question.

Insights calculation takes in consideration the question filters, answer groupings, and weighting scheme that are applied when the Insights are calculated.

Calculating Insights is a highly computational intensive task. Therefore, it is a task that is performed asynchronously.

Insights that the InsightStore™ discovers

The InsightStoreTM compares respondents of each answer in the question against the respondents of other answers in the question for a set of traits, attitudes and behaviors.

It is important to notice that even though the system mines for many insights, it will only report on those that were discovered – i.e., when the comparison yields a significant difference. If a question has few responses, or if the respondents of each options don’t respond differently to the traits, then fewer (or none) insights will be discovered.

For instance, if the question is “What is your favorite color?”, and answers are “Blue”, “Red” and “Yellow”, the system will look for how people that responded blue differ from people that responded not blue (i.e., people that responded red or yellow). The system will also do the same for people that responded red (against those that responded blue or yellow) or yellow (against those that responded blue or red).

The traits are divided into basic and topical. The basic traits that the system compare against are: age, gender, income, education, urbanicity, parental status, US geographic region, and time period. The topical traits are brands, current events, demographics & profile, entertainment, food & beverage, health & wellness, lifestyle, media consumption, personal finance, politics & ideology, shopping habits & behavior, and technology usage.

If Insights with the selected filters and weighting scheme have not yet been computed, then the system will display the message below to indicate as much.

Filters and Weighting scheme can be selected on the top right of the results page, by opening the "Filter or Weight this Data" tab.

Once you have optionally picked filters and a weighting scheme (in the top right of the screen), clicking on the link will trigger the background process that will compute the Insights for that question. Once the computation is finished, you will receive an e-mail notifying you as much, with a link to go back to see the results. The background process can take seconds or minutes, depending on the question that you have selected and the number of respondents.

One example of an insight would be to verify if the respondents that answered blue are different from the respondents that answered yellow or red, according to gender (one of the traits). So, if those that answered blue are 50% women and those that answered yellow and red are 25% women, the system would indicate that women are twice as likely to answer blue.

The system will also look for insights in the reverse direction, i.e., how the traits differ with regarding to the answers of the selected question.

Once you click on the link you received through e-mail, you will be brought back to the same page, where the comparison results will now be available.

Understanding the Insights Results

There are four different Display Modes for Insights. Two are Lists of Insights: Differences in Answers Likelihood and Differences in Trait Likelihood. And two are English Language Narratives produced using Natural Language Generation, a brief format and a long one.

Understanding the Insights Results in List Mode

When Insight results are shown in either of the list modes, the system display a list that contains all insights that were discovered. When many insights were discovered, pagination is automatically added to the list (see the links on the bottom).

For ease of navigation when in list mode, there is a filter on the top that allows for filtering only selected categories.

The list is ordered by strength of association, with highest strengths on the top. Here’s an example of the list:

Insight lists can be added to any Dashboard by clicking on the Add to Dashboard link.

Understanding the Insights Results in Narrative Mode

When brief narrative or full report are selected, the system will use natural language generation to produce an English language text describing the results and the insights that were discovered. The text is hyperlinked to each of the insights, so you can explore each of them in more details if you so choose.

This is extremely powerful, while extremely simple, as the analysis is performed for you. You can add the generated summary text to a dashboard (that can be exported to power point), or you can copy and paste the sections that are relevant to you to any text editor.

The brief summary includes a description of the question results and a few of the basic insights with the highest strengths.

The full report will include all insights discovered, and can become quite long when many insights are present.

Here’s an example of a brief summary:

Do you still need help?

We are always available to answer your questions. For the speediest service, please email support@civicscience.com.