Segment Compare takes any two segments you've created and compares their (potentially weighted) responses to a pre-defined list of several dozen key questions, organized by category. You can also add your own questions to the comparison. Statistically significant differences are then highlighted.
Segment compare is therefore a very powerful research tool, as it allow you to quickly see how a particular segment of respondents differ from another segment of respondents.
For each question in the Segment Compare list, difference of proportions tests are calculated for all answer choices. Results are then screened using the Benjamini-Hochberg false detection rate procedure. Lastly, the remaining statistically significant results are filtered to exceed a preset log odds ratio to ensure a minimum level of statistical meaningfulness.
Comparing segments is a highly computational intensive task. Therefore, it is a task that is performed asynchronously.
Setting up a comparison between two segments
To use segment compare, the first step is to select the first segment. Then, navigate to Details>Compare, which will lead you to the segment compare page.
The first segment will always be the starting segment, and will be in the left side of the comparison results. To select the second segment, use the dropdown on the right. You can either select “Others”, which means that you are comparing respondents on the segment with respondents that are not in the segment(*), or you can pick another segment to compare against from the dropdown.
(*) “Others” will include respondents that are known not to be in the segment, but not those that are not known. Suppose that you defined the segment Cat Lovers as respondents that have responded Cats to the “Are you a cat or a dog person?” question. In this case, “Others” will mean all respondents that have answered “Dogs” or “Neither” to the same question. Respondents that haven’t answered that question yet will not be included in the comparison.
The segment compare page will always show the results of the last comparison computed for the selected segments with the selected weighting scheme. It will display the message below that indicates as much, and will allow you to trigger a new calculation if you so wish.
If a comparison between the segments with the selected weighting scheme has not yet been computed, then the system will display the message below to indicate as much.
Once you have picked the segment to compare against, and optionally picked a weighting scheme (in the top right of the screen), clicking on the link will trigger the background process that will compute the comparison between the segments. Once the comparison is finished, you will receive an e-mail notifying you as much, with a link to go back to see the results. The background process can take seconds or minutes, depending on the complexity of the segments that you have selected.
Once you click on the link you received through e-mail, you will be brought back to the same page, where the comparison results will now be available.
Understanding the Comparison Results
Comparison results will be shown in a table that contains many categories, and each category is divided into questions. Lines in the table will represent each question, and there will be a column for each of the two segments.
If there are not any statistically significant differences between the segments with respect to that question, the system will display a message indicating as much. For instance, if the gender composition of both segments is the same, the system will display “No significant differences were found with respect to Gender.”
If there are statistically significant differences, the system will show them and highlight them. The percent of respondents in each line will be shown, with the lowest number grayed out, and the highest numbers in bold. For instance, Cat Lovers (weighted by U.S. Census) are 40% Male and Others are 52% Male.
As there are many comparison sections, InsightStore™ provides a way to hide sections that one might not be interested on. Simply click on the arrow on the right of a category, and all questions in that category will be collapsed. Clicking on the arrow again will show the category again.
Adding questions to segment compare
In addition to the pre-selected categories and questions, users have the ability to add their own questions to the comparison. In order to do so, users should navigate to the bottom of the comparison page and click on the “Add Another Question” link. This will open a question search page where the user will be able to filter and search for the questions to ask, and then make a selection.
Once a question is added, it will be queued to be used in the next comparison.