The export is a multi-part, tab-delimited, compressed file set with a name prefixed with observations, e.g., observations.20221221_1_7_3.csv.gz. A sentinel file, _SUCCESS, should be found prior to ingestion by the media partner as it indicates that the entire set is complete.
The full export, as well as the daily incremental file, contains the following.
- EXT_ID: This is the external ID or the media partner ID that we are receiving through IDSync.
- HASHED_CS_ID: The CivicScience User ID or alias, which is hashed and unique to each media partner account
- TARGET_ID: The identifier for this instance of the poll unit, which can be mapped to the target look-up table
- QUESTION_ID: Each question in our dataset has a unique identifier. This ID can be mapped to the question text in the key file. Note that in rare circumstances, Q_TEXT can be changed. Any mapping should be to the QUESTION_ID and not the text.
- ANSWER_ID: Each answer in our dataset has a unique identifier. This ID can be mapped to the Answer ID in the key file. The answer text can change (albeit rarely), so the ANSWER_ID should be used for mapping. Additionally, a value of 0 for ANSWER_ID indicates that the respondent has rescinded the answer. Thus, for a given alias, questions should be windowed for time, and the most recent should be selected to have the most recent attribute.
- ISO_REGION: State abbreviation
- METRO_ID: Metro region ID
- EPOCH_TS: Unix timestamp
Mapping Files
There are also three mapping files to help translate the question, answer, and target IDs.
Question
- ID: The question identifier. This file will contain all eligible questions within the account’s permission.
- TEXT: The question text.
Answer
- ID: The answer identifier. Answers should be associated with Question IDs. A zero value indicates that a user has rescinded the answer.
- TEXT: The answer text.
Target
- ID: The poll unit instance (target) identifier. A zero value indicates that the target in which the question was answered is from an account outside of the media partner’s network. Media partners have access to all questions and answers from respondents who have answered at least one question on targets within their account, regardless of the source of that response.
- TEXT: The name of the poll unit instance (target).