Results collection and processing

Once the survey deadline has passed, the next step is to collect all responses, process them and generate visualizations. This guide covers the full workflow from closing the survey forms to downloading responses, building an aggregated dataset and producing visualizations.

1. Stop accepting responses

Before downloading any data, all forms must be closed to prevent new submissions after the deadline and ensure consistency of the dataset.

Run:

uv run scripts/stop_accepting_responses.py \
  --forms_json data/forms.json \
  --secrets_file credentials.json \
  --token_file token.json

Arguments

• --forms_json
  Path to the JSON file containing metadata about all generated forms.

• --secrets_file
  Path to Google API OAuth credentials (credentials.json).

• --token_file
  Path where the OAuth token is stored and reused.

2. Download responses

After forms are closed, responses can be downloaded and converted into a unified Pandas DataFrame and exported to a Parquet file.

Run:

uv run scripts/gather_responses.py \
  --teacher_data FI-81.json ... FF-82.json \
  --forms_json data/forms.json \
  --secrets_file credentials.json \
  --token_file token.json \
  --out_path results_2526_2.parquet

Arguments

• --teacher_data
  Paths to JSON files containing teacher metadata collected during the Information collection phase.

• --forms_json
  Path to the JSON file with generated forms metadata.

• --secrets_file
  Path to Google API OAuth credentials (credentials.json).

• --token_file
  Path where the Google API OAuth token is stored and reused.

• --out_path
  Output path for the resulting Parquet dataset containing all responses.

Each question response is parsed using a predefined mapping that converts raw Google Forms output into a structured data format.

If new questions are added or the current question set is modified, the corresponding parser mapping must be updated to ensure correct field extraction and prevent missing or misaligned data.

Recommendation: It is highly recommended to run gather_responses.py periodically during the survey period to create backups of all responses collected so far. This provides a safeguard against unexpected issues with the Google account or Google Forms that could otherwise result in data loss.

3. Aggregate responses

The dataset produced by the previous step contains one row per submitted form. While this representation preserves all individual responses, it is not suitable for generating visualizations or summarizing survey results.

The aggregation step combines responses for each teacher into a single record. Depending on the question type, different aggregation strategies are applied:

• Rating questions are aggregated by computing the mean score.
• Yes/No questions are aggregated as the percentage of positive ("Yes") responses.
• Open-ended questions are aggregated into lists of responses.
• Paired questions (e.g. "Які недоліки є у викладанні?" and "Які шляхи їх вирішення ви бачите?") are aggregated into lists of (problem, solution) tuples to preserve the correspondence between answers.

To perform the aggregation, run:

uv run scripts/aggregate_responses.py \
  --raw_df_path results_2526_2.parquet \
  --out_path aggregated_results.parquet

Arguments

• --raw_df_path
  Path to the Parquet file produced by gather_responses.py.

• --out_path
  Path where the aggregated dataset will be saved.

During aggregation, the script also:

• filters out empty open-ended responses;
• replaces swear words with asterisks.

Note: If the survey question set changes, the mapping question to aggregator in the script must be updated accordingly.

4. Filter teachers by the number of responses

Before publishing the results, it is necessary to exclude teachers with an insufficient number of responses. This helps ensure that the published results are reasonably representative. For example, it is generally not meaningful to publish results for a teacher who taught hundreds of students but received responses from only a couple of them.

The current filtering logic combines both relative and absolute thresholds. A teacher is included if they satisfy either of the following conditions:

• the number of responses is at least 15% of all students taught and no fewer than 5 responses were received;
• or at least 10 responses were received, regardless of the response rate (primarily intended for teachers of large streams).

To apply this filter, run:

uv run scripts/num_responses_filter.py \
  --teacher_data FI-81.json ... FF-82.json \
  --df_path aggregated_results.parquet \
  --out_path filtered_agg_results.parquet

Arguments

• --teacher_data
  Paths to JSON files containing teacher information.

• --df_path
  Path to the aggregated dataset produced by aggregate_responses.py.

• --out_path
  Path where the filtered dataset will be saved.

Note: The filtering thresholds (minimum response percentage, minimum number of responses, and the absolute response threshold) are defined in the script and can be adjusted if needed.

5. Generate visualizations

The final step is to generate visualization cards for all teachers whose results passed the filtering stage.

Each visualization includes:

• the teacher's photo and full name;
• a radar chart summarizing the rating questions;
• bar charts for the questions:
  • "Наскільки ви в загальному задоволені викладанням дисципліни цим викладачем?";
  • "Як ви оціните власні знання з дисципліни?";
• the percentage of positive responses to "Чи хочете ви, щоб викладач продовжував викладати?", highlighted with color depending on the value;
• the semester in which the survey was conducted;
• the number of responses received together with the total number of students taught by the teacher.

To generate the visualizations, run:

uv run scripts/generate_vizualizations.py \
  --teacher_data FI-81.json ... FF-82.json \
  --aggr_df_path filtered_agg_results.parquet \
  --photo_dir <photo_dir> \
  --save_dir <save_dir>

Arguments

• --teacher_data
  Paths to JSON files containing teacher information.

• --aggr_df_path
  Path to the filtered aggregated dataset.

• --photo_dir
  Directory containing teacher photos (from step 2 of Information collection)

• --save_dir
  Directory where generated visualizations will be saved.

Note: The appearance of the visualizations (colors, fonts, displayed questions, layout, etc.) can be customized by modifying the corresponding script.

Optional: Monitor response activity during the survey

While the survey is running, it is useful to monitor response activity in order to identify teachers who have not yet received enough responses to pass the publication threshold. This makes it possible to remind students to complete questionnaires for those teachers before the survey closes.

The same helper bot used for form distribution can also be used for response tracking.

To start the bot, run:

uv run src/bot/helper_bot.py \
  --teacher_data <all_teacher_data_files> \
  --forms_json <forms_data_from_prev_script> \
  --secrets_file credentials.json \
  --token_file token.json \
  --token <TG_TOKEN>

Note: Instructions for obtaining a Telegram bot token are available here:
https://github.com/ShkalikovOleh/ipt_survey/wiki/Results-posting#prerequisites

Arguments

• --teacher_data
  Paths to JSON files containing teacher information.

• --forms_json
  Path to the JSON file with generated forms metadata.

• --secrets_file
  Path to Google API OAuth credentials (credentials.json).

• --token_file
  Path where the OAuth token is stored and reused.

• --token
  Telegram bot token.

Bot commands

The bot provides two families of commands:

• \s... — show the current number of responses for all teachers in the selected audience.
• \n... — show only teachers that currently do not satisfy the minimum response threshold.

The audience is selected by the command suffix:

Command	Audience
\sgroup <GROUP> / \ngroup <GROUP>	Study group
\sstream <STREAM> / \nstream <STREAM>	Stream
\sspec <SPECIALTY> / \nspec <SPECIALTY>	Specialty
\sall / \nall	Entire faculty

The amount of detail available for response tracking depends on both the form granularity and whether the optional statistics question (--stats_granularity) is enabled.

If forms are generated at the group level, the bot can determine the number of responses for each group directly from the submitted form. Likewise, stream and specialty granularities allow tracking responses at their respective levels without any additional information.

When forms are generated at a coarser granularity (e.g. faculty), this information is not available from the form itself. Since the survey is anonymous, responses cannot be linked to a specific student or group after submission. In this case, tracking response activity for groups, streams, or specialties is only possible if the optional statistics question was included in the questionnaire. Responses where this question was not answered are grouped under the Anonymous category.