Updated EA parse to remove button press during scan response #298
Updated EA parse to remove button press during scan response #298DESm1th merged 5 commits intoTIGRLab:masterfrom
Conversation
|
Hello @ThomasHMAC, Thank you for updating!
To test for issues locally, Comment last updated at 2021-02-22 14:44:01 UTC |
| for index, row in log_file.iterrows(): | ||
| if ("rating" in log_file["Code"][index]) and any( | ||
| resp in log_file["Code"][index - 2] for resp in scan_response | ||
| ): | ||
| indexes_to_drop.append(index) | ||
| indexes_to_drop.append(index - 2) |
There was a problem hiding this comment.
Can you add a comment describing why the current index and 2 indices beforehand need to be dropped? Mostly so we don't have a magic number in there that people in the future wouldn't understand!
| logger.warning( | ||
| "Removed registered rating occurred before or after actual rating" | ||
| ) |
There was a problem hiding this comment.
Maybe include the number of "chunks" being dropped here for debugging purposes (i.e if an abnormally large number of indices got dropped).
| logger.warning( | |
| "Removed registered rating occurred before or after actual rating" | |
| ) | |
| logger.warning( | |
| f"Removed {len(indexes_to_drop)/2} registered ratings that occurred before or after actual rating" | |
| ) |
| indexes_to_drop_1 = [] | ||
|
|
||
| for index, row in log_file_cleaned.iterrows(): | ||
| if ("rating" in log_file_cleaned["Code"][index]) and any( | ||
| resp in log_file_cleaned["Code"][index - 1] | ||
| for resp in scan_response | ||
| ): | ||
| indexes_to_drop_1.append(index) | ||
| indexes_to_drop_1.append(index - 1) |
There was a problem hiding this comment.
I think a "why" comment is needed here as well, might just be me, but I'm not really sure why we dropped i, i-2, and now i,i-1!
| final_log_file = final_log_file.reset_index(drop=True) | ||
| logger.warning("Removed rating registered followed scanner responses") |
There was a problem hiding this comment.
Same as above logging comment re: including the number of ratings dropped
| df["duration"] = df["movie_name"].apply( | ||
| lambda x: int(vid_info[x]["duration"]) * 10000 | ||
| if x in vid_info | ||
| else "n/a" |
There was a problem hiding this comment.
maybe it might be a good idea to use to use pd.NA here to avoid having a mixed data-type column
| ) | ||
|
|
||
| df["stim_file"] = df["movie_name"].apply( | ||
| lambda x: vid_info[x]["stim_file"] if x in vid_info else "n/a" |
There was a problem hiding this comment.
same as above pd.NA comment
| # Reads in and clean the log, skipping the first three preamble lines | ||
| try: | ||
| log = read_in_logfile(log_file) | ||
| log_cleaned = clean_logfile(log) |
There was a problem hiding this comment.
could this be outside the try/catch block? It looks like clean_logfile is mostly if not entirely safe
| try: | ||
| os.mkdir(dest_folder) | ||
| except OSError: | ||
| pass |
There was a problem hiding this comment.
| try: | |
| os.mkdir(dest_folder) | |
| except OSError: | |
| pass | |
| os.makedirs(dest_folder, exist_ok=True) |
Unless you want it to error out if intermediate directories don't exist!
5 participants
No description provided.