KeyError: 'opensmile'

[jiminbot20]

KeyError: 'opensmile'

the error occurs with simple run sh.

Traceback (most recent call last):
File "/opt/conda/bin/paiprun", line 8, in
sys.exit(main())
File "/opt/conda/lib/python3.8/site-packages/paips/run_paips.py", line 50, in main
main_task.run()
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 459, in run
out_dict = self.__serial_run(run_async=run_async)
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 346, in __serial_run
outs = self.process()
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 654, in process
out_dict = task.run()
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 459, in run
out_dict = self.__serial_run(run_async=run_async)
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 346, in __serial_run
outs = self.process()
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 654, in process
out_dict = task.run()
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 471, in run
out_dict = self.__serial_map(iteration=iteration,run_async=run_async)
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 395, in __serial_map
outs.append(self.__serial_run(run_async=run_async))
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 346, in __serial_run
outs = self.process()
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 654, in process
out_dict = task.run()
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 459, in run
out_dict = self.__serial_run(run_async=run_async)
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 346, in __serial_run
outs = self.process()
File "/home/emotion/ser-with-w2v2/tasks/normalize.py", line 75, in process
grouped_data = data.loc[data[normalization_by] == g][col]
File "/opt/conda/lib/python3.8/site-packages/pandas/core/frame.py", line 3455, in getitem
indexer = self.columns.get_loc(key)
File "/opt/conda/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3363, in get_loc
raise KeyError(key) from err
KeyError: 'opensmile'

So I start debugging and tried to print every necessary variable.

ser-with-w2v2/tasks/normalize.py

Line 75 in f3983fa

grouped_data = data.loc[data[normalization_by] == g][col]

this line has 4 necessary variables data, normalized_by, g, col

data is dataframe of Session1 IEMOCAP data.

below is sample of data

subject	session	start	end	duration	emotion	valence_raw	activation_raw	dominance_raw	emotion_pair	valence	arousal	annotations_selfreport	annotations	annotations_proportion	sad_times	sad_dur	words_data	syl_avg_dur	syl_avg_rate	syl_times	annotations_entropy	annotations_entropy_self_report	annotations_full_agreement	sentences	classID
Ses01F	1	0	1.9455625	1.9455625	neutral	2.5	2.5	2.5	-,-	,	-	['neutral']	[['neutral'], ['neutral'], ['neutral']]	{'neutral': 1.0, 'anger': 0.0, 'fear': 0.0, 'excited': 0.0, 'sadness': 0.0, 'frustration': 0.0, 'surprise': 0.0, 'other': 0.0, 'happiness': 0.0, 'disgusted': 0.0}	[(0.45, 0.79), (0.8, 1.05)]	59	[[0.45, 0.79, 'EXCUSE'], [0.8, 1.05, 'ME']]	0.1933333333	5.172413793	[(0.45, 0.5), (0.51, 0.79), (0.8, 1.05)]	0	0	TRUE	EXCUSE ME	2
Ses01F	1	0	1.3824375	1.3824375	neutral	2.5	2.5	2.5	-,-	,	-	['neutral', 'anger']	[['neutral'], ['neutral'], ['neutral']]	{'neutral': 1.0, 'anger': 0.0, 'fear': 0.0, 'excited': 0.0, 'sadness': 0.0, 'frustration': 0.0, 'surprise': 0.0, 'other': 0.0, 'happiness': 0.0, 'disgusted': 0.0}	[(0.53, 0.9)]	37	[[0.53, 0.9, 'YEAH']]	0.37	2.702702703	[(0.53, 0.9)]	0	0	TRUE	YEAH	2
Ses01F	1	0	3.13025	3.13025	neutral	2.5	2.5	2.5	-,-	,	-	['neutral', 'anger']	[['neutral'], ['surprise'], ['neutral']]	{'neutral': 0.6666666666666666, 'anger': 0.0, 'fear': 0.0, 'excited': 0.0, 'sadness': 0.0, 'frustration': 0.0, 'surprise': 0.3333333333333333, 'other': 0.0, 'happiness': 0.0, 'disgusted': 0.0}	[(2.1, 2.15), (2.16, 2.28), (2.29, 2.32), (2.33, 2.82)]	69	[[2.1, 2.15, 'IS'], [2.16, 2.28, 'THERE'], [2.29, 2.32, 'A'], [2.33, 2.82, 'PROBLEM']]	0.136	7.352941176	[(2.1, 2.15), (2.16, 2.28), (2.29, 2.32), (2.33, 2.53), (2.54, 2.82)]	0.6365141683	0.6365141683	FALSE	IS THERE A PROBLEM	2

normalized_byis 'subject'(str).
g is indexed of groups which is ['Ses01F' 'Ses01M'].

data[normalization_by] == g

so this line means selecting data if the 'subject' column identificate with g which can be 'Ses01F' or 'Ses01M'.

Here comes the problem. I look into col and column

column is a list; ['opensmile']. So col is str; 'opensmile'

this doesn't make sense.

Dataframe data doesn't have any of 'opensmile'.

[jiminbot20]

similar problem happens, when running from this line below

ser-with-w2v2/run_paper_experiments.sh

Line 23 in c9be8a9

#-----------------------------------------Wav2Vec2-PT-----------------------------------------------

Traceback (most recent call last):
File "/opt/conda/bin/paiprun", line 8, in
sys.exit(main())
File "/opt/conda/lib/python3.8/site-packages/paips/run_paips.py", line 50, in main
main_task.run()
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 459, in run
out_dict = self.__serial_run(run_async=run_async)
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 346, in __serial_run
outs = self.process()
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 654, in process
out_dict = task.run()
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 459, in run
out_dict = self.__serial_run(run_async=run_async)
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 346, in __serial_run
outs = self.process()
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 654, in process
out_dict = task.run()
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 471, in run
out_dict = self.__serial_map(iteration=iteration,run_async=run_async)
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 395, in __serial_map
outs.append(self.__serial_run(run_async=run_async))
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 346, in __serial_run
outs = self.process()
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 654, in process
out_dict = task.run()
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 459, in run
out_dict = self.__serial_run(run_async=run_async)
File "/opt/conda/lib/python3.8/site-packages/paips/core.py", line 346, in __serial_run
outs = self.process()
File "/home/emotion/ser-with-w2v2/tasks/normalize.py", line 100, in process
grouped_data = data.loc[data[normalization_by] == g][col]
File "/opt/conda/lib/python3.8/site-packages/pandas/core/frame.py", line 3455, in getitem
indexer = self.columns.get_loc(key)
File "/opt/conda/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3363, in get_loc
raise KeyError(key) from err
KeyError: 'wav2vec2'

[jiminbot20]

I cannot solve this problem. Could you please help?

[jiminbot20]

It was my mistake that I erased
data[output_column] = data['filename'].progress_apply(extract_embedding)

becaues this line occurs an error
AttributeError: 'Series' object has no attribute 'progress_apply'

It solved with undergrading tqdm version into 1.2.5.