audio input not working in pyttitools-PYTTI.ipynb

[paulhaskett]

Hi when I try to use the audio input I get TypeError: 'NoneType' object is not subscriptable, I am using the pyttitools-PYTTI.ipynb notebook in colab. Thanks for any help

[dmarx]

Can you share the logging message that reports your settings? Thanks!

[paulhaskett]

Hi thanks for your quick reply, do you mean this?

SETTINGS:
{'defaults': ['self', {'conf': '_empty'}], 'scenes': ' | a colourful Mandelbrot set : audioreact | ', 'scene_prefix': '| faces:-1:-.95 | animals:-1:-.95 | text:-1:-.95 | people:-1:-.95 | watermark:-1:-.95 |', 'scene_suffix': '', 'direct_image_prompts': '', 'init_image': '/content/drive/MyDrive/fsod.jpg', 'direct_init_weight': '0', 'semantic_init_weight': '1', 'image_model': 'VQGAN', 'vqgan_model': 'sflckr', 'animation_mode': '3D', 'width': 854, 'height': 480, 'steps_per_scene': 45120, 'steps_per_frame': 20, 'interpolation_steps': 1, 'learning_rate': None, 'reset_lr_each_frame': True, 'seed': 5722604420188579041, 'cutouts': 4, 'cut_pow': 1, 'cutout_border': 0.2, 'border_mode': 'clamp', 'field_of_view': 60, 'near_plane': 0, 'far_plane': 200, 'input_audio': 'southseajourney.mp3', 'input_audio_offset': 0, 'input_audio_filters': [{'variable_name': 'audioreact', 'f_center': 1000, 'f_width': 1990, 'order': 5}], 'pre_animation_steps': 250, 'lock_camera': True, 'translate_x': '0', 'translate_y': '0', 'translate_z_3d': '.0', 'rotate_3d': '[1,0,0,0]', 'rotate_2d': '0', 'zoom_x_2d': '0', 'zoom_y_2d': '0', 'sampling_mode': 'bicubic', 'infill_mode': 'black', 'pixel_size': 1, 'smoothing_weight': 0.02, 'random_initial_palette': False, 'palette_size': 6, 'palettes': 9, 'gamma': 1, 'hdr_weight': 0.01, 'palette_normalization_weight': 0.2, 'show_palette': False, 'target_palette': '', 'lock_palette': False, 'frames_per_second': 12, 'direct_stabilization_weight': '1', 'semantic_stabilization_weight': '1', 'depth_stabilization_weight': '', 'edge_stabilization_weight': '1', 'flow_stabilization_weight': '', 'video_path': '', 'frame_stride': 0, 'reencode_each_frame': False, 'flow_long_term_samples': 1, 'ViTB32': True, 'ViTB16': False, 'ViTL14': False, 'ViTL14_336px': False, 'RN50': False, 'RN101': False, 'RN50x4': False, 'RN50x16': False, 'RN50x64': False, 'use_mmc': False, 'file_namespace': 'readtoflyversion19video', 'allow_overwrite': True, 'display_every': 20, 'clear_every': 20, 'display_scale': 1, 'save_every': 20, 'backups': 5, 'show_graphs': False, 'approximate_vram_usage': False, 'use_tensorboard': False, 'models_parent_dir': '.', 'gradient_accumulation_steps': 1, 'device': 0, 'resume': False, 'mmc_models': [], 'base_name': 'readtoflyversion19video'}

here is a screenshot of the error

I have tried to run it from a fresh notebook with the audio settings too, the MP3 file is in the pytti_tools folder on my gdrive. Thanks again for your help.

[dmarx]

can you click on the "8 frames" thing? It'll expand the error to reveal a more detailed traceback :)

[dmarx]

here's a potential fix you can try: on you prompt "a colourful Mandelbrot set", instead of using just audioreact for the weight, maybe try 0.001 + audioreact or something like that. Totally shooting from the hip here without that more detailed traceback though

[paulhaskett]

thanks

[paulhaskett]

I think I have it working now thanks, there was a parameter with .0 which I think caused some problems. I also put the prompt as you suggested in brackets like (0.001 + audioreact) and it is running ok now, thanks again.

[paulhaskett]

in case anyone else is having this trouble I think the cause was having pre_animation_step set different to input_audio_offset. Having them both set to 0 fixed the problem for me.

[dmarx]

nice, I'm gonna open an issue to document and fix that, thanks!