Audio detection question: how to trigger on any noise?

[fahnoe]

Frigate 0.16.1, rpi5, hailo8

I've been using ZoneMinder for years and have recently been switching over to Frigate. One of my cameras is configured to contact ZoneMinder's zmtrigger when the camera "hears" sounds over a certain volume threshold. So, it provides low-tech alerting on noises in an otherwise quiet environment.

In Frigate I have the following camera config for the above camera (an Axis M1054)

  loft:
    enabled: true
    ffmpeg:
      inputs:
        - path: 
            rtsp://user:password@axis-m1054:554/axis-media/media.amp?fps=5
          roles:
            - detect
            - audio
    audio:
      enabled: true
      min_volume: 300
    motion:
      mask: 0,0,1,0,1,0.300,0.225,0.300,0,0.585
    objects:
      track:
        - person
      filters:
        person:
          min_area: 15000

ffmpeg is seeing the two streams from the camera so a believe the audio detector should be getting data:

[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"codec_long_name":"H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10","width":1280,"height":800,"display_aspect_ratio":"8:5","avg_frame_rate":"0/0"},{"codec_long_name":"AAC (Advanced Audio Coding)","avg_frame_rate":"0/0"}]}}]

I've been gradually lowering the min_volume to try to get it to trigger on the same events that ZoneMinder is currently picking up, but have not been successful yet. Is the reason that whatever sounds are being detected do not match the audio detector's default listen for bark, fire_alarm, scream, speech, and yell, or simply that the sounds are not yet above the min_volume threshold?

I'd be okay with the audio detector simply letting me know that it heard something, bonus points if it labeled the event with what it thought it heard. I don't think I want to explicitly list the contents of its audio-labelmap.txt though.

--Larry

[inkeep-ai-assistant[bot]]

The issue is likely that the detected sounds don't match Frigate's default audio detection labels. By default, Frigate only listens for specific audio types: bark, fire_alarm, scream, speech, and yell (1). If the sounds in your environment don't match these categories, no audio events will be triggered regardless of volume.

The included audio model has over 500 different types of audio that can be detected (1). To trigger on a broader range of sounds, you'll need to explicitly configure additional audio labels in your listen configuration.

Here's how to expand your audio detection:

audio:
  enabled: true
  min_volume: 300
  listen:
    - bark
    - fire_alarm
    - scream
    - speech
    - yell
    - door_slam
    - footsteps
    - glass_break
    # Add other relevant labels from the labelmap

(1)

You can find the complete list of available audio labels in the audio-labelmap.txt file referenced in the documentation (1).

The audio detector uses volume levels similar to how motion detection works - Frigate will not run audio detection unless the audio volume is above the configured min_volume level (1). However, even if the volume threshold is met, the audio must still match one of the configured listen labels to create an event.

If you want broader detection without listing hundreds of labels, you might need to experiment with adding common environmental sounds from the labelmap that are likely to occur in your quiet environment.

📚 Sources:

1. Audio Detectors

---

Have another question? Just tag @inkeep.

[hawkeye217]

I'll add to the AI's correct response above - the next version of Frigate (0.17) will add a feature to the Debug view to make debugging audio levels easier.

[fahnoe]

the next version of Frigate (0.17) will add a feature to the Debug view to make debugging audio levels easier.

That will be a nice addition. I was thinking I'd need to check MQTT's audio/dBFS and audio/rms to get an idea of what the levels are.

Based on the AI answer I need to be explicit about what labels I want detected. I infer that there is no wildcard, regex, or ALL macro available. Does one need to be cautious about the number of labels added (I assume they consume resources/add overhead)?

--Larry

[NickM-27]

(I assume they consume resources/add overhead)?

No, they don't. The model detects everything in one go, and then the list is checked for what should be monitored. There is no performance difference there.

[hawkeye217]

That will be a nice addition. I was thinking I'd need to check MQTT's audio/dBFS and audio/rms to get an idea of what the levels are.

That's great. This is basically all the UI in the next version is doing as well.

Based on the AI answer I need to be explicit about what labels I want detected. I infer that there is no wildcard, regex, or ALL macro available. Does one need to be cautious about the number of labels added (I assume they consume resources/add overhead)?

Yes, just be more explicit with the labels you want detected. There is no wildcard. Additional labels will not consume any additional overhead - model inference is done regardless, and the labels returned from the inference are all you are selecting.