ALSA: pcm: fix divide error in snd_pcm_lib_ioctl

Zubin Mithra · tiwai · commit f3eef46f0518 · 2021-08-27T22:34:12.000+02:00
Syzkaller reported a divide error in snd_pcm_lib_ioctl. fifo_size is of type snd_pcm_uframes_t(unsigned long). If frame_size is 0x100000000, the error occurs. Fixes: a9960e6 ("ALSA: pcm: fix fifo_size frame calculation") Signed-off-by: Zubin Mithra <zsm@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Cc: <stable@vger.kernel.org> Link: https://lore.kernel.org/r/20210827153735.789452-1-zsm@chromium.org Signed-off-by: Takashi Iwai <tiwai@suse.de>
diff --git a/sound/core/pcm_lib.c b/sound/core/pcm_lib.c
@@ -1746,7 +1746,7 @@ static int snd_pcm_lib_ioctl_fifo_size(struct snd_pcm_substream *substream,
 		channels = params_channels(params);
 		frame_size = snd_pcm_format_size(format, channels);
 		if (frame_size > 0)
-			params->fifo_size /= (unsigned)frame_size;
+			params->fifo_size /= frame_size;
 	}
 	return 0;
 }

Original file line number	Diff line number	Diff line change
`@@ -1746,7 +1746,7 @@ static int snd_pcm_lib_ioctl_fifo_size(struct snd_pcm_substream *substream,`
`1746`	`1746`	`channels = params_channels(params);`
`1747`	`1747`	`frame_size = snd_pcm_format_size(format, channels);`
`1748`	`1748`	`if (frame_size > 0)`
`1749`		`- params->fifo_size /= (unsigned)frame_size;`
	`1749`	`+ params->fifo_size /= frame_size;`
`1750`	`1750`	`}`
`1751`	`1751`	`return 0;`
`1752`	`1752`	`}`