This project demonstrates how to display audio files with subtitles using IIIF viewers. The example used is 「日本のアクセントと言葉調子（下）」, available from the National Diet Library's Historical Recordings Collection. Transcriptions are generated using OpenAI's Speech to Text service. Please note that the transcription results may contain errors.
Follow the instructions in this article to obtain mp4 files:
Transcribe audio using OpenAI's API:
from openai import OpenAI
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
audio_file = open(output_mp4_path, "rb")
transcript = client.audio.transcriptions.create(
model="whisper-1",
file=audio_file,
response_format="vtt")
with open(output_vtt_path, "w", encoding="utf-8") as file:
file.write(transcript)Example code to create a manifest file:
from iiif_prezi3 import Manifest, AnnotationPage, Annotation, ResourceItem, config
from moviepy.editor import VideoFileClip
def get_video_duration(filename):
with VideoFileClip(filename) as video:
return video.duration
config.configs['helpers.auto_fields.AutoLang'].auto_lang = "ja"
duration = get_video_duration(mp4_path)
manifest = Manifest(id=f"{prefix}/manifest.json", label=label)
canvas = manifest.make_canvas(id=f"{prefix}/canvas", duration=duration)
anno_body = ResourceItem(id=mp4_url, type="Sound", format="audio/mp4", duration=duration)
anno_page = AnnotationPage(id=f"{prefix}/canvas/page")
anno = Annotation(id=f"{prefix}/canvas/page/annotation", motivation="painting", body=anno_body, target=canvas.id)
anno_page.add_item(anno)
canvas.add_item(anno_page)
# Adding VTT URL
vtt_body = ResourceItem(id=vtt_url, type="Text", format="text/vtt")
vtt_anno = Annotation(
id=f"{prefix}/canvas/annotation/webvtt",
motivation="supplementing",
body=vtt_body,
target=canvas.id,
label="WebVTT Transcript (machine-generated)"
)
vtt_anno_page = AnnotationPage(id=f"{prefix}/canvas/page/2")
vtt_anno_page.add_item(vtt_anno)
canvas.annotations = [vtt_anno_page]
with open(output_path, "w") as f:
f.write(manifest.json(indent=2))This project uses the iiif-prezi3 library for manifest creation. For more detailed information, please refer to this article:
This README aims to assist those interested in applying IIIF to enhance their audio and video projects. Should you have any further inquiries or need more details, feel free to reach out.