[WIP] Automatically import recipes from videos (like youtube ones) into mealie

[SR-G]

Hey.

I'm new to mealie + i have many recipes from youtube, tiktok, .. links that i was willing to import.

So i've started (and beware, it's a work-in-progress for now !) an external importer (AI-powered) allowing to do nearly 100% of the work regarding these imports from videos (with as few manual modifications as possible).

What is already covered :

• Creation of the recipe
• Adding all the ingredients (if needed, grouped)
• Adding all the steps (if needed / relevant, grouped by sections)
• Detecting the number of "servings"
• Adding a short summary for the recipe
• Adding automatically the main image at the top of the recipe
• Storing the video URL as "original URL"
• Storing a few extra metadata in "Extras" (youtube id, channel, import date ...)
• Adding automatically proper tags related to the imports (like "import", "youtube", "youtube/") for easy retrieval of these imported recipes
• Adding the video in a "remark" at the bottom, through an "iframe", for integration of the source video (see the example at the end)
• If the video has some chapters : add the chapters (with timestamped links) in the remark (just under the video) to easily be able to reach directly the source video at the proper chapter
• If the video has some chapters AND if the names of these chapters are somehow matching the "sections", then extract some screenshots from the video file and add them in the description of the relevant instructions/steps (note : here i take the image at "30% of the length of the chapter", without any guarantees that it will be the most adequate image, or even a proper one, of course)

Regarding my very first tests, it seems to be basically working (but more tests and some adjustements are still needed).

What really needs to be finished :

• Multi-language (for now i'm only testing with "french" recipes, but i also have "english" ones to convert, so i'll definitely make the small adjustements needed for EN video imports)
• TikTok imports (for now it's definitely only working with youtube ones, i'll have to do several adjustments for tiktok ones)
• Work on the ingredients proper integration : here i'm struggling because for now with Mealie 2.8.0, the ingredients available as reference inside mealie data are only in english, and not translated/translatable in other languages ... i'm not sure how to handle this (so for now it's only "pure text")
• Work on the cooking / preparation time : for now this is really not working well, i'm not able to have accurate numbers
• Package the tool in some way (regarding how to publish / use it) if there are some interest for this

Regarding "how to use it", for now it's only available on my side as an external tool, with a few configuration and prerequesites.

• Configuration :
  • Mealie URL (= where mealie is hosted) and mealie token (API token)
  • An AI key (i'm using GEMINI)
• Pre-requesites :
  • Have the "yt-dlp" package at OS level (to rip the videos from youtube or tiktok)
  • Have the "ffmpeg" package at OS level (to extract the "chapters thumbnails")
• Launch is for now like mealie-importer "https://www.youtube.com/watch?v=ABCDEFGH

That importer for the moment is just a single binary (will work on linux, mac, windows, as long as the pre-requesites are there) (being written in GOLANG).

At this time and with that set of features, the import is somehow fast (15-20 seconds of processing on top of the video download time).

The principle is basically just :

1. Extract the video with "yt-dlp" + extra informations (thumbnail, JSON metadata including chapters, etc., sub-titles)
2. Use AI from all this to extract the summary, ingredients, steps, ... (with proper queries and some adjustements)
3. Use the MEALIE API (REST endpoints) to create the recipe, attach the assets (images) to it, etc.

Maybe in the end it could be usable in another way, but for now it's like this.

One test recipe from my own mealie instance (the only one being "public") : so, everything inside that recipe has been automatically generated (no manual adjustements anywhere, except "public" state + "public" added in the title).
https://mealie.tensin.org/g/home/r/steak-hache-frites-2-0-public

A full screenshot for reference :

(and this is somehow related to discussions here : #4501 and here : #819)

[8AH]

Hi! Do you have any info/news on this project?
Looks quite interesting!

[SR-G]

Sadly there has been no traction/zero interest for this projet, so i thought at that time that as no-one seems interested (*), it would be a waste of time and energy to package/document it/explain it/etc.

(* : maybe one reason is that it's an "external" tool - to be launched manually outside mealie - whereas most people may expect an "embedded" tool (i.e., all the work being done in the mealie importer))

[andjules]

I just want to add that I'm grateful for your work and interested in any kind automatable workflow that gets video recipes into mealie

[andrew-manger]

Also grateful, this looks great, but yeah, it would need to be embedded into mealie to gain traction. Have you thought about putting together a PR to add this as an import option?

Also may the Gemini API key may deter users, are you just using their free key?

[SR-G]

It's a very different stack on my side (GOLANG + yt-dlp for the youtube downloader, as in the end, grabbing youtube videos is somehow a complex topic) compared to the MEALIE one (which is python based), so it would probably be difficult to integrate the two - but the tooling here is just using the MEALIE API in an external way, and, i'd say, it's also why an external API is there ... but i fully agree that a fully embedded solution would be easier / better for most people.

Regarding the GEMINI API, i was using their beta (free) key, which was working really fine for some time (regarding results and speed), but it seems things have changed (now it's very slow, maybe they have introduced some limitations, i'll have to re-investigate that part).

[Goeste]

looking exactly for something like this....

[michael-genson]

FYI this is now implemented in some capacity: #6764
Not everything mentioned here is supported just yet, but you can transcribe/import videos from most sites