From 41ed0d9394061f5c15a8381b19d1d508bdca4229 Mon Sep 17 00:00:00 2001 From: John Smith Date: Sun, 31 Aug 2025 21:06:48 +0100 Subject: [PATCH 1/5] feat(posts): add ytx .NET global tool tutorial blog post MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add comprehensive tutorial about building ytx, a YouTube transcript extractor as a .NET Global Tool. Covers architecture, implementation, CI/CD, and NuGet publishing following established blog style with emojis and code examples. ๐Ÿค– Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude --- ...cript-extractor-as-a-dotnet-global-tool.md | 404 ++++++++++++++++++ 1 file changed, 404 insertions(+) create mode 100644 _posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md diff --git a/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md b/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md new file mode 100644 index 00000000..9eb44410 --- /dev/null +++ b/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md @@ -0,0 +1,404 @@ +--- +layout: post +title: Building ytx - A YouTube Transcript Extractor as a .NET Global Tool +description: How to build a .NET Global Tool that extracts YouTube video metadata and transcripts as structured JSON, from concept to NuGet publication +summary: A tutorial on creating ytx, a .NET global tool that extracts YouTube video titles, descriptions, and transcripts as JSON using YoutubeExplode and automated CI/CD. +tags: +- dotnet-global-tools +- youtube +- transcripts +- csharp +- dotnet +- json +- ci-cd +- nuget + +--- +**Overview** โ˜€ + +Sometimes you need to extract structured data from YouTube videos for analysis, documentation, or automation. While there are various web-based solutions, having a command-line tool that outputs clean JSON makes integration with scripts and pipelines much easier. + +I built `ytx` - a .NET Global Tool that extracts YouTube video metadata and transcripts as structured JSON. The tool takes a YouTube URL and returns the video title, description, and full transcript with timestamps in both raw text and markdown formats. + +**The Problem** ๐ŸŽฏ + +I wanted a simple way to: +- Extract YouTube video transcripts for analysis +- Get structured JSON output for easy parsing +- Handle different caption languages and auto-generated captions +- Create timestamped links back to specific moments in videos +- Package everything as a portable command-line tool + +**Project Architecture** ๐Ÿ—๏ธ + +The tool is built as a single-file .NET console application with a simple but effective architecture: + +```csharp +record Input(string url); + +class Output +{ + public string url { get; set; } = ""; + public string title { get; set; } = ""; + public string description { get; set; } = ""; + public string transcriptRaw { get; set; } = ""; + public string transcript { get; set; } = ""; +} +``` + +The data flow is straightforward: +1. Input validation (command-line args or JSON via stdin) +2. YouTube video data extraction via YoutubeExplode +3. Caption track discovery and selection +4. Transcript formatting (raw + markdown with timestamps) +5. JSON serialization to stdout + +**Getting Started** ๐Ÿš€ + +First, I created the project structure: + +```powershell +dotnet new console -n Ytx +cd Ytx +``` + +The key to making this a global tool is the `.csproj` configuration: + +```xml + + + Exe + net8.0;net9.0 + enable + enable + + + true + ytx + solrevdev.ytx + 1.0.2 + + + solrevdev + Extract YouTube title, description, and transcript (raw + Markdown) as JSON. + YouTube;transcript;captions;cli;dotnet-tool;json + https://github.com/solrevdev/solrevdev.ytx + MIT + README.md + ../../nupkg + + + + + + + + + + +``` + +The crucial elements are: +- `PackAsTool>true` - Makes this a global tool +- `ToolCommandName` - The command users will type +- `TargetFrameworks` - Multi-targeting for compatibility +- `PackageOutputPath` - Consistent build artifacts location + +**Core Implementation** โš™๏ธ + +The main challenge was handling YouTube's various caption formats and languages. Here's the core logic: + +```csharp +static async Task Main(string[] args) +{ + try + { + string? url = null; + + // Handle both command-line args and JSON stdin + if (args.Length == 1 && !string.IsNullOrWhiteSpace(args[0])) + { + url = args[0]; + } + else + { + string stdin = Console.IsInputRedirected ? await Console.In.ReadToEndAsync() : ""; + if (!string.IsNullOrWhiteSpace(stdin)) + { + var input = JsonSerializer.Deserialize(stdin.Trim(), + new JsonSerializerOptions { PropertyNameCaseInsensitive = true }); + url = input?.url; + } + } + + if (string.IsNullOrWhiteSpace(url)) + { + Console.Error.WriteLine("Usage: ytx \n or: echo '{\"url\":\"https://...\"}' | ytx"); + return 2; + } + + var client = new YoutubeClient(); + var videoId = VideoId.TryParse(url) ?? throw new ArgumentException("Invalid YouTube URL/ID."); + var video = await client.Videos.GetAsync(videoId); + + // Extract basic metadata + var title = video.Title ?? ""; + var description = video.Description ?? ""; + + // The transcript extraction logic + var transcriptResult = await ExtractTranscript(client, video); + + var output = new Output + { + url = url, + title = title, + description = description, + transcriptRaw = transcriptResult.raw, + transcript = transcriptResult.markdown + }; + + // Output clean JSON with proper Unicode handling + var json = JsonSerializer.Serialize(output, new JsonSerializerOptions + { + WriteIndented = true, + Encoder = System.Text.Encodings.Web.JavaScriptEncoder.UnsafeRelaxedJsonEscaping + }); + + Console.OutputEncoding = Encoding.UTF8; + Console.WriteLine(json); + return 0; + } + catch (Exception ex) + { + Console.Error.WriteLine($"Error: {ex.Message}"); + return 1; + } +} +``` + +**Smart Caption Detection** ๐Ÿง  + +One of the trickiest parts was handling YouTube's various caption formats. The tool needs to: +- Prefer English captions when available +- Fall back to any available language +- Handle both manual and auto-generated captions +- Gracefully handle videos without captions + +```csharp +var manifest = await client.Videos.ClosedCaptions.GetManifestAsync(video.Id); +var track = manifest.Tracks + .OrderByDescending(t => t.Language.Name.Contains("English", StringComparison.OrdinalIgnoreCase)) + .ThenByDescending(t => t.IsAutoGenerated) + .FirstOrDefault(); + +if (track != null) +{ + var captions = await client.Videos.ClosedCaptions.GetAsync(track); + + var rawSb = new StringBuilder(); + var mdSb = new StringBuilder(); + + foreach (var c in captions.Captions) + { + var text = NormalizeCaption(c.Text); + if (string.IsNullOrWhiteSpace(text)) continue; + + // Build raw transcript + if (rawSb.Length > 0) rawSb.Append(' '); + rawSb.Append(text); + + // Build markdown with timestamped links + var ts = ToHhMmSs(c.Offset); + var link = $"https://www.youtube.com/watch?v={video.Id}&t={(int)c.Offset.TotalSeconds}s"; + mdSb.AppendLine($"- [{ts}]({link}) {text}"); + } + + transcriptRaw = rawSb.ToString().Trim(); + transcriptMd = mdSb.ToString().TrimEnd(); +} +``` + +**Utility Functions** ๐Ÿ”ง + +The tool includes helper functions for formatting and text normalization: + +```csharp +static string ToHhMmSs(TimeSpan ts) +{ + int h = (int)ts.TotalHours; + int m = ts.Minutes; + int s = ts.Seconds; + return h > 0 ? $"{h:00}:{m:00}:{s:00}" : $"{m:00}:{s:00}"; +} + +static string NormalizeCaption(string text) +{ + if (string.IsNullOrWhiteSpace(text)) return ""; + text = Regex.Replace(text, @"\s+", " ").Trim(); + text = text.Replace(" ", " "); + return text; +} +``` + +**Local Development and Testing** ๐Ÿงช + +During development, I used this workflow: + +```powershell +# Restore dependencies +dotnet restore src/Ytx + +# Build the project +dotnet build src/Ytx -c Release + +# Test with a YouTube URL +dotnet run --project src/Ytx --framework net8.0 "https://www.youtube.com/watch?v=dQw4w9WgXcQ" + +# Package for local installation testing +dotnet pack src/Ytx -c Release +dotnet tool install -g solrevdev.ytx --add-source ./nupkg +``` + +This allowed me to test the tool end-to-end before publishing to NuGet. + +**Output Format** ๐Ÿ“„ + +The tool produces clean, structured JSON: + +```json +{ + "url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ", + "title": "Rick Astley - Never Gonna Give You Up (Official Music Video)", + "description": "The official video for \"Never Gonna Give You Up\" by Rick Astley...", + "transcriptRaw": "We're no strangers to love You know the rules and so do I...", + "transcript": "- [00:17](https://www.youtube.com/watch?v=dQw4w9WgXcQ&t=17s) We're no strangers to love\n- [00:20](https://www.youtube.com/watch?v=dQw4w9WgXcQ&t=20s) You know the rules and so do I..." +} +``` + +The markdown transcript format makes it easy to create documentation with clickable timestamps that jump directly to specific moments in the video. + +**Automated CI/CD Pipeline** ๐Ÿค– + +To streamline releases, I set up GitHub Actions to automatically: +- Build and test the project +- Increment version numbers +- Publish to NuGet +- Create GitHub releases + +The workflow file (`.github/workflows/publish.yml`) handles version bumping: + +```yaml +name: Publish NuGet (ytx) + +on: + push: + branches: [ master ] + workflow_dispatch: + inputs: + version_bump: + description: 'Version bump type' + required: true + default: 'patch' + type: choice + options: + - patch + - minor + - major + +jobs: + publish: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Setup .NET + uses: actions/setup-dotnet@v4 + with: + dotnet-version: | + 8.0.x + 9.0.x + + - name: Bump version + run: | + # Script to increment version in .csproj + VERSION_TYPE="${{ github.event.inputs.version_bump || 'patch' }}" + ./scripts/bump-version.sh "$VERSION_TYPE" + + - name: Build and Pack + run: | + dotnet restore src/Ytx + dotnet build src/Ytx -c Release + dotnet pack src/Ytx -c Release + + - name: Publish to NuGet + run: | + dotnet nuget push nupkg/*.nupkg \ + --api-key ${{ secrets.NUGET_API_KEY }} \ + --source https://api.nuget.org/v3/index.json +``` + +**Installation and Usage** ๐Ÿ“ฆ + +Once published to NuGet, users can install the tool globally: + +```powershell +# Install the tool +dotnet tool install -g solrevdev.ytx + +# Basic usage +ytx "https://www.youtube.com/watch?v=dQw4w9WgXcQ" + +# Via JSON input for scripting +echo '{"url":"https://www.youtube.com/watch?v=dQw4w9WgXcQ"}' | ytx + +# Save to file for further processing +ytx "https://www.youtube.com/watch?v=dQw4w9WgXcQ" > video-data.json + +# Update to latest version +dotnet tool update -g solrevdev.ytx +``` + +**Error Handling and Edge Cases** โš ๏ธ + +The tool handles various error scenarios gracefully: + +- Invalid YouTube URLs (exit code 2) +- Private or region-restricted videos (exit code 1) +- Videos without captions (returns explanatory message) +- Network errors (exit code 1) + +This makes it suitable for use in scripts and automation pipelines. + +**Key Learnings** ๐Ÿ’ก + +Building this tool taught me several valuable lessons: + +1. **YoutubeExplode Evolution**: The library has improved significantly - version 6.5.4 resolved transcript extraction issues that existed in earlier versions +2. **Global Tool Packaging**: The `PackageReadmeFile` and proper NuGet metadata are crucial for discoverability +3. **Multi-targeting**: Supporting both .NET 8 and 9 ensures broader compatibility +4. **JSON Input/Output**: Supporting both CLI args and stdin makes the tool more versatile for automation +5. **Caption Prioritization**: Smart ordering logic for captions improves user experience significantly + +**Future Enhancements** ๐Ÿ”ฎ + +Potential improvements for future versions: + +- Support for downloading specific time ranges +- Batch processing multiple URLs +- Custom output formats (CSV, XML) +- Integration with subtitle file formats (SRT, VTT) +- Translation support for non-English captions + +**The Development Journey** ๐Ÿ“ˆ + +The project evolved through several key commits: + +1. **Initial Implementation** (`2a2c702`) - Core functionality with basic transcript extraction +2. **NuGet Packaging Fixes** (`a22d4ce`) - Resolved packaging and dependency issues +3. **Version Management** (`0ce044f`) - Added automated version bumping +4. **Documentation** (`8b28691`) - Added CLAUDE.md and comprehensive docs +5. **HTML Viewer** (`52e08aa`) - Added self-contained HTML viewer for JSON output + +Each iteration improved the tool's reliability and usability. + +Success! ๐ŸŽ‰ \ No newline at end of file From d1f92c19ac9ad80ba6444ea684007ac439540702 Mon Sep 17 00:00:00 2001 From: John Smith Date: Mon, 29 Sep 2025 12:04:17 +0100 Subject: [PATCH 2/5] feat: add cover image for ytx blog post MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add ytx-dotnet-tool-cover.svg with terminal interface design - Include .NET and YouTube branding with purple gradient - Add cover_image front matter to blog post - Fix Liquid syntax errors in GitHub Actions YAML blocks ๐Ÿค– Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- ...cript-extractor-as-a-dotnet-global-tool.md | 5 +- images/ytx-dotnet-tool-cover.svg | 70 +++++++++++++++++++ 2 files changed, 73 insertions(+), 2 deletions(-) create mode 100644 images/ytx-dotnet-tool-cover.svg diff --git a/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md b/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md index 9eb44410..224d821c 100644 --- a/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md +++ b/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md @@ -3,6 +3,7 @@ layout: post title: Building ytx - A YouTube Transcript Extractor as a .NET Global Tool description: How to build a .NET Global Tool that extracts YouTube video metadata and transcripts as structured JSON, from concept to NuGet publication summary: A tutorial on creating ytx, a .NET global tool that extracts YouTube video titles, descriptions, and transcripts as JSON using YoutubeExplode and automated CI/CD. +cover_image: /images/ytx-dotnet-tool-cover.svg tags: - dotnet-global-tools - youtube @@ -321,7 +322,7 @@ jobs: - name: Bump version run: | # Script to increment version in .csproj - VERSION_TYPE="${{ github.event.inputs.version_bump || 'patch' }}" + VERSION_TYPE="{% raw %}${{ github.event.inputs.version_bump || 'patch' }}{% endraw %}" ./scripts/bump-version.sh "$VERSION_TYPE" - name: Build and Pack @@ -333,7 +334,7 @@ jobs: - name: Publish to NuGet run: | dotnet nuget push nupkg/*.nupkg \ - --api-key ${{ secrets.NUGET_API_KEY }} \ + --api-key {% raw %}${{ secrets.NUGET_API_KEY }}{% endraw %} \ --source https://api.nuget.org/v3/index.json ``` diff --git a/images/ytx-dotnet-tool-cover.svg b/images/ytx-dotnet-tool-cover.svg new file mode 100644 index 00000000..ebcb42c6 --- /dev/null +++ b/images/ytx-dotnet-tool-cover.svg @@ -0,0 +1,70 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ytx - YouTube Transcript Extractor + + + $ + ytx "https://youtube.com/watch?v=..." + + + { + "title": + "Building Amazing Apps" + "url": + "https://youtube.com/..." + "transcript": + "Welcome to today's..." + "transcriptRaw": + "[00:15] Welcome..." + } + + + + + + + + + + + + + ytx + + + YouTube Transcript Extractor โ€ข .NET Global Tool + + + + + + + + + \ No newline at end of file From 16c8c60be62efeccd2a65ed6faf3f1d8d7f4b361 Mon Sep 17 00:00:00 2001 From: John Smith Date: Mon, 29 Sep 2025 12:17:44 +0100 Subject: [PATCH 3/5] feat: enhance ytx blog post ending with AI development insights MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Replace commit list with modern development velocity section - Highlight Claude Code, GitHub Copilot, and MCP collaboration - Add human-AI partnership perspective - Maintain idiomatic Success! ๐ŸŽ‰ ending ๐Ÿค– Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- ...cript-extractor-as-a-dotnet-global-tool.md | 136 +++++++++++------- 1 file changed, 86 insertions(+), 50 deletions(-) diff --git a/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md b/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md index 224d821c..215dce05 100644 --- a/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md +++ b/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md @@ -77,6 +77,8 @@ The key to making this a global tool is the `.csproj` configuration: true ytx solrevdev.ytx + + 1.0.2 @@ -84,8 +86,11 @@ The key to making this a global tool is the `.csproj` configuration: Extract YouTube title, description, and transcript (raw + Markdown) as JSON. YouTube;transcript;captions;cli;dotnet-tool;json https://github.com/solrevdev/solrevdev.ytx + https://github.com/solrevdev/solrevdev.ytx MIT README.md + + ../../nupkg @@ -107,7 +112,7 @@ The crucial elements are: **Core Implementation** โš™๏ธ -The main challenge was handling YouTube's various caption formats and languages. Here's the core logic: +The main challenge was handling YouTube's various caption formats and languages. Here's the complete Main method: ```csharp static async Task Main(string[] args) @@ -116,7 +121,6 @@ static async Task Main(string[] args) { string? url = null; - // Handle both command-line args and JSON stdin if (args.Length == 1 && !string.IsNullOrWhiteSpace(args[0])) { url = args[0]; @@ -141,24 +145,64 @@ static async Task Main(string[] args) var client = new YoutubeClient(); var videoId = VideoId.TryParse(url) ?? throw new ArgumentException("Invalid YouTube URL/ID."); var video = await client.Videos.GetAsync(videoId); - - // Extract basic metadata var title = video.Title ?? ""; var description = video.Description ?? ""; - // The transcript extraction logic - var transcriptResult = await ExtractTranscript(client, video); + string transcriptRaw = ""; + string transcriptMd = ""; + + try + { + var manifest = await client.Videos.ClosedCaptions.GetManifestAsync(video.Id); + var track = manifest.Tracks + .OrderByDescending(t => t.Language.Name.Contains("English", StringComparison.OrdinalIgnoreCase)) + .ThenByDescending(t => t.IsAutoGenerated) + .FirstOrDefault(); + + if (track != null) + { + var captions = await client.Videos.ClosedCaptions.GetAsync(track); + + var rawSb = new StringBuilder(); + var mdSb = new StringBuilder(); + + foreach (var c in captions.Captions) + { + var text = NormalizeCaption(c.Text); + if (string.IsNullOrWhiteSpace(text)) continue; + + if (rawSb.Length > 0) rawSb.Append(' '); + rawSb.Append(text); + + var ts = ToHhMmSs(c.Offset); + var link = $"https://www.youtube.com/watch?v={video.Id}&t={(int)c.Offset.TotalSeconds}s"; + mdSb.AppendLine($"- [{ts}]({link}) {text}"); + } + + transcriptRaw = rawSb.ToString().Trim(); + transcriptMd = mdSb.ToString().TrimEnd(); + } + else + { + transcriptRaw = ""; + transcriptMd = "_No transcript/captions available for this video._"; + } + } + catch + { + transcriptRaw = ""; + transcriptMd = "_No transcript/captions available or captions retrieval failed._"; + } var output = new Output { url = url, title = title, description = description, - transcriptRaw = transcriptResult.raw, - transcript = transcriptResult.markdown + transcriptRaw = transcriptRaw, + transcript = transcriptMd }; - // Output clean JSON with proper Unicode handling var json = JsonSerializer.Serialize(output, new JsonSerializerOptions { WriteIndented = true, @@ -185,39 +229,11 @@ One of the trickiest parts was handling YouTube's various caption formats. The t - Handle both manual and auto-generated captions - Gracefully handle videos without captions -```csharp -var manifest = await client.Videos.ClosedCaptions.GetManifestAsync(video.Id); -var track = manifest.Tracks - .OrderByDescending(t => t.Language.Name.Contains("English", StringComparison.OrdinalIgnoreCase)) - .ThenByDescending(t => t.IsAutoGenerated) - .FirstOrDefault(); - -if (track != null) -{ - var captions = await client.Videos.ClosedCaptions.GetAsync(track); - - var rawSb = new StringBuilder(); - var mdSb = new StringBuilder(); - - foreach (var c in captions.Captions) - { - var text = NormalizeCaption(c.Text); - if (string.IsNullOrWhiteSpace(text)) continue; - - // Build raw transcript - if (rawSb.Length > 0) rawSb.Append(' '); - rawSb.Append(text); - - // Build markdown with timestamped links - var ts = ToHhMmSs(c.Offset); - var link = $"https://www.youtube.com/watch?v={video.Id}&t={(int)c.Offset.TotalSeconds}s"; - mdSb.AppendLine($"- [{ts}]({link}) {text}"); - } - - transcriptRaw = rawSb.ToString().Trim(); - transcriptMd = mdSb.ToString().TrimEnd(); -} -``` +The caption selection logic is embedded in the Main method above (lines 32-62), where it: +1. Gets the caption manifest for the video +2. Orders tracks by English language preference, then by auto-generated status +3. Downloads the selected caption track and formats both raw and markdown output +4. Handles error cases gracefully with appropriate fallback messages **Utility Functions** ๐Ÿ”ง @@ -390,16 +406,36 @@ Potential improvements for future versions: - Integration with subtitle file formats (SRT, VTT) - Translation support for non-English captions -**The Development Journey** ๐Ÿ“ˆ +**Modern Development Velocity** โšก + +What strikes me most about this project is the development speed enabled by modern AI tooling. From initial concept to published NuGet package took just a few hours - a timeframe that would have been unthinkable just a few years ago. + +**The AI-Assisted Workflow** ๐Ÿค– -The project evolved through several key commits: +This project showcased the power of combining multiple AI tools: -1. **Initial Implementation** (`2a2c702`) - Core functionality with basic transcript extraction -2. **NuGet Packaging Fixes** (`a22d4ce`) - Resolved packaging and dependency issues -3. **Version Management** (`0ce044f`) - Added automated version bumping -4. **Documentation** (`8b28691`) - Added CLAUDE.md and comprehensive docs -5. **HTML Viewer** (`52e08aa`) - Added self-contained HTML viewer for JSON output +- **Claude Code**: Handled the core architecture decisions, error handling patterns, and CI/CD pipeline setup. Particularly valuable for getting the .csproj packaging configuration right on the first try. +- **GitHub Copilot**: Excelled at generating repetitive code patterns, JSON serialization boilerplate, and regex text normalization functions. +- **MCPs (Model Context Protocol)**: Provided seamless integration between different AI tools and development contexts, making the workflow feel natural rather than fragmented. -Each iteration improved the tool's reliability and usability. +**The Human-AI Partnership** ๐Ÿค + +The most interesting aspect wasn't that AI wrote the code, but how it changed the development process itself: + +1. **Design-First Thinking**: Instead of iterating through implementation details, I could focus on the user experience and data flow +2. **Documentation-Driven Development**: Writing this blog post in parallel with coding helped clarify requirements and catch edge cases early +3. **Confidence in Exploration**: Having AI assistance made it easy to try different approaches without the usual "sunk cost" feeling + +**Looking Forward** ๐Ÿ”ฎ + +This project represents a new normal in software development - where the bottleneck shifts from typing code to thinking through problems and user needs. The combination of AI coding assistants, intelligent toolchains, and human creativity is genuinely transformative. + +For developers hesitant about AI tools: they're not replacing us, they're amplifying our ability to solve meaningful problems quickly. The future belongs to developers who can effectively collaborate with AI to build better software faster. + +Ready to extract some YouTube transcripts? ๐ŸŽฌ + +```powershell +dotnet tool install -g solrevdev.ytx +``` Success! ๐ŸŽ‰ \ No newline at end of file From b6108b35ad23fab8e42ca3c5f36461e44ba34e0b Mon Sep 17 00:00:00 2001 From: John Smith Date: Sat, 25 Oct 2025 19:18:28 +0100 Subject: [PATCH 4/5] feat(docs): update tutorial for building ytx .NET global tool MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ๐Ÿ“ Revised the tutorial to enhance clarity and provide updated information on building the `ytx` tool. ๐Ÿ“ Modified: _posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md ๐Ÿ”ง Improved descriptions and summaries to better reflect the extraction of YouTube transcripts and metadata as JSON โš™๏ธ Expanded CI/CD pipeline details with GitHub Actions for production readiness and best practices ๐Ÿ“ฆ Included insights on using AI tools to accelerate the development process and emphasized design-first and documentation-driven approaches --- ...cript-extractor-as-a-dotnet-global-tool.md | 255 ++++++++++++++---- 1 file changed, 197 insertions(+), 58 deletions(-) diff --git a/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md b/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md index 215dce05..3458f8ba 100644 --- a/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md +++ b/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md @@ -1,8 +1,8 @@ --- layout: post title: Building ytx - A YouTube Transcript Extractor as a .NET Global Tool -description: How to build a .NET Global Tool that extracts YouTube video metadata and transcripts as structured JSON, from concept to NuGet publication -summary: A tutorial on creating ytx, a .NET global tool that extracts YouTube video titles, descriptions, and transcripts as JSON using YoutubeExplode and automated CI/CD. +description: Build a .NET Global Tool to extract YouTube transcripts and metadata as JSON. Learn YoutubeExplode, CLI argument parsing, NuGet packaging, and GitHub Actions automation from concept to publication. +summary: Complete tutorial on creating ytx, a .NET global tool for extracting YouTube video titles, descriptions, and transcripts as JSON. Covers YoutubeExplode library, caption selection logic, JSON serialization, NuGet packaging, and automated CI/CD with GitHub Actions. cover_image: /images/ytx-dotnet-tool-cover.svg tags: - dotnet-global-tools @@ -19,7 +19,7 @@ tags: Sometimes you need to extract structured data from YouTube videos for analysis, documentation, or automation. While there are various web-based solutions, having a command-line tool that outputs clean JSON makes integration with scripts and pipelines much easier. -I built `ytx` - a .NET Global Tool that extracts YouTube video metadata and transcripts as structured JSON. The tool takes a YouTube URL and returns the video title, description, and full transcript with timestamps in both raw text and markdown formats. +I built **ytx** - a .NET Global Tool that extracts YouTube video metadata and transcripts as structured JSON. The tool takes a YouTube URL and returns the video title, description, and full transcript with timestamps in both raw text and markdown formats. This post walks you through building your own .NET global tool from scratch, covering architecture design, caption handling, JSON serialization, NuGet packaging, and setting up automated CI/CD with GitHub Actions. **The Problem** ๐ŸŽฏ @@ -294,66 +294,199 @@ The tool produces clean, structured JSON: The markdown transcript format makes it easy to create documentation with clickable timestamps that jump directly to specific moments in the video. -**Automated CI/CD Pipeline** ๐Ÿค– +**Production-Ready CI/CD Pipeline with GitHub Actions** ๐Ÿค– -To streamline releases, I set up GitHub Actions to automatically: -- Build and test the project -- Increment version numbers -- Publish to NuGet -- Create GitHub releases +To streamline releases and reduce manual work, I set up GitHub Actions to automatically handle the entire release pipeline. Unlike simple workflows, this production pipeline: +- Runs on every push to `master` (only when source files change, avoiding redundant builds) +- Allows manual triggers with version bump selection (patch, minor, or major) +- Automatically increments semantic versions in your `.csproj` file +- Commits version changes back to the repository with git tags +- Builds and publishes to NuGet with proper error handling +- Creates GitHub releases with auto-generated release notes +- Supports multiple .NET versions (8.x and 9.x) for maximum compatibility -The workflow file (`.github/workflows/publish.yml`) handles version bumping: +The complete workflow file (`.github/workflows/publish.yml`) handles all of this: ```yaml name: Publish NuGet (ytx) on: - push: - branches: [ master ] workflow_dispatch: inputs: - version_bump: - description: 'Version bump type' + bump: + description: 'Version bump type (major|minor|patch)' required: true default: 'patch' - type: choice - options: - - patch - - minor - - major + push: + branches: [ "master" ] + paths: + - 'src/Ytx/**' + - '.github/workflows/publish.yml' + +permissions: + contents: write + packages: read + +env: + PROJECT_DIR: src/Ytx + CSPROJ: src/Ytx/Ytx.csproj + NUPKG_DIR: nupkg + NUGET_SOURCE: https://api.nuget.org/v3/index.json jobs: - publish: + build-pack-publish: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 - + - name: Checkout + uses: actions/checkout@v4 + - name: Setup .NET uses: actions/setup-dotnet@v4 with: dotnet-version: | - 8.0.x - 9.0.x + 9.x + 8.x - - name: Bump version - run: | - # Script to increment version in .csproj - VERSION_TYPE="{% raw %}${{ github.event.inputs.version_bump || 'patch' }}{% endraw %}" - ./scripts/bump-version.sh "$VERSION_TYPE" + - name: Restore + run: dotnet restore $PROJECT_DIR - - name: Build and Pack + - name: Determine and bump version + id: bump + shell: bash run: | - dotnet restore src/Ytx - dotnet build src/Ytx -c Release - dotnet pack src/Ytx -c Release + set -euo pipefail + CURR=$(grep -oPm1 '(?<=)[^<]+' "$CSPROJ") + echo "Current version: $CURR" + IFS='.' read -r MAJ MIN PAT <<< "$CURR" + BUMP="${{ github.event.inputs.bump || 'patch' }}" + case "$BUMP" in + major) MAJ=$((MAJ+1)); MIN=0; PAT=0 ;; + minor) MIN=$((MIN+1)); PAT=0 ;; + patch|*) PAT=$((PAT+1)) ;; + esac + NEW="$MAJ.$MIN.$PAT" + echo "New version: $NEW" + sed -i "s|$CURR|$NEW|" "$CSPROJ" + echo "version=$NEW" >> "$GITHUB_OUTPUT" + + - name: Commit version bump + if: ${{ github.ref == 'refs/heads/master' }} + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + git add ${{ env.CSPROJ }} + git commit -m "chore: bump version to ${{ steps.bump.outputs.version }}" + git tag "v${{ steps.bump.outputs.version }}" + git push --follow-tags + + - name: Build + run: dotnet build $PROJECT_DIR -c Release --no-restore + + - name: Pack + run: dotnet pack $PROJECT_DIR -c Release --no-build - name: Publish to NuGet + env: + NUGET_API_KEY: ${{ secrets.NUGET_API_KEY }} run: | - dotnet nuget push nupkg/*.nupkg \ - --api-key {% raw %}${{ secrets.NUGET_API_KEY }}{% endraw %} \ - --source https://api.nuget.org/v3/index.json + dotnet nuget push $NUPKG_DIR/*.nupkg \ + --api-key "$NUGET_API_KEY" \ + --source "$NUGET_SOURCE" \ + --skip-duplicate + + - name: Create GitHub Release + uses: softprops/action-gh-release@v2 + with: + tag_name: v${{ steps.bump.outputs.version }} + name: ytx v${{ steps.bump.outputs.version }} + generate_release_notes: true ``` +**Understanding the Workflow Architecture** ๐Ÿ—๏ธ + +This workflow implements several production best practices that help .NET developers distribute global tools effectively: + +**Environment Variables for DRY Principle:** +The `env:` block defines reusable values (`PROJECT_DIR`, `CSPROJ`, `NUPKG_DIR`, `NUGET_SOURCE`) referenced throughout the workflow. This approach keeps configuration centralizedโ€”change a directory path once, and it updates everywhere. This is crucial when managing complex multi-project solutions or adjusting package output locations. + +**Permissions Block:** +The `permissions:` section restricts the workflow to only what it needs: +- `contents: write` โ€” Required to create commits, tags, and push back to the repository +- `packages: read` โ€” Required for accessing NuGet package data + +This follows the principle of least privilege, improving security by preventing the workflow from performing unauthorized actions. + +**Smart Trigger Configuration:** +```yaml +on: + push: + branches: [ "master" ] + paths: + - 'src/Ytx/**' + - '.github/workflows/publish.yml' +``` + +The `paths:` filter prevents unnecessary builds when only documentation or other non-source files change. This saves CI/CD minutes and reduces feedback latency. + +**Semantic Version Bumping with bash:** +The version bump step demonstrates how to parse and manipulate semantic versions programmatically: + +```bash +IFS='.' read -r MAJ MIN PAT <<< "$CURR" # Parse 1.0.2 into components +case "$BUMP" in + major) MAJ=$((MAJ+1)); MIN=0; PAT=0 ;; # 1.0.2 โ†’ 2.0.0 + minor) MIN=$((MIN+1)); PAT=0 ;; # 1.0.2 โ†’ 1.1.0 + patch|*) PAT=$((PAT+1)) ;; # 1.0.2 โ†’ 1.0.3 +esac +``` + +This approach ensures version consistency without manually editing `.csproj` files. The `echo "version=$NEW" >> "$GITHUB_OUTPUT"` sends the new version to subsequent stepsโ€”a key pattern in GitHub Actions workflows. + +**Git Automation for Reproducible Releases:** +```bash +git config user.name "github-actions[bot]" +git add ${{ env.CSPROJ }} +git commit -m "chore: bump version to ${{ steps.bump.outputs.version }}" +git tag "v${{ steps.bump.outputs.version }}" +git push --follow-tags +``` + +This creates an immutable audit trail. Every NuGet release corresponds to: +1. A specific git commit (with the bumped version) +2. A git tag (for easy checkout: `git checkout v1.0.3`) +3. A GitHub release (with release notes) + +This traceability is essential for troubleshooting issues and understanding what code produced which package version. + +**Optimized Build Pipeline:** +Notice the careful use of build flags: +```bash +dotnet restore $PROJECT_DIR # Explicit restore +dotnet build $PROJECT_DIR -c Release --no-restore # Skip redundant restore +dotnet pack $PROJECT_DIR -c Release --no-build # Skip redundant build +``` + +The `--no-restore` and `--no-build` flags prevent repeating expensive operations. For .NET global tools especially, proper dependency isolation mattersโ€”you want to ensure your tool works across different .NET SDK versions, which is why this workflow tests against both 8.x and 9.x. + +**NuGet Publishing with Idempotency:** +```bash +dotnet nuget push $NUPKG_DIR/*.nupkg \ + --skip-duplicate +``` + +The `--skip-duplicate` flag means you can safely re-run the workflow without errors if a version was already published. This is crucial for reliabilityโ€”sometimes you need to retry a build due to temporary network issues or API timeouts. + +**Automated GitHub Releases:** +```yaml +- name: Create GitHub Release + uses: softprops/action-gh-release@v2 + with: + tag_name: v${{ steps.bump.outputs.version }} + generate_release_notes: true +``` + +This automatically creates a GitHub release with auto-generated release notes based on commit messages since the last release. Users see a clear changelog without manual effort, and the release is properly associated with the NuGet package version. + **Installation and Usage** ๐Ÿ“ฆ Once published to NuGet, users can install the tool globally: @@ -386,15 +519,16 @@ The tool handles various error scenarios gracefully: This makes it suitable for use in scripts and automation pipelines. -**Key Learnings** ๐Ÿ’ก +**Key Learnings & Best Practices** ๐Ÿ’ก -Building this tool taught me several valuable lessons: +Building this .NET global tool taught me several valuable lessons applicable to any command-line tool project: -1. **YoutubeExplode Evolution**: The library has improved significantly - version 6.5.4 resolved transcript extraction issues that existed in earlier versions -2. **Global Tool Packaging**: The `PackageReadmeFile` and proper NuGet metadata are crucial for discoverability -3. **Multi-targeting**: Supporting both .NET 8 and 9 ensures broader compatibility -4. **JSON Input/Output**: Supporting both CLI args and stdin makes the tool more versatile for automation -5. **Caption Prioritization**: Smart ordering logic for captions improves user experience significantly +1. **YoutubeExplode Library Maturity**: Version 6.5.4 resolved transcript extraction issues that plagued earlier versions. Always verify library versions match your use case requirements. +2. **.NET Global Tool Packaging**: The `PackAsTool` property, `ToolCommandName`, and `PackageReadmeFile` are crucial for NuGet discoverability. Missing these makes your tool harder to find. +3. **Multi-targeting Strategy**: Supporting both .NET 8 and 9 simultaneously ensures broader compatibility across development environments and CI/CD pipelines. +4. **Flexible Input/Output Design**: Supporting both command-line arguments and stdin (JSON) makes your tool more versatile for automation, scripting, and pipeline integration. +5. **Intelligent Caption Selection**: Smart ordering logic (English preference โ†’ auto-generated fallback) dramatically improves user experience compared to simple "first available" approaches. +6. **Semantic Versioning in CI/CD**: Automating patch/minor/major version bumps reduces manual work and ensures consistency across releases. **Future Enhancements** ๐Ÿ”ฎ @@ -406,36 +540,41 @@ Potential improvements for future versions: - Integration with subtitle file formats (SRT, VTT) - Translation support for non-English captions -**Modern Development Velocity** โšก +**Development Velocity with Modern AI Tooling** โšก -What strikes me most about this project is the development speed enabled by modern AI tooling. From initial concept to published NuGet package took just a few hours - a timeframe that would have been unthinkable just a few years ago. +What stands out about this project is the development speed enabled by modern AI assistance. From initial concept through architecture, implementation, testing, and NuGet publication took just a few hours - something that would have required days of work just five years ago. -**The AI-Assisted Workflow** ๐Ÿค– +**The AI-Assisted Development Workflow** ๐Ÿค– -This project showcased the power of combining multiple AI tools: +This .NET global tool project showcased the power of combining multiple AI tools effectively: -- **Claude Code**: Handled the core architecture decisions, error handling patterns, and CI/CD pipeline setup. Particularly valuable for getting the .csproj packaging configuration right on the first try. -- **GitHub Copilot**: Excelled at generating repetitive code patterns, JSON serialization boilerplate, and regex text normalization functions. +- **Claude Code**: Handled core architecture decisions, .NET-specific patterns, error handling strategies, and GitHub Actions CI/CD pipeline configuration. Particularly valuable for getting the `.csproj` packaging configuration correct on the first attempt. +- **GitHub Copilot**: Excelled at generating repetitive code patterns, JSON serialization boilerplate, regex text normalization functions, and test scaffold code. - **MCPs (Model Context Protocol)**: Provided seamless integration between different AI tools and development contexts, making the workflow feel natural rather than fragmented. -**The Human-AI Partnership** ๐Ÿค +**The Human-AI Partnership in Practice** ๐Ÿค + +The most interesting insight wasn't that AI wrote the code, but how it transformed the development process itself: -The most interesting aspect wasn't that AI wrote the code, but how it changed the development process itself: +1. **Design-First Development**: Instead of iterating through implementation details, focus shifted to user experience and clean data flow architecture. +2. **Documentation-Driven Development**: Writing this technical blog post in parallel with coding helped clarify requirements and catch edge cases early. +3. **Risk-Free Exploration**: AI assistance made it easy to try different architectural approaches without the usual "sunk cost" hesitation. -1. **Design-First Thinking**: Instead of iterating through implementation details, I could focus on the user experience and data flow -2. **Documentation-Driven Development**: Writing this blog post in parallel with coding helped clarify requirements and catch edge cases early -3. **Confidence in Exploration**: Having AI assistance made it easy to try different approaches without the usual "sunk cost" feeling +**What This Means for .NET Developers** ๐Ÿš€ -**Looking Forward** ๐Ÿ”ฎ +This project represents a new normal in software developmentโ€”where the bottleneck shifts from typing code to thinking through problems and user needs. The combination of AI coding assistants, intelligent build toolchains (GitHub Actions, NuGet), and human creativity is genuinely transformative. -This project represents a new normal in software development - where the bottleneck shifts from typing code to thinking through problems and user needs. The combination of AI coding assistants, intelligent toolchains, and human creativity is genuinely transformative. +For developers hesitant about AI tools: they're not replacing you; they're amplifying your ability to solve meaningful problems quickly. The future belongs to developers who can effectively collaborate with AI to build better software faster. -For developers hesitant about AI tools: they're not replacing us, they're amplifying our ability to solve meaningful problems quickly. The future belongs to developers who can effectively collaborate with AI to build better software faster. +**Get Started Building Your Own .NET Global Tool** ๐Ÿ“ฆ -Ready to extract some YouTube transcripts? ๐ŸŽฌ +Ready to create your own command-line tool and publish it to NuGet? Install ytx to see a working example: ```powershell dotnet tool install -g solrevdev.ytx +ytx "https://www.youtube.com/watch?v=dQw4w9WgXcQ" ``` +Or [explore the source code on GitHub](https://github.com/solrevdev/solrevdev.ytx) to see the complete implementation. + Success! ๐ŸŽ‰ \ No newline at end of file From c0070e73bc608befa954f36ecb3597fccd118307 Mon Sep 17 00:00:00 2001 From: John Smith Date: Mon, 27 Oct 2025 12:39:07 +0000 Subject: [PATCH 5/5] feat(docs): add raw markdown syntax for GitHub Actions workflow in tutorial --- ...tx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md b/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md index 3458f8ba..9e6c4104 100644 --- a/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md +++ b/_posts/2025-08-31-building-ytx-a-youtube-transcript-extractor-as-a-dotnet-global-tool.md @@ -307,6 +307,7 @@ To streamline releases and reduce manual work, I set up GitHub Actions to automa The complete workflow file (`.github/workflows/publish.yml`) handles all of this: +{% raw %} ```yaml name: Publish NuGet (ytx) @@ -401,6 +402,7 @@ jobs: name: ytx v${{ steps.bump.outputs.version }} generate_release_notes: true ``` +{% endraw %} **Understanding the Workflow Architecture** ๐Ÿ—๏ธ