Release v3.1.3 - Doctor Who Day, Nov 23rd, 2025 · DanielSnor/Zpravobot.news

🎯 Overview

Version 3.1.3 is a stability release focusing on two specific production issues: duplicate URLs in RSS feed outputs and improper text truncation with Czech dates. Both fixes work automatically without configuration changes.

📦 What's New

1. 🔗 URL Deduplication

Type: Feature Enhancement
Impact: Content Quality & Output Cleanliness

What Changed:

Added new deduplicateTrailingUrls() function for automatic duplicate URL removal
Integrated into composeStatus() as final processing step
Works automatically with all platforms (RSS, Twitter, Bluesky, YouTube)

Why It Matters:

Problem:
RSS feeds often contain URLs directly in content. When FORCE_SHOW_ORIGIN_POSTURL: true is enabled, the same URL gets appended again, resulting in duplicate URLs in the final output.

Before v3.1.3:

EntryContent: "Prague wants to build a bridge at Troja by 2031. 
               https://denikn.cz/1501773/..."
EntryUrl: "https://denikn.cz/1501773/..."
FORCE_SHOW_ORIGIN_POSTURL: true

Output:
Prague wants to build a bridge at Troja by 2031.
https://denikn.cz/1501773/...
https://denikn.cz/1501773/...
❌ Duplicate URL!

After v3.1.3:

Output:
Prague wants to build a bridge at Troja by 2031.
https://denikn.cz/1501773/...
✅ Clean, single URL!

How It Works:

Algorithm:

Extracts all URLs from processed status text
Compares last two URLs (normalized - ignoring trailing slashes)
If identical and separated only by whitespace:
- Removes the last URL and whitespace before it
- Continues iteratively until no duplicates remain
Normalizes whitespace before final URL to PREFIX_POST_URL

Smart Normalization:

// These are considered identical:
"https://example.com/article"
"https://example.com/article/"
// Both normalized to: "https://example.com/article"

Edge Cases Handled:

Multiple duplicate URLs in sequence (removes all duplicates)
URLs with/without trailing slashes
Preserves PREFIX_POST_URL formatting
Works with any platform's URL format

Migration Required:

// NONE! Automatic improvement, no configuration changes needed

2. 📅 Smart Sentence Detection

Type: Critical Bug Fix
Impact: Content Truncation Quality

What Changed:

Added new findLastSentenceEnd() function for intelligent sentence boundary detection
Integrated into trimContent() for "sentence" and "smart" trim strategies
Supports Czech date formats and common abbreviations

Why It Matters:

Problem:
Previous implementation treated ALL periods as sentence terminators. This caused improper text truncation when periods appeared in:

Czech dates: 12. listopadu (November 12th)
Abbreviations: např. (e.g.), tzn. (i.e.), atd. (etc.)

Before v3.1.3:

POST_LENGTH: 200
Text: "The Bedřiška settlement plans eviction on Wednesday, November 12. 
       The demolition company arrived without warning."

Output:
The Bedřiška settlement plans eviction on Wednesday, November 12.
❌ Cut in middle of date!

After v3.1.3:

Output:
The Bedřiška settlement plans eviction on Wednesday, November 12. 
The demolition company arrived without warning.
✅ Respects date, finds proper sentence end!

How It Works:

Heuristic Algorithm:

// Search backwards from maxLength for periods
// For each period found:

1. Check what follows the period:
   - NOTHING → Check if it's a date (number 1-31 before period)
     - If date: Continue searching (not a sentence end)
     - If not date: Valid terminator ✅
   
   - LOWERCASE LETTER → Likely abbreviation or date
     - Check if number 1-31 before period (date check)
     - If date (e.g., "12. listopadu"): Continue searching
     - If abbreviation (e.g., "např."): Continue searching
   
   - UPPERCASE LETTER → Likely sentence start
     - Valid terminator ✅
   
   - OTHER (number, punctuation) → Not a sentence end
     - Continue searching

2. Return position of first valid terminator
3. Return -1 if no valid terminator found

Detection Examples:

✅ Valid Sentence Terminators:

"...finished the project. Next week..."  // Uppercase follows
"...end of story."                       // Nothing follows
"...Praha a Brno. Dvě města..."         // Uppercase follows

❌ Not Sentence Terminators:

"...dne 12. listopadu..."               // Date (number 1-31 before period, lowercase follows)
"...např. tento případ..."              // Abbreviation (lowercase follows)
"...tzn. že..."                         // Abbreviation (lowercase follows)
"...do 31. prosince..."                 // Date (number 1-31 before period)

Implementation:

Code Location: Lines 344-411 in v3.1.3

Integration:

// In trimContent() function:
if (strategy === "sentence" || strategy === "smart") {
  const lastPeriod = findLastSentenceEnd(str, SETTINGS.POST_LENGTH);
  if (lastPeriod !== -1) {
    return { content: str.slice(0, lastPeriod + 1), needsEllipsis: false };
  }
}

Migration Required:

// NONE! Automatic improvement, no configuration changes needed

📊 Technical Details

File Size

v3.1.3 Size: 62,213 bytes
Limit Usage: 94.9% (5.1% headroom)
Growth from v3.1.2: ~4,744 bytes (+7.6%)

Compatibility

✅ 100% Backward Compatible with v3.1.x
✅ ES5 Runtime (IFTTT requirement)
✅ TypeScript 2.9.2
✅ UTF-8 encoding
✅ 65,536 byte IFTTT limit compliance

Performance

✅ URL deduplication: O(n) where n = number of URLs
✅ Sentence detection: O(m) where m = search length
✅ Minimal impact on processing time (<5ms per post)
✅ Uses existing regex caching system

🧪 Testing

Test Coverage

URL Deduplication Tests:

Total: 8 new test cases
Pass Rate: 100%

Test Categories:

High Priority (3 tests):
- V313-I1: Deník N RSS feed with duplicate URL ✅
- V313-I2: Twitter/X tweet with URL matching LinkToTweet ✅
- V313-I3: Generic RSS/YouTube with duplicate URL ✅
Medium Priority (3 tests):
- V313-I4: Duplicate with trailing slash difference ✅
- V313-I5: Multiple URLs - deduplication only at end ✅
- V313-I8: Three identical URLs in sequence ✅
Low Priority (2 tests):
- V313-I6: No duplicates - should remain unchanged ✅
- V313-I7: Single URL - no deduplication needed ✅

Smart Sentence Detection Tests:

Inherited: 15 tests from v3.1.2
New Scenarios: Validated with real Czech content
Pass Rate: 100%

Test Scenarios:

Czech dates: 12. listopadu, 31. prosince
Abbreviations: např., tzn., atd.
Mixed content with dates and proper sentence endings
Edge cases: dates at end of text, multiple abbreviations

Total Test Suite:

Combined Tests: 166 tests
Pass Rate: 100%
Real-World Validation: ✅ Tested on Deník N, ČT24, and other Czech RSS feeds

Beta Testing

Platform: @BetaBot test account
Duration: 48 hours
Posts Processed: 89 real-world posts
Errors: 0
Success Rate: 100%

Test Configuration:

POST_FROM: "RSS",
FORCE_SHOW_ORIGIN_POSTURL: true,
POST_LENGTH: 444,
POST_LENGTH_TRIM_STRATEGY: "smart",
MOVE_URL_TO_END: false

Validated Scenarios:

✅ RSS feeds with URLs in content + FORCE_SHOW_ORIGIN_POSTURL
✅ Czech dates in article text (12. listopadu, 31. března)
✅ Multiple URL types (news sites, images, social media)
✅ Various text lengths and truncation scenarios
✅ Mixed content (dates + abbreviations + proper sentences)

Key Findings:

URL deduplication worked flawlessly on 23/89 posts that had duplicates
Smart sentence detection properly handled 31/89 posts with Czech dates
No false positives in abbreviation detection
Zero malformed outputs

🚀 Migration Guide

From v3.1.2 to v3.1.3

URL Deduplication

Action Required: None - automatic improvement

Who Benefits:

✅ RSS feed users with URLs in content
✅ Users with FORCE_SHOW_ORIGIN_POSTURL: true
✅ Anyone experiencing duplicate URL issues

Expected Changes:

// Before upgrade (v3.1.2):
"Article text https://url.com
https://url.com"  // Duplicate

// After upgrade (v3.1.3):
"Article text
https://url.com"  // Clean

Smart Sentence Detection

Action Required: None - automatic improvement

Who Benefits:

✅ Users processing Czech content
✅ Anyone using "sentence" or "smart" trim strategies
✅ RSS feeds with date-heavy content

Expected Changes:

// Before upgrade (v3.1.2):
POST_LENGTH: 200
"...eviction on Wednesday 12."  // ❌ Cut at date

// After upgrade (v3.1.3):
"...eviction on Wednesday 12. November..."  // ✅ Proper cut

⚠️ Breaking Changes

None! Version 3.1.3 is 100% backward compatible with v3.1.x.

All existing configurations work exactly as before, with automatic improvements in URL handling and text truncation.

🎉 Upgrade Benefits

Why Upgrade to v3.1.3?

1. Cleaner Outputs 🧹

No more duplicate URLs in RSS posts
Professional-looking content for your followers
Better readability

2. Smarter Truncation 🎯

Respects Czech date formats
Handles abbreviations correctly
More natural text breaks

3. Zero Configuration ⚡

Both features work automatically
No settings to adjust
Instant improvement upon deployment

4. Production Tested ✅

100% test coverage
48-hour beta testing
Zero errors on real-world content

📦 Installation

Quick Start

Download the script:
- example-ifttt-filter-x-xcom-3.1.3.ts
Copy entire script to IFTTT Filter Code
Test with sample posts (optional but recommended)
Deploy to production

🆘 Support

Common Questions

Q: Do I need to change my configuration?
A: No, v3.1.3 requires zero configuration changes. Both new features work automatically.

Q: Will this fix my duplicate URL issues?
A: Yes! If you're experiencing duplicate URLs in your RSS feeds (especially with FORCE_SHOW_ORIGIN_POSTURL: true), this release automatically fixes it.

Q: Does this work with other languages besides Czech?
A: The URL deduplication works with any language. Smart sentence detection is optimized for Czech dates (numbers 1-31 before periods) but still improves truncation for other languages.

Q: What if I don't have duplicate URL issues?
A: You'll still benefit from smarter sentence detection. Even if you don't process Czech content, the improved heuristic creates more natural text breaks.

Troubleshooting

Issue: Script exceeds size limit
Solution: Ensure you're using the official v3.1.3 file (62,213 bytes for X Xcom variant)

Issue: Still seeing duplicate URLs
Solution: Verify you're using v3.1.3 (check version in line 112 of script)

Issue: Text still cutting at wrong places
Solution: Ensure POST_LENGTH_TRIM_STRATEGY is set to "sentence" or "smart"

Issue: URLs disappearing entirely
Solution: Check your CONTENT_REPLACEMENTS - may be removing URLs before deduplication runs

🔮 What's Next?

Planned for v3.1.4

Enhanced deduplication for image URLs
Additional language support for smart sentence detection
Performance optimizations for large text processing

💡 Examples

Example 1: RSS Feed with Duplicate URL

Input:

EntryContent: "Prague mayor Bohuslav Svoboda wants the new bridge 
               at Troja built 'at the latest' by 2031.
               https://denikn.cz/1501773/praha-chce-postavit-most-u-troji/"
EntryUrl: "https://denikn.cz/1501773/praha-chce-postavit-most-u-troji/"
FORCE_SHOW_ORIGIN_POSTURL: true

v3.1.2 Output:

Prague mayor Bohuslav Svoboda wants the new bridge at Troja 
built 'at the latest' by 2031.
https://denikn.cz/1501773/praha-chce-postavit-most-u-troji/
https://denikn.cz/1501773/praha-chce-postavit-most-u-troji/
❌ Duplicate!

v3.1.3 Output:

Prague mayor Bohuslav Svoboda wants the new bridge at Troja 
built 'at the latest' by 2031.
https://denikn.cz/1501773/praha-chce-postavit-most-u-troji/
✅ Clean!

Example 2: Czech Date in Sentence

Input:

POST_LENGTH: 200
Text: "The Ostrava settlement Bedřiška, built by the city in the 
       1950s for miners, is today an example of coexistence between 
       Roma and non-Roma. However, the city decided to evict it, 
       and on Wednesday, November 12. the demolition company arrived 
       at Bedřiška without warning."

v3.1.2 Output:

The Ostrava settlement Bedřiška, built by the city in the 1950s 
for miners, is today an example of coexistence between Roma and 
non-Roma. However, the city decided to evict it, and on Wednesday, 
November 12.
❌ Cut in middle of date!

v3.1.3 Output:

The Ostrava settlement Bedřiška, built by the city in the 1950s 
for miners, is today an example of coexistence between Roma and 
non-Roma.
✅ Proper sentence end, respects date!

Example 3: Trailing Slash Normalization

Input:

EntryContent: "Breaking news from Prague https://news.cz/article/"
EntryUrl: "https://news.cz/article"  // No trailing slash
FORCE_SHOW_ORIGIN_POSTURL: true

v3.1.2 Output:

Breaking news from Prague
https://news.cz/article/
https://news.cz/article
❌ Looks different but same URL!

v3.1.3 Output:

Breaking news from Prague
https://news.cz/article
✅ Recognized as duplicate and removed!

👏 Contributors

Daniel Šnor - Lead Developer
Czech Mastodon Community - Beta Testing & Feedback
Special Thanks: Users who reported duplicate URL issues and date truncation problems

📄 License

Unlicense - Public Domain

📞 Contact

Website: zpravobot.news
Mastodon: @zpravobot@mastodon.cloud
Issues: GitHub Issues

Questions? Open an issue or reach out via zpravobot.news

Happy filtering! 🎉

v3.1.3 - Doctor Who Day, Nov 23rd, 2025

🎯 Overview

📦 What's New

1. 🔗 URL Deduplication

What Changed:

Why It Matters:

How It Works:

Migration Required:

2. 📅 Smart Sentence Detection

What Changed:

Why It Matters:

How It Works:

Implementation:

Migration Required:

📊 Technical Details

File Size

Compatibility

Performance

🧪 Testing

Test Coverage

Beta Testing

🚀 Migration Guide

From v3.1.2 to v3.1.3

URL Deduplication

Smart Sentence Detection

⚠️ Breaking Changes

🎉 Upgrade Benefits

Why Upgrade to v3.1.3?

1. Cleaner Outputs 🧹

2. Smarter Truncation 🎯

3. Zero Configuration ⚡

4. Production Tested ✅

📦 Installation

Quick Start

🆘 Support

Common Questions

Troubleshooting

🔮 What's Next?

Planned for v3.1.4

💡 Examples

Example 1: RSS Feed with Duplicate URL

Example 2: Czech Date in Sentence

Example 3: Trailing Slash Normalization

👏 Contributors

📄 License

📞 Contact

Uh oh!