Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@ export default defineConfig({
"/configs": "/feed-directory/",
"/components/html2rss-web": "/web-application/",
"/components/html2rss": "/ruby-gem/",
"/components/html2rss-configs": "/html2rss-configs/",
"/components/html2rss-configs": "/creating-custom-feeds/",
"/html2rss-configs": "/creating-custom-feeds/",
"/html2rss-configs/": "/creating-custom-feeds/",
"/components": "/",
},
build: {
Expand Down Expand Up @@ -251,6 +253,10 @@ export default defineConfig({
label: "Home",
link: "/",
},
{
label: "Getting Started",
link: "/getting-started/",
},
{
label: "Common Use Cases",
link: "/common-use-cases",
Expand All @@ -263,8 +269,8 @@ export default defineConfig({
label: "Web Application",
collapsed: true,
items: [
"web-application",
"web-application/getting-started",
"web-application",
{
label: "How-to",
autogenerate: { directory: "web-application/how-to" },
Expand Down Expand Up @@ -301,7 +307,7 @@ export default defineConfig({
},
{
label: "Write Your Own Feed Configs",
link: "/html2rss-configs",
link: "/creating-custom-feeds",
},
{
label: "About",
Expand Down
2 changes: 1 addition & 1 deletion src/content/docs/common-use-cases.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ Follow multiple open source projects and their updates.
1. **Identify the websites** you want to follow
2. **Check our [Feed Directory](/feed-directory/)** to see if feeds already exist
3. **Try the [Web App](/web-application/getting-started)** to create feeds easily
4. **Learn advanced techniques** with our [Config Guide](/html2rss-configs/)
4. **Learn advanced techniques** with our [Config Guide](/creating-custom-feeds)

---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,19 +1,21 @@
---
title: "Write Your Own Feed Configs"
description: "Step-by-step guide to writing YAML configuration files for custom RSS feeds. Take control when ready-made feeds aren't enough."
title: "Creating Custom Feeds"
description: "Learn to write custom YAML configurations for RSS feeds when auto-sourcing isn't enough."
sidebar:
order: 2
---

When ready-made feeds aren't enough, you can write your own configuration files to create custom RSS feeds for any website. This guide shows you how to take full control with YAML configs.
When auto-sourcing isn't enough, you can write your own configuration files to create custom RSS feeds for any website. This guide shows you how to take full control with YAML configs.

**Prerequisites:** You should be familiar with the [Web Application](/web-application/getting-started) or [Ruby Gem](/ruby-gem/installation) before diving into custom configurations.
**Prerequisites:** You should be familiar with the [Getting Started](/getting-started) guide before diving into custom configurations.

---

## When to Use Custom Configs

**Use custom configs when:**

- **Ready-made feeds don't exist** for the website you want to follow
- **Auto-sourcing doesn't work** for the website you want to follow
- **Existing feeds are incomplete** or missing important content
- **You need specific formatting** or data extraction
- **The website has complex structure** that requires custom selectors
Expand All @@ -23,27 +25,6 @@ When ready-made feeds aren't enough, you can write your own configuration files

---

## Quick Start

**Need a feed right now?**

1. **[Try the web app](/web-application/getting-started)** - No coding required
2. **[Browse ready-made feeds](/feed-directory/)** - Use what others created
3. **Create a config** - When you need custom control (see below)

**Ready to create a config?** Jump to [Your First Config](#your-first-config).

### For html2rss-web Users

If you're using html2rss-web, you can add your config to the `feeds.yml` file:

1. **Find the example** - Check out the [`example` feed config](https://github.com/html2rss/html2rss-web/blob/master/config/feeds.yml#L9)
2. **Add your config** to the `feeds.yml` file
3. **Test it** by visiting your html2rss-web instance
4. **Debug** using browser developer tools if needed

---

## How It Works

A config file is a simple "recipe" that tells html2rss:
Expand Down Expand Up @@ -87,15 +68,6 @@ This says: "Find each article, get the title from the h2 anchor, and get the lin

**Need more details?** Check our [complete guide to selectors](/ruby-gem/reference/selectors/) for all the options.

## Configuration Options

html2rss-web supports all the same configuration options as the html2rss Ruby gem:

- **Basic selectors** for title, description, and links
- **Advanced features** like custom headers and dynamic parameters
- **Multiple strategies** for different types of websites
- **Post-processing** to clean up extracted content

---

## Your First Config
Expand All @@ -119,41 +91,63 @@ selectors:
attribute: href
```

**Step 3:** Test it with the [web app](/web-application/) or [Ruby gem](/ruby-gem/installation).
**Step 3:** Test it with your html2rss-web instance or the [Ruby gem](/ruby-gem/installation).

**Need help?** See our [detailed tutorial](/ruby-gem/tutorials/your-first-feed/) or [troubleshooting guide](/troubleshooting/troubleshooting) for common issues.
**Need help?** See our [troubleshooting guide](/troubleshooting/troubleshooting) for common issues.

---

## Troubleshooting
## Configuration Options

**Common issues when writing configs:**
html2rss supports many configuration options:

- **No items found?** Check your selectors with browser tools (F12) - the `items.selector` might not match the page structure
- **Invalid YAML?** Use spaces, not tabs, and ensure proper indentation
- **Website not loading?** Check the URL and try accessing it in your browser
- **Missing content?** Some websites load content with JavaScript - you may need to use the `browserless` strategy
- **Wrong data extracted?** Verify your selectors are pointing to the right elements
- **Basic selectors** for title, description, and links
- **Advanced features** like custom headers and dynamic parameters
- **Multiple strategies** for different types of websites
- **Post-processing** to clean up extracted content

**Need more help?** See our [comprehensive troubleshooting guide](/troubleshooting/troubleshooting) or ask in [GitHub Discussions](https://github.com/orgs/html2rss/discussions).
**See our [Ruby Gem Reference](/ruby-gem/reference/)** for complete documentation.

---

## Advanced Features
## Testing Your Config

**Before sharing your config, test it:**

**Dynamic parameters, custom headers, and more:** See our [advanced features guide](/ruby-gem/how-to/advanced-features/).
1. **Test with Ruby gem:**
```bash
html2rss feed your-config.yml
```

2. **Test with web app:** Add your config to the `feeds.yml` file and restart your instance

3. **Check the output:** Make sure all items have titles, links, and descriptions

---

## Contributing
## Sharing Your Config

**Share your config with the community:**
**Help the community by sharing your config:**

1. Go to [html2rss-configs on GitHub](https://github.com/html2rss/html2rss-configs)
2. Click "Fork" → "Add file" → Create `domain.com.yml`
3. Paste your config → "Commit new file" → "Open pull request"

**Need help?** See our [contribution guide](https://github.com/html2rss/html2rss-configs/blob/main/CONTRIBUTING.md).
**Need help?** See our [contribution guide](/get-involved/contributing) for detailed instructions.

---

## Troubleshooting

**Common issues when writing configs:**

- **No items found?** Check your selectors with browser tools (F12) - the `items.selector` might not match the page structure
- **Invalid YAML?** Use spaces, not tabs, and ensure proper indentation
- **Website not loading?** Check the URL and try accessing it in your browser
- **Missing content?** Some websites load content with JavaScript - you may need to use the `browserless` strategy
- **Wrong data extracted?** Verify your selectors are pointing to the right elements

**Need more help?** See our [comprehensive troubleshooting guide](/troubleshooting/troubleshooting) or ask in [GitHub Discussions](https://github.com/orgs/html2rss/discussions).

---

Expand All @@ -166,23 +160,11 @@ selectors:
**For Beginners:**

- **[Browse the Feed Directory](/feed-directory/)** - See real-world examples
- **[Try html2rss-web](/web-application/getting-started)** - Create feeds without coding
- **[Learn more about selectors](/ruby-gem/reference/selectors/)** - Master CSS selectors
- **[Submit your config via GitHub Web](https://github.com/html2rss/html2rss-configs)** - No Git knowledge required!

**For Contributors:**

- **[Submit your config via GitHub Web](https://github.com/html2rss/html2rss-configs)** - No Git knowledge required!
- **[Browse existing configs](https://github.com/html2rss/html2rss-configs/tree/master/lib/html2rss/configs)** - See real examples
- **[Join discussions](https://github.com/orgs/html2rss/discussions)** - Connect with other users

**For Developers:**

- **[Ruby Gem Documentation](/ruby-gem/)** - Full API reference
- **[Advanced Features](/ruby-gem/how-to/advanced-features/)** - Learn advanced techniques
- **[Custom HTTP Requests](/ruby-gem/how-to/custom-http-requests/)** - Handle complex scenarios

### Need More Help?

- **[Troubleshooting](/troubleshooting/)** - Get help with common issues
- **[Community Forum](https://github.com/html2rss/html2rss-configs/discussions)** - Ask questions and share ideas
- **[Report Issues](https://github.com/html2rss/html2rss-configs/issues)** - Found a bug? Let us know!
- **[Learn advanced features](/ruby-gem/how-to/advanced-features/)** - Take your configs to the next level
39 changes: 17 additions & 22 deletions src/content/docs/get-involved/contributing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,44 +15,39 @@ Before you begin, please read our [Code of Conduct](https://github.com/html2rss/

## How to Contribute

Here are some of the ways you can contribute to the `html2rss` project:
Here are the main ways you can contribute to the `html2rss` project:

### 1. Create a Feed Config
### 1. Create a Feed Config (Most Popular!)

Are you missing an RSS feed for a website? You can create your own feed config and share it with the community. It's a great way to get started with `html2rss` and help other users.

The html2rss "ecosystem" is a community project. We welcome contributions of all kinds. This includes new feed configs, suggesting and implementing features, providing bug fixes, documentation improvements, and any other kind of help.
**The easiest way to contribute:**

Which way you choose to add a new feed config is up to you. You can do it manually. Please [submit a pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork)!
1. Go to [html2rss-configs on GitHub](https://github.com/html2rss/html2rss-configs)
2. Click "Fork" → "Add file" → Create `domain.com.yml`
3. Paste your config → "Commit new file" → "Open pull request"

After you're done, you can test your feed config by running `bundle exec html2rss feed lib/html2rss/configs/<domainname.tld>/<path>.yml`.
**Need help writing configs?** See our [Creating Custom Feeds](/creating-custom-feeds) guide.

#### Preferred way: manually
**Want to test your config first?** Use the [Ruby gem](/ruby-gem/installation) to test it locally:

1. Fork the `html2rss-config` git repository and run `bundle install` (you need to have Ruby >= 3.3 installed).
2. Create a new folder and file following this convention: `lib/html2rss/configs/<domainname.tld>/<path>.yml`
3. Create the feed config in the `<path>.yml` file.
4. Add this spec file in the `spec/html2rss/configs/<domainname.tld>/<path>_spec.rb` file.

```ruby
RSpec.describe '<domainname.tld>/<path>' do
include_examples 'config.yml', described_class
end
```bash
html2rss feed your-config.yml
```

### 2. Improve this Website
### 2. Host a Public Instance

This website is built with Astro and Starlight and is hosted on GitHub Pages. If you have any ideas for improving the documentation or the design, we'd love to hear from you.
The [`html2rss-web`](https://github.com/html2rss/html2rss-web) project is a web application that provides RSS feeds with stable HTTPS URLs. You can host your own public instance to help other users access feeds.

[**Find the source code on GitHub**](https://github.com/html2rss/html2rss.github.io)
[**Learn how to host a public instance**](/web-application/how-to/deployment)

### 3. Host a Public Instance
### 3. Improve the Website

The [`html2rss-web`](https://github.com/html2rss/html2rss-web) project is a web application that allows you to create and manage your RSS feeds through a user-friendly interface. You can host your own public instance to help other users create feeds.
This website is built with Astro and Starlight and is hosted on GitHub Pages. If you have any ideas for improving the documentation or the design, we'd love to hear from you.

[**Learn how to host a public instance**](/web-application/how-to/deployment)
[**Find the source code on GitHub**](https://github.com/html2rss/html2rss.github.io)

### 4. Improve the `html2rss` Gem
### 4. Improve the `html2rss` Engine

Are you a Ruby developer? You can help us improve the core `html2rss` gem. Whether you're fixing a bug, adding a new feature, or improving the documentation, your contributions are welcome.

Expand Down
1 change: 1 addition & 0 deletions src/content/docs/get-involved/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ sidebar:
Engage with the `html2rss` project. Contribute and connect with the community.

- [**Project Roadmap**](https://github.com/orgs/html2rss/projects/3/views/1): View current work, plans, and priorities.
- [**Self-Host Your Own Instance**](/get-involved/self-hosting): Take control of your information diet and join the decentralized web movement.
- [**Report Bugs & Discuss Features**](/get-involved/issues-and-features): Report bugs or propose features.
- [**Join Community Discussions**](/get-involved/discussions): Connect with users and contributors.
- [**Contribute to html2rss**](/get-involved/contributing): Contribute code, documentation, or feed configurations.
Expand Down
40 changes: 40 additions & 0 deletions src/content/docs/get-involved/self-hosting.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
title: "Self-Host Your Own Instance"
description: "Take control of your information diet. Host your own html2rss instance and join the decentralized web movement."
sidebar:
order: 3
---

Turn any website into an RSS feed. Self-host your own instance and take back control of your information diet.

## Quick Start

1. Install [Docker](https://docs.docker.com/get-docker/).
2. Create a directory for html2rss:
```bash
mkdir html2rss-web && cd html2rss-web
```
3. Download the configuration:
```bash
curl -O https://raw.githubusercontent.com/html2rss/html2rss-web/master/docker-compose.yml
curl -O https://raw.githubusercontent.com/html2rss/html2rss-web/master/config/feeds.yml
```
4. Start the instance:
```bash
docker compose up -d
```
5. Visit [http://localhost:3000](http://localhost:3000)
You should see the html2rss-web interface.

## Next Steps

- [Secure your instance](/web-application/how-to/deployment#security)
- [Enable HTTPS](/web-application/how-to/deployment#https)
- [Troubleshoot issues](/troubleshooting/troubleshooting)
- [Join the community](https://github.com/orgs/html2rss/discussions)

---

## License

[MIT](https://github.com/html2rss/html2rss/blob/main/LICENSE)
33 changes: 33 additions & 0 deletions src/content/docs/getting-started.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
title: "Getting Started"
description: "Choose the fastest path to start using html2rss and jump to the detailed setup that fits your needs."
sidebar:
order: 1
---

html2rss makes it easy to follow any website through RSS. Use this page as your launchpad and choose the path that matches the amount of control and effort you want.

## Pick Your Path

### Try html2rss instantly

- **Use a hosted instance.** Visit a [community-maintained html2rss-web instance](https://github.com/html2rss/html2rss-web/wiki/Instances) and start generating feeds directly in your browser—no installation required.
- **Browse existing feeds.** Our [Feed Directory](/feed-directory/) curates examples you can import into your reader right away.

### Run your own instance

- **Follow the full installation walkthrough.** The [html2rss-web getting started guide](/web-application/getting-started/) covers prerequisites, Docker setup, and first-run checks step by step.
- **Bring your own configuration.** Once your server is up, expand it with the [Creating Custom Feeds](/creating-custom-feeds) guide or explore the [How-to collection](/web-application/how-to/) for common tasks.

### Grow with the community

- **Troubleshoot or ask for help.** Visit the [Troubleshooting guide](/troubleshooting/troubleshooting) or join the [community discussions](https://github.com/orgs/html2rss/discussions) when you need a hand.
- **Share and collaborate.** Learn how to contribute and self-host with the [Get Involved](/get-involved/) section.

## Next steps

1. Decide whether you want to try a public instance or self-host.
2. Follow the detailed instructions linked above.
3. Add the generated RSS URLs to your favourite feed reader and stay up to date.

Ready for the full walkthrough? Head over to the [html2rss-web installation guide](/web-application/getting-started/) whenever you're set to deploy your own instance.
Loading
Loading