Skip to content

Commit

Permalink
Update documentation.
Browse files Browse the repository at this point in the history
  • Loading branch information
funilrys committed Aug 20, 2023
1 parent 6789b99 commit 0605e6a
Show file tree
Hide file tree
Showing 26 changed files with 1,885 additions and 177 deletions.
614 changes: 484 additions & 130 deletions PyFunceble/data/infrastructure/.PyFunceble_production.yaml

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions README.md
Expand Up @@ -8,9 +8,9 @@
[![image](https://img.shields.io/pypi/v/pyfunceble-dev.png)](https://pypi.org/project/pyfunceble-dev)
[![image](https://img.shields.io/github/issues/funilrys/PyFunceble.png)](https://github.com/funilrys/PyFunceble/issues)

[![image](https://pepy.tech/badge/pyfunceble-dev)](https://pepy.tech/project/pyfunceble-dev)
[![image](https://pepy.tech/badge/pyfunceble-dev/month)](https://pepy.tech/project/pyfunceble-dev)
[![image](https://pepy.tech/badge/pyfunceble-dev/week)](https://pepy.tech/project/pyfunceble-dev)
[![image](https://static.pepy.tech/badge/pyfunceble-dev)](https://pepy.tech/project/pyfunceble-dev)
[![image](https://static.pepy.tech/badge/pyfunceble-dev/month)](https://pepy.tech/project/pyfunceble-dev)
[![image](https://static.pepy.tech/badge/pyfunceble-dev/week)](https://pepy.tech/project/pyfunceble-dev)

**PyFunceble** aims to provide an accurate availability check through
the usage of multiple sources which are for example - to only list a
Expand Down
2 changes: 1 addition & 1 deletion docs/contributing/contributing.md
Expand Up @@ -17,7 +17,7 @@ All types of contributions are encouraged and valued. See the Table of Contents
## Code of Conduct

This project and everyone participating in it is governed by the
[PyFunceble Code of Conduct](./Code-Of-Conduct.md).
[PyFunceble Code of Conduct](./code-of-conduct.md).
By participating, you are expected to uphold this code. Please report unacceptable behavior
to <contact@funilrys.com>.

Expand Down
23 changes: 0 additions & 23 deletions docs/develop/api-references/SUMMARY.md
Expand Up @@ -152,29 +152,6 @@
* [url2netloc](converter/url2netloc.md)
* [wildcard2subject](converter/wildcard2subject.md)
* [data](data/index.md)
* [alembic](data/alembic/index.md)
* [mysql](data/alembic/mysql/index.md)
* [env](data/alembic/mysql/env.md)
* [versions](data/alembic/mysql/versions/index.md)
* [35c79626ecb9_fix_some_columns](data/alembic/mysql/versions/35c79626ecb9_fix_some_columns.md)
* [3a4c55a9320d_add_continue_table](data/alembic/mysql/versions/3a4c55a9320d_add_continue_table.md)
* [3d6f4a33cdb2_add_inactive_table](data/alembic/mysql/versions/3d6f4a33cdb2_add_inactive_table.md)
* [45713fea8097_deletion_uneeded_columns_from_whois_](data/alembic/mysql/versions/45713fea8097_deletion_uneeded_columns_from_whois_.md)
* [459a0d7b8f09_add_idna_subject_column_into_whois](data/alembic/mysql/versions/459a0d7b8f09_add_idna_subject_column_into_whois.md)
* [6f4729deaf03_delete_inactive_source_column](data/alembic/mysql/versions/6f4729deaf03_delete_inactive_source_column.md)
* [7bcf7fa64ba1_rename_created_to_created_at_and](data/alembic/mysql/versions/7bcf7fa64ba1_rename_created_to_created_at_and.md)
* [83ada95132bf_delete_the_file_table](data/alembic/mysql/versions/83ada95132bf_delete_the_file_table.md)
* [912bbcb77a6c_add_registrar_column](data/alembic/mysql/versions/912bbcb77a6c_add_registrar_column.md)
* [95dc17ddd729_introduction_of_the_session_id_column](data/alembic/mysql/versions/95dc17ddd729_introduction_of_the_session_id_column.md)
* [ade87195b0a0_base](data/alembic/mysql/versions/ade87195b0a0_base.md)
* [bef7bcaac3f2_make_id_a_bigint](data/alembic/mysql/versions/bef7bcaac3f2_make_id_a_bigint.md)
* [d8893cd406db_allow_whois_record_to_be_empty_null](data/alembic/mysql/versions/d8893cd406db_allow_whois_record_to_be_empty_null.md)
* [e04e8301d1a2_deletion_of_the_mined_table](data/alembic/mysql/versions/e04e8301d1a2_deletion_of_the_mined_table.md)
* [postgresql](data/alembic/postgresql/index.md)
* [env](data/alembic/postgresql/env.md)
* [versions](data/alembic/postgresql/versions/index.md)
* [a32ac5d66eee_initial_version](data/alembic/postgresql/versions/a32ac5d66eee_initial_version.md)
* [infrastructure](data/infrastructure/index.md)
* [database](database/index.md)
* [credential](database/credential/index.md)
* [base](database/credential/base.md)
Expand Down
9 changes: 6 additions & 3 deletions docs/gen_ref_pages.py
Expand Up @@ -4,6 +4,7 @@
Everything under /develop/api-references is actually generated by this script.
"""


import os
from pathlib import Path

Expand All @@ -30,15 +31,17 @@
elif parts[-1] == "__main__":
continue

nav[parts] = doc_path.as_posix()

identifier = ".".join(parts)

if any(f"{x}." in identifier for x in EXCLUDE_MODULES):
if os.path.exists(full_doc_path):
os.remove(full_doc_path)
if os.path.exists(full_doc_path.absolute()):
os.remove(full_doc_path.absolute())

continue

nav[parts] = doc_path.as_posix()

with mkdocs_gen_files.open(full_doc_path, "w") as file_stream:
print("::: " + identifier, file=file_stream)

Expand Down
10 changes: 5 additions & 5 deletions docs/index.md
Expand Up @@ -25,13 +25,13 @@
</p>
<p align="center">
<a href="https://pepy.tech/project/pyfunceble-dev">
<img src="https://pepy.tech/badge/pyfunceble-dev" alt="image">
<img src="https://static.pepy.tech/badge/pyfunceble-dev" alt="image">
</a>
<a href="https://pepy.tech/project/pyfunceble-dev">
<img src="https://pepy.tech/badge/pyfunceble-dev/month" alt="image">
<img src="https://static.pepy.tech/badge/pyfunceble-dev/month" alt="image">
</a>
<a href="https://pepy.tech/project/pyfunceble-dev">
<img src="https://pepy.tech/badge/pyfunceble-dev/week" alt="image">
<img src="https://static.pepy.tech/badge/pyfunceble-dev/week" alt="image">
</a>
</p>

Expand All @@ -47,9 +47,9 @@ PyFunceble can be included in your existing project through:

- its standard built-in CLI implementation.
- its [Python
API](https://pyfunceble.readthedocs.io/en/dev/api/index.html).
API](./develop/api-references).
- the [PyFunceble
web-worker](https://github.com/pyfunceble/web-worker) project that
web-worker](https://github.com/pyfunceble/web-worker) (beta) project that
provides the core functionalities of PyFunceble behind a web API.

The PyFunceble CLI can test from a hosts file, a plain list of subjects,
Expand Down
6 changes: 3 additions & 3 deletions docs/use/configuration/environment-variables.md
Expand Up @@ -74,11 +74,11 @@ read and use them. #MayTheForceBeWithYou :star:
| GITLAB_CI | Used to detect whether we are running under a GitLab CI/CD runner. |
| GITLAB_USER_ID | Used to detect whether we are running under a GitLab CI/CD runner. |
| TRAVIS_BUILD_DIR | Used to detect whether we are running under a Travis CI instance. |
| JENKINS_URL | Used to detect whether we are running under a Jenkins CI worker. |
| JENKINS_HOME | Used to detect whether we are running under a Jenkins CI worker. |
| JENKINS_URL | Used to detect whether we are running under a Jenkins CI worker. |
| JENKINS_HOME | Used to detect whether we are running under a Jenkins CI worker. |

## Beta Features

## Beta Features
!!! danger "Beware!!!"

This section is documented, but most of the features below them are not open to everyone - yet.
Expand Down
14 changes: 14 additions & 0 deletions docs/use/configuration/parameters/SUMMARY.md
@@ -0,0 +1,14 @@
- [](index.md)
- [`cli_decoding`](cli-decoding.md)
- [`cli_testing`](cli-testing.md)
- [`collection`](collection.md)
- [`debug`](debug.md)
- [`dns`](dns.md)
- [`http_codes`](http-codes.md)
- [`links`](links.md)
- [`lookup`](lookup.md)
- [`max_http_retries`](max-http-retries.md)
- [`proxy`](proxy.md)
- [`share_logs`](share-logs.md)
- [`user_agent`](user-agent.md)
- [`verify_ssl_certificates`](verify-ssl-certificates.md)
124 changes: 124 additions & 0 deletions docs/use/configuration/parameters/cli-decoding.md
@@ -0,0 +1,124 @@
# `cli_decoding`

When using the CLI, you can use some of the parameters listed below to control
the way inputed files are being decoded or interpreted.

## Overview

```yaml title=".PyFunceble.overwrite.yaml"
cli_decoding:
# Provides everything related to the decoding of input files from the CLI.

# Enable/Disable the aggressive decoding.
#
# The aggressive mode is a mode that tries to decode as much as possible without
# trying to follow any technical conventions.
#
# When decoding AdBlock filter lists, it will try to decode almost every
# domains it finds.
#
# When decoding hosts file or plain text files, it will try to convert URLS
# (e.g https://example.org/hello/world) into domains to test (e.g example.org).
#
# CLI Argument: --aggressive
aggressive: no

# Enable/Disable the (exclusive) decoding using the adblock decoder.
#
# WARNING:
# Activating this parameter in your configuration file, will make
# PyFunceble assume that it has to decode adblock files - exclusively.
#
# CLI Argument: --adblock
adblock: no

# Enable/Disable the (exclusive) decoding using the RPZ decoder.
#
# WARNING:
# Activating this parameter in your configuration file, will make
# PyFunceble assume that it has to decode RPZ files - exclusively.
#
# CLI Argument: --rpz
rpz: no

# Enable or disable the reduction of wildcards.
#
# Meaning that any caught wildcards (e.g. *.example.org) will be reduced so
# that we have a "valid" domain to test (e.g. example.org).
#
# WARNING:
# Activating this parameter will reduce wildcards (e.g *.example.org) to
# match domains (e.g ecample.org)
#
# CLI Argument: --wildcard
wildcard: no
```

## `aggressive`

Enable or disable the aggressive mode. The aggressive mode is a mode that tries
to decode as much as possible without trying to follow any technical conventions.

When using this parameter while decoding AdBlock filter lists, it will try to
decode almost every domains it finds.

However, when using this parameter while decoding hosts or plain lists, it will
also convert URLs (e.g `https://example.com/hello/world`) into domains to test
(e.g `example.com`).

**Type:** boolean

**Default Value:** `no`

**Available Values:** `yes`, `no`

**CLI Argument:** `--aggressive`

## `adblock`

Enable or disable the (exclusive) decoding of AdBlock filter lists.

!!! danger "Beware!!!!"

If you choose to activate this parameter in your configuration file,
PyFunceble will assume that any inputed files are AdBlock filter lists to
decode.

**Type:** boolean

**Default Value:** `no`

**Available Values:** `yes`, `no`

**CLI Argument:** `--adblock`

## `rpz`

Enable or disable the (exclusive) decoding of RPZ files.

!!! danger "Beware!!!!"

If you choose to active this parameter in your configuration file,
PyFunceble will assume that any inputed files are RPZ files to decode.

**Type:** boolean

**Default Value:** `no`

**Available Values:** `yes`, `no`

**CLI Argument:** `--rpz`

## `wildcard`

Enable or disable the reduction of wildcards. Meaning that any caught wildcards
(e.g. \*.example.org) will be reduced so that we have a "valid" domain to test
(e.g. example.org).

**Type:** boolean

**Default Value:** `no`

**Available Values:** `yes`, `no`

**CLI Argument:** `--wildcard`

0 comments on commit 0605e6a

Please sign in to comment.