-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #5 from caffeine-addictt/dev
v0.1.0 release
- Loading branch information
Showing
9 changed files
with
566 additions
and
5 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,166 @@ | ||
# Exceptions | ||
|
||
> [!NOTE] | ||
> Exceptions from Python's `threading` module are not included | ||
<br /> | ||
<details> | ||
<summary>Jump to</summary> | ||
<ul> | ||
<li><a href='#ignoring-exceptions'> Ignoring Exceptions</a></li> | ||
<li><a href='#exceptions-1'> Exceptions </a></li> | ||
</ul> | ||
</details> | ||
|
||
|
||
Don't have the thread library? [See here](./getting-started.md) for installing thread | ||
|
||
--- | ||
|
||
## Ignoring exceptions | ||
|
||
When initializing a thread, you can parse a [**suppress_errors**](./threading.md#parameters) bool.<br /> | ||
By default it is false, but if set to true, exceptions will not be propagated but just stored within `Thread._errors` | ||
|
||
When initializing a thread, you can parse a [**ignore_errors**](./threading.md#parameters) sequence.<br /> | ||
By default it is an empty tuple.<br /> | ||
Ignored errors will not be propagated and not stored within `Thread._errors` | ||
|
||
<br /> | ||
|
||
### Example | ||
```py | ||
from thread import Thread | ||
|
||
def bad_function(): | ||
raise RuntimeError('>:cc') | ||
|
||
|
||
# Normal behaviour | ||
thread0 = Thread( | ||
target = bad_function | ||
) | ||
thread0.start() | ||
thread0.join() | ||
# exit(1) RuntimeError(':<<') | ||
|
||
|
||
# Suppress exceptions | ||
thread1 = Thread( | ||
target = bad_function, | ||
suppress_errors = True | ||
) | ||
thread1.start() | ||
thread1.join() | ||
print(thread1._errors) # list[RuntimeError('>:cc')] | ||
|
||
|
||
# Ignore error | ||
thread2 = Thread( | ||
target = bad_function, | ||
ignore_errors = [RuntimeError] | ||
) | ||
thread2.start() | ||
thread2.join() | ||
print(thread2._errors) # list[] | ||
|
||
|
||
# Non-ignored error | ||
thread3 = Thread( | ||
target = bad_function, | ||
ignore_errors = [ValueError] | ||
) | ||
thread3.start() | ||
thread3.join() | ||
# exit(1) RuntimeError(':<<') | ||
|
||
|
||
# Non-ignored error with suppressing | ||
thread4 = Thread( | ||
target = bad_function, | ||
ignore_errors = [ValueError], | ||
suppress_errors = True | ||
) | ||
thread4.start() | ||
thread4.join() | ||
print(thread4._errors) # list[RuntimeError(':<<')] | ||
|
||
|
||
# Ignored error with suppressing | ||
thread5 = Thread( | ||
target = bad_function, | ||
ignore_errors = [RuntimeError], | ||
suppress_errors = True | ||
) | ||
thread5.start() | ||
thread5.join() | ||
print(thread5._errors) # list[] | ||
``` | ||
|
||
<br /> | ||
|
||
|
||
## Exceptions | ||
|
||
The list of exceptions that can be thrown | ||
|
||
<br /> | ||
|
||
|
||
### ThreadErrorBase | ||
|
||
This is the base exception class that all exceptions inherit from | ||
|
||
<br /> | ||
|
||
|
||
### ThreadStillRunningError | ||
|
||
This is raised when you attempt to invoke a method which requries the thread to not be running, but is running. | ||
> You can wait for the thread to terminate with [**Thread.join()**](./threading.md#methods) before invoking the method | ||
> You can check if the thread is running with [**Thread.is_alive()**](threading.md#methods) before invoking the method | ||
<br /> | ||
|
||
|
||
### ThreadNotRunningError | ||
|
||
This is raised when you attempt to invoke a method which requires the thread to be running, but isn't | ||
> You can run the thread with [**Thread.start()**](threading.md#methods) before invoking the method | ||
<br /> | ||
|
||
|
||
### ThreadNotInitializedError | ||
|
||
This is raised when you attempt to invoke a method which requires the thread to be initialized, but isn't | ||
> You can initialize and start the thread with [**Thread.start()**](threading.md#methods) before invoking the method | ||
<br /> | ||
|
||
|
||
### HookRuntimeError | ||
|
||
This is raised when hooks raise an exception<br /> | ||
Conforms to when thread is ran with errors suppressed or ignored | ||
|
||
Example traceback | ||
```text | ||
HookRuntimeError: Encountered runtime errors in hooks | ||
1. my_function | ||
>>>>>>>>>> | ||
/usr/home/proj/main.py:50 | ||
ZeroDivisionError: | ||
<<<<<<<<<< | ||
2. my_otherfunction | ||
>>>>>>>>>> | ||
ImportError: | ||
<<<<<<<<<< | ||
``` | ||
|
||
<br /> | ||
|
||
[See here](./threading.md) for how to using the `thread.Thread` class! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
# Getting started with thread | ||
|
||
Thanks for using thread! I hope you find it useful for your projects. | ||
|
||
Here's to you get started. | ||
|
||
--- | ||
|
||
## Prerequisites | ||
|
||
* Python 3.11+ | ||
|
||
The project is quite heavily type-annotated, and we use `Concatenate[Any, ...]` in some function declarations. | ||
However `Python <=3.10` does not support `...` being the last argument as laid out in [this stack overflow question](https://stackoverflow.com/questions/74893354/is-literal-ellipsis-really-valid-as-paramspec-last-argument). | ||
|
||
If possible, I may release a sister version of thread that is compatible with `Python 3.9+` in the future, but for the time being, | ||
support will extend only from Python 3.11+ | ||
|
||
<br /> | ||
|
||
|
||
## Installing | ||
|
||
### From pip (Recommended) | ||
```sh | ||
pip install thread | ||
``` | ||
|
||
### Building from source (Not Recommended) | ||
```sh | ||
# Clone this repository | ||
git clone https://github.com/caffeine-addictt/thread | ||
|
||
# Install dependencies | ||
pip install poetry | ||
|
||
# Build the distribution | ||
python3 -m poetry build | ||
|
||
# Install the distribution | ||
pip install -e . | ||
``` | ||
|
||
<br /> | ||
|
||
|
||
## Importing thread | ||
|
||
Import thread into your .py file | ||
```py | ||
import thread | ||
``` | ||
|
||
Now you have successfully installed thread! | ||
|
||
[See here](./threading.md) for how to using the `thread.Thread` class! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,143 @@ | ||
# Parallel Processing Documentation | ||
|
||
I will lay out how to use the `thread.ParallelProcessing` class! | ||
|
||
<br /> | ||
<details> | ||
<summary>Jump to</summary> | ||
<ul> | ||
<li><a href='#how-does-it-work'> How it works </a></li> | ||
<li><a href='#initializing-a-parallel-process'> Initialize a Parallel Process </a></li> | ||
<li><a href='#parameters'> Parameters </a></li> | ||
<li><a href='#attributes'> Attributes </a></li> | ||
<li><a href='#methods'> Class Methods </a></li> | ||
</ul> | ||
</details> | ||
|
||
|
||
Don't have the thread library? [See here](./getting-started.md) for installing thread | ||
|
||
--- | ||
|
||
## Importing the class | ||
|
||
```py | ||
from thread import ParallelProcessing | ||
``` | ||
|
||
<br /> | ||
|
||
|
||
## How does it work? | ||
|
||
Parallel Processing works best by optimizing data processing with large datasets. | ||
|
||
What it does: | ||
```py | ||
dataset = [1, 2, 3, ..., 2e10] | ||
|
||
# Splits into chunks as evenly as possible | ||
# thread_count = min(max_threads, len(dataset)) | ||
# n == len(chunks) == len(thread_count) | ||
chunks = [[1, 2, 3, ...], [50, 51, 52, ...], ...] | ||
|
||
# Initialize and run n threads | ||
# each thread handles 1 chunk of data and parses it into the function | ||
|
||
# processed data is arranged back in order | ||
|
||
# processed data is returned as a list[Data_Out] | ||
``` | ||
|
||
<br /> | ||
|
||
|
||
## Initializing a parallel process | ||
|
||
A simple example | ||
```py | ||
def my_data_processor(Data_In) -> Data_Out: ... | ||
|
||
# Reccommended way | ||
my_processor = ParallelProcessing( | ||
function = my_data_processor, | ||
dataset = [i in range(0, n)] | ||
) | ||
|
||
# OR | ||
# Not the reccommended way | ||
my_processor = ParallelProcessing(my_data_processor, [i in range(0, n)]) | ||
``` | ||
|
||
It can be ran by invoking the `start()` method | ||
```py | ||
my_processor.start() | ||
``` | ||
|
||
> [!NOTE] | ||
> The **threading.ParallelProcessing()** class from python will only be initialized when **start()** is invoked | ||
<br /> | ||
|
||
|
||
### Parameters | ||
|
||
* function : (DataProcessor, dataset, *args, **kwargs) -> Any | Data_Out | ||
> This should be a function that takes in a dataset and/or anything and returns Data_Out and/or anything | ||
* dataset : Sequence[Data_In] = () | ||
> This should be an interable sequence of arguments parsed to the `DataProcessor` function<br /> | ||
> (e.g. tuple('foo', 'bar')) | ||
* *overflow_args : Overflow_In | ||
> These are arguments parsed to [**thread.Thread**](./threading.md#parameters) | ||
* **overflow_kwargs : Overflow_In | ||
> These are arguments parsed to [**thread.Thread**](./threading.md#parameters)<br /> | ||
> [!NOTE] | ||
> If `args` is present, then it will automatically be removed from kwargs and joined with `overflow_args` | ||
* **Raises** AssertionError: max_threads is invalid | ||
|
||
<br /> | ||
|
||
|
||
### Attributes | ||
|
||
These are attributes of [`ParallelProcessing`](#importing-the-class) class | ||
|
||
* results : List[Data_Out] | ||
> The result value | ||
> **Raises** [`ThreadNotInitializedError`](./exceptions.md#threadNotInitializedError) | ||
> **Raises** [`ThreadNotRunningError`](./exceptions.md#threadnotrunningerror) | ||
> **Raises** [`ThreadStillRunningError`](./exceptions.md#threadStillRunningError) | ||
<br /> | ||
|
||
|
||
### Methods | ||
|
||
These are methods of [`ParallelProcessing`](#importing-the-class) class | ||
|
||
* start : () -> None | ||
> Initializes the threads and starts it<br /> | ||
> **Raises** [`ThreadStillRunningError`](./exceptions.md#threadStillRunningError) | ||
* is_alive : () -> bool | ||
> Indicates whether the thread is still alive<br /> | ||
> **Raises** [`ThreadNotInitializedError`](./exceptions.md#threadNotInitializedError) | ||
* get_return_values : () -> Data_Out | ||
> Halts the current thread execution until the thread completes | ||
* join : () -> JoinTerminatedStatus | ||
> Halts the current thread execution until a thread completes or exceeds the timeout | ||
> **Raises** [`ThreadNotInitializedError`](./exceptions.md#threadNotInitializedError) | ||
> **Raises** [`ThreadNotRunningError`](./exceptions.md#threadnotrunningerror) | ||
<br /> | ||
|
||
|
||
Now you know how to use the [`ParallelProcessing`](#importing-the-class) class! | ||
|
||
[See here](./parallel-processing.md) for how to using the `thread.ParallelProcessing` class! |
Oops, something went wrong.