Skip to content

Latest commit

 

History

History
235 lines (134 loc) · 6.04 KB

Parallel.md

File metadata and controls

235 lines (134 loc) · 6.04 KB
Raw

Parallel

Tweak of joblib.Parallel that propagates the scikit-learn configuration.

This subclass of joblib.Parallel ensures that the active configuration (thread-local) of scikit-learn is propagated to the parallel workers for the duration of the execution of the parallel tasks.

The API does not change and you can refer to joblib.Parallel documentation for more details.

Python Reference

Constructors

constructor()

Signature

new Parallel(opts?: object): Parallel;

Parameters

Name Type
opts? object

Returns

Parallel

Defined in: generated/utils/parallel/Parallel.ts:25

Methods

dispatch_next()

Dispatch more data for parallel processing

This method is meant to be called concurrently by the multiprocessing callback. We rely on the thread-safety of dispatch_one_batch to protect against concurrent consumption of the unprotected iterator.

Signature

dispatch_next(opts: object): Promise<any>;

Parameters

Name Type
opts object

Returns

Promise<any>

Defined in: generated/utils/parallel/Parallel.ts:99

dispatch_one_batch()

Prefetch the tasks for the next batch and dispatch them.

The effective size of the batch is computed here. If there are no more jobs to dispatch, return false, else return true.

The iterator consumption and dispatching is protected by the same lock so calling this function should be thread safe.

Signature

dispatch_one_batch(opts: object): Promise<any>;

Parameters

Name Type
opts object

Returns

Promise<any>

Defined in: generated/utils/parallel/Parallel.ts:129

dispose()

Disposes of the underlying Python resources.

Once dispose() is called, the instance is no longer usable.

Signature

dispose(): Promise<void>;

Returns

Promise<void>

Defined in: generated/utils/parallel/Parallel.ts:80

format()

Return the formatted representation of the object.

Signature

format(opts: object): Promise<any>;

Parameters

Name Type
opts object

Returns

Promise<any>

Defined in: generated/utils/parallel/Parallel.ts:155

init()

Initializes the underlying Python resources.

This instance is not usable until the Promise returned by init() resolves.

Signature

init(py: PythonBridge): Promise<void>;

Parameters

Name Type
py PythonBridge

Returns

Promise<void>

Defined in: generated/utils/parallel/Parallel.ts:43

print_progress()

Display the process of the parallel execution only a fraction of time, controlled by self.verbose.

Signature

print_progress(opts: object): Promise<any>;

Parameters

Name Type
opts object

Returns

Promise<any>

Defined in: generated/utils/parallel/Parallel.ts:181

Properties

_isDisposed

boolean = false

Defined in: generated/utils/parallel/Parallel.ts:23

_isInitialized

boolean = false

Defined in: generated/utils/parallel/Parallel.ts:22

_py

PythonBridge

Defined in: generated/utils/parallel/Parallel.ts:21

id

string

Defined in: generated/utils/parallel/Parallel.ts:18

opts

any

Defined in: generated/utils/parallel/Parallel.ts:19

Accessors

py

Signature

py(): PythonBridge;

Returns

PythonBridge

Defined in: generated/utils/parallel/Parallel.ts:30

Signature

py(pythonBridge: PythonBridge): void;

Parameters

Name Type
pythonBridge PythonBridge

Returns

void

Defined in: generated/utils/parallel/Parallel.ts:34