reliure.offline

reliure.offline.main()

Small run usage exemple

reliure.offline.run(pipeline, input_gen, options={})

Run a pipeline over a input generator

>>> # if we have a simple component
>>> from reliure.pipeline import Composable
>>> @Composable
... def print_each(letters):
...     for letter in letters:
...         print(letter)
...         yield letter
>>> # that we want to run over a given input:
>>> input = "abcde"
>>> # we just have to do :
>>> res = run(print_each, input)
a
b
c
d
e

it is also possible to run any reliure pipeline this way: >>> import string >>> pipeline = Composable(lambda letters: (l.upper() for l in letters)) | print_each >>> res = run(pipeline, input) A B C D E

reliure.offline.run_parallel(pipeline, input_gen, options={}, ncpu=4, chunksize=200)

Run a pipeline in parallel over a input generator cutting it into small chunks.

>>> # if we have a simple component
>>> from reliure.pipeline import Composable
>>> # that we want to run over a given input:
>>> input = "abcde"
>>> import string
>>> pipeline = Composable(lambda letters: (l.upper() for l in letters))
>>> res = run_parallel(pipeline, input, ncpu=2, chunksize=2)
>>> #Note: res should be equals to [['C', 'D'], ['A', 'B'], ['E']]
>>> #but it seems that there is a bug with py.test and mp...