morphogen.parallel

Parallel runs: processes on one machine, job arrays on clusters, MPI.

A run is a list of realizations (items), each with its own seed derived from the seed of the run and its indices (morphogen.seeding): the results do not depend on how the items are spread over processes, tasks or nodes.

  • Every realization leaves a record (records/<name>.json: its row of results and its rows of apparent properties) as soon as it is done. A run started again with resume skips the realizations that have a record, so that an interrupted task (a time limit on a cluster) loses at most the realizations in progress.

  • jobs processes run the realizations of a task on one machine (spawn start method, on every platform); the threads (threads, by default all the cores) are shared between them, so that the compiled kernels and the linear algebra do not oversubscribe the cores.

  • A run split in n_tasks tasks (task_id = 0 … n - 1) gives task i the items i, i + n, i + 2n...: a job array of a scheduler, or the ranks of an MPI program (run_mpi()). merge() gathers the records of all the tasks into the results tables and writes the RVE report; status() counts them.

  • job_script() writes the script of a job array for SLURM, PBS Pro, Torque, SGE, OAR or LSF, and the script of the merging job.

morphogen.parallel.THREAD_VARIABLES = ('OMP_NUM_THREADS', 'OPENBLAS_NUM_THREADS', 'MKL_NUM_THREADS', 'VECLIB_MAXIMUM_THREADS', 'NUMEXPR_NUM_THREADS')

Environment variables bounding the threads of the linear algebra libraries.

class morphogen.parallel.Item(index: int, key: tuple[int, ...], stem: str, size: float | None, ident: dict = <factory>)[source]

Bases: object

One realization of a run: its position, seed key, file stem and domain size (None: the domain of the configuration), and the columns identifying its rows.

morphogen.parallel.items(cfg) → list[Item][source]

Realizations of a run (a batch, or the sizes and realizations of an ensemble RVE study).

morphogen.parallel.task_items(all_items: list[Item], task_id: int, n_tasks: int) → list[Item][source]

The items of a task among n_tasks (task_id from 0): one item in n_tasks, so that the tasks share them evenly.

morphogen.parallel.records_dir(outdir) → Path[source]

The folder of the records of the items of a run, in its output folder.

morphogen.parallel.record_path(outdir, item: Item) → Path[source]

The record of an item of a run (records/<item>.json in the output folder).

morphogen.parallel.write_record(outdir, item: Item, record: dict) → Path[source]

Write a record atomically (a partial file never looks complete).

morphogen.parallel.read_records(outdir) → list[dict][source]

All the records of a run directory, in the order of the run.

morphogen.parallel.realize(cfg, item: Item, outdir) → dict[source]

Generate and process one realization; returns its record.

morphogen.parallel.threads_per_job(jobs: int, threads: int | None = None) → int[source]

Threads of each of jobs processes sharing threads (default: the cores).

morphogen.parallel.process_pool(jobs: int, threads: int) → Iterator[ProcessPoolExecutor][source]

Pool of jobs processes (spawn) with threads threads each.

morphogen.parallel.execute(cfg, todo: list[Item], outdir, *, jobs: int = 1, resume: bool = False) → list[dict][source]

Records of the items todo (in their order), computed by jobs processes; with resume, the items that have a record are not computed again. Each realization is a checkpoint of morphogen.stopping (a stop cancels those the processes have not started).

morphogen.parallel.rows_of(records: list[dict]) → tuple[list[dict], list[dict]][source]

Results rows and homogenization rows of records.

morphogen.parallel.run_mpi(cfg, *, jobs: int = 1, resume: bool = False) → Path[source]

Run with the ranks of an MPI program (mpirun -n N morphogen run config --mpi): rank i of N is task i; rank 0 merges the results when all are done.

morphogen.parallel.merge(outdir, *, report: bool = True) → Path[source]

Gather the records of all the tasks of a run into results.csv and homogenization.csv, and write the RVE report of an ensemble study.

morphogen.parallel.status(outdir) → dict[source]

Realizations done and missing in a run directory.

morphogen.parallel.SCHEDULERS = {'lsf': {'account': '-P {account}', 'after': ['-w', 'done({job})'], 'array': None, 'cores': '-n {cores}', 'index': '$((LSB_JOBINDEX - 1))', 'log': ['-o {logs}/%J_%I.log'], 'memory': '-M {memory}', 'name': '-J {name}[1-{count}]', 'prefix': '#BSUB', 'queue': '-q {queue}', 'submit': ['bsub'], 'time': '-W {time}'}, 'oar': {'account': '--project {account}', 'after': ['--anterior={job}'], 'array': '--array {count}', 'cores': '-l /nodes=1/core={cores},walltime={time}', 'index': '$((OAR_ARRAY_INDEX - 1))', 'log': ['-O {logs}/%jobid%.log', '-E {logs}/%jobid%.log'], 'memory': None, 'name': '-n {name}', 'prefix': '#OAR', 'queue': '-q {queue}', 'submit': ['oarsub', '-S'], 'time': None}, 'pbs': {'account': '-A {account}', 'after': ['-W', 'depend=afterok:{job}'], 'array': '-J 0-{last}', 'cores': '-l select=1:ncpus={cores}', 'index': '${PBS_ARRAY_INDEX}', 'log': ['-o {logs}/', '-j oe'], 'memory': '-l mem={memory}', 'name': '-N {name}', 'prefix': '#PBS', 'queue': '-q {queue}', 'submit': ['qsub'], 'time': '-l walltime={time}'}, 'sge': {'account': '-A {account}', 'after': ['-hold_jid', '{job}'], 'array': '-t 1-{count}', 'cores': '-pe smp {cores}', 'index': '$((SGE_TASK_ID - 1))', 'log': ['-o {logs}/', '-j y'], 'memory': '-l h_vmem={memory}', 'name': '-N {name}', 'prefix': '#$', 'queue': '-q {queue}', 'submit': ['qsub', '-terse'], 'time': '-l h_rt={time}'}, 'slurm': {'account': '--account={account}', 'after': ['--dependency=afterok:{job}'], 'array': '--array=0-{last}', 'cores': '--cpus-per-task={cores}', 'index': '${SLURM_ARRAY_TASK_ID}', 'log': ['--output={logs}/%x_%A_%a.log'], 'memory': '--mem={memory}', 'name': '--job-name={name}', 'prefix': '#SBATCH', 'queue': '--partition={queue}', 'submit': ['sbatch', '--parsable'], 'time': '--time={time}'}, 'torque': {'account': '-A {account}', 'after': ['-W', 'depend=afterokarray:{job}'], 'array': '-t 0-{last}', 'cores': '-l nodes=1:ppn={cores}', 'index': '${PBS_ARRAYID}', 'log': ['-o {logs}/', '-j oe'], 'memory': '-l mem={memory}', 'name': '-N {name}', 'prefix': '#PBS', 'queue': '-q {queue}', 'submit': ['qsub'], 'time': '-l walltime={time}'}}

Array directive, task index (0-based) and submission command of each scheduler.

class morphogen.parallel.JobOptions(tasks: int, jobs: int = 1, threads: int = 1, time: str = '01:00:00', memory: str | None = None, queue: str | None = None, account: str | None = None, name: str = 'morphogen', setup: list[str] = <factory>)[source]

Bases: object

Resources of the tasks of a job array.

property cores: int
morphogen.parallel.job_script(scheduler: str, config, outdir, opts: JobOptions, *, seed: int | None = None, executable: str | None = None) → tuple[str, str][source]

Scripts of the job array running a configuration in opts.tasks tasks (each with opts.jobs processes of opts.threads threads, resumable), and of the job merging the results.

morphogen.parallel.submit(scheduler: str, array_script: Path, merge_script: Path) → list[str][source]

Submit the job array, then the merging job after it; returns their ids.