morphogen.parallel¶
Parallel runs: processes on one machine, job arrays on clusters, MPI.
A run is a list of realizations (items), each with its own seed derived from the seed
of the run and its indices (morphogen.seeding): the results do not depend on how
the items are spread over processes, tasks or nodes.
Every realization leaves a record (
records/<name>.json: its row of results and its rows of apparent properties) as soon as it is done. A run started again withresumeskips the realizations that have a record, so that an interrupted task (a time limit on a cluster) loses at most the realizations in progress.jobsprocesses run the realizations of a task on one machine (spawnstart method, on every platform); the threads (threads, by default all the cores) are shared between them, so that the compiled kernels and the linear algebra do not oversubscribe the cores.A run split in
n_taskstasks (task_id= 0 … n - 1) gives taskithe itemsi, i + n, i + 2n...: a job array of a scheduler, or the ranks of an MPI program (run_mpi()).merge()gathers the records of all the tasks into the results tables and writes the RVE report;status()counts them.job_script()writes the script of a job array for SLURM, PBS Pro, Torque, SGE, OAR or LSF, and the script of the merging job.
- morphogen.parallel.THREAD_VARIABLES = ('OMP_NUM_THREADS', 'OPENBLAS_NUM_THREADS', 'MKL_NUM_THREADS', 'VECLIB_MAXIMUM_THREADS', 'NUMEXPR_NUM_THREADS')¶
Environment variables bounding the threads of the linear algebra libraries.
- class morphogen.parallel.Item(index: int, key: tuple[int, ...], stem: str, size: float | None, ident: dict = <factory>)[source]¶
Bases:
objectOne realization of a run: its position, seed key, file stem and domain size (
None: the domain of the configuration), and the columns identifying its rows.
- morphogen.parallel.items(cfg) list[Item][source]¶
Realizations of a run (a batch, or the sizes and realizations of an ensemble RVE study).
- morphogen.parallel.task_items(all_items: list[Item], task_id: int, n_tasks: int) list[Item][source]¶
The items of a task among
n_tasks(task_idfrom 0): one item inn_tasks, so that the tasks share them evenly.
- morphogen.parallel.records_dir(outdir) Path[source]¶
The folder of the records of the items of a run, in its output folder.
- morphogen.parallel.record_path(outdir, item: Item) Path[source]¶
The record of an item of a run (
records/<item>.jsonin the output folder).
- morphogen.parallel.write_record(outdir, item: Item, record: dict) Path[source]¶
Write a record atomically (a partial file never looks complete).
- morphogen.parallel.read_records(outdir) list[dict][source]¶
All the records of a run directory, in the order of the run.
- morphogen.parallel.realize(cfg, item: Item, outdir) dict[source]¶
Generate and process one realization; returns its record.
- morphogen.parallel.threads_per_job(jobs: int, threads: int | None = None) int[source]¶
Threads of each of
jobsprocesses sharingthreads(default: the cores).
- morphogen.parallel.process_pool(jobs: int, threads: int) Iterator[ProcessPoolExecutor][source]¶
Pool of
jobsprocesses (spawn) withthreadsthreads each.
- morphogen.parallel.execute(cfg, todo: list[Item], outdir, *, jobs: int = 1, resume: bool = False) list[dict][source]¶
Records of the items
todo(in their order), computed byjobsprocesses; withresume, the items that have a record are not computed again. Each realization is a checkpoint ofmorphogen.stopping(a stop cancels those the processes have not started).
- morphogen.parallel.rows_of(records: list[dict]) tuple[list[dict], list[dict]][source]¶
Results rows and homogenization rows of records.
- morphogen.parallel.run_mpi(cfg, *, jobs: int = 1, resume: bool = False) Path[source]¶
Run with the ranks of an MPI program (
mpirun -n N morphogen run config --mpi): rankiofNis taski; rank 0 merges the results when all are done.
- morphogen.parallel.merge(outdir, *, report: bool = True) Path[source]¶
Gather the records of all the tasks of a run into
results.csvandhomogenization.csv, and write the RVE report of an ensemble study.
- morphogen.parallel.SCHEDULERS = {'lsf': {'account': '-P {account}', 'after': ['-w', 'done({job})'], 'array': None, 'cores': '-n {cores}', 'index': '$((LSB_JOBINDEX - 1))', 'log': ['-o {logs}/%J_%I.log'], 'memory': '-M {memory}', 'name': '-J {name}[1-{count}]', 'prefix': '#BSUB', 'queue': '-q {queue}', 'submit': ['bsub'], 'time': '-W {time}'}, 'oar': {'account': '--project {account}', 'after': ['--anterior={job}'], 'array': '--array {count}', 'cores': '-l /nodes=1/core={cores},walltime={time}', 'index': '$((OAR_ARRAY_INDEX - 1))', 'log': ['-O {logs}/%jobid%.log', '-E {logs}/%jobid%.log'], 'memory': None, 'name': '-n {name}', 'prefix': '#OAR', 'queue': '-q {queue}', 'submit': ['oarsub', '-S'], 'time': None}, 'pbs': {'account': '-A {account}', 'after': ['-W', 'depend=afterok:{job}'], 'array': '-J 0-{last}', 'cores': '-l select=1:ncpus={cores}', 'index': '${PBS_ARRAY_INDEX}', 'log': ['-o {logs}/', '-j oe'], 'memory': '-l mem={memory}', 'name': '-N {name}', 'prefix': '#PBS', 'queue': '-q {queue}', 'submit': ['qsub'], 'time': '-l walltime={time}'}, 'sge': {'account': '-A {account}', 'after': ['-hold_jid', '{job}'], 'array': '-t 1-{count}', 'cores': '-pe smp {cores}', 'index': '$((SGE_TASK_ID - 1))', 'log': ['-o {logs}/', '-j y'], 'memory': '-l h_vmem={memory}', 'name': '-N {name}', 'prefix': '#$', 'queue': '-q {queue}', 'submit': ['qsub', '-terse'], 'time': '-l h_rt={time}'}, 'slurm': {'account': '--account={account}', 'after': ['--dependency=afterok:{job}'], 'array': '--array=0-{last}', 'cores': '--cpus-per-task={cores}', 'index': '${SLURM_ARRAY_TASK_ID}', 'log': ['--output={logs}/%x_%A_%a.log'], 'memory': '--mem={memory}', 'name': '--job-name={name}', 'prefix': '#SBATCH', 'queue': '--partition={queue}', 'submit': ['sbatch', '--parsable'], 'time': '--time={time}'}, 'torque': {'account': '-A {account}', 'after': ['-W', 'depend=afterokarray:{job}'], 'array': '-t 0-{last}', 'cores': '-l nodes=1:ppn={cores}', 'index': '${PBS_ARRAYID}', 'log': ['-o {logs}/', '-j oe'], 'memory': '-l mem={memory}', 'name': '-N {name}', 'prefix': '#PBS', 'queue': '-q {queue}', 'submit': ['qsub'], 'time': '-l walltime={time}'}}¶
Array directive, task index (0-based) and submission command of each scheduler.
- class morphogen.parallel.JobOptions(tasks: int, jobs: int = 1, threads: int = 1, time: str = '01:00:00', memory: str | None = None, queue: str | None = None, account: str | None = None, name: str = 'morphogen', setup: list[str] = <factory>)[source]¶
Bases:
objectResources of the tasks of a job array.
- property cores: int¶
- morphogen.parallel.job_script(scheduler: str, config, outdir, opts: JobOptions, *, seed: int | None = None, executable: str | None = None) tuple[str, str][source]¶
Scripts of the job array running a configuration in
opts.taskstasks (each withopts.jobsprocesses ofopts.threadsthreads, resumable), and of the job merging the results.