aiida_fireworks_scheduler package

Subpackages

Submodules

aiida_fireworks_scheduler.awareness module

Runtime scheduler awareness :noindex:

class aiida_fireworks_scheduler.awareness.DummyAwareness(*args, **kwargs)[source]

Bases: aiida_fireworks_scheduler.awareness.SchedulerAwareness

DummyAwareness for running jobs locally

DEFAULT_REMAINING_TIME = 2592000
__init__(*args, **kwargs)[source]

Instantiate an DummyAwareness - as if we have lots of time to run

__module__ = 'aiida_fireworks_scheduler.awareness'
get_n_cpus()[source]

Get the number of CPUS

get_remaining_seconds()[source]

Get the remaining time. Default to 30 days

property is_in_job

Are we inside an scheduler job - always true for a dummy

property job_id

The id of the job

class aiida_fireworks_scheduler.awareness.SGEAwareness(*args, **kwargs)[source]

Bases: aiida_fireworks_scheduler.awareness.SchedulerAwareness

SGE runtime awareness

__init__(*args, **kwargs)[source]

Initialise the SGEAwareness object

__module__ = 'aiida_fireworks_scheduler.awareness'
_readtask_info()[source]

Read more detailed task infomation

get_end_time(refresh=False)[source]

Return the time when the job is expected to finish

get_max_run_seconds()[source]

Return the maximum run time in seconds

get_n_cpus()[source]

Get the number of CPUS

get_remaining_seconds()[source]

Return the remaining time in seconds

get_start_time(refresh=False)[source]

Return the start time of this job

property job_id

ID of the job

class aiida_fireworks_scheduler.awareness.SchedulerAwareness(*args, **kwargs)[source]

Bases: object

Scheduler object

__dict__ = mappingproxy({'__module__': 'aiida_fireworks_scheduler.awareness', '__doc__': 'Scheduler object', '__init__': <function SchedulerAwareness.__init__>, 'get_n_cpus': <function SchedulerAwareness.get_n_cpus>, 'user_name': <property object>, 'get_remaining_seconds': <function SchedulerAwareness.get_remaining_seconds>, 'is_in_job': <property object>, 'job_id': <property object>, 'get_awareness': <classmethod object>, '__dict__': <attribute '__dict__' of 'SchedulerAwareness' objects>, '__weakref__': <attribute '__weakref__' of 'SchedulerAwareness' objects>, '__annotations__': {}})
__init__(*args, **kwargs)[source]

SchedulerAwareness object for accessing information from the scheduler

__module__ = 'aiida_fireworks_scheduler.awareness'
__weakref__

list of weak references to the object (if defined)

classmethod get_awareness()[source]

Automatically get the specialised Awareness instance

get_n_cpus()[source]

Return the number of CPUS in this job

get_remaining_seconds()[source]

Get the remaining time before this job gets killed

property is_in_job

Return wether I am in a remote job

property job_id

ID of the current job

property user_name

Return the name of the current user

class aiida_fireworks_scheduler.awareness.SlurmAwareness[source]

Bases: aiida_fireworks_scheduler.awareness.SchedulerAwareness

SlurmAwareness object for storing and extracting information in slurm

__init__()[source]

Initialise and SlurmAwareness instance

__module__ = 'aiida_fireworks_scheduler.awareness'
_readtask_info()[source]

A function to extract information from environmental variables SLURM_JOB_ID unique to each job Return an dictionnary contain job information. If not in slurm, return None TODO Refactor avoid saving intermediate file

_task_info = None
_warning = 0
get_end_time()[source]

Query the end time of an job Return a datetime object

get_n_cpus()[source]

Return number of CPU allocated

get_remaining_seconds()[source]

Return the remaining time in seconds

property is_in_job

Wether I am in a job

property job_id

Id of the job

aiida_fireworks_scheduler.cmdline module

aiida_fireworks_scheduler.common module

Module for common stuff like global variables…

aiida_fireworks_scheduler.fworker module

Class for specialised worker for running AiiDA jobs include multiprocessing (mpinp) and walltime limit in the query

class aiida_fireworks_scheduler.fworker.AiiDAFWorker(computer_id, mpinp, *args, username='AIIDA_USER', **kwargs)[source]

Bases: fireworks.core.fworker.FWorker

Specialised worker for running AiiDA related jobs

SECONDS_SAFE_INTERVAL = 60
__abstractmethods__ = frozenset({})
__init__(computer_id, mpinp, *args, username='AIIDA_USER', **kwargs)[source]

Instantiate a AiiDAFWorker object. The worker selects the jobs to run using the criteria defined in the constructor

Parameters
  • computer_id – Hostname of the computer

  • username – User name for the computer

  • mpinp – the number of MPI processes to be launched. this constraint will be ignored if is is set to -1 or 0.

The rest of the arguments will be passed to the FWorker.

__module__ = 'aiida_fireworks_scheduler.fworker'
_abc_impl = <_abc_data object>
classmethod from_dict(*args, **kwargs)
property query

Query used for selecting fireworks

property seconds_left

How long this job is going to be alive.

to_dict(*args, **kwargs)

aiida_fireworks_scheduler.fwscheduler module

Specialised scheduler to interface with Fireworks

class aiida_fireworks_scheduler.fwscheduler.FwJobResource(**kwargs)[source]

Bases: aiida.schedulers.datastructures.ParEnvJobResource

JobResource for the FwScheduler based on ParEnvJobResource. The difference is that the default parallel_env file default to “mpi” here, and it is OK to have it not set.

__abstractmethods__ = frozenset({})
__module__ = 'aiida_fireworks_scheduler.fwscheduler'
_abc_impl = <_abc_data object>
classmethod validate_resources(**kwargs)[source]

Validate the resources against the job resource class of this scheduler.

Parameters

kwargs – dictionary of values to define the job resources

Returns

attribute dictionary with the parsed parameters populated

Raises

ValueError – if the resources are invalid or incomplete

class aiida_fireworks_scheduler.fwscheduler.FwScheduler(launchpad=None)[source]

Bases: aiida.schedulers.scheduler.Scheduler

Scheduler that interfaces with fireworks.LaunchPad

FRESH_ENV = True
__abstractmethods__ = frozenset({})
__init__(launchpad=None)[source]
__module__ = 'aiida_fireworks_scheduler.fwscheduler'
_abc_impl = <_abc_data object>
_features = {'can_query_by_user': False}
_get_joblist_command(jobs=None, user=None)[source]

Return the command to get the most complete description possible of currently active jobs.

Note

Typically one can pass only either jobs or user, depending on the specific plugin. The choice can be done according to the value returned by self.get_feature(‘can_query_by_user’)

Parameters
  • jobs – either None to get a list of all jobs in the machine, or a list of jobs.

  • user – either None, or a string with the username (to show only jobs of the specific user).

_get_kill_command(jobid)[source]

Return the command to kill the job with specified jobid.

_get_submit_command(submit_script)[source]

Return the string to execute to submit a given script.

Warning

the submit_script should already have been bash-escaped

Parameters

submit_script – the path of the submit script relative to the working directory.

Returns

the string to execute to submit a given script.

_get_submit_script_header(job_tmpl)[source]

Return the submit script header, using the parameters from the job_tmpl.

Args:

job_tmpl: an JobTemplate instance with relevant parameters set.

Modified and simplied based on that of the SGE scheduler. We need to write the scheduler headers because they will be read in the upload step later to submit the job to fireworks’ MongoDB server.

_job_resource_class

alias of aiida_fireworks_scheduler.fwscheduler.FwJobResource

_logger = <Logger aiida.scheduler.Fw (REPORT)>
_lpad = None
_parse_joblist_output(retval, stdout, stderr)[source]

Parse the joblist output as returned by executing the command returned by _get_joblist_command method.

Returns

list of JobInfo objects, one of each job each with at least its default params implemented.

_parse_kill_output(retval, stdout, stderr)[source]

Parse the output of the kill command.

Returns

True if everything seems ok, False otherwise.

_parse_submit_output(retval, stdout, stderr)[source]

Parse the output of the submit command returned by calling the _get_submit_command command.

Returns

a string with the job ID.

get_detailed_job_info(job_id)[source]

Getting detailed job information. Does not make sense for this scheduler

get_jobs(jobs=None, user=None, as_dict=False)[source]

Return the list of currently active jobs

kill(jobid)[source]

Defuse a job in the LaunchPad

Note, for fireworks this only works for queued jobs. Need to think about how to kill running ones….

submit_from_script(working_directory, submit_script)[source]

Submit the submission script to the scheduler

This will create a WorkFlow for the job using the provided script and working directory and submit it to the LaunchPad.

Returns

return a string with the job ID in a valid format to be used for querying.

class aiida_fireworks_scheduler.fwscheduler.FwSchedulerKeepEnv(launchpad=None)[source]

Bases: aiida_fireworks_scheduler.fwscheduler.FwScheduler

A FW-base scheduler that runs the _aiidasubmit.sh in the same environment as the submission script.

This mode of operation is needed by SLURM which uses srun directives to launch job steps.

FRESH_ENV = False
__abstractmethods__ = frozenset({})
__module__ = 'aiida_fireworks_scheduler.fwscheduler'
_abc_impl = <_abc_data object>
aiida_fireworks_scheduler.fwscheduler.parse_sge_script(local_script_path)[source]

Parse the SGE script

Returns

A dictionary of the options for constructing AiiDAJobFirework

aiida_fireworks_scheduler.jobs module

Mapping AiiDA scheduler jobs to Firework

class aiida_fireworks_scheduler.jobs.AiiDAJobFirework(computer_id, username, remote_work_dir, job_name, submit_script_name, mpinp, walltime, stdout_fname, stderr_fname, fresh_env=True, priority=100)[source]

Bases: fireworks.core.firework.Firework

A Firework that encapsulate AiiDA jobs

__abstractmethods__ = frozenset({})
__init__(computer_id, username, remote_work_dir, job_name, submit_script_name, mpinp, walltime, stdout_fname, stderr_fname, fresh_env=True, priority=100)[source]

Instantiate a Firework to run jobs prepared by AiiDA daemon on the remote computer

__module__ = 'aiida_fireworks_scheduler.jobs'
_abc_impl = <_abc_data object>

Module contents

aiida_fireworks_scheduler

AiiDA plugin to allow using fireworks as the execution engine for CalcJob.