aiida_fireworks_scheduler package¶
Subpackages¶
Submodules¶
aiida_fireworks_scheduler.awareness module¶
Runtime scheduler awareness :noindex:
- class aiida_fireworks_scheduler.awareness.DummyAwareness(*args, **kwargs)[source]¶
Bases:
aiida_fireworks_scheduler.awareness.SchedulerAwareness
DummyAwareness for running jobs locally
- DEFAULT_REMAINING_TIME = 2592000¶
- __init__(*args, **kwargs)[source]¶
Instantiate an DummyAwareness - as if we have lots of time to run
- __module__ = 'aiida_fireworks_scheduler.awareness'¶
- property is_in_job¶
Are we inside an scheduler job - always true for a dummy
- property job_id¶
The id of the job
- class aiida_fireworks_scheduler.awareness.SGEAwareness(*args, **kwargs)[source]¶
Bases:
aiida_fireworks_scheduler.awareness.SchedulerAwareness
SGE runtime awareness
- __module__ = 'aiida_fireworks_scheduler.awareness'¶
- property job_id¶
ID of the job
- class aiida_fireworks_scheduler.awareness.SchedulerAwareness(*args, **kwargs)[source]¶
Bases:
object
Scheduler object
- __dict__ = mappingproxy({'__module__': 'aiida_fireworks_scheduler.awareness', '__doc__': 'Scheduler object', '__init__': <function SchedulerAwareness.__init__>, 'get_n_cpus': <function SchedulerAwareness.get_n_cpus>, 'user_name': <property object>, 'get_remaining_seconds': <function SchedulerAwareness.get_remaining_seconds>, 'is_in_job': <property object>, 'job_id': <property object>, 'get_awareness': <classmethod object>, '__dict__': <attribute '__dict__' of 'SchedulerAwareness' objects>, '__weakref__': <attribute '__weakref__' of 'SchedulerAwareness' objects>, '__annotations__': {}})¶
- __init__(*args, **kwargs)[source]¶
SchedulerAwareness object for accessing information from the scheduler
- __module__ = 'aiida_fireworks_scheduler.awareness'¶
- __weakref__¶
list of weak references to the object (if defined)
- property is_in_job¶
Return wether I am in a remote job
- property job_id¶
ID of the current job
- property user_name¶
Return the name of the current user
- class aiida_fireworks_scheduler.awareness.SlurmAwareness[source]¶
Bases:
aiida_fireworks_scheduler.awareness.SchedulerAwareness
SlurmAwareness object for storing and extracting information in slurm
- __module__ = 'aiida_fireworks_scheduler.awareness'¶
- _readtask_info()[source]¶
A function to extract information from environmental variables SLURM_JOB_ID unique to each job Return an dictionnary contain job information. If not in slurm, return None TODO Refactor avoid saving intermediate file
- _task_info = None¶
- _warning = 0¶
- property is_in_job¶
Wether I am in a job
- property job_id¶
Id of the job
aiida_fireworks_scheduler.cmdline module¶
aiida_fireworks_scheduler.common module¶
Module for common stuff like global variables…
aiida_fireworks_scheduler.fworker module¶
Class for specialised worker for running AiiDA jobs include multiprocessing (mpinp) and walltime limit in the query
- class aiida_fireworks_scheduler.fworker.AiiDAFWorker(computer_id, mpinp, *args, username='AIIDA_USER', **kwargs)[source]¶
Bases:
fireworks.core.fworker.FWorker
Specialised worker for running AiiDA related jobs
- SECONDS_SAFE_INTERVAL = 60¶
- __abstractmethods__ = frozenset({})¶
- __init__(computer_id, mpinp, *args, username='AIIDA_USER', **kwargs)[source]¶
Instantiate a AiiDAFWorker object. The worker selects the jobs to run using the criteria defined in the constructor
- Parameters
computer_id – Hostname of the computer
username – User name for the computer
mpinp – the number of MPI processes to be launched. this constraint will be ignored if is is set to -1 or 0.
The rest of the arguments will be passed to the FWorker.
- __module__ = 'aiida_fireworks_scheduler.fworker'¶
- _abc_impl = <_abc_data object>¶
- classmethod from_dict(*args, **kwargs)¶
- property query¶
Query used for selecting fireworks
- property seconds_left¶
How long this job is going to be alive.
- to_dict(*args, **kwargs)¶
aiida_fireworks_scheduler.fwscheduler module¶
Specialised scheduler to interface with Fireworks
- class aiida_fireworks_scheduler.fwscheduler.FwJobResource(**kwargs)[source]¶
Bases:
aiida.schedulers.datastructures.ParEnvJobResource
JobResource for the FwScheduler based on ParEnvJobResource. The difference is that the default parallel_env file default to “mpi” here, and it is OK to have it not set.
- __abstractmethods__ = frozenset({})¶
- __module__ = 'aiida_fireworks_scheduler.fwscheduler'¶
- _abc_impl = <_abc_data object>¶
- classmethod validate_resources(**kwargs)[source]¶
Validate the resources against the job resource class of this scheduler.
- Parameters
kwargs – dictionary of values to define the job resources
- Returns
attribute dictionary with the parsed parameters populated
- Raises
ValueError – if the resources are invalid or incomplete
- class aiida_fireworks_scheduler.fwscheduler.FwScheduler(launchpad=None)[source]¶
Bases:
aiida.schedulers.scheduler.Scheduler
Scheduler that interfaces with fireworks.LaunchPad
- FRESH_ENV = True¶
- __abstractmethods__ = frozenset({})¶
- __module__ = 'aiida_fireworks_scheduler.fwscheduler'¶
- _abc_impl = <_abc_data object>¶
- _features = {'can_query_by_user': False}¶
- _get_joblist_command(jobs=None, user=None)[source]¶
Return the command to get the most complete description possible of currently active jobs.
Note
Typically one can pass only either jobs or user, depending on the specific plugin. The choice can be done according to the value returned by self.get_feature(‘can_query_by_user’)
- Parameters
jobs – either None to get a list of all jobs in the machine, or a list of jobs.
user – either None, or a string with the username (to show only jobs of the specific user).
- _get_submit_command(submit_script)[source]¶
Return the string to execute to submit a given script.
Warning
the submit_script should already have been bash-escaped
- Parameters
submit_script – the path of the submit script relative to the working directory.
- Returns
the string to execute to submit a given script.
- _get_submit_script_header(job_tmpl)[source]¶
Return the submit script header, using the parameters from the job_tmpl.
- Args:
job_tmpl: an JobTemplate instance with relevant parameters set.
Modified and simplied based on that of the SGE scheduler. We need to write the scheduler headers because they will be read in the upload step later to submit the job to fireworks’ MongoDB server.
- _job_resource_class¶
alias of
aiida_fireworks_scheduler.fwscheduler.FwJobResource
- _logger = <Logger aiida.scheduler.Fw (REPORT)>¶
- _lpad = None¶
- _parse_joblist_output(retval, stdout, stderr)[source]¶
Parse the joblist output as returned by executing the command returned by _get_joblist_command method.
- Returns
list of JobInfo objects, one of each job each with at least its default params implemented.
- _parse_kill_output(retval, stdout, stderr)[source]¶
Parse the output of the kill command.
- Returns
True if everything seems ok, False otherwise.
- _parse_submit_output(retval, stdout, stderr)[source]¶
Parse the output of the submit command returned by calling the _get_submit_command command.
- Returns
a string with the job ID.
- get_detailed_job_info(job_id)[source]¶
Getting detailed job information. Does not make sense for this scheduler
- kill(jobid)[source]¶
Defuse a job in the LaunchPad
Note, for fireworks this only works for queued jobs. Need to think about how to kill running ones….
- submit_from_script(working_directory, submit_script)[source]¶
Submit the submission script to the scheduler
This will create a WorkFlow for the job using the provided script and working directory and submit it to the LaunchPad.
- Returns
return a string with the job ID in a valid format to be used for querying.
- class aiida_fireworks_scheduler.fwscheduler.FwSchedulerKeepEnv(launchpad=None)[source]¶
Bases:
aiida_fireworks_scheduler.fwscheduler.FwScheduler
A FW-base scheduler that runs the _aiidasubmit.sh in the same environment as the submission script.
This mode of operation is needed by SLURM which uses srun directives to launch job steps.
- FRESH_ENV = False¶
- __abstractmethods__ = frozenset({})¶
- __module__ = 'aiida_fireworks_scheduler.fwscheduler'¶
- _abc_impl = <_abc_data object>¶
aiida_fireworks_scheduler.jobs module¶
Mapping AiiDA scheduler jobs to Firework
- class aiida_fireworks_scheduler.jobs.AiiDAJobFirework(computer_id, username, remote_work_dir, job_name, submit_script_name, mpinp, walltime, stdout_fname, stderr_fname, fresh_env=True, priority=100)[source]¶
Bases:
fireworks.core.firework.Firework
A Firework that encapsulate AiiDA jobs
- __abstractmethods__ = frozenset({})¶
- __init__(computer_id, username, remote_work_dir, job_name, submit_script_name, mpinp, walltime, stdout_fname, stderr_fname, fresh_env=True, priority=100)[source]¶
Instantiate a Firework to run jobs prepared by AiiDA daemon on the remote computer
- __module__ = 'aiida_fireworks_scheduler.jobs'¶
- _abc_impl = <_abc_data object>¶
Module contents¶
aiida_fireworks_scheduler
AiiDA plugin to allow using fireworks as the execution engine for CalcJob.