exasol_integration_test_docker_environment.lib.base.base_task#
Classes
|
|
|
Represents a future for a statically ("requires") registered task. Details: - The RegisterTaskFuture is used with dependencies returned in luigis required method - In the required method you return tasks and usually you would to get the output target of your child tasks via luigis input() method (check https://luigi.readthedocs.io/en/stable/tasks.html#task-input) - However, if you have multiple child tasks which return in requires, for example in a dict or list, you need address each task in the input first and then you address the output - output can also return a nested dict or list of targets - requires can return a nested dict or list of tasks - we reduced the output to a single completion target with the list of return value targets and the child tasks are collected in the registered_tasks list - The RegisterTaskFuture gets the position of the task in the registered_tasks as input, such as current task - If the user call get_output the RegisterTaskFuture call the input() method of the current task and uses the position of child task to retrieve the completion target of the child task - It then reads the completion target to get the list of return value targets and reads the requested one. |
|
Represents a future for a dynamically registered task. Details: - run_dependencies uses luigi dynamic task by yielding the tasks - luigi returns for each yielded task the return value of that tasks output method (see https://luigi.readthedocs.io/en/stable/tasks.html#dynamic-dependencies) - the output method returns targets and in case of the BaseTask this is always the completion target which contains the list of return value targets - the RunTaskFuture encapsulate the target returned by the yield and handles reading the target for the user, such that the user get directly the return values of the child task. |