Summary
The user is asking about creating a task called ConfigurableFunction that multiplies an integer by a stored value and how to demonstrate it in a workflow function, questioning if this is similar to writing a custom task plugin. The response clarifies that it is somewhat different, highlighting the role of flytekit in task execution. It provides examples, a link to a default implementation, and details on the loader_args and load_task methods. The recommendation includes creating a decorator class like task_cls for further development. The user also seeks clarification on the task resolver's appearance on the Flyte pod and the importability of flykit.core.python_auto_container.default_task_resolver. The response explains that Flyte propeller only needs the container and arguments to run, and as long as the task and resolver are defined and loadable, it should work. The best approach for iteration is to use the Flyte sandbox for local workflow development, and the user suggests that documentation on this topic would be helpful.
denis.shvetsov
Thank you! I'll try
niels.bantilan
As a sidenote <@UNZB4NW3S> <@U04H6UUE78B> <@U0635LZ5LFM> it would awesome if we had some docs around this topic
niels.bantilan
Flyte propeller only knows what container to run and what arguments to pass into it. The resolver and task are loaded in at the entrypoint.py : https://github.com/flyteorg/flytekit/blob/15dee9579a000696539b63745d0d036647987c0e/flytekit/bin/entrypoint.py#L392-L396
So as long as the task and its resolver at defined and loadable via pyflyte run or pyflyte register it should work.
Best way to iterate on this is using the <https://docs.flyte.org/en/latest/getting_started_with_workflow_development/running_a_workflow_locally.html#id2|flyte sandbox>
denis.shvetsov
Does this work because flykit.core.python_auto_container.default_task_resolver is importable in the pod?
How can I achieve the same with custom task resolver?
niels.bantilan
which is what ends up in the Flyte UI right sidebar:
niels.bantilan
denis.shvetsov
I don’t understand how the task resolver appear on the Flyte pod. Do you know maybe where I can find information about that?
niels.bantilan
I’d recommend creating a decorator class like task_cls or something and iterating from there
class task_cls:
niels.bantilan
loader_args : method that outputs a list of strings, which provides metadata on which module, contains the task and what the attribute name is in the top-level scope of the module
load_task : takes the output from loader_args , imports the module, and plucks the f10 and f25 variables from the module, which is assumed to be initialized instances of ConfigurableFunction
niels.bantilan
It’s slightly different. It tells flytekit how to find and load the task when it runs on the Flyte pod.
See the default implementation https://github.com/flyteorg/flytekit/blob/15dee9579a000696539b63745d0d036647987c0e/flytekit/core/python_auto_container.py#L258-L268|here, and a few examples: • <https://github.com/flyteorg/flytekit/blob/15dee9579a000696539b63745d0d036647987c0e/plugins/flytekit-airflow/flytekitplugins/airflow/task.py#L46|Airflow task resolver> • <https://github.com/flyteorg/flytekit/blob/15dee9579a000696539b63745d0d036647987c0e/flytekit/extras/cloud_pickle_resolver.py#L7|CloudPickle resolver>
denis.shvetsov
Is it the same as writing Custom task plugin?
kumare
You will have to work a custom task resolver
denis.shvetsov
Hi, is it possible to make a task based on class?
Something like
class ConfigurableFunction:
def __init__(self, x:int):
self._x = x
def __call__(self, y:int) -> int:
return self._x * y```
And then usage
```f10 = ConfigurableFunction(10)
f25 = ConfigurableFunction(25)
@workflow
def wf():
a = 10
b = f10(a)
c = f25(b)
return c```