F

Flyte enables you to build & deploy data & ML pipelines, hassle-free. The infinitely scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks. Explore and Join the Flyte Community!

Confusion on Variable Parameters in Flytekit

Summary

The user is confused about using tasks with variable parameters in Flytekit due to errors encountered. They provide an example of a task function with variable keyword arguments and mention that variable parameters are unsupported, recommending dataclasses instead. The user seeks clarification on how dataclasses can help, particularly in adding new attributes without needing code updates, and questions if this method would allow for creating a task that accepts input from varying numbers of other nodes.

Status
resolved
Tags
    Source
    #ask-the-community
      k

      kumare

      9/27/2024

      I guess it can or you can simply pass a dict right

      d

      denis.shvetsov

      9/27/2024

      Does it help me, if I want a task, that can have input from different amount of other nodes?

      k

      kumare

      9/26/2024

      As in you can add new attributes to a data class without having to update your code

      d

      denis.shvetsov

      9/26/2024

      Could you elaborate how dataclasses can help me?

      k

      kumare

      9/26/2024

      Variable parameters not supported please use dataclasses

      d

      denis.shvetsov

      9/26/2024

      Apparently variable parameters are not supported yet. At least I get errors from https://github.com/flyteorg/flytekit/blob/master/flytekit/core/promise.py#L1282-L1290|here.

      d

      david.espejo

      9/25/2024

      flytekit <https://github.com/flyteorg/flytekit/pull/2522|supports positional args>, but not sure if already covers variable parameters

      d

      denis.shvetsov

      9/25/2024

      Hi, tasks with variable parameters are possible, impossible, not-recommended?

      Something like

      def jsonify(**args: str) -&gt; str:
         return json.dumps(str)```