F

Flyte enables you to build & deploy data & ML pipelines, hassle-free. The infinitely scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks. Explore and Join the Flyte Community!

Function Passing in Workflows Inquiry

Summary

The user inquires about the feasibility of passing a function from a main workflow to a subworkflow, emphasizing the use of conditionals based on previous outputs and the @dynamic feature for dynamic workflows. They provide an example of a user-defined function and seek clarification on the purpose of this method, questioning if the example is adequate for creating a workflow that permits others to implement their own versions. Additionally, they mention that task resolvers can be utilized for customizing function loading in a container, suggesting that while the desired functionality is achievable, further exploration and experimentation are necessary as there is no ready-made solution available.

Status
open
Tags
    Source
    #ask-the-community
      k

      kumare

      10/11/2024

      ohh there is one way, i realized - @eager

      k

      kumare

      10/11/2024

      we do not support tasks as parameters today

      c

      csv8674xn

      10/11/2024

      what she is trying to do is more like this

      def workflow_with_udf_input(user_passed_in_func: Type):
        // some logic
        result=user_passed_in_func()```
      user's perspective
      ```def user_defined_function():
      
      @workflow
      def user_invoke_our_workflow():
        // some logic
        workflow_with_udf_input(user_defined_function)```
      
      k

      kumare

      10/11/2024

      Use callable type, I’ll try later today

      k

      kumare

      10/11/2024

      The only way to pass would be to use python to trigger

      k

      kumare

      10/11/2024

      You can use pickle

      h

      habuelfutuh

      10/11/2024

      there are task resolvers (advanced topic :slightly_smiling_face:) that can be used to allow you to customize how to load the function in a given container... so you can pretty much do what you are trying to do but will need to explore/experiment with that as we don't have a ready-made recipe

      h

      habuelfutuh

      10/11/2024

      I get that but I'm trying to uncover what is the purpose of that?

      Would something like this suffice?

          # do something
      
      @workflow
      def subworkflow(custom_function_name: Any):
          conditional("func")._if(custom_function_name == "user_defined_function").then(custom_function())...
          # do other things
      
      @workflow
      def workflow():
          subworkflow(custom_function_name="user_defined_function")
          # do more things```
      Are you trying to build a workflow that others can use and substitute their own implementation for a portion of it?
      
      a

      aishi

      10/11/2024

      Not really, more like

          # do something
      
      @workflow
      def subworkflow(custom_function: Any):
          custom_function()
          # do other things
      
      @workflow
      def workflow():
          subworkflow(custom_function=user_defined_function)
          # do more things```
      
      h

      habuelfutuh

      10/11/2024

      Are you trying to dynamically choose a function to execute? There are conditionals that you can use to decide based on a prior step's output or you can use @dynamic to run a fully dynamic workflow...

      def my_dynamic_wf(inputs...):
       ... call this or that```
      The dynamic workflow won't immediately call `this` or `that`  but instead figure out the workflow graph then execute it after the function returns
      
      a

      aishi

      10/11/2024

      Hi :wave:, is there a way to pass in a function from a workflow into the subworkflow, and execute the user defined function inside the subworkflow?