F

Flyte enables you to build & deploy data & ML pipelines, hassle-free. The infinitely scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks. Explore and Join the Flyte Community!

Custom Decorator for Flyte Tasks

Summary

The user is creating a custom decorator for tagging tasks in the Flyte remote client but is struggling to find code references to modify the tags field in the Flyte console. They mention a specific line in the Flyte GitHub repository that appears inaccessible for workflows or tasks. Additionally, the infrastructure team is focused on performing sanity checks on input/output types and setting up a contract for operational pipelines to allocate the decorated task and obtain user output. The user expresses a desire to eliminate outdated elements from Python 2.

Status
open
Tags
    Source
    #ask-the-community
      c

      csv8674xn

      10/18/2024

      thanks for the help. We'll change our design accordingly :smile:

      y

      ytong

      10/18/2024

      yeah we really need to get rid of them. they’re a vestige of python 2 :disappointed:

      y

      ytong

      10/18/2024

      oh sorry

      c

      csv8674xn

      10/18/2024

      that gave me confusion when I see tags under metadata and did not see anything available on the client side :smile:

      c

      csv8674xn

      10/18/2024

      thanks for confirming

      c

      csv8674xn

      10/18/2024

      exactly

      y

      ytong

      10/18/2024

      so if you want to use it’ll still need to be updated.

      y

      ytong

      10/18/2024

      we’re trying to move away from model files, but haven’t had the time.

      y

      ytong

      10/18/2024
      y

      ytong

      10/18/2024

      I don’t think tags is used by flyte, feel free to use it.

      c

      csv8674xn

      10/18/2024

      i would need to output of the task, hence below. We are adjusting the design based on what flyte can provide. For now, it looks like node searching based on custom tag is not feasible so we are trying to confirm with the community if there's a path to do this :smile:

      y

      ytong

      10/18/2024

      wait why is the custom decorator below the task decorator?

      c

      csv8674xn

      10/15/2024
      @task
      @my_decorator
      def user_function() -> Foo
      
      // flyte remote
      execution = remote.get....
      node = execution.find_node_for_user_function()
      // do something with node.outputs```
      
      c

      csv8674xn

      10/15/2024

      the task is expected to return certain type of object, and we are interested in reading that

      y

      ytong

      10/15/2024

      what do you mean by get user output?

      c

      csv8674xn

      10/15/2024

      we are an infra team developing a decorator to be used on customer's task, which intend to

      1. sanity check such as ensure input/output type
      2. create a contract that can be used in operation pipeline for 2) we'd like to allocate that decorated task and get user output
      y

      ytong

      10/15/2024

      would have to look closer to make sure that field is safe to use. could you help us understand what the end use case is here btw?

      c

      csv8674xn

      10/14/2024

      Hey team, we are trying to identify a user selected task in flyte remote client if they apply some decorated we created. One way to do it is to inject a tag in the task metadata in our decorator. However, after looking around, I can see there's a metadata tags field under task metadata section in our flyte console, but I don't seem to find any code reference to update it :thinking_face: The closest is https://github.com/flyteorg/flyte/blame/c899586448f842ae361b91ccb610d3d2c612546c/flyteidl/protos/flyteidl/core/tasks.proto#L128|here while again, don't think it is somehow exposed to workflow or task