F

Flyte

Flyte enables you to build & deploy data & ML pipelines, hassle-free. The infinitely scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks. Explore and Join the Flyte Community!

Issues with Flyte Workflow Creation

Summary

The user is attempting to create a Flyte workflow that includes running a task, launching dynamic workflows with for loops, and calling additional tasks within those workflows. They have encountered issues with the UI not displaying parallelism after the second dynamic task and not breaking down specific tasks into sub-tasks, which complicates understanding the workflow's structure. The user is looking for insights on these problems and has been advised to check out @eager workflows, which are experimental but will soon be productionized. They note that a task cannot invoke another task unless it is dynamic and suggest that the user should utilize eager workflows for now.

Status
open
Tags
  • Workflow
  • flyte
  • Flyte
  • User
  • UI Issue
  • Question
  • Developer Help
  • Dynamic Tasks
  • @eager
Source
#ask-the-community
    e

    eric901201

    10/24/2024

    eager docs: https://docs.flyte.org/en/latest/user_guide/advanced_composition/eager_workflows.html#id1

    k

    kumare

    10/24/2024

    so what you really want is eager, for now you will have to make do

    k

    kumare

    10/24/2024

    > • This task has several additional tasks called from within it A task cannot invoke a task unless its dynamic

    k

    kumare

    10/24/2024

    these are experimental, but getting productionized soon

    k

    kumare

    10/24/2024

    check out @eager workflows

    s

    samuelrshowalter

    10/23/2024

    :wave: I am trying to make a Flyte workflow that does the following: • Runs a task • Launches a dynamic workflow via a for loop • This calls another dynamic workflow, also with a for loop • (*) For each of these calls, call a task • This task has several additional tasks called from within it • One of these tasks, midway through, is another dynamic task I noticed a few things and had some questions:

    1. When I create this workflow initially, the UI does not show any parallelism after the second dynamic task. That is, the third dynamic task does not appear
    2. The UI also does not break apart the task marked with a (*) into its sub-tasks, making the UI more obscure to what is happening Any insight here appreciated!