Summary
The user is exploring the construction of a workflow akin to a while loop, focusing on tasks and considering dynamic workflows with recursion. They express concerns about the possibility of deep graphs with numerous iterations and are questioning whether Flyte can manage this or if they should choose eager workflows, while also inquiring about the stability of the latter. They note that eager workflows are currently under development and expect a more stable version by the end of the year, with authentication being a challenge. Additionally, they mention that dynamic workflows should function well up to 100 levels deep, although visualization may pose a challenge.
pim
Thanks! Will definitely try
kumare
From system pov it should work
kumare
I can try later today too
kumare
But first why don’t you try - a simple experiment
kumare
No dynamic is the way, you can always launch child workflows nested
pim
Okay, thanks, that's very helpful. Would there be another pattern to create a while loop-like control flow with dynamic workflows?
kumare
Eager is being worked on right now. We believe by eoy, we will have a more stable version of eager. Auth is the problem for eager
For dynamic 100 levels deep should work, challenge might be visualization
pim
A second question: I'd like to construct a workflow that's like a while loop, in which the body is a task. It looks like I could do that with a dynamic workflow with recursion, like in https://docs.flyte.org/projects/cookbook/en/v0.3.66/auto/core/control_flow/run_merge_sort.html If there are e.g. 100s of iterations, this can lead to very deep graphs. Can flyte handle that? Or should I use eager workflows for this application? (if so, how stable are these?)