F

Flyte enables you to build & deploy data & ML pipelines, hassle-free. The infinitely scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks. Explore and Join the Flyte Community!

Out of Memory Issue with Dynamic Workflow

Summary

The user is facing an Out of Memory (OOM) issue with a dynamic workflow that processes approximately 25,000 documents from Elasticsearch in chunks of 1,000. They are asking if resources can be assigned to a dynamic workflow like they are to tasks. The response indicates that while it is possible to assign resources to a dynamic workflow, it is advisable to avoid heavy processing within the dynamic itself and instead handle it in tasks.

Status
resolved
Tags
    Source
    #ask-the-community