F

Flyte enables you to build & deploy data & ML pipelines, hassle-free. The infinitely scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks. Explore and Join the Flyte Community!

Safety of Increasing Pods in Flyte

Summary

The user is asking about the safety of increasing the number of pods in the flyte-binary deployment and whether this requires migrating to flyte-core with separate deployments for components like propeller and datacatalog. It is suggested that using flyte-core is recommended for scaling out, and while adding binary is possible, it may involve changing some configurations that are not well documented, which could lead to issues.

Status
open
Tags
    Source
    #ask-the-community
      k

      kumare

      9/23/2024

      It’s just splitting up the binary

      k

      kumare

      9/23/2024

      It’s non destructive migration

      d

      dubovikov.kirill

      9/23/2024

      Ok, understood. We’ll evaluate using core then

      k

      kumare

      9/23/2024

      Exactly what I said not documented to scale out

      d

      dubovikov.kirill

      9/23/2024

      I meant an example for flyte-binary if there is any

      k

      kumare

      9/23/2024

      So please use core

      k

      kumare

      9/23/2024

      Wdym, core is documented

      d

      dubovikov.kirill

      9/23/2024

      <@UNZB4NW3S> thanks! Is there any example for this?

      d

      dubovikov.kirill

      9/23/2024

      Hi Everyone. Could you tell me if it is safe to scale out flyte-binary deployment by increasing the number of pods in the deployment? Or if we want scaling out, we need to migrate to using flyte-core and separate deployments for propeller, datacatalog and etc?

      k

      kumare

      9/23/2024

      It is recommended to use core to scale out. You should be able to add binary, the challenge is some config has to be changed but, if it’s not documented well, that might cause trouble