Summary
The user is new to Flyte and is facing latency issues with a simple workflow involving a ContainerTask and a Python task. While the Python task runs quickly locally, it takes over 2 minutes when executed in the Flyte sandbox. They are questioning if this delay is typical and if there is a configurable timeout. They also note that the flyte-copilot-sidecar container seems idle after the task container completes, indicating that the delay may be due to waiting for the sidecar to exit. Suggestions include configuring timeouts, considering the time it takes to spin up pods, and using Union for better performance by reusing containers. The user is advised to run the task repeatedly, as it may take a few seconds initially due to container downloads. Performance considerations include balancing overhead, runtime, reproducibility, and ephemerality, especially for non-latency sensitive applications.
kumare
and multiple folks can run different version concurrently with low resources
kumare
But the way to think about performance is overhead vs runtime and reproducibility and ephemerality. For example for non latency sensitive applications this is ok - especially it tears down the entire cluster when no one is using it
kumare
once the container has been downloaded
kumare
also can you try running it repeatedly, it might take a few seconds still
kumare
if you want performance, then I recommend to use Union - it has ways to reuse containers and hence can run things in under a second
kumare
it can take time depending on if it is spinning up pods
kumare
you can always configure timeouts
perrygeo
Looking at the k3s pods while it's running and I see the flyte-copilot-sidecar container which appears to be idle. The actual task container itself is completed, it looks like the extra time is waiting for the sidecar to exit?
perrygeo
Hi everyone :wave: just getting started with Flyte and I have a question about task latency. I have a basic workflow consisting of one ContainerTask and one regular python task. Running it with pyflyte run locally finishes in under a second. But running with the flyte sandbox as a remote pyflyte run --remote it takes over 2 minutes according to the UI. Is this expected? It does eventually finish, Is there a timeout or something that can be configured? Details -> https://gist.github.com/perrygeo/05b438430ab776786bdc85185360c327