Summary
The user discusses an internal Flytekit plugin designed for managing Ray tasks, created to address limitations of the open-source Ray plugin. This plugin operates a Ray Cluster on a separate GKE cluster, executing jobs and then deleting the cluster, while the Flyte GKE cluster manages the Ray cluster's lifecycle. The user is considering using Flyte Agents for performance improvements, as they could reduce pod creation and allow for autoscaling. They are assessing whether their use case is suitable for testing with Flyte Agents and questioning why the OSS Ray plugin remains a backend plugin rather than being upgraded to an Agent. Additionally, the user seeks clarification on how their plugin's lifecycle management methods (pre_execute, execute, post_execute) interact with the Agent's methods (create, get, delete), particularly regarding process alignment. They note that only execute will be invoked if no pod is created for tasks run through an agent and express doubt that pre_execute and post_execute will be called in the agent server during remote execution, but may be called in local execution. The user also inquires about any planned work related to this topic and offers assistance if needed.