F

Flyte enables you to build & deploy data & ML pipelines, hassle-free. The infinitely scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks. Explore and Join the Flyte Community!

Spark Plugin Compatibility Inquiry

Summary

The user is asking about experiences with the Spark plugin and its compatibility with Volcano or other batch schedulers. They found documentation for Volcano with Kubeflow but not for the Spark plugin. The user has created a custom plugin for scheduling Python tasks using the Armada batch scheduler and mentions that Joshua is interested in gang scheduling with Spark, which is available in their union.

Status
open
Tags
    Source
    #ask-the-community