F

Flyte enables you to build & deploy data & ML pipelines, hassle-free. The infinitely scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks. Explore and Join the Flyte Community!

Help with Flyte combine function implementation

Summary

The user is looking for help with implementing the combine function in a Flyte workflow to merge two directories from tasks t1 and t2. They highlight the importance of streaming write for data training and offer to share a sample if needed. The user mentions challenges with documentation generation and refers to a specific Flyte directory link. They prefer using crawl to create a new directory for writing Flyte files and also mention the option to use listdir. The user shares a basic solution for the combine function but expresses dissatisfaction with it, noting that it requires downloading and re-uploading both directories and only works if their paths are identical. They also mention being away from the keyboard.

Status
open
Tags
    Source
    #ask-the-community