Summary
The user is looking for help with implementing the combine
function in a Flyte workflow to merge two directories from tasks t1
and t2
. They highlight the importance of streaming write for data training and offer to share a sample if needed. The user mentions challenges with documentation generation and refers to a specific Flyte directory link. They prefer using crawl to create a new directory for writing Flyte files and also mention the option to use listdir. The user shares a basic solution for the combine
function but expresses dissatisfaction with it, noting that it requires downloading and re-uploading both directories and only works if their paths are identical. They also mention being away from the keyboard.