Summary
The user is looking for guidance on how to return a list of files in Flyte from a zip file, which they need to split into batches for a map_task. They are facing a "permission denied" error when trying to return List[FlyteFile] while saving in the /tmp directory. They mention that using a list of FlyteDirectories worked, but they are unsure why a list of FlyteFiles did not.
kumare
Can share a snippet in a bit once at the keyboard
lior.greenberg
Interesting, and that will make each pod only download each needed batches?
kumare
Hmm weird - you can also use an iterator to stream write
lior.greenberg
they are already in a zip, i want to unzip them to chunk into batches for a map_task.
A list of FlyeDIrectorys actually worked, not sure why a list of FlyteFiles didn't.
kumare
Also list of Flytefile should work
kumare
You can still split it later just return the zip for now
kumare
You can return a zip as a single file
lior.greenberg
Hi, What is the correct Flyte way to return a list of files? (they are currently in a zip, and i want to split them into batches after). When i return List[FlyeFile] i get permission denied for the files (saving in /tmp)