F

Flyte enables you to build & deploy data & ML pipelines, hassle-free. The infinitely scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks. Explore and Join the Flyte Community!

Simplifying FlyteDirectory File Access

Summary

The user is seeking a simpler way to access a specific existing file from a FlyteDirectory, finding FlyteDirectory.listdir cumbersome. They express confusion over the functionality of dir.new_file() and suggest that a more straightforward method is needed. Additionally, they propose a join method, akin to os.path.join, to enhance the process, and mention the existence of a crawl method.

Status
resolved
Tags
  • FlyteDirectory.listdir
  • flyte
  • crawl
  • os.path.join
  • User
  • Support Need
  • dir.new_file()
  • Question
  • Support Request
  • Feature Request
Source
#ask-the-community
    p

    pim

    11/13/2024

    Thanks!

    k

    kumare

    11/13/2024

    I like the idea of join - cc <@U0265RTUJ5B> where are you with the flytedir updates?

    k

    kumare

    11/13/2024

    that allows you to crawl all the members

    k

    kumare

    11/13/2024

    there is also a crawl method

    k

    kumare

    11/13/2024

    hmm i think that exists

    p

    pim

    11/13/2024

    Indeed, it'd be nice not to have to DL the entire dir (otherwise going to Path solves it). Maybe a join method, like os.path.join?

    k

    kumare

    11/13/2024

    I guts you don’t want to download all, what suggestion do you have for the API

    k

    kumare

    11/13/2024

    It can be a posix path

    p

    pim

    11/13/2024

    Hi! Simple question: How do I access one particular existing file from a FlyteDirectory? I can do FlyteDirectory.listdir and the pick out the particular child that I want, but that is a bit cumbersome. Alternatively, I could do dir.new_file(), which should work, but that usage seems contradicting the method name.

    Shouldn't such a function exist? Thanks!