F

Flyte enables you to build & deploy data & ML pipelines, hassle-free. The infinitely scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks. Explore and Join the Flyte Community!

Local Drive Mounting for Flyte Demo

Summary

The user is asking about mounting their local drive when creating a demo cluster with the command "flytectk demo start" using a custom Docker image, specifically to allow Flyte tasks to access the '/data' directory on their Ubuntu system with read-write permissions. They are informed that this is possible by using k3s's LocalProvisioner to create a PersistentVolume backed by the host machine and by mounting a local volume to the Docker container. The response also suggests modifying the default PodTemplate to mount the volume into the task pod and recommends mounting the PersistentVolume to back the minio deployment for better data availability. Additionally, the user inquires about using their local Ubuntu GPU with a specific Docker image and seeks help with a task error related to a grace period exceeding 3 minutes, despite attempts to adjust the timeout settings.

Status
resolved
Tags
  • Docker
  • Demo Cluster
  • Workflow Configuration
  • flyte
  • Flyte
  • flytectk demo start
  • Ubuntu
  • PersistentVolume
  • Question
  • Developer Help
  • Workflow Orchestration
  • k3s
  • Cluster
Source
#ask-the-community
    r

    roman.kazinnik

    10/26/2024

    Hi team - how can I solve the problem "currentAttempt done. Last Error: USER::Grace period [3m0s] exceeded|containers with unready status"? I tried to set timeout=20 and the task ended after 20 seconds. I tried timeout=600 but the task ends after 3mins.

    r

    roman.kazinnik

    10/25/2024

    How can I make the local flyte demo to use my local Ubuntu gpu? I want to use this image in Flyte: docker run --gpus all --rm -it --rm -it nvidia/cuda:12.5.0-devel-ubuntu22.04 /bin/bash

    h

    habuelfutuh

    10/25/2024

    Hey <@U07MS09EZ47>, I've not tried this but I believe this is possible. • You can leverage k3s's LocalProvisioner to create a PersistentVolume backed by your host machine. <https://dev.to/bbende/k3s-on-raspberry-pi-volumes-and-storage-1om5|Here is a guide>. • You will need to also mount a local volume to the docker container running flyte demo. • You can then modify the default PodTemplate (or use Pod Template Tasks) to mount that volume into the task pod. Flyte demo uses http://min.io|min.io to provide s3-compatible storage for tasks. I think a better approach is to mount the Persistent volume to back the minio deployment. This way the tasks can access s3-like FlyteFiles which will help you avoid making wrong assumptions about data availability when you run the tasks in a real cluster.

    r

    roman.kazinnik

    10/24/2024

    My custom container runs with "docker run -v /data:/data" with an existing /data on my Ubuntu. Can I have Flyte tasks to access read-write the '/data' on my Ubuntu?

    r

    roman.kazinnik

    10/24/2024

    Hi team - can I mount my local drive when I create a new demo cluster (flytectk demo start) with my custom docker image?