F

Flyte

Flyte enables you to build & deploy data & ML pipelines, hassle-free. The infinitely scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks. Explore and Join the Flyte Community!

Docker Image Build Issues for Flyte Workflows

Summary

The user is building a Docker image for Flyte workflows using Bazel and encountered a ModuleNotFoundError for the 'pydantic' package while attempting to use the pyflyte command, which also led to issues with registering ContainerTasks due to a missing 'kubernetes' package. This problem did not arise with Flytekit. The user resolved the issue by exporting PYTHONPATH. They are currently busy with other tasks but expect to be available to contribute in January and invited someone to contribute Bazel rules, suggesting a GitHub link as a starting point.

Status
resolved
Tags
  • Support Need
  • Bug Report
  • Support Request
  • flyte
  • Developer
  • Question
  • Developer Help
Source
#ask-the-community
    d

    david.espejo

    1/20/2025

    hey <@U07R0ED4PK8> coming back to this, just in case you have some time to contribute Bazel rules you use. this may be a good starting point https://github.com/flyteorg/bazelrules_flyte

    v

    vasani.ashwin

    11/13/2024

    Yes, I can contribute. I'm currently working on other tasks, but I should have some availability to complete it in January.

    d

    david.espejo

    11/13/2024

    uh thanks for sharing. Do you think you could contribute bazel rules? It would be very beneficial for the community

    v

    vasani.ashwin

    11/12/2024

    exporting PYTHONPATH fixed this issue.

    v

    vasani.ashwin

    11/12/2024

    Hi Flyte Community,

    I'm building a Docker image using bazel that includes workflows, tasks, and launch plans. I'm registering to Flyte using pyflyte with the following command:

    pyflyte register --copy none --image "${FLYTE_WORKFLOW_IMAGE}" --version "${VERSION}" "${FLYTE_WORKFLOW_DIR}" The workflow in the image imports a package that, in turn, imports other pip packages like Pydantic, k8s. However, when I register with the above command, I encounter the error like below:

    ModuleNotFoundError: No module named 'pydantic' Can someone help me resolve this issue?

    Due to this error, I'm also unable to register the ContainerTasks with PodSpec, which requires the kubernetes package. pyflyte register is also unable to find the kubernetes package even though it's present locally and in bazel. Oddly enough, this error doesn't occur when registering with Flytekit which just looks for r<https://github.com/flyteorg/flytekit/blob/750e4fbdc2035c4bf12365473f04292655fdf04b/flytekit/tools/repo.py#L49|egistrable entities>.