F

Flyte

Flyte enables you to build & deploy data & ML pipelines, hassle-free. The infinitely scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks. Explore and Join the Flyte Community!

Issues with pyflyte register command

Summary

The user is experiencing issues with the pyflyte register command using the --non-fast option, which reports successful registration of 5 entities but skips registering Tasks, resulting in outdated task code during workflow execution. They received an error about a task with a different structure already existing. After removing the --skip-errors option, they found the actual error related to a task with a different Docker image. The user is inquiring if this is a known issue and if a fix exists. They identified that the COMMIT_HASH was a static string causing the error, and appending the current timestamp resolved it. They also noted that the message "Successfully registered 5 entities" is misleading and suggested that it should specify the exact number of entities registered. Additionally, they believe that if registering the same version is not allowed, the registration should fail for Launchplans and workflows as well, not just tasks.

Status
resolved
Tags
  • different Docker image
  • flyte
  • Bug
  • User
  • Task Error
  • pyflyte register
  • Developer
  • --non-fast
  • --skip-errors
  • Question
  • task with a different structure
  • static string
  • Support Request
  • task with a different structure already exists
  • current timestamp
  • Registration
  • Bug Report
Source
#ask-the-community
    v

    vasani.ashwin

    11/12/2024

    COMMIT_HASH is the issues. It was static string, hence seeing the error. I append current timestamp which fixed the issue. But Successfully registered 5 entities was misleading. If it registers 4, then message should explicitly say Successfully registered 4 entities . Also, if registering will same version is not allowed, then registration should also fail for Launchplans, & workflows and not just tasks.

    k

    kumare

    11/6/2024

    It should has the code to generate a new version

    k

    kumare

    11/6/2024

    Can you send the code snippet

    v

    vasani.ashwin

    11/6/2024

    I think we need to pass unique uuid as version everytime. Not sure if this is expected behavior.

    v

    vasani.ashwin

    11/6/2024

    I did some more debugging. If the version string is same, then Task registration is failing after second run. But, then why only Task fails? Seems like a bug in pyflyte. pyflyte register --non-fast --image "${FLYTE_WORKFLOW_IMAGE}" --version "${COMMIT_HASH}" "${FLYTE_BINARY_DIR}"

    k

    kumare

    11/6/2024

    really odd

    v

    vasani.ashwin

    11/6/2024
    RPC Failed, with Status: StatusCode.INVALID_ARGUMENT
	Details: <task_name> task with different structure already exists:```
After removing `--skip-errors` , above it the actual error. I am not sure why I am seeing this. This is just registration of task with different docker image.
    v

    vasani.ashwin

    11/5/2024

    :wave: Hi Flyte community!! I am seeing a weird behavior with pyflyte register with --non-fast. It says Successfully registered 5 entities but actually skips to register Tasks. On workflow execution it uses old task code. Is this a known issue? Do we know the fix?

    [DEBUG][selector_events.py:54] Using selector: EpollSelector
[x] Registration <task_name> type TASK skipped!
[✔] Registration <wf_name> type WORKFLOW successful with version 18c812f241-dirty
[✔] Registration <wf_name> type LAUNCH_PLAN successful with version 18c812f241-dirty
[✔] Registration <lp_name> type LAUNCH_PLAN successful with version 18c812f241-dirty
[✔] Registration <lp_name> type LAUNCH_PLAN successful with version 18c812f241-dirty
Successfully registered 5 entities```
Thanks in advance!!