Summary
The user is seeking assistance in programmatically creating a launch plan in Flyte using a code snippet. They are experiencing issues with their Python code not producing results and have had no success with pyflyte launchplan
and pyflyte run --remote
. The user is working on a GitHub pull request and is trying to create a launch plan with an invalid cron string to validate recent changes. They provided a code snippet for a workflow involving tasks for calculating mean, standard deviation, and standard scaling. Despite their efforts, the launch plan creation did not work as expected, and they are looking for insights or guidance, particularly regarding the raw_register
method for registering a launch plan.
michaellai901026
I think lhs
is only used for imported instances, which is not really ideal. One way to bypass this error is to move all wf & task definitions into another file and import them into your main module.
kumare
thank you
michaellai901026
Sure, I’ll give it a look when I have time
kumare
it seems we broke the __main__
handling?
kumare
cc <@U06SFQHEJ3H> can you please take a look, the above script fails with
│ /Users/ketanumare/src/ketan-examples/extras/register_main.py:38 in <module> │
│ │
│ ❱ 38 │ remote_standard_scale_workflow = remote.register_workflow( │
│ │
│ /Users/ketanumare/src/flytekit/flytekit/remote/remote.py:939 in │
│ register_workflow │
│ │
│ ❱ 939 │ │ │ _, _, _, module_file = extract_task_module(entity) │
│ │
│ /Users/ketanumare/src/flytekit/flytekit/core/tracker.py:354 in │
│ extract_task_module │
│ │
│ ❱ 354 │ │ │ name = f.lhs │
│ │
│ /Users/ketanumare/src/flytekit/flytekit/core/tracker.py:120 in lhs │
│ │
│ ❱ 120 │ │ return self.find_lhs() │
│ │
│ /Users/ketanumare/src/flytekit/flytekit/core/tracker.py:172 in find_lhs │
│ │
│ ❱ 172 │ │ raise _system_exceptions.FlyteSystemException(f"Error looking │
╰──────────────────────────────────────────────────────────────────────────────╯
FlyteSystemException: SYSTEM:Unknown: error=Error looking for LHS in
register_main```
kumare
i think this seems like a bug in the tracker
kumare
aah i see its there
kumare
yes will try to add the method once i get a chance
peterxcli
Hi <@UNZB4NW3S> Thanks for answering:wave:
I'm currently working on <https://github.com/flyteorg/flyte/pull/5951|this PR> and am trying to create a launch plan with an intentionally invalid cron string to validate the recent changes.
I attempted the following setup:
@task
def mean(values: List[float]) -> float:
return sum(values) / len(values)
@task
def standard_deviation(values: List[float], mu: float) -> float:
variance = sum([(x - mu) ** 2 for x in values])
return sqrt(variance)
@task
def standard_scale(values: List[float], mu: float, sigma: float) -> List[float]:
return [(x - mu) / sigma for x in values]
@workflow
def standard_scale_workflow(values: List[float]) -> List[float]:
mu = mean(values=values)
sigma = standard_deviation(values=values, mu=mu)
return standard_scale(values=values, mu=mu, sigma=sigma)
if __name__ == "__main__":
remote = FlyteRemote(config=Config.auto())
standard_scale_launch_plan = LaunchPlan.get_or_create(
workflow=standard_scale_workflow,
name="standard_scale_lp_haha",
schedule=CronSchedule(schedule="* * * * MON#1"),
)
remote_standard_scale_workflow: FlyteWorkflow = remote.register_workflow(
standard_scale_workflow,
)
remote.register_launch_plan(standard_scale_launch_plan, version="v1", project="flytesnacks", domain="development")```
Unfortunately, this approach didn’t work as expected, and I suspect there might be an issue with my process. I also explored the `raw_register` method you mentioned, but I couldn’t find much documentation on using it specifically to register a launch plan.
If you have any insights or additional guidance, I would greatly appreciate it! Thank you very much.
kumare
you can use raw_register
kumare
you can , we should write a method in remote.py to do https://github.com/flyteorg/flytekit/blob/0662bc6546278154605f5f544782db0110bd463d/flytekit/remote/remote.py#L808-L820|this (its currently behind raw_register) sadly
peterxcli
Anyone know how to create a launch plan programmatically in Flyte?
I've been trying to run this code:
name="my_launchplan",
workflow=wf,
schedule=CronSchedule(
schedule="0 6 * * MON#1", # UTC, see <https://docs.flyte.org/en/latest/concepts/schedules.html#cron-expression-table>
)
)```
but running it directly with Python doesn't seem to do anything. I've tried `pyflyte launchplan` and `pyflyte run --remote`, but neither seems to support this use case.
Any help would be much appreciated! :pray:
(I'm running the example from here: <https://github.com/flyteorg/flytesnacks/blob/0ec8388759d34566a0ffc0c3c2d7443fd4a3a46f/examples/basics/basics/launch_plan.py>)