COPY Command with ImageSpec Inquiry

Summary

The user is asking if they can use the COPY command with ImageSpec, similar to how they can use the RUN command. They attempted to use the COPY command but encountered an error indicating that COPY was not found. They found that using image_spec.copy works, but they express frustration that extra RUN commands are executed before COPY commands. The user suggests that the design pattern should allow for better composition and ordering, proposing that each with method should return a new ImageSpec with the additional change while pointing to the original. They request assistance in improving this implementation.

Status

resolved

Tags

Workflow Issue
Inquiry
flyte
ImageSpec
RUN
Command Execution
Developer
COPY Command
COPY
User Frustration
Execution
Question
Developer Help
Command

Source

#ask-the-community

denis.shvetsov

10/22/2024

Thank you!

pingsutw

10/21/2024

yup, we don’t support ordering for now. To wrokaround it, you can use nested imageSpec image_spec = ImageSpec(base_image=ImageSpec(copy={...})).with_commands(...) flytekit will build the first image that copy packages to the image, and then use it as base image for the second imageSpec

denis.shvetsov

10/21/2024

Sure, I’ll make a more careful look in a spare time

habuelfutuh

10/21/2024

The original intention of the design pattern is to allow composition and therefore ordering. I don't think the current implementation lives up to the premise. Would it be something you can help with? The simplest fix in my opinion is that every with should return a new ImageSpec with just the additional change (and points to the original as its base) At image build time, we just run through the linked list in order.

<@USU6W5ATA> <@U0265RTUJ5B>

denis.shvetsov

10/21/2024

https://github.com/flyteorg/flytekit/blob/master/flytekit/image_spec/default_builder.py#L79-L80

denis.shvetsov

10/21/2024

Though it is not very convenient that extra run commands are executed before extra copy commands. Is there any reason for that?

david.espejo

10/21/2024

Sorry, for that one you should use .with_copy

denis.shvetsov

10/21/2024

According to the code This command works

image_spec.copy = [
    './packages'
]```

denis.shvetsov

10/21/2024

I tried

    'COPY ./packages /packages'
)```
Got the error

```=&gt; ERROR [stage-2  8/10] RUN COPY ./packages /packages                      0.2s
------
 &gt; [stage-2  8/10] RUN COPY ./packages /packages:
0.142 /bin/sh: 1: COPY: not found
------
Dockerfile:29
--------------------
  27 |
  28 |
  29 | &gt;&gt;&gt; RUN COPY ./packages /packages
  30 |
  31 |
--------------------
ERROR: failed to solve: process "/bin/sh -c COPY ./packages /packages" did not complete successfully: exit code: 127```

Looks like it is still `RUN` command

david.espejo

10/21/2024

Hey Denis, yes Using .with_commands you should be able to

denis.shvetsov

10/21/2024

Hi all, with IamgeSpec I can execute RUN command, I also want to execute COPY command, am I able to do so?

Flyte

COPY Command with ImageSpec Inquiry