Summary
The user is upgrading their workflow to tensorflow=2.16.2, which has breaking changes in keras, now a separate package. They are facing an error in a Flyte task related to the TensorFlowModelTransformer using the old Keras 2 style. The user proposes a code modification for Keras 3, noting it is untested and may need a new transformer. They haven't found related issues on GitHub and are asking if others have faced similar problems. Additionally, they mention a potential issue with the protobuf version as a reason for the upgrade not being completed.
kumare
or maybe we just did not get to it
kumare
we would love to take this contribution. but <@U0265RTUJ5B> may know why the upgrade is not done, maybe problem with protobuf version?
dennis
I am experimenting with upgrading my workflow to use tensorflow=2.16.2. This includes some breaking changes with how keras works. It's no longer bundled as tensorflow.keras and is instead it's own package. It also has some breaking changes about how models are saved and loaded. Details in this <https://keras.io/guides/migrating_to_keras_3/|migration guide>.
I'm getting an error in a flyte task that returns a trained model. I think this is caused by the https://github.com/flyteorg/flytekit/blob/master/flytekit/extras/tensorflow/model.py|TensorFlowModelTransformer using the keras 2 style. To make this work with keras 3, I think the fix would be something like this:
+ local_path = ctx.file_access.get_random_local_path() + ".keras"
pathlib.Path(local_path).parent.mkdir(parents=True, exist_ok=True)
# save model in SavedModel format
- tf.keras.models.save_model(python_val, local_path)
+ keras.saving.save_model(python_val, local_path)```
This is untested code BTW. And it would require similar changes in loading the model. It might require a new transformer for Keras 3 specifically.
I didn't see anything related to this in the github issues. I'm wondering if others have had this problem as well.