Summary
The user is setting up a service on GCP and has configured a DNS record but is facing issues with gRPC traffic. The certificate challenge has been pending for two days, and the acme-http-solver service is encountering validation errors. Attempts to bypass SSL have been unsuccessful, leading to uncertainty about whether the certificate challenge is the main issue. Additionally, the user experiences a connection error when using pyflyte, despite the FLYTECTL_CONFIG being set correctly with insecure settings. The A record points from a non-GCP provider to the Ingress, and the browser does not trust the certificate, indicating ongoing certificate challenge issues. The user suggests changing the endpoint settings to make insecure false while keeping insecureSkipVerify true.
artyom
Ahh, interesting! Thank you for the pointer; didn't see this page and it seems very relevant. Gonna take a look Will let you know how things go!
david.espejo
ok, AFAICT there's a different setup process if you want to use auth and the IAP, where you shouldn't be using the flyte-provided Ingress, for example. For the IAP, there's a plugin, so I was wondering if you have had the chance to follow the process described here: https://github.com/flyteorg/flytekit/tree/master/plugins/flytekit-identity-aware-proxy#flytekit-identity-aware-proxy
david.espejo
<@U06AXT9B763> what version of flytekitare you using?
artyom
We have the IAP setup, but the network path for the flyte deployment doesn't seem to be behind it (at least I can access the console without going through the IAP). Not 100% sure if it has any effect on k8s cluster. Very happy to check/provide additional info, but will need pointers :slightly_smiling_face:
david.espejo
oh that's interesting. Is your setup running behind a proxy?
artyom
Another thing from nginx ingress logs, perhaps this could be useful as well:
│ ingress-nginx-controller-86jhn 172.16.0.18 - - [12/Sep/2024:19:05:37 +0000] "GET /.well-known/acme-challenge/<TOKEN> HTTP/1.1" 502 150 "-" "cert-manager-challenges/v1.13.2 (linux/amd64) cert-manager/432a489f5be77e3f4e2043564991a80e3bff6047" │
│ 277 0.001 [flyte-cm-acme-http-solver-6b4lg-8089] [] 192.168.0.136:8089 0 0.001 502 6bf36d69ceac8cc177f5b10fccca3a83 ```
artyom
1.13.5