F

Flyte

Flyte enables you to build & deploy data & ML pipelines, hassle-free. The infinitely scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks. Explore and Join the Flyte Community!

Centralized Configuration Discussion

Summary

The user message discusses the centralized configuration of a system, specifically the contents of configuration files in /etc/flyte/config, including symbolic links to YAML files like admin.yaml and agent_service.yaml. It details the content of agent_service.yaml, which outlines the default agent endpoint and supported task types, as well as enabled_plugins.yaml, which lists supported task types and enabled plugins. The user asks for an example configuration that a decentralized team would implement, noting that the custom agent owner team will expect to configure aspects mentioned in a provided Slack link.

Status
resolved
Tags
    Source
    #flyte-agents
      s

      shuliang

      10/3/2024

      Yes, this is showing the what centralized team is managing. the custom agent owner team will expect to configure things mentioned in https://flyte-org.slack.com/archives/C06SYN9QJ5N/p1727908859362339|here

      h

      habuelfutuh

      10/3/2024

      These two should merge fine... but these are the configs you are centrally managing, right? Can you show me an example config some decentralized team will push?

      s

      shuliang

      10/2/2024

      The centralized configuration • The propeller reads the config from /etc/flyte/config (from configmap)

      total 0
0 lrwxrwxrwx 1 root 65534 17 Oct  2 04:54 admin.yaml -> ..data/admin.yaml
0 lrwxrwxrwx 1 root 65534 25 Oct  2 04:54 agent_service.yaml -> ..data/agent_service.yaml
0 lrwxrwxrwx 1 root 65534 17 Oct  2 04:54 cache.yaml -> ..data/cache.yaml
0 lrwxrwxrwx 1 root 65534 19 Oct  2 04:54 catalog.yaml -> ..data/catalog.yaml
0 lrwxrwxrwx 1 root 65534 19 Oct  2 04:54 copilot.yaml -> ..data/copilot.yaml
0 lrwxrwxrwx 1 root 65534 16 Oct  2 04:54 core.yaml -> ..data/core.yaml
0 lrwxrwxrwx 1 root 65534 27 Oct  2 04:54 enabled_plugins.yaml -> ..data/enabled_plugins.yaml
0 lrwxrwxrwx 1 root 65534 15 Oct  2 04:54 k8s.yaml -> ..data/k8s.yaml
0 lrwxrwxrwx 1 root 65534 20 Oct  2 04:54 kingkong.yaml -> ..data/kingkong.yaml
0 lrwxrwxrwx 1 root 65534 18 Oct  2 04:54 logger.yaml -> ..data/logger.yaml
0 lrwxrwxrwx 1 root 65534 16 Oct  2 04:54 mufn.yaml -> ..data/mufn.yaml
0 lrwxrwxrwx 1 root 65534 28 Oct  2 04:54 resource_manager.yaml -> ..data/resource_manager.yaml
0 lrwxrwxrwx 1 root 65534 19 Oct  2 04:54 storage.yaml -> ..data/storage.yaml
0 lrwxrwxrwx 1 root 65534 21 Oct  2 04:54 task_logs.yaml -> ..data/task_logs.yaml ```
• centralized agent-service.yaml

      [flyte@flytepropeller-f6d98fbd8-bqdgc /]$ cat /etc/flyte/config/agent_service.yaml plugins: agent-service: defaultAgent: endpoint: flyteagent:8000 insecure: true supportedTaskTypes: - sensor``` • centralized enabled_plugins

      plugins:
  agent-service:
    supportedTaskTypes:
    - sensor
tasks:
  task-plugins:
    default-for-task-types:
      container: container
      container_array: k8s-array
      echo: echo
      mpi: mpi
      pytorch: pytorch
      ray: ray
      sensor: agent-service
      sidecar: sidecar
      tensorflow: tensorflow
    enabled-plugins:
    - container
    - sidecar
    - k8s-array
    - tensorflow
    - mufn
    - mpi
    - pytorch
    - ray
    - echo
    - agent-service```