Caddy runs an administration endpoint on 127.0.0.1:2019 by default. It exposes the full JSON configuration for reading and replacement, plus PKI and metrics endpoints. Anything able to reach that port can replace the entire server configuration, so exposing it beyond loopback is equivalent to handing over the server.
Binding it elsewhere is done with the admin global option, and admin off disables it entirely. Disabling it also disables caddy reload, which works by POSTing the adapted configuration to that endpoint, so a container with the admin API off must be restarted to pick up changes. In the official container image the admin endpoint is bound to all interfaces inside the container namespace so caddy reload works from another process; publishing port 2019 to the host or to a shared network is the mistake to avoid.
Because the running configuration is JSON, a Caddyfile is only an input format adapted at load time. Changes pushed through the API are not written back to the Caddyfile, and the autosaved config in Caddy's config directory takes precedence on restart when Caddy is started with caddy run --resume.