nix build creates a result symlink in the working directory, and that symlink is registered as an indirect garbage collection root. Deleting it makes the build output eligible for deletion by the next nix-collect-garbage, even though the derivation is still in the store and the build appears cached.
nix build --no-link produces no root at all, and nix build --print-out-paths combined with deleting the link leaves an output path that works until the next collection and then does not. This is the usual explanation for a build that was instant yesterday and takes twenty minutes today.
Roots are listed by nix-store --gc --print-roots. nix-collect-garbage -d additionally deletes old profile generations, which removes the roots holding previous system or user environments and is what makes it free so much more space than the plain form. For a result that should survive collection permanently, use nix build --out-link /nix/var/nix/gcroots/per-user/NAME/thing or add it to a profile. A development shell entered with nix develop roots its dependencies only while the shell is open.