Unlike most configuration formats, ssh_config does not let a later entry override an earlier one. For every keyword, the first value obtained while reading the file top to bottom wins. A 'Host *' block placed at the top of ~/.ssh/config therefore silences every host-specific setting below it, and the failure is silent: ssh neither warns nor errors.
The rule is: put the most specific Host patterns at the top and the catch-all 'Host *' at the very bottom. This also explains why command-line options always win, since they are read before the file.
Inspect the effective result with 'ssh -G hostname', which prints every resolved keyword and value without connecting. That is the definitive answer to 'which key is it actually using'. The Include directive, added in OpenSSH 7.3, is expanded in place, so an included file's position in the parent file determines its precedence; putting 'Include ~/.ssh/config.d/*' at the top gives the included files priority, which is usually what you want.