Is solidity EVM-complete?

Is it possible to use solidity to do anything that can be done with bytecode? Or, is there something that the EVM supports that the solidity compiler does not support?

I guess that depends on how pedantic you want to be.
Technically, there are patterns expressible as EVM bytecode that can’t be expressed as solidity code - you can quickly get there by using fully dynamic jumps, which Solidity avoids, but which you can do on the level of EVM opcodes (but which also may be disallowed with “EOF” in upcoming EVM versions).

In practice, you should be able to express pretty much anything in Solidity that can be expressed on the EVM level, especially since (apart from low-level control flow instructions like jumps) it supports using any EVM opcode directly via inline assembly.

OK that’s good. I was mainly wondering what happens when Ethereum adds a new feature to the EVM. It would be annoying to have to switch languages to use the newest features, but that’s not the case.

Yes, we usually try to stay ahead of any EVM changes - there should only be minor delays in exceptional cases, if at all - at least for basic support via inline assembly.