Take into account if is a search server of any type and move the topology, this is a cool powershell tool, free, and works great for indexes that are in the standard location. You will need to use powershell, otherwise.
If you have a vanity url for central admin, you might need to back that bus up and here is how you got the bus to where it was in the first place, click here.
After you have handled search, run psconfig from the server that has central admin, after you have taken the vanity url out, if you had one. If you have a load balancer in play, stop the routing of the calls for the vanity url, and you will need to either leave ca on the other server in that load balanced pool, or remove it from both servers in the pool and then add it to one. the steps are simple and do differe
run psconfig wizard (psconfiggui.exe) to remove central admin and stay in the farm
run psconfig wizard to add central admin to the server you want to move it onto
Note: if you have central admin load balanced, stop the routing, and remove central admin from both servers as you remove the vanity url, then add it back to one or more and put the vanity url back in place, if you truly do have a lbal with affinity set for sessions, balancing traffic to two central admin servers.
And if that is the case, do not forget about those host.ini files (type drivers on the run bar, then open the etc directory at that location, there is that hosts file)
Always consider search before you take a server out of the farm. Do the same for Distributed cache and take the service off the server and out of the Distributed Cache cluster, after making sure there are already some Distributed Cache cluster hosts in the cluster and not just this server that you’re taking out of the farm, assuming that your reason for moving central admin was to remove the server from the farm.
And note, when you move central admin and want to remove a server from the farm, I recommend first move central admin to a new server, then go back and take the former home of central admin out!