I would like to understand globally how to update the server. I'm also not sure on how to do this for a Windows dedicated server. Surprisingly this is not well documented.
EDIT - Here's what I found for others, however it still is not clear for Windows servers -
https://wiki.vintagestory.at/Special:MyLanguage/Setting_up_a_Multiplayer_Server#Updating_Servers
Updating Servers
If you want to install a newer version, remove all installation files first, or you'll get unforseeable behavior. If you chose a data folder inside the installation folder, make sure to not delete that one, or you'll loose your save games, settings and player data. Download the "Tar.gz Archive/Linux (server only)" file and extract it. If you have a remote installation and only a FTP connection to your server, you will have to unpack it locally and upload each file.