Replacing a network while people are working on it is not a technical problem but a problem of order. The equipment works; what you have to work out is how to get from one state to the other without a moment where nothing works, and how to get back at every step.
What "without downtime" means here
Not zero interruption. What it does mean: each step takes seconds to minutes, touches one part of the network, and can be undone in the same amount of time. Anyone promising nobody will notice has not yet thought about the step where the gateway changes.
Preparing
- Collect what is there now. On every old device:
/export file=beforeand/system backup save name=before. Download both. You read the export; the backup is your real way back. - Build the new situation in the tool. One site with every device, the VLAN table, the SSIDs, and on the network board the cables as they will be. Use locations with their own address range when several buildings are involved.
- Read the checks across the whole site. A VLAN that stops at a cable, a device without the management VLAN, two devices on one address, an MTU that is not the same everywhere: exactly the faults you do not want to find on a Wednesday evening.
- Print the handover sheet. It lists what hangs off each port, the subnet per VLAN, and where each device is reachable. That is your patching list at the same time.
- Shorten the DHCP lease on the old router, a day in advance, to ten minutes. After the switchover it then takes ten minutes instead of a day before every client has the new gateway.
- Switch the emergency port on. In Bridge & ports you can keep one port out of the bridge with its own address and DHCP. If something goes wrong with the bridge or the VLANs, you can still get in there with a laptop. On an evening spent working on the bridge, that is not a luxury.
The tool's scripts are written for a device at factory defaults. So configure new equipment on the bench, not in the cupboard: connect, reset, paste the script, check the identity and the management address, and only then carry it over.
The order
From the edge inwards. The device handing out addresses is the last one you touch, because while the old router is running everybody keeps working.
- Access points. They are bridged and hand out nothing. Replacing one costs the people on that single unit a brief reconnect. Keep the new SSIDs named the same as the old ones for now, with the same passphrase, and nobody notices the difference.
- The switches, one at a time. Put the new switch next to the old one, run a trunk between them, and move workstations across a handful at a time. Test after each handful. A 24-port switch is 24 small steps, not one large one.
- The management network. Confirm you can reach every new switch and access point at the address on the handover sheet before you go on.
- The router. This is the real switchover and the only step with a window. See below.
The router step
First hang the new router next to the old one in the same network, with a different address and DHCP off. Then check everything you can check without moving anything: does the WAN come up, can you ping out from the router, are the VLANs there, can you reach it from the management network.
Then, inside the window:
- Switch DHCP off on the old router.
- Switch DHCP on on the new one.
- Move the internet cable.
- Wait out the lease time and test from a workstation: address, gateway, DNS, a website, a printer, a shared folder.
Leave the old router powered afterwards, on a spare port, with its old address. That is your way back for the first week, and it costs you one port.
The rollback per step
| Step | How to undo it | How long |
|---|---|---|
| Access point replaced | Put the old AP back, cable back | Minutes |
| Workstations moved to the new switch | Patch cables back to the old switch | Seconds per port |
| Trunk moved | Trunk back into the old switch | Seconds |
| DHCP switched over | DHCP back on on the old router, off on the new | Minutes, plus the lease time |
| Internet cable moved | Cable back into the old router | Seconds |
| Script pasted on a device that already had config | The rollback script, or the backup from step 1 | Minutes |
Agree in advance on the moment you roll back. "If it is not working by eleven, the cable goes back into the old router and we talk about it tomorrow" is a better agreement than carrying on until daylight.
The two ways back the tool gives you
- The rollback script. Next to every script there is a second one that removes what the first one added. Every object the tool creates carries a name or a comment, and the rollback script looks for those, so it can only touch what the tool made itself. What it cannot undo is a
set: the old value was on the device, not in the script. Those are listed at the end, by menu, so you know where to look. - The backup. That is the only truly exact way back, which is why you make it in step 1. A rollback script is meant for a device you set up an hour ago, not for one that has been running for three years.
For a device you are keeping
Sometimes you do not replace everything. For a router or switch that stays but has to take the new VLAN numbering, a full script is too blunt: it is written for factory defaults. Use Compare with the device instead, below the script. Paste the output of /export from the running device there and you get only the lines that bring it to the new configuration, with a count of what is added, changed and removed.
Read those lines before you paste them. Tables where order matters, such as the firewall, are replaced as a whole; the tool names which ones. Rules you once added by hand are not in the configuration and go with them.
What you test at every step
- Does a workstation get an address, and from the right VLAN?
ipconfig /allis enough. - Is every management address reachable? Work down the handover sheet.
- Does the printer work, the till, the time clock. Not only the internet: that nearly always works, and the rest does not.
Further reading: Using the script, Safe mode and Rolling out twenty branches.