Rebuilding After Ransomware: Clean-Room Rebuild or Restore

The ransom note is the middle of the incident, not the end. Recovery is the part that takes days or weeks, and the decisions made in the first day of it decide whether you recover once or twice. The biggest decision is whether each system comes back from a backup or gets built again from installation media.

Here is how to think about that choice, and the order to bring things back so that the environment you rebuild is one the attacker no longer has a key to.

Restore or rebuild: the deciding question

A restore brings back the server as it was at a point in time. If the attacker was already inside at that point, the restore brings them back too, with their scheduled tasks, their accounts and their tools. A rebuild starts from clean media and gets only the data restored onto it.

The deciding question for each system is: do you know when the attacker got in, and do you have a backup from before that? If the answer to either is no, rebuild. Attackers typically dwell in a network for days to weeks before encrypting, so the backup from last night is almost never clean enough for identity systems.

• Domain controllers: rebuild. Always. A restored DC brings back the attacker's accounts, Group Policy changes and any Kerberos ticket material they harvested. Build new DCs and migrate, or restore only if forensics has established the intrusion date and you have a System State backup from well before it.
• Hypervisors: rebuild. The host is one layer below the VMs, and ESXi in particular has been a direct target. Reinstall and reattach the datastores.
• Application and file servers: restore the data to a fresh OS install where practical. Where the application is complex, restore the VM to an isolated network and inspect it before trusting it.
• Workstations: reimage every one. Cleaning is slower than imaging and less certain.

Set up the clean room

A clean room is an environment where nothing from the old network is trusted until it has been checked. It is a new VLAN, or a physically separate switch, with no route to the old network, a firewall in front of it, and EDR on every machine before that machine does anything else.

Everything that enters the clean room is either built from installation media or has been restored from backup and scanned. Nothing crosses from the old network as a running system.

1. Download fresh installation media for Windows Server, the hypervisor and applications from the vendor. Verify hashes.
2. Stand up a new firewall or a factory-reset one with a minimal configuration: clean room VLAN, outbound only to update servers and the EDR cloud, logging on.
3. Build a management workstation first, from media, with EDR. All further work happens from it.
4. Deploy EDR to every new machine as the first step after the OS install, before it joins anything.
5. Keep a separate credential set for the clean room. Every password from the old environment is assumed known to the attacker.

The order of operations

Systems depend on each other, and bringing them back out of order wastes days. Identity comes first because everything authenticates against it. Backups come second because everything else is restored from them. Then the applications the business needs most, in the order the business ranks them, not the order IT finds convenient.

1. Identity: new domain controllers in the clean room. If you are keeping the old domain, seize FSMO roles, remove the old DCs from AD, reset the krbtgt account password twice, and reset every account password. If forensics says the domain is not salvageable, build a new forest.
2. Backup server and repository: rebuild the server, reattach the immutable repository, verify a test restore before restoring anything real.
3. Core infrastructure: DNS, DHCP, certificate authority (rebuild the CA if the attacker had Domain Admin), file server.
4. Line-of-business applications in business priority order: usually accounting or ERP, then email if it is on-premises, then everything else.
5. Workstations, reimaged and rejoined, in waves by department, with the most affected department first.
6. Reconnect the clean room to the internet with full egress rules, then decommission the old network. Do not merge it.

# On a new DC after migration: reset krbtgt (run twice, hours apart)
Import-Module ActiveDirectory
Set-ADAccountPassword -Identity krbtgt -Reset -NewPassword (ConvertTo-SecureString (New-Guid).Guid -AsPlainText -Force)
# Force replication and check
repadmin /syncall /AdeP
Get-ADUser krbtgt -Properties PasswordLastSet | Select-Object PasswordLastSet

Before you call it done

Recovery ends when the business is running, not when the last server boots. The final checks are about proving the attacker is gone and that the same door is shut.

Have the incident response firm confirm the initial access point and that it is closed. Review every account, every scheduled task and every Group Policy object in the new environment. Turn on the monitoring that would have caught the intrusion earlier. And schedule the tabletop exercise that walks through this again in six months, while the memory is fresh.

• Every password reset, including service accounts, the backup encryption key, and every local admin through LAPS.
• MFA enforced everywhere it was missing.
• Backups running to the immutable repository with a verified restore.

Frequently asked questions

Takeaway

Restore the data, rebuild the systems, and do it in a clean environment the attacker has never seen. Identity first, backups second, applications in business order. It is slower on paper than restoring everything, and much faster than recovering twice.