VS Code Remote-SSH Cooked My Server at 96°C

My home server runs about thirty containers: this blog, a NAS dashboard, monitoring, a handful of self-hosted things. On top of that I do my actual development on it over VS Code Remote-SSH, against a 36 GB workspace holding four hundred thousand files.

One afternoon in mid-July it cooked itself until it stopped responding.

Three days of digging later, the culprit turned out to be the editor. Not any of those thirty containers.

The symptom: 90°C pinned for forty minutes#

While chasing an earlier power-loss problem I had thrown together a script that logs CPU temperature and load once a minute. That script caught this:

15:12  cpu=90.1C  load=8.65
15:22  cpu=90.0C  load=32.53
15:31  cpu=90.0C  load=84.42
15:43  cpu=90.0C  load=156.54
15:50  cpu=90.0C  load=196.01   ← cron can no longer get scheduled

Temperature nailed to 90°C, load climbing from 8 all the way to 196, and eventually even a once-a-minute cron job couldn't get a slot. Forty minutes of that and the whole machine stopped answering.

The journal was spitting this out every four seconds right up to the end:

amdgpu 0000:79:00.0: amdgpu: SMU: I'm not done with your previous command

The SMU is the little microcontroller that manages power and temperature on the chip. If even that can't get scheduled, the CPU is as busy as it gets.

Power was easy to rule out: the machine sits behind a UPS, and the UPS logged OL the whole time — mains steady, battery full. This was not a power cut.

The answer, up front#

Three culprits, all of them part of the VS Code Remote-SSH bundle:

The rest of this is how I got there.

This was hard to find because the evidence lies#

The temperature didn't spike, it crept:

07-16 was completely fine, 07-17 was not. Something arrived on 07-17 and never left.

The problem is that the next three leads were all convincing, and all three were wrong.

Trap one: the sensors were lying#

Here's what the machine was reporting at the time:

Tctl: 94.6°C   PPT: 5-22W   load: 10-13

An 8700G drawing 5 to 22 watts — which is idle — should sit around 45 to 55°C. This was 94°C. Power that low with heat that high points in exactly one direction: the heat isn't getting out. Dried-out thermal paste, or a cooler that isn't seated properly. The timeline even cooperated: I'd had the case open in early July while chasing the power problem, so knocking the CPU fan cable loose was entirely plausible.

So the call at the time was to repaste. I never did it, purely out of laziness — the cooler is a Thermalright PS120 EVO, dual-tower coolers are a pain to pull and reseat, and I'd have had to go find where I put the thermal paste.

Not doing it turned out to be the right call. After a reboot, same machine, same cooler:

And that's under load — load average 12, every container running — still holding 55°C.

Trap two: rebooting fixed it#

After the reboot I went back through the process list and found one that had been eating CPU for days:

msedge   395% CPU   ← four cores

The causal chain assembled itself instantly: a runaway Edge tab pins the CPU, the CPU hits Tjmax and throttles hard, throttling backs up the work queue, load goes 8, 32, 84, 196, the SMU gets starved, the system locks up. And since killing Edge via the reboot made everything fine again, it looked verified.

Except I'd been in there myself. I connected over RustDesk, Edge wasn't running, and CPU usage was only 30 to 50%.

If it runs hot without Edge, Edge isn't the cause. "Rebooting fixed it" only proves the culprit isn't a long-running service — it says nothing about which one it is. Treating that as confirmation is the closest I came to going down the wrong road entirely.

Trap three: 30 to 50% CPU doesn't look like overheating#

This is the most counterintuitive one, and it's also the answer. sar unpacks it below.

How to chase an event that already ended#

The event was over and the processes were long dead, and I didn't want to wait for it to happen again before catching it. Two things had been quietly keeping history for me.

07-16 (fine)     %user  7%   %system  3%   %idle 87%
07-20 (94°C)     %user 46%   %system 18%   %idle 32%   runq-sz 20-24   blocked 0

delta        hot       quiet   app
+829.3     834.7        5.4   sshd     ← 8.3 cores
  +5.0      45.5       40.4   dockerd
  +4.7       9.0        4.3   rustdesk

sshd jumping from 5.4% to 834.7% does not mean the SSH service itself is misbehaving. apps.plugin attributes unrecognised processes to their parent's group, and I connect over VS Code Remote-SSH:

sshd
└─ code-server
   ├─ bootstrap-fork --type=fileWatcher
   └─ bootstrap-fork --type=extensionHost
      ├─ tsserver.js ×2
      ├─ typingsInstaller.js
      ├─ jsonServerMain
      └─ claude ×2

As far as netdata is concerned, the entire VS Code Remote-SSH stack is sshd. Grouping by user agrees: my account went from 74% to 928.7% while root barely moved. So does the timeline:

I installed both of these ages ago and never look at them. Reconstructing this without waiting for a repeat came down entirely to them: sysstat for the CPU breakdown, netdata's apps.plugin for the per-process history.

Culprit one: rust-analyzer looping cargo check forever#

Once it was narrowed to the VS Code bundle, I went straight to its own logs. The biggest file was rust-analyzer's, and it was wall-to-wall:

ERROR Flycheck failed to run: cargo check --workspace
error: failed to run custom build command for `soup3-sys`
error: failed to run custom build command for `javascriptcore-rs-sys`
error: failed to run custom build command for `gobject-sys` / `glib-sys` / `gio-sys`

The full command:

cargo check --workspace --keep-going --all-targets \
  --manifest-path ~/Server/tauri-plugin-sidecar/Cargo.toml

That's a Tauri plugin project, and on Linux Tauri draws its windows through the system's GTK and WebKit. At build time the *-sys crates go looking for those libraries' development files on the host.

Except this is a headless server that has never had a desktop environment on it, so none of them are there. Here's what the log was missing:

And the same set of failures repeats thirty-five times in the log. That's the retry count.

In other words this cargo check isn't slow, it's unfinishable: every round falls over in the same place, and then starts again from scratch. That's how the loop closes:

This also explains two things that hadn't made sense: why it only started after 07-17 (the day I started working on that Tauri project) and why a reboot always fixed it (the VS Code session dies, and rust-analyzer with it).

The fix#

The real fix is installing those system dependencies, or simply not opening that project on the server. But the more important part is putting a ceiling in place so that the next loop, whatever causes it, can't cook the machine:

# ~/.cargo/config.toml
[build]
jobs = 6

// rust-analyzer: skip test/bench/example, and cap its own threads
{
  "rust-analyzer.check.allTargets": false,
  "rust-analyzer.numThreads": 4
}

The jobs line is the only one that actually guarantees an upper bound. Everything else treats symptoms.

Worth noting: my files.watcherExclude was already set up properly, with the large directories excluded. So file watching was not the problem here — it was purely cargo. That's worth saying out loud, because in most "VS Code eats resources" discussions file watching gets treated as the only suspect.

Culprit two: Foam spawning hundreds of ripgrep processes#

I thought that was the end of it, but a few days later the CPU started sawing up and down again. Sampling the rg process count repeatedly:

16 → 36 → 6 → 5 → 15 → ... → 245

Every one of them had the same fingerprint:

rg (--files ... .foam ...)
   └── PID 1220855  = the ext host VS Code had just respawned

It was Foam, periodically firing rg --files across the entire workspace. And my workspace looks like this:

After disabling it, rg went to zero and the temperature dropped from 90°C back to 69°C.

This symptom isn't unique to me. VS Code has issues for rg hitting 900% CPU and sustained high rg CPU, and the Cursor community has a thread titled Cursor spawns hundreds of rg processes describing almost exactly my 245.

What those threads have in common is that nobody identifies a root cause; most of the replies speculate that ripgrep itself got slower. But rg isn't broken. It's just being invoked over and over. The thing burning CPU is rg; the hand pressing the button is something else — with this class of problem you go looking for the caller, not the callee.

Culprit three: RustDesk, which is only noise#

I ran a thirty-second process-spawn capture, recording everything new that appeared:

11× sh   ← spawned by rustdesk
 7× ps   ← spawned by rustdesk

RustDesk 1.3.1 forks one or two sh processes per second to run ps aux, using them to manage its own children and detect user sessions. Over four days that added up to nineteen hours of CPU time, and it does this with nobody connected. Restarting it changes nothing.

Across a sixteen-core machine that's 1.5%. Nowhere near enough to boil a CPU, and I need it to get a remote desktop into the box, so it stays. It's written up here only because it came third in the netdata delta table, and ruling it out cost more time than it was worth.

Two safety nets added afterwards#

Capping the temperature limit at 85°C#

I went into the BIOS and pulled the CPU's thermal limit down from somewhere around 95°C to 85°C.

Not because cooling is inadequate, but because this incident proved that "something loops forever and saturates every core" is a thing that happens here. A lower ceiling means the chip throttles earlier in that situation: the machine gets slower, but it doesn't sit at 90-plus with every core pegged for forty minutes.

The setting lives under PBO, as Thermal Throttle Limit. Day to day I cannot tell the difference, because nothing normally gets near 85°C — the baseline I measured afterwards was 72.6°C with an 83°C peak, 40W average and 64W peak against a roughly 88W ceiling, 4.6GHz all-core and 4.9GHz single-core. All normal.

Beyond that you could undervolt with Curve Optimizer, which runs cooler and cheaper with no performance loss. But this machine holds data, and an unstable undervolt shows up as random lockups and segfaults. Stability beats optimisation here, so I'm not doing it.

Changing how I do remote development#

I also changed how I work, because neither of the two real culprits was an accident — they're what the remote development architecture implies. Language servers, file indexing, extensions, search: all of it runs on the machine with far less thermal headroom than a desktop. The bigger the workspace the worse it gets, and mine is 36 GB across four hundred thousand files, which is close to worst case.

The split now is: VS Code Remote-SSH at home over the LAN, where I'm sitting next to the machine and can see and kill anything that catches fire; Zed when I'm connecting from somewhere else for hours, because all three culprits above are the kind that build up quietly while you're connected — and not being in the room is exactly the worst case.