HTML-in-Canvas in Chrome 157

The text cards next to the Saturn on my home page (Moments, Notes and the rest) have always been DOM floating on top of the 3D scene. This week Chrome 157 shipped HTML-in-Canvas. Only Chrome 157+ gets it for now, but support will come eventually and a good fallback is all I need, so I decided from the start to move them into Saturn's canvas. The text and the moons are now drawn by the same worker in the same frame, and main-thread work on an idle home page dropped by almost 70%.

Those numbers were measured in Chrome 158 with a 4x CPU slowdown (to simulate a slow phone), on the home page sitting idle for 10 seconds, and in dev mode. Absolute numbers in production will differ, but the ratios should be about the same.

What HTML-in-Canvas is#

I think this is one of the few platform updates in recent years that will actually change how we build for the web. The last time I felt this way was WebGPU (Chrome 113, 2023), which gave browsers direct access to modern GPUs. HTML-in-Canvas fills in the other half: it lets the GPU draw "real HTML". WebMCP has also been big news lately, but that's about letting AI agents operate websites, which has nothing to do with how pixels get drawn. Anyway, I'm really glad to see this; it solves a problem that has been around for more than a decade.

There used to be only two options#

Before this, canvas and HTML were two separate worlds. If you wanted laid-out text or a button inside a canvas (2D, WebGL, WebGPU), there were only two ways to do it, each with its own cost:

My home page used the second one, and every problem I measured later in this post comes from it.

Now there's a third#

HTML-in-Canvas lets the browser take a "snapshot" of a regular DOM element and hand it to the canvas to draw. The key point is that the element itself stays in the DOM, so:

• Everything the browser already does keeps working: clicks, Tab focus, screen readers, text selection, Ctrl+F, translation, form autofill all work as usual.
• Layout goes back to the browser: CSS, fonts, line breaking, multiple languages, style changes like :hover; what gets drawn is the result the browser laid out, no layout code of your own.
• It's a GPU texture when drawn: you can put it in a 3D scene, have objects occlude it, give it perspective, run shaders on it for distortion or glow, none of which a DOM overlay can do.
• It can be handed to a worker: the main thread takes the snapshot, and drawing can happen in a worker's OffscreenCanvas, in the same frame as the rest of the 3D content, so no more drifting.
• Hit testing keeps up: you can tell the browser where it was drawn, so clicks and accessibility positions line up with what's on screen.

The use cases listed in the proposal include chart legends and axes, text boxes in creative tools, game menus, screens or billboards in 3D scenes, processing HTML with shaders, and exporting HTML to images or video. Babylon.js, PlayCanvas and luma.gl already have ways to use it.

It has limits too: for privacy, cross-origin images or iframes, system colors, spellcheck squiggles and visited-link colors are never drawn; and text has no subpixel antialiasing, so it looks slightly lighter than the original DOM.

Check the browser and the API first#

The shipped version was renamed#

After its origin trial the API was renamed, and a lot of the tutorials, articles and even framework implementations I found while researching still use the old names:

The old names are already undefined in Chrome 158, so code written from an old tutorial just silently does nothing. To detect support, set the attribute once and check whether it reflects back:

function supportsHtmlInCanvas(): boolean {
  const canvas = document.createElement('canvas');
  canvas.setAttribute('content', 'drawable');
  return canvas.content === 'drawable' && typeof canvas.captureElementImage === 'function';
}

TypeScript's lib.dom doesn't have these APIs yet, so you declare the types yourself and mark all of them optional, since they don't exist in unsupported browsers.

The simplest usage#

Start with the 2D version; the whole skeleton of the API is here. Add content="drawable" to the canvas and drawable to the child you want drawn:

<canvas id="c" width="600" height="300" content="drawable">
  <div id="card" drawable>
    <h2>手記 NOTES</h2>
    <a href="/notes">最新一篇</a>
  </div>
</canvas>

const canvas = document.querySelector<HTMLCanvasElement>('#c')!;
const ctx = canvas.getContext('2d')!;
const card = document.querySelector<HTMLElement>('#card')!;

// When the element's content or style changes (including :hover) the browser fires paint; redraw here
canvas.addEventListener('paint', () => {
  ctx.clearRect(0, 0, canvas.width, canvas.height);
  ctx.drawElementImage(card, 40, 60);
});
canvas.requestPaint(); // To animate every frame, call it again inside paint

A few rules:

• A drawable is its own drawing unit. Its descendants get drawn along with it, but if a descendant is itself drawable, it's excluded and has to be drawn separately.
• 2D lines up hit testing automatically. From the drawElementImage arguments plus the canvas's current transform, the browser knows where the element was drawn, so clicks, getBoundingClientRect() and the magnifier all follow.
• With WebGL / WebGPU you have to tell the browser. The browser doesn't know where a shader put the texture, so you pass the matrix to updateElementGeometry(); if you don't, the thing rotates away but clicks still land in the old spot. clearElementGeometry() makes it unclickable and marks it offscreen in the accessibility tree.

No browser to test with locally#

The Edge on my server is 148, and the stable version from apt is only 156, neither of which supports it. Edge 157 is only in beta and needs sudo to install. In the end I used npx @puppeteer/browsers install chrome@beta to install Chrome for Testing 158 into my home directory, no sudo needed. The server has no display, so I ran it headed under xvfb with the Vulkan flags to get real WebGPU.

three.js's HTMLTexture doesn't work#

three.js has had HTMLTexture since r184, so in theory I could just use it. I read the r186 source in my project and found two problems:

So I had to write the upload flow myself:

For the destination I use the texture of a three render target. r187 happens to add renderer.initRenderTarget(), which lets you allocate the GPU texture first and then write into it, and about 40 lines replaced HTMLTexture. The core is this:

const rt = new THREE.RenderTarget(texW, texH, { depthBuffer: false });
renderer.initRenderTarget(rt);
const gpuTexture = renderer.backend.get(rt.texture).texture;

device.queue.drawElementImageToTexture(
  { source: image },                       // ElementImage sent from the main thread
  { texture: gpuTexture, size: { width: texW, height: texH } },
);
image.close();                             // Release it when done

I learned image.close() from that PR. My prototype didn't have it at first, and every paint left behind an unreleased snapshot.

Prototype: cards orbiting Saturn#

Before touching the site's code, I built a standalone prototype in a scratch area that fully mimicked the home page's architecture: three.js WebGPU in a worker drawing a Saturn, with a few DOM cards orbiting it.

The card on the right is closer to the camera, so it has perspective and looks bigger. The mouse is hovering over it, so its border turned amber, which shows that CSS :hover goes into the texture in real time. The card behind is hidden by Saturn, which the depth test handles naturally. Clicks work too: click on the projected position and elementFromPoint returns the right card; once the hidden card's geometry is cleared, clicking the same spot only hits the canvas.

The prototype also ran into a few gotchas:

• Texture color space and UVs. At first the background came out brighter and bluish and the text was upside down. I had to handle the color space and flip the plane's UV vertically.
• Intermittent errors. About 1 in 9 calls to drawElementImageToTexture threw "No context found for ElementImage". An element that has just been added and hasn't painted yet can't be captured; catch it and call requestPaint() to re-snapshot.
• Tab focus goes behind Saturn. Keyboard focus can land on the hidden card, so that behavior needs its own decision.
• Distant text is softer. When a card is far from the camera its texture gets minified, so I capture at 2x resolution.

Moving the small labels first: only one benefit#

With the prototype working, I opened a new worktree and, as a first step, tried it out on the five always-on small labels on the home page (WORK, NOTES and so on) to see what wiring it into the real site actually buys. The cards that only appear on hover stayed DOM for now. Compared against ?hic=0 (which forces the original DOM version), the result was a bit surprising:

• When the main thread is blocked, the labels stay attached to the moons. This was the only obvious benefit. I deliberately blocked the main thread for 3.5 seconds while the moons kept orbiting in the worker: the DOM labels froze in place, with dots and text coming apart, while the labels drawn into the canvas followed the moons the whole time.
• Main-thread work only dropped 13%, and style recalculation 20%.
• "Really hidden behind the planet" almost never happens on the home page. The far side of the innermost orbit only grazes the top edge of the planet, and labels always extend away from Saturn. I waited 2 minutes (almost two orbits) and didn't see it once.

There was also a cost: for the drawn labels to be clickable, the three layers above the canvas (.horbit, <main>, and the z-index 10 content container) all had to let the mouse pass through, a change that reached pretty deep.

Where the main thread actually gets stuck#

13% was too little, and I wanted to know where the rest of the time went. I recorded performance traces over CDP with style invalidation tracking on and a 4x CPU slowdown, attributed every style recalculation to the element that triggered it, and removed them one by one to measure:

The big one was the cards, not the labels. The cards accounted for nearly 90% of style recalculation, and about 60% of that came from a really dumb cause: while idle, the cards take turns highlighting, so their opacity changes every frame, and the CSS had transition: opacity 0.3s. Every write canceled the old transition and created a new one.

That part doesn't need HTML-in-Canvas to fix, so I first moved the fades into JavaScript, which benefits every browser. The other half is the cost of writing inline styles on a dozen-plus elements every frame, and the only way to remove that is to let the canvas draw the cards too. That also explains why moving only the small labels saved so little: the elements actually getting restyled every frame were the cards.

Moving the cards in too#

With the transitions fixed, the next step was moving the cards in as well. For now only Chrome 157+ benefits, but support will only grow. With a solid fallback, today it's people on new Chrome who get the win, and later it's everyone.

The rule was "it must look exactly like it does now", so the production version has one key difference from the prototype: the text doesn't go into the 3D scene. It's drawn on the very top layer, in screen coordinates, aligned to device pixels, with the snapshot pasted one pixel to one pixel so the text never gets resampled and blurry. Real occlusion almost never happens on the home page anyway, and in exchange the text is indistinguishable from the DOM version.

Positioning, fades and when to yield, I pulled into a pure-function module (lib/orbitLabels) shared by the DOM version and the worker, so both sides always compute the same thing.

Every line aligns on its own#

Each line in a card aligns according to the moon's position: left-aligned when the moon is on the right, right-aligned when it's on the left, centered when it's below, and it changes continuously. If I kept changing each line's translate every frame like before, it would trigger style recalculation plus a re-snapshot, which defeats the purpose.

What I did was capture the whole card only once and send it to the worker along with each line's top edge and width. When the worker draws, it slices the snapshot into horizontal strips; each line is a sub-region of the same texture and shifts left or right on its own, while the outer margin (where the keyboard focus ring is drawn) stays put. So as long as the card's content doesn't change there's no re-snapshot, and each frame only moves vertices.

When to re-snapshot is up to the paint event. It carries the elements that changed this time (changedElements), so on hover, theme change or card content updates, only the cards that changed get re-captured:

canvas.addEventListener('paint', (e) => {
  const changed = e.changedElements;
  const out = [];
  let missed = false;
  for (const el of labels) {
    if (changed?.length && !changed.includes(el)) continue;
    try {
      out.push({ image: canvas.captureElementImage(el), w: el.offsetWidth, h: el.offsetHeight });
    } catch {
      missed = true; // Just added and not painted yet, capture again next frame
    }
  }
  if (out.length) worker.postMessage({ type: 'labels', labels: out }, out.map((l) => l.image));
  if (missed) canvas.requestPaint();
});

ElementImage is transferable, so sending it to the worker doesn't copy it; the worker calls close() once the upload is done.

Colors were 30% off#

In the first version, the small text came out about 30% darker than the DOM version. Digging in, there were two things:

• The snapshot uses straight alpha, but I treated it as premultiplied, so alpha got applied twice.
• When three outputs to the canvas, it first divides out alpha, converts to sRGB, then multiplies alpha back in. DOM text over the canvas is blended on sRGB values; blending in linear space makes the antialiased edges of light text thinner.

So the text compositing step has to work backwards from three's output rules: first compute the sRGB value that was on the canvas, blend the text on that value, then convert back to the linear value the output expects. I verified it with a translucent patch of known color: white at 50% on top, expected 134.5, actual 132.

One of these is the DOM version and the other is drawn into the canvas, and even zoomed in they're almost impossible to tell apart. The only difference is that DOM text can land on fractional pixels while the canvas version is aligned to whole pixels, so its edges are slightly sharper.

Clicks and keyboard#

Every frame the worker computes each card's on-screen box and sends it back to the main thread for updateElementGeometry, so clicks land where the card was drawn. If the box is the same as last frame, it isn't sent.

Keyboard and screen readers now go through a set of invisible links on the page, with the same Tab order as before; the canvas-drawn cards are marked aria-hidden so they aren't read twice. The focus ring is drawn on the card in the canvas.

There was a small thing I almost missed: to draw the focus ring, the snapshot has an extra margin around it, and as a result the mouse hit area also grew by 7px. I changed it so the margin only counts when focused by keyboard, and now the mouse hit area is exactly the same as the DOM version.

Results#

All the main thread does each frame now is work out "which moon is lit", and it doesn't send anything if the value hasn't changed. After shipping, I confirmed on the production site: Chrome 158 uses HTML-in-Canvas, Edge 148 stays on DOM, and neither has errors. Firefox, Safari, devices with only WebGL2, and cases where 3D didn't start also all get the DOM version.

Once text becomes a texture#

With performance done, what I was actually more curious about was something else: the text is now a texture on the GPU, so can it do effects a DOM overlay can't, and really blend into this universe? I listed five ideas and built a trial version of each. This part is still on an experimental branch and not live yet.

At first there was actually one more: "black hole gravitational lensing". In the black hole theme, text passing near the black hole would bend along with the stars behind it. Technically it was the most interesting, because the lensing math already existed; the text just had to be sampled at the deflected position.

But in practice it bent too much and felt a bit abstract, the text drifted some distance away from its own moon, and the hit area stayed where it was before bending. I ended up dropping it and replacing it with the "spit out" version: getting sucked in is because of gravity, so coming back by being spat out in reverse ties in nicely. When you return to the home page and Saturn grows back, each label, as its own moon pops out, comes out of that point long and thin, then unfolds back to its shape in about 0.7 seconds.

The first version just reused the moon's pop-out progress, but that's only 0.25 seconds, and the label was fading in from transparent, so it looked like nothing happened. I changed it so the label runs its own animation, fully opaque as it comes out, relying only on the shape unfolding from a point. On the way back it's usually just a 10px small label, so being spat out is more understated than being sucked in, and you need it zoomed and slowed to see clearly:

These effects only show up in Chrome 157+, so they can only be a bonus; no functionality can depend on them. When I turn them into the production version, the suck-in, spit-out and decode effects will be turned off under prefers-reduced-motion.

Things to watch out for#

• Treat it as a bonus only. Right now only Chrome / Edge 157+ support it, so you must have a DOM fallback path. In my case, failed detection, WebGL2, 3D not starting, or a URL with ?hic=0 all go to DOM.
• Don't follow old tutorials. Make sure you're using the shipped names; the layoutsubtree and texElementImage2D set no longer exists in 158.
• Hold off on three.js's HTMLTexture. As of r187 it still uses the old names, and if your canvas is in a worker, you can't use it even after the fix.
• Close your snapshots. ElementImage keeps piling up if you don't release it after use.
• Re-snapshot when capture fails. Catch the occasional "No context found for ElementImage" and requestPaint() to capture again next frame.
• Measure before you move. Before moving things into the canvas, make sure the main-thread time is really being spent on those elements. My first step only moved the small labels, and it saved just 13%.

What hit home most for me this time was actually that comparison video with the blocked main thread: once the text is drawn by the worker, no amount of main-thread busyness can drag it down, and "sucked into the moon, spat back out" also runs in the worker, completely unaffected. The stutter in page transitions has exactly the same cause: mounting the new page hogs the main thread, and the transition's dots are drawn on the main thread too. So next I'm doing the same thing and moving the transition dots and comet tail into the worker as well. Work has already started on a branch, and I'll write about it separately when it's done.