Pre-render d3.js charts at server side - part1: proof-of-concept

Out of date - Hi, this article was written back in 2013 and is partly obsolete.

While the method outlined is likely to still be working, hydration techniques might be more suitable in use cases where state has to be transferred from server to client-side.

This guide explains how to use d3.js on the server to pre-render data-visualization, which will be used client-side.

The main goal here is to do the heavy-lifting on the server in order to reduce page load time and other burden for the website visitors, while keeping all the power offered by d3.js, in terms of interactions on the browser.

This first article explains the concept via a very simple example, while a second one will illustrate its benefits when applied to a large chart.

Why pre-render?

For performance

d3.js is great to manipulate data on web pages, but performance quickly becomes an issue when we talk about a large volume of data. Some layout algorithms take an important time to run, and DOM manipulations also are very slow. Functionally, there is no need to do all this work at client side, and moving part of the work to the server is likely to solve these issues, as all the overhead of computing layout and adding DOM nodes is done once and for all on the server, instead of being executed on each visitor’s device, sometimes crashing their web browsers…

Here is an overview of a dashboard we generate at tribalytics, it combines several visualizations, force layout being the most expensive (we even prevent nodes overlapping):

screenshot of tribalytics dashboard

For SEO

A side benefit is that your svg content will be indexed by Google (Google indexes svg since 2010). If your visuals contain much text, then this might be an important boost to your ranking.

How to do it

We will now show how to generate an svg circle at server side with d3, and have it manipulated on the client side. This example is deliberately simple to be easily understandable; the actual use case for pre-rendering svgs is to gain performance on more complex visuals.

A follow-up article will apply this technique to a large data visualization and measure performance effect on the client.

Dependencies

We assume that node.js is already installed. d3.js and jsdom modules are necessary:

npm install d3 jsdom

Generate an html page with jsdom

First step is to create an html document on the server with jsdom

This is the how the skull of our script looks like, we’ll complete it step by step:

var d3 = require('d3')
	, jsdom = require('jsdom')
	, htmlStub = '<html><head></head><body><div id="dataviz-container"></div><script src="js/d3.v3.min.js"></script></body></html>' // html file skull with a container div for the d3 dataviz

// pass the html stub to jsDom
jsdom.env({ features : { QuerySelector : true }, html : htmlStub
	, done : function(errors, window) {
	// process the html document, like if we were at client side
		// code to generate the dataviz and process the resulting html file to be added here
	}
})

The code above creates an html document on the server, kept in memory. We have included base html tags, a container div for the svg, as well as a script loading d3.js on the client.

Thanks to jsdom we can manipulate the html file using d3.js just as if we were at client side. To do so, we have to use jsdom’s querySelector function and pass its result to d3.js.

We will select the dataviz container and the page’s body. Container selection will be used to append the svg, body selection will be used to append the client-side d3.js script.

var el = window.document.querySelector('#dataviz-container')
	, body = window.document.querySelector('body')

Render the dataviz with d3 node.js module

This is where things get interesting.

We will now draw a circle using d3.js, but instead of writing the usual client-side script, we will use d3.js to generate the svg on the server. We will serve a page already filled with the svg drawing, instead of having the visual generated only when the page is loaded.

We create an svg containing a green circle, and append it to the container:

// append the svg to the container selector
d3.select(el)
	.append('svg:svg')
		.attr('width', 600).attr('height', 300)
		.append('circle')
			.attr('cx', 300).attr('cy', 150).attr('r', 30).attr('fill', '#26963c')

Remember, the d3 code above is executed on the server. The html document we have generated contains the svg circle, and not the lines above.

Generate a d3.js client side script

Let’s say we want to make the dataviz interactive, therefore also processed with d3.js on the client. Then we also need to inject the client script inside the html. In our example, the client-side script will select the circle by its Id and transition its fill color to orange.

We have to modify our code this way:

add an Id to the circle (we simulate that the circle was created using dynamically retrieved values)
write a script manipulating the d3 dataviz for the client, and append it to the html body.

var circleId = 'a2324'  // say, this value was dynamically retrieved from a database

// append the svg to the selector
d3.select(el)
	.append('svg:svg')
		.attr('width', 600).attr('height', 300)
		.append('circle')
			.attr('cx', 300).attr('cy', 150).attr('r', 30).attr('fill', '#26963c')
			.attr('id', circleId) // we assign the circle to an Id here

// write the client-side script manipulating the circle
var clientScript = "d3.select('#" + circleId + "').transition().delay(1000).attr('fill', '#f9af26')"

// append the script to page's body
d3.select(body)
	.append('script')
		.html(clientScript)

Note that the clientScript is passed with the value of circleId which is dynamically retrieved on the server.

The solution depicted in this post still would require you to run d3.data() on the client. I am looking around to see whether there would be a way to pass d3 selections directly from the server to the client, and even to bind the DOM __data__ information on the server already.

Use the result

All that is left to do is to consume the html file. We could, according to our requirements and infrastructure, either serve the output directly in node.js, save the svg in a database, to be consumed by the server later on, or save the whole html page and serve it as a static document.

Here is how to save the html file we produced:

// save result in an html file
var fs = require('fs')
	, svgsrc = window.document.innerHTML
	
fs.writeFile('index.html', svgsrc, function(err) {
	if(err) {
		console.log('error saving document', err)
	} else {
		console.log('The file was saved, open index.html to see the result')
	}
})

Wrapping it up

You can get the full code of the above proof-of-concept in this gist.

The technique shown is quite simple and effective, providing the following gains:

performance: layout is already computed on the server
performance: generation of the DOM nodes is already done
SEO: the page is served to Google bots with the content already generated

The second article of this series will measure how it affects the rendering of a force layout chart.

An alternative way to improve performance of d3 at client side, without pre-rendering on the server is via the use of DOM DocumentFragments, read the following thread to know how this can be done.

Further improvement would be to pass the state of server-side d3 to the client-side, I have to investigate about whether this is possible. If you have an idea about this, please comment below.

Resources

The following pages were used as a base to this development:

Matt Baker’s gist generating a pie chart on the server
d3.js tests: load.js.

Category: engineering. Published by Mehdi on October 23rd 2013 , last modified: February 1st 2023 .