Bing Maps and GeoBuf

Background

Geographic data can be large, or huge in size. Lots of application demands offline data or for scenarios where you have limited storage, like mobile devices. GeoJSON is the industry standard for communication from server to web client. It has many advantages, for example it is standardized and readable for humans. A disadvantage is that it can have loads of redundancy, for example it can share the same border which increases the data set. For that there is another format, TopoJSON which can save 80% or even more.

Both formats are text based and the commonality is JSON. When it comes to large datasets: if it is possible using TopoJSON is a good idea depending on how the data sets looks like. It is also possible to down size the data by cutting some decimals. But sometimes it is not enough. Sometimes we need something else. Something that makes a different. GeoBuf is such a format that is not so public known but the format is on uprising the last years.

GeoBuf

GeoBuf is an extension of Protocol Buffers. GeoBuf using GeoJSON when encoding the data. In the example provided it packed the data almost x 10. Since GeoBuf using GeoJSON it is straight forward to apply. I would be surprised if it doesn’t show up in spatial servers soon, there is an initiative for PostGIS. However, the product is from one vendor, not standardized so there is no guarantee that it will not be changed without notice or backword compability. Not likely though, but possible working with propriety formats. GeoBuf comes with ISC license. The downside of geobuf is it takes some time to first unpack the data and then parse the GeoJSON.

Example implementation

The example provided packs the countries in the world, a GeoJSON file with the size of 23.5 MB. The server logic, written in NodeJS, is super easy and straight forward.

var countries = fs.readFileSync("./public/countries.json")

var geoJson = JSON.parse(countries)

var content = geobuf.encode(geoJson, new pbf())

var buffer = Buffer.from(content)

return buffer

That’s it! We are done on the server. The example using Express for structure and routing in the application.

On the client. It is even easier with the GeoJSON module!

var geojson = geobuf.decode(new Pbf(geoBuffer));

var geoms = Microsoft.Maps.GeoJson.read(geojson);

client.map.entities.push(geoms);

So, with 7 lines of code GeoBuf helped downsize the data size with a factor of almost x10!

And network data.

Here is a live example.

And here is the code.

Happy buffering!