Creating a 10,000+ px interactive web canvas

When I began working on the Upper Fort Garry (UFG) app (which you can read more about here) one of the features needed was a way to explore the 400 ft. steel wall on a mobile device. On a device the wall can be 10,000 – 30,000+ pixels wide. I quickly realized it was going to be an interesting challenge to have that scale of imagery perform smoothly. This is the first hybrid app I’ve worked on and the first time using with some JS libraries I’ve never used before and… Well there was a lot of firsts and that’s what made it challenging and exciting.

Default Browser Scrolling

The first thing I tried was to simply plop the images in a <div> and float the images left. Done. Right? Well here is how that turned out:

See the Pen Scrolling Wall V1 by Pattern Interactive (@Pattern-Interactive) on CodePen.

Note this example is “bare bones” without the overhead of the rest of the app. Even so this takes up a signification amount of memory and processing power to render when scrolling quickly. Older and less powerful devices like the an iPod (5th Gen) or iPhone 4 scrolling quickly causes momentarily blank spots as the device tries to keep up with rendering the images. In some cases the application would outright crash.

Well that’s no good 🙁

Leaflet

I had considered using Leaflet as means to pan around the wall. A pretty solid library but when panning there is a slight delay when loading in tiles and we wanted to have it load instantaneously.

Enter HTML5 canvas.

The next idea was what if we drew the images on a canvas? There are some neat interactive applications out there that use this technology. Just check out this one. If that can perform fairly well surely we can make our ‘simple’ scrolling wall perform somersaults.

“Wall versions A through G were complete failures! But wall H feels good on the whole…”

There were actually quite a few iterations of the wall as I experimented before I developed the final version. At first I tried simply drawing the whole wall on a single canvas to see what that would do. It crashed. Canvases have a maximum size depending on the browser – see here. It didn’t matter what was drawn if the canvas element exceeded it’s max dimensions, the end result was still a crash. as well, drawing images beyond the defined boundary of a canvas will still use significantly more resources. For example, drawing a  1’000px by 10’000px  image within a a 300px by 300px canvas, still renders the entire image.

As a solution I would have to figure out how to draw only a portion of the wall, detect when a user was attempting to scroll and redraw the wall with the correction sections showing.

Into the Code

Let’s get more into the code and start with the basics and work our way up. In this example we start by simply drawing an image on the canvas (read the comments in the code for additional details):

See the Pen Scrolling Wall V2.0 by Pattern Interactive (@Pattern-Interactive) on CodePen.

Scaling the Wall

Okay, that was pretty straight forward. Now I want the canvas to fill the entire screen whatever the device size:

Note how I set the canvas dimensions via the canvas HTML element’s attributes and not the CSS attributes.

The element’s attributes are needed for the canvas to know the size of the canvas. The CSS equivalent attributes will tell the browser how large to display the canvas but won’t tell the canvas how big of a box it can draw in. That may be a little confusing at first but if you play around with it you’ll understand. Also, setting both the HTML and CSS width and height attributes can effect the scale and therefore quality of what’s drawn.

See the Pen Scrolling Wall V2.1 by Pattern Interactive (@Pattern-Interactive) on CodePen.


You may have noticed that the image drawn is way to large to fit in the canvas at smaller resolutions. And if you played with the width and height attributes you may have already figured out how to solve that. Unlike an <img>  tag where the browser will automagically scale proportionally if it’s height/width is set to auto, the canvas dimensions needs to be defined.

We know that the dimensions of the wall images and the canvas can be different. But we also know that since the wall will be horizontally scrolling the full height of the wall images will always be the height of the device. Knowing this we can use the actual height of the images and the height of the canvas to create a ratio to proportionally scale the images:

With that we can draw the image like so:

Neat! Now how about “scrolling” to see the rest of the wall?

iScroll:

The idea of creating custom functions to track user gestures to simulate scrolling on the canvas seemed like reinventing the wheel. That’s where iScroll comes in. The library is battle tested and provided all the functionality I could ask for, but how could I combine the two?

After some thought, there are 3 main tasks I could see:

  • Create on overlay with iScroll to capture user gestures
  • load all images before scrolling – we will need to create function to loop through all the image and tell us when they are loaded, And
  • Create some logic in the draw function to know which images to draw and where.

Setting up iScroll

Let’s start by adding the iScroll library (we need the probe version for all it’s features):

Add a an HTML for the overlay (iScroll needs a wrapper element and a child element with all its content inside):

Some styling:

Now to initialize iScroll:

See the Pen Scrolling Wall V2.2 – iScroll by Pattern Interactive (@Pattern-Interactive) on CodePen.

Okay, let’s try to get the wall to move now. We will need to trigger the draw function on scroll and pass it the position scrolled to:

See the Pen Scrolling Wall V2.3 – iScroll by Pattern Interactive (@Pattern-Interactive) on CodePen.

Setting up the images

Great! Now let’s get the rest of wall moving. We have to start by loading up all of the other sections of the wall:

We will want to store the images in an array that maintains the display order – this will make it a bit easier to determine which image needs to be drawn later. One approach this a for loop but the images can finish loading at different times. The smarter method is to load them in sequence in the wall_image_urls variable.

We do this by creating a function to load a single image, then once the onload() event of that image triggers, we load the next image.

Here is what that looks like:

Next we need to add logic in the draw function to know which image to draw when the screen is scrolled. To do this we need to know the position of each image (see comments in the code for details):

Now we have to go through an array of images with conditional statements to see if we need to draw the image at the start or end of the canvas.

The final piece to revise is the resizing function. Since resizing or changing the orientation of your view will change the size of the images and the wall, we will need to adjust accordingly. All we have below is a loop to set to starting position of the images, it is the exact code that is used in the  load_wall_imgs function. And the other bit is simply resizing the iScroll enabled element.

We now have a fully functional image scrolling canvas. And here, of course, is the CodePen for it:

See the Pen Scrolling Wall V2.4 by Pattern Interactive (@Pattern-Interactive) on CodePen.

Enhance, enhance, enhance.

There were a few other ideas that I explored in effort to maximize performance and image quality. One of the performance ‘enhancements’. I experimented with was to load up all the images and base 64 encode them with the canvas.

But even setting the quality parameter to its highest (1.0)  toDataURL() , there was still a lose of image quality. Instead, for the best quality I found that drawing a larger image on to a smaller canvas. Which is actually what we’re already accomplishing this by using the img_posX_start_ratio to scale down the large 1024px images for mobile devices.

I had though since the wall guide in the app would only be landscape enabled, I could draw all the images to the canvas once and take a snapshot with toDataURL() to get the exact sizes. This way I don’t have to worry about the image aspect ratio and draw them as is. Additionally for devices like an iPod which aren’t 1024 px tall in landscape I won’t have to store the larger version of the images. While scrolling seemed silky smooth there was now a much longer initial loading time. And, as I mentioned, even setting the second parameter to 1 which tells it to use high quality the images still ended up a bit pixelated and was quite noticeable on retina screens.

Note: if your having trouble with image quality on canvases this article may be a worth a read: https://www.html5rocks.com/en/tutorials/canvas/hidpi/

Momentum

A quick but important note about iScroll. iScroll has has an attribute called deceleration which adjusts the amount of momentum when scrolling. I found that I decreasing the deceleration time – and therefore decreasing the amount of momentum – had a significant impact improving performance, especially for certain mobile devices.

Finally we have a re-sizable 10,000+ px scrolling wall. In some ways it seems kinda simple now but at the time it was an exploratory process. This of course doesn’t include GPS which was added later. I had a fun getting scrolling to work correctly with all the different mobile devices we needed to comply with and hope this helps you with your project.

Check it out

The final working version of the UFG wall can be seen on the mobile app or on the UFG website. We also went on to use this same technology in the Great War memorial installation where people could look through a list of 65,000+ names. You can read more about that here.

Categories

  • Mobile Applications
  • Prototypes
  • Websites