Generating 360 stereoscopic videos with Processing

It all started with a tweet.

My friend and frequent collaborator Joshua Davis asked about 360 video using processing. Initially I had thought he meant he wanted to run 360 videos inside a Processing sketch which is a simple matter of texturing a sphere or cube with the video file.

Alas, what he actually wanted was the ability to render his generative sketches into a 360 video, so that you can be immerged in the art work.

With a little tweaking of an existing Processing example (and a neat little shader), a day later I had a working sketch that generated a cubemap to an equirectangular image.

BTW... get ready to sh** your pants... Ben is a f***ing jedi... Joshua Davis, 28th Oct 2016, via Slack.

It was short, and a little rough around the edges, but it was a solution. I cleaned up that code and created a couple more examples.

Then Joshua showed me the Fantasynth video and asked "How can we do this?"

Now, generating stereoscopic images is a lot more complicated than a single 360 image. Paul Bourke has a fountain of research and information about it, and after studying that, along with the Unreal Engine code by Kite & Lightning that was used in generating the Fantasynth video, I set to replicating those techniques inside of Processing.

The final code I put together (that is largely a port of the Kite & Lightning plugin), is a mix of Processing and a GLSL shader. The results are pretty incredible and so far the best 360 stereoscopic result I've been able to get out of Processing. It isn't fast though; each frame takes around 30 seconds to generate. The music video I created is ~90 seconds, running at 30fps. The render time for that: 30 x 30 x 90 = 22.5 hours. The actual render time was somewhere in the range of 36 to 48 hours (it was left unmonitored while it ran, and some sections have significantly more geometery to render).

My code is available on github, giving the Processing community a new tool and the ability to turn their art into 360 videos.

For the best experience, if you have access to a VR headset (either a Samsung Gear, a Rift or Vive), download the 4k file, sit back and enjoy.