Musings on “Retina Display” quality for Virtual Reality

tl,dr: How much resolution do VR HMDs need to approach that of a “Retina Display”? Using the current optics setup, about 10x more than they do now. Roughly 22,000 × 10,060 pixels per display.

I work for Framestore’s Virtual Reality Studio. Occasionally I ponder big questions. Sometimes these paths cross. We also make a lot of videos – both for the incorrectly named 360º videos (of which I’m firmly in the “this is NOT Virtual Reality” camp, they just use the same hardware as VR) and as backdrops for VR experiences.

I was wondering just how good we should make our videos in VR without going overboard. You just want to  make the resolution of the video good enough so it either hits the resolution of the display, OR it hits the maximum resolution of the eye – whichever is lower. So this got me to thinking, what IS the resolution that the typical person can discern when wearing an HMD? Hmm… where to start…

Well let’s take a guess that Apple’s definition of a retina display is a good resolution to shoot for. The definition of a Retina Display is generally accepted as 300 ppi  (pixels/inch) @ 10-12 inches from eye. (Really – who holds a display in front of their face? – no matter…)

OK, so let’s convert this into some form of useful VR measurement. If we use the formula;

PPD (Pixels /Degree) = 2 D R tan(½º)

R = resolution in pixels/unit-distance
D = distance in units-distance

 = ~57 PPD

So 57 PPD is going to be our definition of “Retina Display”. We need to convert this to the VR FOV (Field of View) to calculate how big a VR display has to be. Well’ since this is speculative let’s go the full value, the average full FOV.

SO now just how big is the average FOV, both horizontally and vertically? Well, there’s quite a few opinions, but since I’m a US taxpayer, let’s let the US Government – specifically the United Stated Department of Defense – answer this (and many many other human factors questions) in MIL-STD-1473 G – “Human Engineering” or

So if we look through this wonderful, rich, document that contains many many useful items for the aspiring VR/AR UI & UX designer, we arrive at the following diagram:


So we can see that according to the DOD, the average FOV is 190º horizontal and 120º vertically.

But.. the horizontal value is for both eyes.  If we instead use the horizontal FOV of a single eye 95º from the eye center outward, and 35º from eye center inwards, we get about 130º FOV for an eye.

For 57 PPD, this gives us 7,410 x 6840 pixel display – per eye, or a 14,820 x 6840 a full stereo FOV retina display using current lens optics with a single split screen.

I might be wrong, but current HMD’s are nowhere near that….Let’s compare to a DK2.

A Oculus DK2 FOV max is 95º horizontal and 106º vertical (from vrHmdDesc::MaxEyeFov – not using the 100 degree value). So assuming we still have 35º from eye center inwards this gives of 64º per eye horizontal FOV for a DK2.

Thus we’d need a resolution 3648 x 6042 to make a “Retina Display” at this FOV for one eye. A DK2 is 1080 x 1200 per eye. If we use the current split screen setup for left eye/right eye,  7269 x 6042 is the “Retina Display” resolution, while 2160 x 1200 is the DK2s.

But we’re not done. You see, due to the lenses, most HMD’s render out smaller area than the actual resolution. If you plot it out from a standard DK2, you’ll get the actual pixels used (white) vs the unused pixels (black);


Thus only about 68% of the pixels in the display are actually used to render what the eye can see.

So my final tally is that for an HMD to have a retina display and using lenses to bend the flat display, we’d need about 10,900 x 10,060 per eye. This is about 85 times the pixels of a DK2. Or, put another way it’s about a factor of 10x the current resolution.

Considering that I currently have to try to get a render done in about 5 milliseconds with my current target hardware (Oculus’  CV1), it looks like my role as chief performance optimization engineer is pretty safe occupation for the near future.

It’s also a good thing to know, as now when we’re doing high quality rendered video we should keep the high quality stuff archived (at 20K x 30K as the target because, 360!), because while a 10x jump in resolution sounds like a lot, I have no doubt that we’re going to rapidly iterate on VR hardware improvements and that number will soon be well within the realm of possibility.






Posted in Augmented Reality, GearVR, Graphics Hardware, Virtual Reality | Leave a comment

Making the most of your GearVR

The GearVR has a micro USB port on it that can be used for a number of different things. The trick is to plug in the USB device *before* you plug in the phone! We use the following type of adapter in our GearVR’s to connect up any regular USB-Type A device.


This allows us to power the fan for extended game playing or movie watching (makes the battery last longer) or to take screen shots by connecting a keyboard – you just need to Print-Screen and the screenshot will show up in the Phone’s screenshots folder. You can connect up any HID USB device.

Posted in GearVR, Virtual Reality | Leave a comment

Framestore’s VR Studio is pulling ahead in VR Space

My plan for domination of VR content creation space continues …muhahahahahaha

(As told by AdAge)

Production Company Standout 2015: Framestore

Shop Solidified Rep as VR Pro

Virtual reality has been the industry’s darling of late and remains top of mind for agencies and advertisers. Visual effects powerhouse Framestore planted its foot firmly in the space by creating a dedicated division to the field and has now become the destination shop for those seeking to create sophisticated virtual reality experiences.

… the article continues on in a favorable tone 🙂

Really, it’s the excellent team for folks working at Framestore that make this kind of stuff possible. The Framestore reputation for the highest quality work continues in our VR productions. Did I mention that we are hiring?

Posted in Framestore, Virtual Reality | Leave a comment

All work and some play….

Part of the VR team at Framestore New York VR Studios takes a break from virtual things and tries their hand at sculpting something real – with various levels of success.IMG_20151030_153741.2

Posted in Framestore, Virtual Reality | Leave a comment

Uploading your GearVR APKs faster – Part 2

When the $99 GearVR hits the market this year, and as folks learn that they can get a great VR experience I really think that it’s going to take off and build momentum. With the obvious (to consumers) advantage that having Netflix and Hulu and games available on your GearVR, that extra nudge to try VR and get converted to a VR enthusiast  isn’t that tough a call. Pretty much *anyone* who’s tried a GearVR with good VR content comes away desiring one. It takes entertainment to a whole other level. Great for consumers. Great for content creators. Bad for VR devs.

Developing for GearVR is a monumental pain. Take the disadvantage of Android programming in general, throw in the fact that you’re testing on a device that you not only have to upload the APK to, but you need to put the device on your face to see if it actually looks right. Plus the fact that you do NOT have a USB connection to get the debug log spew, but you are forced to either suck it off the device after it runs or (better) create a WiFi connection with your device to watch the spew scroll by. You eventually learn a bunch of tricks to ease the pain. But in the end, you’ll still need to strap it on to do final testing.

Don’t get me wrong, I love the VR aspects of GearVR – Samsung and Oculus are pushing out a high-quality consumer VR product and the mobility of it blows me away. When I used to be in Developer Relations for Intel, I had to tell my in-laws that I went to game companies and made the game code better. That’s easier to understand than optimizing rendering pipeline operations, reducing the number of samplers,  ruthless culling, optimizing cache line alignment, or multi-threading an rendering engine. Pretty much all my in-laws  now know I produce “Virtual Reality Experiences” quickly followed by a GearVR demo. The get it right away. It’s an awesome platform for mobile VR (and regular video entertainment and games as well). Did I mention that developing for it is a real pain?

So one of the inevitable steps of development for GearVR is uploading the APK to the device. Now when we had a huge video file, we were smart and just copied it to the phone and loaded in programmatically, not making it part of the APK till the very end. But eventually you need to turn off all the dev. shortcuts and start testing the entire APK, in the near final form. Building & uploading time becomes critical, especially at 3am.

One thing we did was try to choose the devices that could upload the fastest.  It turns out that, while their GPU power was slow, if you were interested in how things behaved and looked, and not how they performed, we could get away with using an S5 “Moonlight” and running it in USB3 mode.

APK Upload SpeedsI did some performance testing and here’s a graph of relative performance.  Note that prior to S6, there could be some hardware (incl. GPU) variations that would give wildly varying performance, including upload speeds. (Only one variant of the Note 4 was tested) So if you end up in the test-edit-upload-test cycle, you might find you can save yourself some time and frustration by trying to optimize upload speeds to the testing devices.


Posted in Android, GearVR, Virtual Reality | Leave a comment

How to get a job in VR

“Be prepared. Luck is truly where preparation meets opportunity.”

I am giving a talk at the NYVR Meetup in which I’ll be talking about how to maximize your chances to get a job in the VR industry.  Since it’s pretty much a brand new industry, hardly anyone has experience. This means that you can maximize your chances by doing some ground work now.

Here are the basic tenets;

  • Hang out with people who are already doing what you want to do.

These folks are your intellectual stimulation. They know stuff, take the opportunity to learn from them.  Informal groups, meetups, conferences, hackathons, or your own team are all ways to hang with similarly minded folks.

  • Educate yourself on the topic. Become the expert.

Never turn down an opportunity to learn stuff. Find out the answers to questions. Force yourself to learn, ideally by building  a real demo.

  • Commit to working on a project. And finish it.

Ideally as part of a group.

The  tools  you should be using (learning) are;
Unity (C#), Unreal (C++), Maya, Max, Photoshop, VideoStitch.
And the demo should be running on:
Oculus, Vive, GearVR, Cardboard, stereo/360 video rigs

  • Share your results

Publish your results. Blog, Webpage, Facebook, Youtube, etc. Don’t forget to show your passion when you are talking about what you’ve done. If your passion fails to show through, you won’t be a convincing hire.

“Experience is what you get when you didn’t get what you wanted. And experience is often the most valuable thing you have to offer.”

Hardly anybody in VR has multi-year on-the-job experience – so what we look for is self driven folks who’ve gone the extra mile to gain experience on their own time.

I talked about two videos folks should watch.

Randy Pausch Last Lecture: Achieving Your Childhood Dreams

Amy Cuddy: Your body language shapes who you are

Posted in Miscellaneous, Uncategorized, Virtual Reality | Leave a comment

Working in a VR Studio occasionally Gobsmacks me..

Just because…BoxOculusSmall


Posted in Uncategorized | Leave a comment

Samsung announces consumer GearVR details & a $99 price

At Oculus Connect Samsung finally finalized the consumer GearVR plans – They are coming out with an updated $99 GearVR that will universally support any of Samsung’s four new flagship phones: the Galaxy Note 5, the Galaxy S6, the Galaxy S6 Edge, and the Galaxy 6S Edge+.  They’ve added some differentiating contours to the trackpad, so less chance of fumbling around trying to find the trackpad. They’ve gotten rid of the overhead strap and made the side straps a little softer.


Plus Hulu, Netflix,Fox, Lionsgate, Twitch, Vimeo, Facebook join Oculus Video support, to support that movie-on-your-face concept. Lot’s of games, including some retro games from Oculus Arcade (will have to see how that works) and of course, Minecraft. And I’d be remiss if I didn’t mention that the Samsung announcement had a few seconds of Framestore’s ‘Gear VR Test Drive’ video.

Posted in Uncategorized | Leave a comment