Today Nvidia has officially released the 1.0 version of two powerful VR software development kits (SDKs) – the Nvidia GameWorks VR and Nvidia DesignWorks VR that are targeted at headset, game, and application VR developers in order for this relatively new category of display devices to offer better performance and user experience. Delivering good VR games and experiences is a complex challenge, especially since immersive VR can require multiple times the graphics processing power compared to traditional 3D apps and games you not only need a good GPU in terms of performance, but also one that is optimized for VR. With these SDKs developers on Nvidia hardware should now have the tools to create amazing VR experiences, increase performance, reduce latency, improve hardware compatibility and accelerate 360-degree video broadcasts. Both SDKs deliver a comprehensive set of APIs and libraries for headset and app developers, including the new Multi-Res Shading Technology. Available publicly for the first time, Multi-Res Shading is an innovative rendering technique that increases performance by as much as 50 percent while maintaining image quality. Also the 1.0 SDK releases also add support for the new Windows 10 operating system.
For game and application developers, the GameWorks VR SDK includes:
– Multi-Res Shading — an innovative rendering technique for VR in which each part of an image is rendered at a resolution that best matches the pixel density of the warped image required by the headset. It uses the NVIDIA Maxwell chip architecture’s multi-projection capability to render multiple-scaled viewports in a single pass, delivering substantial performance improvements.
– VR SLI — provides increased performance for VR applications where multiple GPUs can be assigned a specific eye to dramatically accelerate stereo rendering.
– Context Priority — provides control over GPU scheduling to support advanced VR features such as asynchronous time warp, which cuts latency and quickly adjusts images as gamers move their heads, without the need to re-render a new frame.
– Direct Mode — treats VR headsets as head-mounted displays accessible only to VR applications, rather than a typical Windows monitor, providing better plug and play support and compatibility for VR headsets.
– Front Buffer Rendering — enables the GPU to render directly to the front buffer to reduce latency.
For developers of professional VR applications in markets such as manufacturing, media and entertainment, oil and gas, and medical imaging, NVIDIA DesignWorks VR builds on the core GameWorks VR SDK with the addition of powerful tools, such as:
– Warp and Blend — new APIs that provide application-independent geometry corrections and intensity adjustments across entire desktops to create seamless VR CAVE environments, without introducing any latency.
– Synchronization — techniques to prevent tearing and image misalignment while creating one large desktop that is driven from multiple GPUs or clusters. Various technologies like Frame Lock, Stereo Lock, Swap Groups and Swap Barriers are available to help developers design seamless and expansive VR CAVE and cluster environments.
– GPU Affinity — provides dramatic performance improvements by managing the placement of graphics and rendering workloads across multiple GPUs.
– Direct for Video — enabling VR and augmented reality environments such as head-mounted displays, CAVES/immersive displays and cluster solutions.
AMD has also been more active on VR support lately with the recent announcement of their AMD LiquidVR Technology for Developers. One of the key technology goals of LiquidVR is to reduce unwanted processing latency (reduce motion-to-photon latency) and deliver a consistent frame rate. AMD recently released the Alpha version of its LiquidVR SDK to select technology partners. The LiquidVR SDK is a platform designed to simplify and optimize VR development.
The four major features of LiquidVR SDK include:
– Asynchronous Shaders: more efficient GPU resource management.
– Affinity Multi-GPU: faster multi-GPU performance.
– Latest Data Latch: reduced motion-to-photon latency.
– Direct-To-Display: seamless plug and play experience.
Now the big question that remains is how soon users are going to have their hands on the new VR headset hardware such as the consumer version of the Oculus Rift that should be released sometime in the Q1 2016 or the alternatives such as HTC VIVE and others that might be coming with their own hardware. The developer hardware that has been available with most notable wider availability of the two generations of dev kits of the Oculus Rift has sparked the interest and demand for VR headset in many users that simply cannot wait to get their hands on the hardware and experience the promised great VR experiences as well as play great games in a new more realistic way.
Here is something that looks very promising and might be able to provide good virtual reality-like gaming experience when playing video games on larger displays such as an HDTV or a projector, especially for FPS games. And although it is not specifically designed for stereo 3D support, it could turn out to be a great addition to a 3D HDTV, by making the experience even more immersive. What the company called IMMERSIX is offering is a solution providing immersive experience with 6 degrees of freedom by being able to track independently markers on your gun and your head movement and this translates into realistic motion inside the game world. What makes this seem very realistic is the use of a phenomenon called moving parallax to render the virtual game world in a way that creates a more realistic feeling of depth even when you are using a 2D display devices.
IMMERSIX claims that they already have plugins for several game engines in order for game developers to quickly implement the required support, however the problem is the hardware that will provide the tracking – the infrared camera along with the gun and glasses with IR markers (passive or active markers?) that will be providing the tracking. The company developing this does not provide much information besides the embedded promo video above, but I get the impression that they are more of a software development company looking to license their technology to hardware manufacturers and this will make it harder for game developers to be convinced to add support. On the other hand if you are a user that is interested in taking advantage of this technology, before buying the hardware you would want to see some good games that are able to support it well. So, IMMERSIX development kits anyone? Anyway this does look promising and is worth keeping a track on their development if you find it interesting.
After I’ve had a day to test out and play with my Oculus Rift dev kit I’m ready to share some of the first impressions I’ve got from the device by trying out some of the still very few demos available as well as the first game that supports it – Team Fortress 2. Since I do wear prescription glasses as I’m a bit nearsighted, with -1.25 diopters what seemed to work best with the Rift was the middle B set of lenses as the A set produces a blurry image for me and the C set is a bit too much. I’ve also tested trying to fit my prescription glasses inside the Rift as they are pretty compact in size (the do fit inside), the effect I get with them inside using the A set is pretty much the same as when using the B set without the glasses. I prefer to use the B set of lenses as it is more comfortable than to try to wear my glasses inside the rift and if you wear larger prescription glasses you may have trouble fitting them inside. You should have in mind that wearing glasses when using the Rift or using the B and C sets of lenses does lead to a slight reduction in the FOV you get. So people with normal vision or if wearing contact lenses with the A set will probably get the best experience in terms of largest FOV achievable, though the difference in the FOV with the B/C sets visually is not that much noticeable.
So after setting up the dev kit with the right lenses it was time to try it out. Obviously the first thing to do when you get your development kit is to register at the Oculus Developer Center (if you still haven’t) and to download the Oculus Tuscany Demo to try out the Rift and this is exactly what I did. As soon as I put on the Rift and started the demo the WOW effect was there, the large FOV you get and the image covering your vision well enough surely makes it feel great. The immersion is very good and the responsiveness of the headtracker makes the head movements result in very natural movement of the image on the screen. A few seconds after that however you may start to notice the not so good things, such as the low resolution (roughly about 580×580 pixels per eye used in TF2 for example) – looking at the Oculus Rift LCD display without the lenses it looks great in terms of detail, but since the lenses zoom it you can clearly see the pixels. So a higher resolution display is indeed a must have for the consumer version of the device, though if you stop paying too much attention to the pixels you can still enjoy what you get. For me personally the lower resolution wasn’t actually the biggest issue, what bothers me more is the motion blur around objects while I move my head around (probably due to slower pixel response time), the blur is not so apparent when looking around with the mouse or moving with the keyboard. This means that it could be possible to improve the situation with just a software update by further improving the translation of the head tracker movements to the image being rendered on the display. Some people may not notice the motion blur that much or it may not bother them as much, but for me it is a bit of a problem, and it probably results in me getting a slight nausea when moving with the keyboard and moving my head looking around. Strangely enough I get nausea fairly quickly only in the Oculus Tuscany Demo and not in any other of the demos I’ve tried or in TF2 (or at least not as fast as in the Tuscany Demo), though I do notice the motion blur in them as well.
The other demos that I’ve tried and that you may want to try out if you still haven’t include the: Space Station Explorer that looks quite well, though not very “playable” yet, the Ogre Oculus that is Ok, even though it does not work very well and is not very complete yet, but the most impressive demo for the moment seems to be the Museum of the Microstar – Oculus Rift Edition. These demos can give you an idea on what to expect and allow you to explore various “virtual worlds” to give you an idea what is possible with the Rift, but you’d want to give Team Fortress 2 a try, it is a free to play game and is currently the only game available to actually feature support for the Rift. The Doom 3 BFG Edition or the Hawken that have Rift support announced are still not yet available to be played with the final version of the Rift dev kits. There is a detailed guide for TF2 on how to enable VR mode in the game. I’ve had a weird problem at first getting an error that the tracker is not being detected, even though it was running fine in the other demos – moving the USB cable from the USB 3.0 port it was connected to to another USB 2.0 has resolved the issue and the game ran fine after that. Playing TF2 with the Rift feels a bit strange at first and requires a bit of getting used to as unlike with the demos here you need to shoot at the enemies. The game works quite well after you get the hang of controlling it, though using the developer console and reading some texts is a it problematic due to the lower image resolution. I still haven’t played with most of the advanced options available in the game, but even the default setup works quite well for me.
I’m sure that many of you are also interested in the stereoscopic 3D experience that the Rift provides. The focus of the stereoscopic 3D support with the Rift is making things seem realistic, that is why by default the settings for the stereoscopic 3D rendering are intended to make virtual objects seem realistic in proportions. And while this works quite well in the demos, they may seem a bit flat for people sued to playing games in stereoscopic 3D mode with a lot of depth – not intended to provide realistic proportions, but just to have a lot of depth. So if realism is your goal, it works quite well even now, though the lower resolution is a bit of a drawback here as well, but virtual reality does not need to always be true to real things, it can be used to provide “unreal” experiences as well. I suppose it can take some time for developers to pick up on stereoscopic 3D support for the Rift to be able to use it as best as possible and also to give adequate user control over the depth levels. From the currently available supported software I cannot say I’m impressed by the stereoscopic 3D support as much as by the VR experience, though both work well together.
Another thing to note, not so bothering, but still a bit of an issue are the color aberrations around objects that can be seen closer to the edges of the lenses, and further improvement to the lenses and the image distortion algorithm could also help in reducing these. By default the display is set to pretty low Brightness of 0 and Contrast setting of 50 and you may want to bring the Contrast even a bit lower as the image can still be quite bright at times, no point in bringing it higher. What you should also be careful about is to keep the lenses clean and not to allow dust to enter inside and fall on the screen as it looks like large dead pixels on the display when you put on the Rift and these are hard to clean. I’ve had a few times the lenses fogging up after some time of use (I guess my face got sweaty). The weight of the unit is Ok and you quickly can get used to wearing it without being bothered at all, though the side straps needed a bit of adjusting in order not to cover my ears so I can use headphones, also the cable between the control unit and the display could be made more flexible.
After trying all of the above demos and TF2 I can say I’m quite pleased with the results and how well the Rift works in overall, lets not forget that this is still a developer kit and it is intended to help developers get to know how to work on VR applications supporting the device and be ready to have them running great with a further improved consumer version. The Rift shows a great potential, even though there are areas that need improvement in order to provide even better user experience, the headtracking is good, but we still need improvement in the display as well as further tweaking the software and improving the support. The Oculus Rift is much better in terms of immersion as compared to all other HDM devices that I’ve tried so far, mostly due to the larger FOV it provides and with a bit more improvement and refining it can really lead to bringing affordable virtual reality experiences to everyone. For now however, unless you are a software developer or an enthusiast interested in researching or programming VR applications you’d probably want to wait for the consumer version of the Rift. If you are a normal consumer and have the chance of trying out the Oculus Rift developer kit you should give it a try, but probably it will not be a good idea to order a kit yourself, better wait for the improved consumer version and for more content supporting the Rift to become available.