The topic of 2D to 3D conversion is getting hot lately, mostly because of a lot of new movies being shot in 2D and then converted to 3D (not so well done) and all that is resulting in not very satisfied viewers that are disappointed from what they see in the cinema. But when I’m talking about 2D to 3D conversion “by hand” most people don’t even have the slightest idea what does that process actually mean, how it is being done and most of all why it is so time consuming and requires big teams with a lot of people working together. Recently I’ve found a nice application by the German company VisuMotion GmbH called z.l.i.c.e. 3D that turns to be a quite good example on how the conversion can be done with the help of Depth Maps. And the best of all is that there is a free trial version of the VisuMotion z.l.i.c.e. 3D software that you can download and try for yourself, you just need to do a quick free registration and you’ll be able to download and test it.
Now let me start with a little bit of an explanation of what is a Depth Map, but before that you should know that there are quite a lot of different methods for creating defining depth maps of and image (photograph or a single frame of a video) and that this is not the only way to convert 2D visual information into 3D. Depth maps are usually grayscale graphic files that define the depth of each pixel of the image so when you combine a 2D still image with a depth map file you can generate a second view of the image with the needed offset as defined in the depth map so that the result is a stereoscopic 3D image. As I’ve mentioned a depth map file is a grayscale file with pixels raging from pure white to complete black, where if a pixel is brighter it is closer and if it is darker it seems far from you. So a pixel with white color seems close to you, a gray one will be in between and a completely black one will be furthest away from you.
What you need to do to convert a single 2D photo into a stereoscopic 3D image is to define one or more layers of depth for the different objects displayed on the image. If you create more layers it means you’ll have more depth information and the depth effect will feel more natural and close to reality, but more layers need more work as a single object (a person in the photo) might even need 5, 10 or even more layers with depth and that requires more time to be done precisely. For example in the VisuMotion z.l.i.c.e. 3D software you are using curves to draw an outline around an object to create a layer with depth information and you need to closely follow the edges in order to not mess up the depth information. After you create the depth layer outline you need to fill it in with “color” – either static value between white and black or a gradient going in some angle and changing between two values of depth. Depending on the complexity of the image and your level of experience it might take you from a few minutes to a few hours to convert just a single image from 2D to 3D.
Now if you need to convert a whole movie that has 24 frames per second and its length is an hour and a half imagine how much time it will be required to process all the lets say 130 thousand frames you’ll have. If you take for example 1 hour per frame you’ll need 130000 hours or somewhere around 15 years of non-stop work for a single person which sounds crazy right. That is why there are some helpful tools that will aid much into converting consecutive frames in a video, that help reduce the time needed greatly, but still it is just too much work for a single person. And that exactly is why when working on a movie there are usually hundreds and maybe even thousand of people working simultaneously to greatly cut the needed time into weeks instead of years. When you define the depth layers of all the objects in a single frame of a scene of the movie you can use the so called trackers that try to automatically follow the borders of the objects you’ve outlined along their movement in the following frames. This requires you to do just some tweaking of the layers where the tracking did not go very well on a number of frames what cover a part of the scene or a whole scene.
Furthermore when we are talking about a movie shot in 2D and then converted to 3D that does not mean that every single frame is in stereoscopic 3D, on the contrary – there are less scenes that are actually being converted in 3D, usually the action scenes or moments with more effects are first on the list for conversion with the idea to try to make you “really” feel the moment or the atmosphere. But when we talking about a movie that has actually been shot with a 3D camera things are a bit different, as you have much more depth information in every recorded frame and then you need to work out additionally on the visual effects scenes as there you need to also bring depth to all the additional effects that you bring up into the actual video footage. Lets consider this as an example to compare a 2D to 3D conversion to a real shot done with 3D cameras – you have a shot of the hero of the movie with some background that is not so important. If you have to convert this from 2D to 3D you’ll need to create layers with depth information that will bring the hero closer to the viewer and the background deeper into the screen. You can use just a single depth layer around the the hero to make his figure stick out of the background, but that will look like too artificial, like a figure cut out of paper as the figure of a person has much more depth that you can define with a single layer.
So in order not to just make a “cheap 3D effect” you’ll need at least a few layers for the hero and then maybe some more for the background, depending on how important it is. For example the conversion of a closeup of a face of a person might take you at least 5-10 layers with depth information just to make it look realistic, but if it was shot in real 3D it will have much more layers and will of course still look more natural and real. And as you can imagine when talking about conversion of videos you cannot just go and say you’ll have 100 or more depth layers for a single frame as that not only sounds unreasonable, but will require much more people and time and of course will cost in times more. That is why usually the movies being converted to 3D don’t go for more than 10 layers of depth information in the more complex scenes and often can go with as little as just two or three layers most of the time. As you can imagine the results are not always so good and you don’t have to imagine them if you’ve watched movies that were shot in 2D and then converted to 3D such as Alice in Wonderland or Clash of the Titans that had too little time and probably resources planned for the conversion process right from the start as there are schedules to keep up with.
And then again there is another factor that usually does not do good for the conversion to 3D and that is the fact that there are people doing and deciding things and when you have a lot of different people working on the same project things might not go so smooth. In the 2D to 3D conversion process you actually don’t have a level of depth you need to stick with, you can decide to go with a level that seems Ok for you, but is not right for most other people. And you may also mess up by willing to create a Wow type of an effect with some object that pops out of the screen, but instead of doing good you might as well mess up things for the viewers completely. Not to mention that due to the factor of human error you may even mess up the depth levels of different objects by ignoring even some of the most basic rules that our eyes and vision use in order to distinguish in reality when one object is actually in front of another one. And when you are messing up depth in a non natural way the audience can get confused by actually seeing contradicting in terms of depth placement things on the screen and this may result not only on bad feedback of the movie, but in actually making the viewers uncomfortable and tire them additionally when they went to watch a movie in order to relax.
So I’ve tried to outline some (not all and not too detailed) of the things that are associated with the 2D to 3D conversion process, including some of the issues and mistakes that can lead to bad results. But now let me get back to the VisuMotion z.l.i.c.e. 3D software that I encourage you to download and try in order for you to see how the process of conversion can be done by you and meet with some of the issues mentioned above firsthand. The video above is an official tutorial from VisuMotion that shows you the basic things you need to do in order to convert a short scene of a video from 2D to 3D, but you can use it to do a simpler conversion of a single photo – just follow the steps for the first frame of the video and skip after the tracking process for the rest of the frames. What is kind of missing from the tutorial in the video above is the end part where you need to export the end result in order to watch it on your PC and see how well did the conversion go. So in order to do that, after you finish the steps shown in the video, you’ll need to add additional Output node called ImageWriter and connect it to the output of the Layer Depth Map node and then you need to just Render the output to have the final image. Unfortunately the software does not show you a realtime preview that supports 3D Vision output, but you can use Anaglyph or any other of the supported modes, provided you have the required hardware and render the output to compare the end result more than once when you make changes. It is certainly not the most convenient thing to do, but this is after all just to get a first hand feel of the conversion process of 2D to 3D that relies on Depth Maps.