3D Vision Blog

After some more experimenting and help from some readers it is time for the new updated version 0.3b of the Avisynth script I wrote about realtime conversion of 2D video to 3D. The first version of the script was a bit rough and not very functional, then the second version was more functional and improved, but still performing slow on 720p or higher resoltion. The new version does not use the slow Reform plugin anymore as it is quite heavy for the CPU for real time processing of frames as Quadrilaterals. This version can also take advantage of multi-core CPUs, just use the MT version of the script with the multithreading support included (version 0.8 for AviSynth 2.5.8). The multithreaded script is recommended if you have a dual or quad core processor and get slower framerates in order to get better framerate. Just as a reminder, this script is to be opened instead of a video file from 3D Vision Stereoscopic Player or the Stereoscopic Player in a Side by Side L/R format by default and is designed to be viewed on a computer that is equipped with a 120Hz display and Nvidia’s GeForce 3D Vision. Thanks to the help of Martin Haverland and based on some parts of eslave’s similar script the conversion can now also be used with anaglyph glasses, on interlaced displays etc. You just need to comment the StackHorizontal command and uncomment the respective mode you need to use in the script code…

## 2D to 3D Realtime Video Conversion Avisynth script v0.3b
## Made by Anton Belev at 3D Vision Blog
## http://www.3dvision-blog.com
## Additionally extended by Martin Haverland at audiomh.de for quality resize
## and different output modes including interlaced format for use in non-3d specific players e.g. windows media player.

## Open the video file for conversion, change the video file name
video2d = DirectShowSource("Avatar_Trailer_HD.avi")

## Increase video brightnes on dark videos, good for 3D Vision owners
# video2d = video2d.Tweak(Bright=10)

## Convert to RGB32 to avoid the width restrictions
video2d = ConvertToRGB32(video2d)

## Optional aspect ratio maintaining quality resize for 3d monitor target resolution.
## Very cpu intensive, may be for offline use only, e.g. in virtualdubmod.
## 2x 3.0ghz cpu may give you a framerate of 16fps while running the whole script in virtualdubmod including xvid compression in HDTV quality saving setting.
## Offers great quality in the result video for fullscreen playback in every .avs capable player later.
## Also reduces ghosting if the original file resolution is smaller than the target resolution.
# videoW = width(video2d)
# videoH = height(video2d)
## For 19" Zalman use 1280, for 22" Zalman Trimon it is 1680 etc.
# hzTargetSize = 1280
# video2d = Lanczos4Resize(video2d, hzTargetsize, hzTargetsize * videoH / videoW)
## Commenting out the above resizing maintains realtime capability!

## Get video width/height and set the frame stretch factor
## Lower the value 100 to increase frame stretch, may introduce ghosting
videoW = width(video2d)
videoH = height(video2d)
ResW = videoW + (videoW / 100)
CropW = (ResW - videoW) / 2

## Create variables for left and right frame with one frame difference
## This is the Plufrich-like simulation that creates illusion of depth from movement
f1 = video2d
f2 = DeleteFrame(video2d, 0)

## Stretch the right frame to further the depth effect
f1 = LanczosResize(f1, ResW, videoH)
f1 = Crop(f1, 0, 0, videoW, videoH)

## Stretch the left frame to further the depth effect
f2 = LanczosResize(f2, ResW, videoH)
f2 = Crop(f2, CropW, 0, videoW, videoH) 

## Output the two video frames in a side-by-side / parallel format
## Use this as a default for playing back on 3D Vision (Side by Side L/R)
StackHorizontal(f2, f1)

## Output the two video frames in a Above/Below format (like Sony?)
# StackVertical(f2,f1)

## Output the two video frames in a page flipping format for shutter glasses etc.
## The Tweak as proposed by eslave is for brighter image, modify the value 30
# f1 = f1.ConvertToYV12.Tweak(Bright=30)
# f2 = f2.ConvertToYV12.Tweak(Bright=30)
# Interleave(f2,f1)

## Output the two video frames in anaglyph red-cyan as proposed by eslave
# MergeRGB(f2.ShowRed, f1.ShowGreen, f1.ShowBlue)

## For reversed anaglyph i.e. cyan-red
# MergeRGB(f1.ShowRed, f2.ShowGreen, f2.ShowBlue)

## Output the two video frames in anaglyph yellow-blue as proposed by eslave (untested)
# f1 = f1.ConvertToYV12(matrix="PC.601").tweak(bright=12.5, sat=1.25, coring=false)
# f1 = f1.ConvertToRGB32
# f1 = f1.Levels(0, 1.05, 255, 0, 255, coring=false)
# MergeRGB(f2.ShowRed, f1.ShowGreen, f1.ShowBlue)

## Output the two video frames in anaglyph blue-Yellow as proposed by eslave (untested)
# f2 = f2.ConvertToYV12(matrix="PC.601").tweak(bright=12.5, sat=1.25, coring=false)
# f2 = f2.ConvertToRGB32
# f2 = f2.Levels(0, 1.05, 255, 0, 255, coring=false)
# MergeRGB(f1.ShowRed, f2.ShowGreen, f2.ShowBlue)

## Output two video frames in Interlaced mode
## Ueed for Zalman Trimon, Acer Aspire 3D etc.
# f1 = SeparateFields(f1)
# f1 = SelectEven(f1)
# f2 = SeparateFields(f2)
# f2 = SelectOdd(f2)
# interleave(f2,f1)
# AssumeFieldBased()
# weave()

The new singlethreaded version of the script requires you to download and install AviSynth on your PC. If you have trouble running it with a high resolution video files (720p or above) you may try the multithreaded version below, just don’t forget that you should have a multicore system for it. And don’t forget to save the code above with AVS file extension and to replace the Avatar_Trailer_HD.avi with the name of the video file you want to play.

## 2D to 3D Realtime Video Conversion Avisynth script v0.3b MultiThreading
## Made by Anton Belev at 3D Vision Blog
## http://www.3dvision-blog.com
## Additionally extended by Martin Haverland at audiomh.de for quality resize
## and different output modes including interlaced format for use in non-3d specific players e.g. windows media player.

## For this to work you need to have AviSynth with MT mod 0.7 installed!!!
SetMTMode(2,0)

## Open the video file for conversion, change the video file name
video2d = DirectShowSource("Avatar_Trailer_HD.avi")

## Increase video brightnes on dark videos, good for 3D Vision owners
# video2d = video2d.Tweak(Bright=10)

## Convert to RGB32 to avoid the width restrictions
video2d = ConvertToRGB32(video2d)

## Optional aspect ratio maintaining quality resize for 3d monitor target resolution.
## Very cpu intensive, may be for offline use only, e.g. in virtualdubmod.
## 2x 3.0ghz cpu may give you a framerate of 16fps while running the whole script in virtualdubmod including xvid compression in HDTV quality saving setting.
## Offers great quality in the result video for fullscreen playback in every .avs capable player later.
## Also reduces ghosting if the original file resolution is smaller than the target resolution.
# videoW = width(video2d)
# videoH = height(video2d)
## For 19" Zalman use 1280, for 22" Zalman Trimon it is 1680 etc.
# hzTargetSize = 1280
# video2d = Lanczos4Resize(video2d, hzTargetsize, hzTargetsize * videoH / videoW)
## Commenting out the above resizing maintains realtime capability!

## Get video width/height and set the frame stretch factor
## Lower the value 100 to increase frame stretch, may introduce ghosting
videoW = width(video2d)
videoH = height(video2d)
ResW = videoW + (videoW / 100)
CropW = (ResW - videoW) / 2

## Create variables for left and right frame with one frame difference
## This is the Plufrich-like simulation that creates illusion of depth from movement
f1 = video2d
f2 = DeleteFrame(video2d, 0)

## Stretch the right frame to further the depth effect
f1 = LanczosResize(f1, ResW, videoH)
f1 = Crop(f1, 0, 0, videoW, videoH)

## Stretch the left frame to further the depth effect
f2 = LanczosResize(f2, ResW, videoH)
f2 = Crop(f2, CropW, 0, videoW, videoH) 

## Output the two video frames in a side-by-side / parallel format
## Use this as a default for playing back on 3D Vision (Side by Side L/R)
StackHorizontal(f2, f1)

## Output the two video frames in a Above/Below format (like Sony?)
# StackVertical(f2,f1)

## Output the two video frames in a page flipping format for shutter glasses etc.
## The Tweak as proposed by eslave is for brighter image, modify the value 30
# f1 = f1.ConvertToYV12.Tweak(Bright=30)
# f2 = f2.ConvertToYV12.Tweak(Bright=30)
# Interleave(f2,f1)

## Output the two video frames in anaglyph red-cyan as proposed by eslave
# MergeRGB(f2.ShowRed, f1.ShowGreen, f1.ShowBlue)

## For reversed anaglyph i.e. cyan-red
# MergeRGB(f1.ShowRed, f2.ShowGreen, f2.ShowBlue)

## Output the two video frames in anaglyph yellow-blue as proposed by eslave (untested)
# f1 = f1.ConvertToYV12(matrix="PC.601").tweak(bright=12.5, sat=1.25, coring=false)
# f1 = f1.ConvertToRGB32
# f1 = f1.Levels(0, 1.05, 255, 0, 255, coring=false)
# MergeRGB(f2.ShowRed, f1.ShowGreen, f1.ShowBlue)

## Output the two video frames in anaglyph blue-Yellow as proposed by eslave (untested)
# f2 = f2.ConvertToYV12(matrix="PC.601").tweak(bright=12.5, sat=1.25, coring=false)
# f2 = f2.ConvertToRGB32
# f2 = f2.Levels(0, 1.05, 255, 0, 255, coring=false)
# MergeRGB(f1.ShowRed, f2.ShowGreen, f2.ShowBlue)

## Output two video frames in Interlaced mode
## Ueed for Zalman Trimon, Acer Aspire 3D etc.
# f1 = SeparateFields(f1)
# f1 = SelectEven(f1)
# f2 = SeparateFields(f2)
# f2 = SelectOdd(f2)
# interleave(f2,f1)
# AssumeFieldBased()
# weave()

For the multithreading version, besides the need for Avisynth, you’ll also need to download the latest Avisynth MT mod and then apply it. This means extracting the MT.dll library in the plugins folder of the Avisynth installation and then overwriting the original avisynth.dll with the modified one (\Windows\System32 for 32-bit OS or \Windows\SysWOW64 for 64-bit OS). After that you’ll be ready to run the script and see that all the available cores of the CPU are being used for processing and thus get a better framerate than with the singlethreading version of the script.

You can download the two scripts along with Avisynth and the MT mod in a single archive file to save you the trouble of getting all the separate files you need. And as always don’t forget to share your comments if you try the scripts and any ideas for further improvements are more than welcome too, especially if they are related to improving the performance as this is a bit of an issue at the moment.

- Download the 2D to 3D conversion scripts and all the needed installation files…

Maybe I should start by wishing you all a Happy New Year 2010 and right after that comes the new updated version 0.2b of the Avisynth script I wrote about realtime conversion of 2D video to 3D. The first version of the script was a bit rough and not very functional, but it was more like a proof of concept. The new version is a bit more complete as it works not only with a 720p videos, but with just about anything you give it in terms of resolution. I’ve also addressed another drawback of the first version and that is the fact that the script can now take advantage of multi-core CPUs as the first version was only using just one of the cores. The multithreaded script however is a different one, because it requires a bit of additional work on Avisynth in order to be used and maybe a bit unstable at times, but if you have a dual or quad core processor and get slower framerates you should definitely try it. Just as a reminder, this script is to be opened instead of a video file from 3D Vision Stereoscopic Player or the Stereoscopic Player in a Side by Side format and is designed to be viewed on a computer that is equipped with a 120Hz display and Nvidia’s GeForce 3D Vision. You may however be able to play it on other stereoscopic 3D setup you have if it is capable of playing back videos in side by side stereosopic format…

# 2D to 3D Realtime Video Conversion Avisynth script v0.2b
# Made by Anton Belev at 3D Vision Blog
# www.3dvision-blog.com

# Load the Reform plugin for the skew function
LoadPlugin ("Reform.dll")

# Open the video file for conversion, change the video file name
video2d = DirectShowSource("avatar_trailer_720p.mov")
video2d = ConvertToRGB32(video2d)
videoW = width(video2d)
videoH = height(video2d)
ResW = videoW + (videoW / 16)
CropW = (ResW - videoW) / 2

# Create variables for left and right frame with one frame difference
f1 = video2d
f2 = DeleteFrame(video2d, 0)

# Add the perspective effect to the second frame and crop it
f1 = Spline16Resize(f1, ResW, videoH)
bg = BlankClip(f1, color = $000000)
f1 = skew(f1, bg, ltopx=2, ltopy=2, lbotx=CropW-4, lboty=videoH-2, rtopx=ResW-2, rtopy=2, rbotx=ResW-CropW, rboty=videoH-2, resize = "point")
f1 = Crop(f1, CropW, 0, videoW, videoH) 

# Output the two video frames in a side by side format
StackHorizontal(f2, f1)

The new singlethreaded version of the script requires you to download and install AviSynthon your PC, and then you also need to download V. C. Mohan ‘s Reform plugin for AviSynth and extract it at the same folder when you have the AVS script and the video file you are going to play and convert from 2D to 3D in real time. In version 0.2b of the script you can replace the video file name to just about any file, no matter what is the resolution, although with HD videos over 720p you might have some trouble playing them with normal framerate, so for this the multithreading version of the script might help if you have a processor with multiple cores or even multiprocessor PC. And don’t forget to save the code above with AVS file extension and to replace the avatar_trailer_720p.mov with the name of your video file.

# 2D to 3D Realtime Video Conversion Avisynth script v0.2b MultiThreading
# Made by Anton Belev at 3D Vision Blog
# www.3dvision-blog.com

# For this to work you need to have AviSynth with MT mod 0.7 installed!!!
SetMTMode(2,0)

# Load the Reform plugin for the skew function
LoadPlugin ("Reform.dll")

# Open the video file for conversion, change the video file name
video2d = DirectShowSource("avatar_trailer_720p.mov")
video2d = ConvertToRGB32(video2d)
videoW = width(video2d)
videoH = height(video2d)
ResW = videoW + (videoW / 16)
CropW = (ResW - videoW) / 2

# Create variables for left and right frame with one frame difference
f1 = video2d
f2 = DeleteFrame(video2d, 0)

# Add the perspective effect to the second frame and crop it
f1 = Spline16Resize(f1, ResW, videoH)
bg = BlankClip(f1, color = $000000)
f1 = skew(f1, bg, ltopx=2, ltopy=2, lbotx=CropW-4, lboty=videoH-2, rtopx=ResW-2, rtopy=2, rbotx=ResW-CropW, rboty=videoH-2, resize = "point")
f1 = Crop(f1, CropW, 0, videoW, videoH) 

# Output the two video frames in a side by side format
StackHorizontal(f2, f1)

For the multithreading version, besides the need for Avisynth and the Reform plugin, you’ll also need to download the latest Avisynth MT 0.7 mod and then apply it. This means extracting the MT.dll library in the plugins folder of the Avisynth installation and then overwriting the original avisynth.dll with the modified one (\Windows\System32 for 32-bit OS or \Windows\SysWOW64 for 64-bit OS). After that you’ll be ready to run the script and see that all the available cores of the CPU are being used for processing and thus get a better framerate than with the singlethreading version of the script.

I’ll continue to work on the 2D to 3D conversion script with Avisynth, although I’m actually quite new to using this tool, but it is really very easy to use and can be quite useful at times for different kinds of video processing. So next will probably be a version of the conversion script that will output directly an anglyph video and then maybe even an interlaced image so that you’ll be able to watch these in your favorite video player and not have to use a specific stereoscopic video player. And as always don’t forget to share your comments if you try the scripts and any ideas for further improvements are more than welcome too, especially if they are related to improving the performance as this is a bit of an issue at the moment.

- Download the 2D to 3D conversion scripts and all the needed installation files…

I few days ago when I’ve written about the 3Dfier DirectShow filter Philip Heggie suggested that similar thing can be achieved with the help of a simple AviSynth script. So I’ve developed a bit on his idea and prepared a bit more advanced (still quite simple actually) AviSynth script that takes an 720p 2D video file and converts it in real time to a side by side stereoscopic 3D video. The end result is much like what you can achieve with the help of the 3Dfier, and I’m sharing the code here so that everyone may experiment with it to further improve it…

In order for you to be able to run this script you’ll first have to download and install AviSynthon your PC. Then you also need to download V. C. Mohan’s Reform plugin for AviSynth and extract it at the same folder when you have the AVS script and the 720p video file you are going to convert from 2D to 3D in real time. What the script does is to create one frame difference between the frames for the left and right eye, makes quite good illusion of depth in objects… kind of Pulfrich effect. The second thing is to create a perspective distortion for one of the frames by stretching the upper part of the image and then cropping the extensive visual information to maintain the rectangular frame size. Two very simple things that create very impressive results, although it is still far from perfect as some objects are at wrong depth and there are some weird visual artefacts on some fast moving scenes, but still the end result is very good.

# Load the Reform plugin for the skew function
LoadPlugin ("Reform.dll")

# Open the video file for conversion
video2d = DirectShowSource("avatar_trailer_720p.mov")

# Create variables for left and right frame with one frame difference
f1=video2d
f2=DeleteFrame(video2d, 0)

# Add the perspective effect to the second frame and crop it
f1 = PointResize(f1, 1360, 720)
bg = BlankClip(f1, color = $000000)
f1 = skew(f1, bg, ltopx=2, ltopy=2, lbotx=32, lboty=718, rtopx=1358, rtopy=2, rbotx=1318, rboty=718, resize = "point")
f1 = Crop(f1, 40, 0, 1280, 720) 

# Output the two video frames in a side by side format
StackHorizontal(f2, f1)

You need to copy the above code and save it to a file lets say called 2d-to-3d.avs and then load this file in the video player for example 3D Vision Video Player instead of directly loading the video file. Instead of the avatar_trailer_720p.mov you should put the file name to the video you want to convert from 2D to 3D, you can just put the name of the file if it is in the same folder or also include the full path to the file. When asked what stereoscopic format to use for the file you need to choose Side by Side in order for the player to display it correctly, don’t forget to switch to fullscreen with 3D vision Video Player of Stereoscopic Player. As you can see in the script I’m using the simplest and fastest Point resize method in order to achieve the best possible speed, other methods can also be used for better quality, but that will lead to more load on the CPU and can make the framerate too slow tor normal video playback. Even with point resize if you try to use the script on a slow CPU you might still get slower framerate, but this is to be expected, because although the script is quite simple the calculations done on the video are actually quite heavy and they have to be done on each frames a lot of times each second.

As usual if you try the above script please don’t forget to leave your comments, ideas, suggestions etc. This is something intended for more advanced users and as a proof of concept is not optimized for performance or ease of use in any way!