3D Vision Blog

################################################################
#                           2D to 3D CONVERSION 
#  Copyright (C) 2010   under GPL by    Branko Jermanis        <branko.jermanis@hi.hinet.hr>
#  My web pages: "Nikola Tesla and My Thoughts":   http://free-ri.htnet.hr/Branko/index.html
#
# This codes are based on 2d-to-3d-03b.avs script from Anton Belev at 3D Vision Blog (http://www.3dvision-blog.com),
# and some ideas from fauxD code from eslave,
# and Caleb Davis ideas with light depth detection codes,
# with indirect help of all that create this Avisynth language and useful plugin functions. Thanks to all...
# Tools that I use:
#  1.  Avisynth video scripting language
#  2.  AvsP script editor
#  3.  VirtualDub video editor (I use xvid compression for avi output)
#  4.  Media Player Classic
#  5. Download helper for Firefox browser (download from YouTube...)
#  6. WinFF convertor (flv to xvid conversion ...)
# For multimedia linux, I sugest Mint audio/video distribution:
# (I add real time kernel, JACK, Ardour, Hidrogen, Rosegarden... with M-Audio 1010LT sound card)
# Tip: press ESC for Grub, add gksudo in properties of icons, disable Timidity server on start and add starting line in JACK
################################################################

#LoadPlugin ("MT.dll")
#SetMTMode(4, 1)  # For multicore procesor (you need copy MT.dll and its avisynth.dll in .../windows/system32 directory)
# It can crash...
#Technical info from MT documentation:
#It contains the two new functions SetMTMode() and GetMTMode() and is needed by MT.dll.
#Install it by overwriting avisynth.dll in your c:\windows\system32 (and remember to take a backup of the old file first)
#On linux with Wine, I have with standard Avisynth version around 4 frame for second (Phenom II 4x)  without MT version (MT don't work on linux for me) with DVD quality (~480p)

LoadPlugin ("mvtools2.dll")   # Movement detector (some old version has problem with blksize=32)



fL = DirectShowSource("Voyager-S1-10.avi").ConvertToYV12     # Input file ( minimum DVD: 480p video), DirectShow work only on windows OS
# Input on linux (it need Wine for VirtualDub, and disable DirectX in preference if you have black screen like me):
#fL = AVISource("Voyager-S1-10.avi").ConvertToYV12    # For linux input I use xvid avi format, because DirectShow don't work. With dvd-rip I transcode to xvid with 480p quality. Sometimes it need recompress XVID or 'normal recompress mode' conversion,  if 'full processing mode' don't work.

# Try to chose right bright, contrast and saturation for bad looking video:
fL = Tweak(fL, bright=10, cont=1.25, sat=1.25, coring=false)       # Dark video correction ( disable this line if video look good, and is not dark)

W = width(fL)
H = height(fL)

fR = DeleteFrame(fL, 0)  # Set the right frame with one frame difference

# If W is less than 640, algorithm for light depth will not work good enough (integer number will eat pixels, because it depend of dW/2).
dW =  W / 128   # More depth perception, but little more artifact on details (it is usefull for me on 640x480 video)

fL = ConvertToRGB32(fL)   # Input in RGB, because of 'crop-even' problem with YV12 (on some movie)


######################    M O T I O N    D E P T H     D E T E C T I O N    ###############


## Motion mask:
super = MSuper(fR)
bvectors = MAnalyse(super, isb = true, blksize=32)  #  greatest blocksize, less problem with picture and best speed (I have been reported that on Vista 64 bit has error with 32 with some old version of mvtools2.dll).

m = MMask(fR, bvectors, kind=3, ml=1)
m = RGBAdjust(ConvertToRGB32(m),1,1,1,1,-127,-255,-255)
m1 = RGBAdjust(ShowRed(m), .5, .5, .5, 1,0,0,0,0,255,255,255,1).Invert

fR = ConvertToRGB32(fR)

# Replace moving object on frame with oposite movement from next frame (copy/paste from R to L):
m = Layer( fL, Mask(fR, m1) )
fL = Layer( fR, Mask(fL, m1) )
fR = m

# Posible improvement in this part of code :
# 1. Some detection when  camera  rotate around object create oposite Pulfrich effect, and then it need oposite switching left/right frames (rare condition)
# 2. With high speed vertical movement, frames are too different and create bad visual overlap, and it will be better switch off this cut/paste system


######################  LIGHTING    DEPTH    DETECTION   ( ideas from Caleb Davis code)  ####################

# On 0.5 version I replace Overlay with Layer function. It is faster function (2x)...

# Add more pixels for less than HD video resolution because quality and depth improvement (for HD can be disabled, because of speed):
W = 2*W
fL = LanczosResize(fL, W, H)
fR = LanczosResize(fR, W, H)

#Create L mask:
fG = Greyscale(fL)
m1 = RGBAdjust(fG, 1,1,1,1, -127,-255,-255)  # For light  scenes
m2 = RGBAdjust(fG, 1,1,1,1, -60,-255,-255)    # For dark scenes
m = ConditionalFilter(m1, m1, m2, "AverageLuma(fG.ConvertToYV12)", ">", "60")  # Switch light/dark scenes

m1 = RGBAdjust(ShowRed(m), .5, .5, .5, 1,0,0,0,0,255,255,255,1)
m2 = Layer(m1, Mask(m1, m1), x= -1*dW, op="add")       # Resize mask to left
m12 = Layer(m2, Mask(m1, m1), x= dW, op="add")          # Resize mask to right

fL = Layer(fL, Mask(fL, m12), x= dW/2)                  # move light part of picture to right on L frame (and remove borders too)
fL = Layer(fL, Mask(fL, m12.Invert), x= -1*dW/2)  # move dark part of picture to left on L frame

#Create R mask:
fG = Greyscale(fR)
m1 = RGBAdjust(fG, 1,1,1,1, -127,-255,-255)
m2 = RGBAdjust(fG, 1,1,1,1, -70,-255,-255)
m = ConditionalFilter(m1, m1, m2, "AverageLuma(fG.ConvertToYV12)", ">", "70")

m1 = RGBAdjust(ShowRed(m), .5, .5, .5, 1,0,0,0,0,255,255,255,1)
m2 = Layer(m1, Mask(m1, m1), x= -1*dW, op="add")       # Resize mask to left
m12 = Layer(m2, Mask(m1, m1), x= dW, op="add")          # Resize mask to right

fR = Layer(fR, Mask(fR, m12.Invert), x= dW/2)  # move dark part of picture to right on R frame
fR = Layer(fR, Mask(fR, m12), x= -1*dW/2)       # move light part of picture to left on R frame

# Going back to standard input Width (for HD can be disabled, because of speed):
W = W/2
fL = LanczosResize(fL, W, H)
fR = LanczosResize(fR, W, H)

# Posible improvement in this part of code :
# 1. Sometimes background is more white than foreground. Then it will be better if this is in reverse.

####################  OUTPUT TO GLASSES:  #############################


### UNIVERZAL FILE FORMAT OUTPUT (FOR ENCODING):
# If you need ColorCode format output from this file, you can use my YouTube3D.avs script to convert with VirtualDub
#return StackHorizontal(fL, fR)
#return StackVertical(fL, fR)
# Or found some standard format (Colorcode?) with some differences information (put around picture)...

### ColorCode, Intel InTrue3d ( brown/blue glasses for best color viewing with standard 60-70 Hz LCD monitors):
# Plastic glasses from xinan8888 (China) on eBay is 9 $ for 6 pair (I didn't try it yet, but look good. ColorCode plastic is around 40$ for 1 pair.)
# Improvement in blue color: Need less blue color  when blue is not part of other color ( R and/or G ).
# Example: blue sky with white clouds.

fRa = MergeRGB(fR.ShowBlue, fR.ShowBlue, fR.ShowBlue)
m = Subtract(fRa, Greyscale(fR)).Levels(220, 1, 222, 0, 255)  # Diference of blue and greyscale can extract just blue places
fRb=RGBAdjust(fR, 1, 1, 1, 1, 0, 0, -50)     # Corection on blue intensity if blue is not part of other colors
fR = Layer(fR, Mask(fRb, m))

# Mask for reducing yellow and blue ghosting:
m1=RGBAdjust(Subtract(fL, fR), 1, 1, 1, 1, 0, 0, 0).Levels(200, 1, 202, 0, 255).Greyscale  # Yellow
m2=RGBAdjust(Subtract(fL, fR).Invert, 1, 1, 1, 1, 0, 0, 0).Levels(200, 1, 202, 0, 255).Greyscale   # Blue

fL=RGBAdjust(fL, 1, 1, 1, 1, 0, -10, -255) # Going down with all green color because of difference on amber filter
fR=RGBAdjust(fR, 1, 1, 1, 1, -255, -255, 10) # Going up with all blue color because of dark blue filter
#fR = Greyscale(fR)

out = MergeRGB(fL.ShowRed, fL.ShowGreen, fR.ShowBlue)   # Output before blue/yellow line intensity correction

out1=RGBAdjust(out, 1, 1, 1, 1, -20, -20, 0).Levels(0, 1, 255, 30, 250)   # Correction inside yellow depth lines
out2 = Layer(out, Mask(out1, m1))

out3=RGBAdjust(out, 1, 1, 1, 1, 0, 0, -30).Levels(0, 1, 255, 0, 220)      # Correction inside blue depth lines
out = Layer(out2, Mask(out3, m2))  # Main output after blue/yellow line intensity correction

# Posible improvement in this part of code :
# Need better filters for blue (green passing problem) and yellow (blue passing problem).
# I am optimize this code for card glasses. I wait for china plastic glasses, to see its filters.
# China glasses are come after 1 month of waiting. Nice look and feel, better transparancy, but little more ghosting. On version 0.75, I try to optimize it.
# On my 2Dto3D conversion look good, but with real 3D it has ghost problem with high depth part of scenes.
# Because of that ghosting, I reduce blue and yellow line intensity. Scenes in focus look good and for my low-level depth conversion this glasses are good enough.
# Solutions: maybe metalic filters on glass (like filters in Dolby glasses with 6 color) will improve this viewing system. But I can't find this 'perfect' filter glasses...

return out

### Red-cyan glasses (because of red color problem, I don't use them):
#fL = Greyscale(fL)
#return MergeRGB(fL.ShowRed, fR.ShowGreen, fR.ShowBlue)


##################      TECHNOLOGICAL INFO:       #########################


### Good FPS conversion algorithm:
#super = MSuper(video)
#bvectors = MAnalyse(super, isb = true, blksize=4, chroma=true, truemotion=true, search = 3)
#fvectors = MAnalyse(super, isb = false, blksize=4, chroma=true, truemotion=true, search = 3 )
#video = MFlowFps(video, super, bvectors, fvectors, num=60, den=1)     # going to  60 fps


# Interesting setup:
# 1. DLP projector (ViewSonic 120Hz PJD6211 or better)
# 2. DLP-link active switching glasses without connection or emitters: XpanD X102
# 3. We need white LED emmiters (and electronic) for simulate this white flash from projector for possible using this kind of glasses on 120 Hz monitors.
# (white flash sync is when active glasses are in switching position, and both eyes are black)
# VirtualDub can open this script. Chose video-compress-xvid, and than Save as AVI, can create useful output file.
# You need lot of CPU power...

################################################################
#                           2D to 3D CONVERSION 
#  Copyright (C) 2010   under GPL by    Branko Jermanis        <branko.jermanis@hi.hinet.hr>
#  My web pages: "Nikola Tesla and My Thoughts":   http://free-ri.htnet.hr/Branko/index.html
#
# This codes are based on 2d-to-3d-03b.avs script from Anton Belev at 3D Vision Blog (http://www.3dvision-blog.com),
# and some ideas from fauxD code from eslave,
# and Caleb Davis ideas with light depth detection codes,
# with indirect help of all that create this Avisynth language and useful plugin functions. Thanks to all...
# Tools that I use:
#  1.  Avisynth video scripting language
#  2.  AvsP script editor
#  3.  VirtualDub video editor (I use xvid compression for avi output)
#  4.  Media Player Classic
#  5. Download helper for Firefox browser (download from YouTube...)
#  6. WinFF convertor (flv to xvid conversion ...)
# For multimedia linux, I sugest Mint audio/video distribution:
# (I add real time kernel, JACK, Ardour, Hidrogen, Rosegarden... with M-Audio 1010LT sound card)
# Tip: press ESC for Grub, add gksudo in properties of icons, disable Timidity server on start and add starting line in JACK
################################################################

#LoadPlugin ("MT.dll")
#SetMTMode(4, 1)  # For multicore procesor (you need copy MT.dll and its avisynth.dll in .../windows/system32 directory)
# It can crash...
#Technical info from MT documentation:
#It contains the two new functions SetMTMode() and GetMTMode() and is needed by MT.dll.
#Install it by overwriting avisynth.dll in your c:\windows\system32 (and remember to take a backup of the old file first)
#On linux with Wine, I have with standard Avisynth version around 4 frame for second (Phenom II 4x)  without MT version (MT don't work on linux for me) with DVD quality (~480p)

LoadPlugin ("mvtools2.dll")   # Movement detector (some old version has problem with blksize=32)



fL = DirectShowSource("teste.avi").ConvertToYV12     # Input file ( minimum DVD: 480p video), DirectShow work only on windows OS
# Input on linux (it need Wine for VirtualDub, and disable DirectX in preference if you have black screen like me):
#fL = AVISource("Voyager-S1-10.avi").ConvertToYV12    # For linux input I use xvid avi format, because DirectShow don't work. With dvd-rip I transcode to xvid with 480p quality. Sometimes it need recompress XVID or 'normal recompress mode' conversion,  if 'full processing mode' don't work.

# Try to chose right bright, contrast and saturation for bad looking video:
fL = Tweak(fL, bright=10, cont=1.25, sat=1.25, coring=false)       # Dark video correction ( disable this line if video look good, and is not dark)

W = width(fL)
H = height(fL)

fR = DeleteFrame(fL, 0)  # Set the right frame with one frame difference

# If W is less than 640, algorithm for light depth will not work good enough (integer number will eat pixels, because it depend of dW/2).
dW =  W / 128   # More depth perception, but little more artifact on details (it is usefull for me on 640x480 video)

fL = ConvertToRGB32(fL)   # Input in RGB, because of 'crop-even' problem with YV12 (on some movie)


######################    M O T I O N    D E P T H     D E T E C T I O N    ###############


## Motion mask:
super = MSuper(fR)
bvectors = MAnalyse(super, isb = true, blksize=32)  #  greatest blocksize, less problem with picture and best speed (I have been reported that on Vista 64 bit has error with 32 with some old version of mvtools2.dll).

m = MMask(fR, bvectors, kind=3, ml=1)
m = RGBAdjust(ConvertToRGB32(m),1,1,1,1,-127,-255,-255)
m1 = RGBAdjust(ShowRed(m), .5, .5, .5, 1,0,0,0,0,255,255,255,1).Invert

fR = ConvertToRGB32(fR)

# Replace moving object on frame with oposite movement from next frame (copy/paste from R to L):
m = Layer( fL, Mask(fR, m1) )
fL = Layer( fR, Mask(fL, m1) )
fR = m

# Posible improvement in this part of code :
# 1. Some detection when  camera  rotate around object create oposite Pulfrich effect, and then it need oposite switching left/right frames (rare condition)
# 2. With high speed vertical movement, frames are too different and create bad visual overlap, and it will be better switch off this cut/paste system


######################  LIGHTING    DEPTH    DETECTION   ( ideas from Caleb Davis code)  ####################

# On 0.5 version I replace Overlay with Layer function. It is faster function (2x)...

# Add more pixels for less than HD video resolution because quality and depth improvement (for HD can be disabled, because of speed):
W = 2*W
fL = LanczosResize(fL, W, H)
fR = LanczosResize(fR, W, H)

#Create L mask:
fG = Greyscale(fL)
m1 = RGBAdjust(fG, 1,1,1,1, -127,-255,-255)  # For light  scenes
m2 = RGBAdjust(fG, 1,1,1,1, -60,-255,-255)    # For dark scenes
m = ConditionalFilter(m1, m1, m2, "AverageLuma(fG.ConvertToYV12)", ">", "60")  # Switch light/dark scenes

m1 = RGBAdjust(ShowRed(m), .5, .5, .5, 1,0,0,0,0,255,255,255,1)
m2 = Layer(m1, Mask(m1, m1), x= -1*dW, op="add")       # Resize mask to left
m12 = Layer(m2, Mask(m1, m1), x= dW, op="add")          # Resize mask to right

fL = Layer(fL, Mask(fL, m12), x= dW/2)                  # move light part of picture to right on L frame (and remove borders too)
fL = Layer(fL, Mask(fL, m12.Invert), x= -1*dW/2)  # move dark part of picture to left on L frame

#Create R mask:
fG = Greyscale(fR)
m1 = RGBAdjust(fG, 1,1,1,1, -127,-255,-255)
m2 = RGBAdjust(fG, 1,1,1,1, -70,-255,-255)
m = ConditionalFilter(m1, m1, m2, "AverageLuma(fG.ConvertToYV12)", ">", "70")

m1 = RGBAdjust(ShowRed(m), .5, .5, .5, 1,0,0,0,0,255,255,255,1)
m2 = Layer(m1, Mask(m1, m1), x= -1*dW, op="add")       # Resize mask to left
m12 = Layer(m2, Mask(m1, m1), x= dW, op="add")          # Resize mask to right

fR = Layer(fR, Mask(fR, m12.Invert), x= dW/2)  # move dark part of picture to right on R frame
fR = Layer(fR, Mask(fR, m12), x= -1*dW/2)       # move light part of picture to left on R frame

# Going back to standard input Width (for HD can be disabled, because of speed):
W = W/2
fL = LanczosResize(fL, W, H)
fR = LanczosResize(fR, W, H)

# Posible improvement in this part of code :
# 1. Sometimes background is more white than foreground. Then it will be better if this is in reverse.

####################  OUTPUT TO GLASSES:  #############################


### UNIVERZAL FILE FORMAT OUTPUT (FOR ENCODING):
# If you need ColorCode format output from this file, you can use my YouTube3D.avs script to convert with VirtualDub
#return StackHorizontal(fL, fR)
#return StackVertical(fL, fR)
# Or found some standard format (Colorcode?) with some differences information (put around picture)...

### ColorCode, Intel InTrue3d ( brown/blue glasses for best color viewing with standard 60-70 Hz LCD monitors):
# Plastic glasses from xinan8888 (China) on eBay is 9 $ for 6 pair (I didn't try it yet, but look good. ColorCode plastic is around 40$ for 1 pair.)
# Improvement in blue color: Need less blue color  when blue is not part of other color ( R and/or G ).
# Example: blue sky with white clouds.

#fRa = MergeRGB(fR.ShowBlue, fR.ShowBlue, fR.ShowBlue)
#m = Subtract(fRa, Greyscale(fR)).Levels(220, 1, 222, 0, 255)  # Diference of blue and greyscale can extract just blue places
#fRb=RGBAdjust(fR, 1, 1, 1, 1, 0, 0, -50)     # Corection on blue intensity if blue is not part of other colors
#fR = Layer(fR, Mask(fRb, m))

# Mask for reducing yellow and blue ghosting:
#m1=RGBAdjust(Subtract(fL, fR), 1, 1, 1, 1, 0, 0, 0).Levels(200, 1, 202, 0, 255).Greyscale  # Yellow
#m2=RGBAdjust(Subtract(fL, fR).Invert, 1, 1, 1, 1, 0, 0, 0).Levels(200, 1, 202, 0, 255).Greyscale   # Blue

#fL=RGBAdjust(fL, 1, 1, 1, 1, 0, -10, -255) # Going down with all green color because of difference on amber filter
#fR=RGBAdjust(fR, 1, 1, 1, 1, -255, -255, 10) # Going up with all blue color because of dark blue filter
#fR = Greyscale(fR)

#out = MergeRGB(fL.ShowRed, fL.ShowGreen, fR.ShowBlue)   # Output before blue/yellow line intensity correction

#out1=RGBAdjust(out, 1, 1, 1, 1, -20, -20, 0).Levels(0, 1, 255, 30, 250)   # Correction inside yellow depth lines
#out2 = Layer(out, Mask(out1, m1))

#out3=RGBAdjust(out, 1, 1, 1, 1, 0, 0, -30).Levels(0, 1, 255, 0, 220)      # Correction inside blue depth lines
#out = Layer(out2, Mask(out3, m2))  # Main output after blue/yellow line intensity correction

# Posible improvement in this part of code :
# Need better filters for blue (green passing problem) and yellow (blue passing problem).
# I am optimize this code for card glasses. I wait for china plastic glasses, to see its filters.
# China glasses are come after 1 month of waiting. Nice look and feel, better transparancy, but little more ghosting. On version 0.75, I try to optimize it.
# On my 2Dto3D conversion look good, but with real 3D it has ghost problem with high depth part of scenes.
# Because of that ghosting, I reduce blue and yellow line intensity. Scenes in focus look good and for my low-level depth conversion this glasses are good enough.
# Solutions: maybe metalic filters on glass (like filters in Dolby glasses with 6 color) will improve this viewing system. But I can't find this 'perfect' filter glasses...

#return out

### Red-cyan glasses (because of red color problem, I don't use them):
#fL = Greyscale(fL)
#return MergeRGB(fL.ShowRed, fR.ShowGreen, fR.ShowBlue)

################ emmerick add #########################################

### cyan-Red ( Inverted) glasses (because of red color problem, I don't use them):
#fL = Greyscale(fL)
#return MergeRGB(fR.ShowRed, fL.ShowGreen, fL.ShowBlue)


## Output the two video frames in a side-by-side / parallel format
## Use this as a default for playing back on 3D Vision (Side by Side L/R)
#StackHorizontal(fR, fL)

## Output the two video frames in a Above/Below format (like Sony?)
#StackVertical(fR,fL)

## Output the two video frames in a page flipping format for shutter glasses etc.
## The Tweak as proposed by eslave is for brighter image, modify the value 30
#fL = fL.ConvertToYV12.Tweak(Bright=30)
#fR = fR.ConvertToYV12.Tweak(Bright=30)
#Interleave(fR,fL)

## Output the two video frames in anaglyph yellow-blue as proposed by eslave (untested)
#fL = fL.ConvertToYV12(matrix="PC.601").tweak(bright=12.5, sat=1.25, coring=false)
#fL = fL.ConvertToRGB32
#fL = fL.Levels(0, 1.05, 255, 0, 255, coring=false)
#return MergeRGB(fR.ShowRed, fL.ShowGreen, fL.ShowBlue)

## Output the two video frames in anaglyph blue-Yellow as proposed by eslave (untested)
# fR = fR.ConvertToYV12(matrix="PC.601").tweak(bright=12.5, sat=1.25, coring=false)
# fR = fR.ConvertToRGB32
# fR = fR.Levels(0, 1.05, 255, 0, 255, coring=false)
# return MergeRGB(fL.ShowRed, fR.ShowGreen, fR.ShowBlue)


## Output the two video frames in anaglyph Ochry-blue as proposed by emmerick (untested)
fL = fL.ConvertToYV12(matrix="PC.601").tweak(bright=12.5, sat=1.25, coring=false)
fL = fL.ConvertToRGB32
fL = fL.Levels(0, 1.05, 255, 0, 255, coring=false)
return MergeRGB(fR.ShowRed, fR.ShowGreen, fL.ShowBlue)

## Output the two video frames in anaglyph Ochry-Green as proposed by emmerick (untested)
#fL = fL.ConvertToYV12(matrix="PC.601").tweak(bright=12.5, sat=1.25, coring=false)
#fL = fL.ConvertToRGB32
#fL = fL.Levels(0, 1.05, 255, 0, 255, coring=false)
#return MergeRGB(fR.ShowRed, fR.ShowGreen, fL.ShowGreen)


##################      TECHNOLOGICAL INFO:       #########################


### Good FPS conversion algorithm:
#super = MSuper(video)
#bvectors = MAnalyse(super, isb = true, blksize=4, chroma=true, truemotion=true, search = 3)
#fvectors = MAnalyse(super, isb = false, blksize=4, chroma=true, truemotion=true, search = 3 )
#video = MFlowFps(video, super, bvectors, fvectors, num=60, den=1)     # going to  60 fps


# Interesting setup:
# 1. DLP projector (ViewSonic 120Hz PJD6211 or better)
# 2. DLP-link active switching glasses without connection or emitters: XpanD X102
# 3. We need white LED emmiters (and electronic) for simulate this white flash from projector for possible using this kind of glasses on 120 Hz monitors.
# (white flash sync is when active glasses are in switching position, and both eyes are black)
# VirtualDub can open this script. Chose video-compress-xvid, and than Save as AVI, can create useful output file.
# You need lot of CPU power...