Tutorial "How to create a 3D panorama with a single camera and a fisheye lens"

Welcome to Nodal Ninja Forum

Results 1 to 15 of 26

Thread: Tutorial "How to create a 3D panorama with a single camera and a fisheye lens"

Share/Bookmark

Threaded View

  1. Tutorial "How to create a 3D panorama with a single camera and a fisheye lens"

    #1

    --------------------------------------------------------------------------------------------------
    You can find a revised tutorial here:

    http://www.nodalninja.com/forum/show...heye-lens-quot

    --------------------------------------------------------------------------------------------------


    Hello Forum,

    3D images (also called stereo images) are not new, they exist for over 100 year, but 3D spherical panos are new and unknown to a lot of people.
    In this tutorial I will explain how to shoot and process a 3D spherical pano with the use of a DSLR camera, a fisheye lens, a Nodal Ninja panohead with a 24 or 36 clickstop rotator, a stable tripod, PTGui Pro, the free Windows application StereoPhoto Maker (works great in OSX with Parallels or Wine) and an anaglyph viewer. The tutorial contains all steps of the workflow and at the end there are links to an example and screenshots of the processing.

    But first some theory.
    You already know that the brain is getting different data from the left and right eyes, the data is interpreted by the visual system of the brain and presented to the other parts of the brain as a 3D image with "depth".
    We can see depth because the eyes are delivering images that are not the same and can have a lot of parallax.
    It is the same parallax that we try to avoid as much as possible when we take our panos.
    We will use this knowledge that parallax is needed for depth to create 3D panos.
    Getting the parallax we need is very simple, all we have to do is to shift the lens forwards in an offset NPP position.

    To get a better understanding about "how it works" we will follow 2 imaginary objects that are both positioned somewhere in the middle of the right half of the image, so at approx. 75% of the width of the image.
    One object is placed in the foreground of the scene, close to the camera, and the other object is placed in the far distance, in the background of the scene.
    Now we rotate the panohead with one clickstop (=10 degree for a 36 degree detentring).

    Due to the small angle of rotation the parallax error between the objects will be small so we continue rotating with more clickstops and while doing this the objects are "moving" with each clickstop from the right part to the left part of the image and after 6 - 9 clickstops (the number is depending on the focal lenght of your lens) both objects are in the center of the left half of the image at approx. 25% of the image width.
    Obviously by turning the panohead we "moved" the objects from the right to the left side of the image.

    The parallax between the objects in the first shot (at 75%) and the last shot shot (at 25%) is huge and it is this huge parallax that we will use to create a 3D pano.
    For this we will use StereoPhoto Maker (SPM), this app will take small strips of the left and the right half of all images (at 25% and 75% of the image width) and connect all "left" strips to make a "right eye" pano and connects the "right" strips for the "left eye" pano. Note that the left/right positions of the strips and the panos are crossed.

    With SPM we can output the "left eye" and "right eye" pano in several ways, as a combined anaglyph, for crossed eye viewing, side by side, etc.

    For this tutorial I will output to anaglyph to make a "full screen" pano. The other output options you can explore for yourself.
    I used a Nodal Ninja 5 with a 36 stop detentring but it is also possible to use a Nodal Ninja 3 of 5 with a R-D16 rotator.
    If you have a Nodal Ninja MKII you can use it with a 24 stop detentring instead but then you have to limit the forward out of NPP shift of the lens to 40 mm as described in step 2 below.
    When using a 24 stop detentring you have to change "36" in "24" and "10" in "15" wherever you see "36" and "10" numbers in the steps below.

    The workflow in 14 steps:

    1) Set your rotator to 36 clickstops. Make sure that the free play of the rotator is less then 0.1 degree. If your rotator has a lot of free play then you can't use it with the method I describe in this tutorial.
    Choose a test scene that is suited, take care that there are no objects in the scene closer then 1 m to the lens.
    Only when there is very little wind outside, or better no wind at all, you can shoot outdoors, otherwise shoot indoors.
    It is important that their is no variation in the light otherwise you will get a visible seem at the 360 degee border.
    It is also important that you don't shoot in a dynamic scene otherwise you get lots of errors, you really need a "frozen" scene.
    Mount the panohead on a sturdy tripod and level the panohead.

    2) Set the upper rail in the horizontal position (zero tilt) and shift your lens forwards on the upper rail.
    When shooting in a small room with objects relatively close by then shift approx. 30 mm forwards, when shooting in a larger room then shift approx 45 mm forwards and when shooting outside then shift approx. 60 mm forwards (see also the formula in the addendum of this posting).

    For most cameras and lenses the maximum possible shift forwards is around 60-70mm, when turning the camera mounting adapter 180 degree you can gain some extra shift.

    3) Set the camera in M mode for a fixed exposure of all images, set the white balance to a fixed setting and the focus of the lens to Manual with a fixed distance.
    Shoot all 36 images around and take care that you don't move the gear by accident, you really need a stable tripod and a cable or wireless shutter release to avoid displacement of the tripod and variations in tilt and roll.

    4) In PTGui Pro you load all images and apply a lens calibrated template.
    In the Image Parameters tab you set a fixed increment for Yaw of 10 degree for all images. It is very important that your rotator is precise because you will not optimize the Yaw of any image.
    Let PTGui automaticly place the CP's.
    Make sure that you are using the Advanced mode in PTGui to get full control over the optimizing.
    In the Optimizer tab you enable the linking for Roll and Pitch for all images, this is to make sure that if there are any images with wrong placed CP's that those images will be aligned properly despite their wrong placed CP's.
    Enable optimizing Roll and Pitch for all images and start optimizing.
    For sure you will get much larger CP errors then usual but that is normal.
    You then optimize for the lens parameters D and E and optimize again.
    Delete all CP's with errors larger then 10 px and optimize as many times as needed to get all CP's errors just below 10 px while keeping an eye on the Pitch and the lens Shift params.
    If the values for D and E differ more then approx. 20 px. from the normal values for your lens then set them back to this normal value.
    Also check the value of the Pitch, when it differ more then 1 degree compared to the normal Pitch value then set it back for all images to the normal value and optimize again.
    There is a tendency that optimizing will increase the values for the lens shift D and E and the pitch parameters so you have to take care that they don't go wild.
    When this happens you have to go to the Control Points tab and check if there are any CP's in the scene that are close to the setup. In an ideal situation you will only have CP's around the horizon of the scene.
    When the CP max is below 10 px and the lens params D and E and the Pitch of the images are more or less normal then the optimizing is done.
    Now set the Yaw of all images to an increment of 0 (zero) degree and stop optimizing.
    Congratulations, when you have come to here you have finished the hardest part.

    5) Open the Panorama Editor window, all images are stacked on top of each other, this is because the Yaw of all images is 0 degree.
    Now reduce the width of the pano with the slider at the bottom of the panorama from 360 degree to the value that is needed to get a nice cropped image, this will be 90-180 degree, depending on the focal length of your lens and the sensor size of your camera.

    6) Go to the Create Panorama tab.
    Make sure that the Link width and height option is enabled.
    Enter the height for a normal full screen pano.
    Because the width and heigth are linked the width will be calculated by PTGui.
    Make a note of the width, you need it later.
    Now some values must be calculated, the corrected height and the shift (you need the value for the shift in step 10).
    To show you how the math is done I will use the values of the tutorial example.

    Height of image: 3500 px (height for a normal full screen pano)
    Width of image: 2625 px (the width as calculated by PTGui, note the value)
    Total size: 36*2625=94500 px.
    Width of equirectangular: 2*3500=7000 px.
    Total overlap: 94500-7000=87500 px.
    Overlap per image: 87500/36=2430.55 px, rounded to 2431 px.
    Shift per image pair: 2625-2431=194 px.
    Corrected width of the equirectangular: 36*194= 6984 px.
    Corrected height of image: 6984/2=3492 px.

    Replace the normal height of the pano by the corrected height you calculated, the width will change to a wrong value.
    To correct the width first disable the Link and width option and then replace the wrong value of the width by the value you noted earlier.
    It is very important that you keep the order of the entering of the values exactly as described above otherwise the height of the output is not 180 degree !
    To make sure that you didn't make a mistake go to the Panorama Settings tab and check that the vertical height is indeed 180 degree. If it is not 180 degree then repeat step 6 (all except for the math).

    7) Go back to the Create Panorama tab and set the output format to JPEG and Individual layers only.
    Hit the Create Panorama button, ignore the message that you didn't optimize after changing CP's and after some time you will end up with 36 cropped equirectangular images.

    8) The next steps are all done in StereoPhoto Maker (http://stereo.jpn.org/eng/stphmkr/)
    Choose the menu option "Make slide show list..." from the File menu, check the option "For panorama", browse to the location of your cropped equirectangulars, enter a name for the list, press "Add all files" and let SPM create the list, you will get a confirmation, press "OK" and close the "Make Slde Show List" window.

    9) Choose the menu option "Open slide show list" from the File menu and SPM will directly show the location of the slide show list file you just created.
    Uncheck the " Auto start Slide Show" option, check the option "Color Anaglyph" and press "Open".

    10) In the image window you see the first of the 2 cropped equirectangular images on top of each other.
    Now they need to be shifted to a position that all objects are in line.
    With the file menu "Easy adjust" you get a window with a slider on top of your images. Use the slider and the arrow buttons at the left and right side of the slider to precisely set the calculated shift of step 6 and press the OK button in the Easy adjust window.
    It is very well possible that some objects are not in line with eachother when you have set the calculated shift but this is normal.
    Press the keys "Shift" and "Y" simultaneously and now the shift of all images is set and the slide show list is updated and saved.

    11) Choose the menu option "Mosaic images" from the File menu. Choose from the popup menu "Files of type" the option "Slide show list" and select the slide show list.
    Check the options "Crop" and "Symmetrical Cropping (for Stereo pano)".
    Now you enter the values needed for taking the small strips out of the images.
    Enter in the first box a value equal of 25% of the width of your images, in the second box you enter 0 (zero), in the third you enter nothing (the app will fill in this value, based on the data in the slideshow list) and in the fourth box you enter the height of the images.

    Here is the calculation for the Crop settings for my tutorial example:
    value first box: 25%*2625 = 656.25, rounded to 656 px.
    value second box: 0 px.
    value third box: leave whatever is inside
    value fourth box: 3492 px

    Press "Connect Files", you get a confirmation window and press "yes".

    12) After a short time the strips are connected into a left and a right eye equirectangular and the results will be showed in the image window.
    Choose from the View menu the option "Panorama Mode (360 degree)" and the option "Fit panorama height to screen".
    If you see the left and right stereo images side by side then use the menu Stereo > Color Anaglyph > color(red/cyan) to combine them into an anaglyph image.
    As you can see the stereo seperation is way to much so both images must be shifted closer to eachother.
    With the Easy Adjustment menu option you align the close by objects in the foreground by shifting the images in the horizontal direction, after zooming in to 50-100% you can check and fine tune the alignment of the images with the left and right arrow keys.

    Put on your anaglyph viewer and check the depth.
    When you are satisfied about the result you can save the 3D pano, if not then you can output new left and right panos by setting a different value in the first Crop box (step 11).
    With a smaller value in the first box the left and right strips are taken out from a position with more parallax in the images and with a larger value the parallax will be smaller.
    Tip: don't put to many parallax in the 3D images, it is better to use to little then to much parallax.

    13) You can save the anaglyph with the "File" menu option "Save Stereo Image ...", you will get several options how you want to save the 3D image.
    The app will also save the left and right pano.

    14) Convert the equirectangular anaglyph to an interactive pano just as you do with any pano.
    Put it full screen and enjoy your self made interactive 3D pano.

    Here is the 3D pano that I shot and processed for this tutorial.
    http://www.dmmdh.nl/panos/3D_tutoria...image_001.html

    The first pano (image 1) needs a red/cyan viewer and the second one (image 2) an amber/blue viewer.
    Image 3 and 4 are pictures of the setup.

    Here are some screenshots of the processing of the images:
    http://www.dmmdh.nl/panos/3D_tutoria...gle_camera.pdf

    In this tutorial I described the use of StereoPhoto Maker to make and connect the strips for the left and right equirectangulars.
    It is also possible to blend strips with PTGui but then you have to cut out the strips, taken from the left and the right part of the equirectangular images, with Photoshop.
    For blending the width of the strips must be much wider then when they are connected because you need an overlap for the blending. The blending of the strips is done in PTGui (without warping them).
    Combining the blended equirectangulars into an anaglyph can be done with SPM.
    Making blended (instead of connected) left and right equirectangulars is much more work while the quality is hardly any better.

    Whatever method you use or prefer, connecting or blending, it all comes down to the same method, take parts of the left half of the image, take parts of the right half of the image, make 2 panos of the parts, combine them and you will have a 3D pano with depth.

    Success in making your own 3D panos.

    Wim

    ---------------------------------------------------------------------------------
    ADDENDUM:

    In the first version of this tutorial (version 01-08-2009) you had to resize the anaglyph equirectangular to get the proper 2:1 ratio, in this version (19-03-2010) this is no longer neeeded
    The last screenshot in the series of the processing is therefore obsolete.

    Here is a formula to calculate the number of images that are needed for 3D with a one camera system, the formula is not not based on any science but as a rule of thumb it can help you to calculate how many images are needed in relation with the forward out of NPP shift of the lens.

    Number of images needed: A x (B + forward shift out of NPP) x C / size of stitched pano

    A - reference number of images for a shoot in NPP=4
    B - reference forward shift out of NPP (cm)=1
    C - reference number of pixels of stitched pano (mpx)=20

    To make clear how to use the formula here is a table with the calculations for a pano of 7000x3500 px (=25 mpx)

    shift 2 cm > 4 x (1+2) x 25 / 20 = 15
    shift 3 cm > 4 x (1+3) x 25 / 20 = 20
    shift 4 cm > 4 x (1+4) x 25 / 20 = 25
    shift 5 cm > 4 x (1+5) x 25 / 20 = 30
    shift 6 cm > 4 x (1+6) x 25 / 20 = 35
    shift 7 cm > 4 x (1+7) x 25 / 20 = 40
    shift 8 cm > 4 x (1+:001_cool: x 25 / 20 = 45
    shift 9 cm > 4 x (1+9) x 25 / 20 = 50
    shift 10 cm > 4 x (1+10) x 25 / 20 = 55

    As you can see with 36 images around it is best to limit the forward shift to approx. 6 cm and with 24 images around to approx. 4 cm.
    ---------------------------------------------------------------------------------

    In case you want to easy make a 3D partial panorama:
    http://www.nodalninja.com/forum/show...rtial-panorama

    Please no copying/pasting of the complete text of this tutorial on other forums, it makes it impossible for other people to read the replies, clarifications and updates.
    It is much better to post a link to the tutorial like this:

    http://tinyurl.com/22kxzyu
    Last edited by Wim.Koornneef; 10-15-2010 at 04:26 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts