|
Manifest Technology Making Sense of Digital Media Technology By Douglas Dixon |
|
|
Articles: | PC Video | Web Media | DVD & CD | Portable Media | Digital Imaging | Wireless Media | Home Media | Tech & Society | Home Media: | Home Media Articles | Home Networked Media Gallery | 3D - The Next Dimension (11/2010)
by Douglas Dixon Are you ready for 3D? Have you enjoyed 3D movies? Interested in enjoying 3D movies or games at home? Or intrigued by the possibilities of shooting, creating, and sharing your own 3D pictures and videos? The consumer electronics industry is ready to help out, with 3D HDTVs, 3D Blu-ray Disc players, and 3D video games -- Plus early cameras, camcorders, and lenses for shooting your own 3D material. Even better, software tools are becoming available to create your own 3D images and video -- and you can get started at minimal cost with your existing equipment. This article and associated presentation explores these possibilities from two viewpoints:
3D - the Next DimensionHave you seen Avatar or other 3D movies? And did you enjoy them? The consumer electronics industry certainly hopes so, as 3D is being positioned as the next great revolution in home entertainment, as you can bring those movies home to enjoy in on your living room screen. There's no rest for consumers from the advance of technology, no pause to enjoy what you have, no time for upgrade fatigue. We've just completed the digital television transition in the U.S., thrown out our tube sets for thinner flat panels, and stepped up to the glory of high definition widescreen home theater with HD widescreen displays, HD cable boxes, and HD Blu-ray Disc players. So it must be time to immediately move on to the next great technology revolution -- 3D TV.
With this push to 3D starting earlier this year, you already can dive in to 3D with new 3D HDTVs and 3D Blu-ray, plus explore other ways of experiencing and experimenting with 3D, including 3D on PCs, 3D cameras and camcorders, and editing software. So is 3D poised to be the next great success like HD? The Consumer Electronics Association (CEA) trade association sees this transition playing out much like the deployment of HD TV, driven initially by sports programming and action / adventure entertainment. However, the development of 3D is going to be a more gradual process over a period of years. But the groundwork is being prepared: the CEA projects that by 2013 over a quarter of TV sets sold will be 3D TVs. And you don't need to wait to get started, or make a big investment in new equipment -- you can begin experimenting with 3D and thinking about possible applications at minimal cost and using your existing gear.
3D Movies in the TheaterAccording to the CEA, consumers have gotten excited about 3D from seeing 3D movies in theaters, and therefore are interested in enjoying 3D entertainment at home. The CEA reports that over 27 percent of U.S. adults saw a 3D movie or event in 2009.
According to the MPAA, the 3D market was a key growth driver in 2009:
If you saw a 3D movie in the theater, then you probably used glasses from RealD, described as the leading 3D technology provider for movie theaters, used by 17 of the 18 largest exhibitors in the world. The RealD Cinema System was introduced in 2005, and, as of June 2010, RealD 3D was in use on some 7,500 screens worldwide, with an additional 4,000 screens under contract to be installed. Beyond the theater, RealD also provides its 3D format, active and passive eyewear, and display and gaming technologies for 3D in the home. Its 3D technologies also have been used for piloting the Mars Rover, heads-up displays for military jets, and robotic medical procedures. References:
3D TV and Blu-ray 3D in the Living RoomThe first step in building momentum for 3D requires getting 3D displays in the home so consumers can actually view in 3D. The CEA estimates that some 8 to 9 percent of U.S. households will have a "3D-ready" display by the end of 2011. However, these are "ready" in that they can display a stereo image, but you still need to add the appropriate accessories to complete the job, including 3D glasses and the associated emitter box. Then to deliver the 3D content, there's a new Blu-ray 3D format for Full HD 3D on spinning discs, since, unlike with the advent of HD, broadcasters and cable services are not leading the market to the new format. DEG: The Digital Entertainment Group reports that nearly 40 Blu-ray titles will be available by the end of the year. In addition, some 3D video formats can be piggybacked on existing devices like set-top boxes and existing cabling without requiring changing the entire delivery infrastructure to the home.
As a result, the movement to 3D will be primed by purchases of higher-end displays and Blu-ray players that have 3D capabilities built in. Even if consumers are not initially using the 3D capabilities, they then will have the necessary components ready when they do get interested, since the price premium for 3D support is already small and growing smaller. Futuresource reports that at least 10% of all Blu-ray devices shipped this year are expected to offer 3D playback, rising to more than 25% in 2011. By 2014, nearly 40% of homes across the three key regions – Western Europe, the USA and Japan – will own a 3D Blu-ray player, recorder or home theatre. Jack Wetherill, research consultant at Futuresource, says that "sales of HD-capable and 3D-capable TVs, coupled with dramatic reductions in Blu-ray player prices are continuing to fuel interest. Add to that the burgeoning 3D Blu-ray market segment, and we'll see the format continue to gather momentum in all major markets across the globe." Jason Blackwell, director for digital home at ABI Research is less optimistic, commenting that while consumers are interested in 3D for the home, it will be important to avoid overdoing the use of 3D and having it appear gimmicky. He says that while everything looks great in HD, 3D production is more of a different look that should be reserved for special events and maybe even portions of events. References:
3D Video Games on the PC3D also adds a new dimension to video games on the PC. The NVIDIA 3D Vision system combines the 3D graphics hardware for generating the 3D signal, a 3D capable monitor for displaying the 3D imagery, and compatible 3D glasses. NVIDIA has converted some 500 games and applications to 3D, so the software generates both the left and right eye perspective views for each frame. 3D PC systems also support additional uses, including watching HD movies on Blu-ray 3D, viewing 3D on the Web on sites including YouTube, and enjoying personal videos and photos in 3D.
The performance of PC-based games continues to increase dramatically, with faster PC systems, multi-core processors, enhanced graphics processing unit (GPU) chips, and improvements in the graphics rendering software. References:
3D Deployment -- the Next HD?However, even more than the beginnings of HD, 3D will still be a product for early-adopter enthusiasts. Besides the usual questions for new formats -- how much consumers will lust for the format, high initial cost, and limited early content -- there are additional issues with the 3D format including the logistics of using glasses and technical obsolescence. ContentThe need for content can be partially addressed by converting existing 2D format to 3D. As with the earlier enthusiasm for colorizing back and white movies, "dimensionalizing" a film into 3D can be an expensive and painstaking process, especially in busy scenes, and may or may not produce satisfying results. Or the player hardware can convert 2D to 3D on the fly, as with DVD players that took advantage of HD displays by upscaling from standard definition. At CES 2010, both Samsung and Toshiba announced 2D to 3D conversion features for their new product lines. This kind of processing can produce at least moderately interesting results, but typically uses a manual adjustment to tune the effect for the type of video material. Glasses
Watching entertainment with glasses also requires dedicated focus, as in the movie theater, and not informal viewing, as when switching back and forth between the game on TV and chatting with friends. And while sports are a key driver for new display technologies, hosting a Super Bowl or World Cup party with a 3D TV will be problematic when you need to buy thirty extra pairs of expensive glasses for all your friends. TechnologyAnother possible concern, a "format war" between different 3D display formats, has seemingly been finessed through the use of display agnostic media (Blu-ray) and compatible HDMI cable interconnects. Basically, consumers should regard the early glasses as an accessory specific to the display, and not expect to be able to use the same glasses with different products. But even without a format war, technical obsolescence is always a concern for early adapters, as purchasers of first-generation products like Blu-ray players without Internet and BD-Live capability can attest. And we're even earlier in the development of both 3D technology and the associated market. For content delivery, there's a Blu-ray format, but no products for satellite or cable delivery, no established paths for Internet delivery, and, oh yes, no standards for broadcast 3D -- so early-generation 3D TVs will not be compatible with these new delivery channels to come. (Image: RealD polarized theater glasses) 3D History - Viewing in StereoSeeing Depth - 2D PerspectiveSo how do we see in 3D? Part of the answer is depth cues from a scene -- which is why we see depth in 2D photos and paintings, as demonstrated by the development of geometric perspective by Renaissance artists in the early 1400's.
References:
Seeing in Stereo - 3D StereoscopyBeyond depth cues, human stereo vision adds the perspective of combining the information from slightly different images at the two eyes for full stereoscopic views. The eyes are positioned about two inches apart, which allows us to see spatial depth and dimension. In addition, the brain uses the focus of the eye to complete the illusion of depth.
Stereoscopy (also called stereoscopic or 3-D imaging) is the technique of recording three-dimensional visual information or creating the illusion of depth in an image. Charles Wheatstone developed the scientific basis for stereography in 1838, showing that the brain unifies the slightly different two-dimensional images from each eye into a single object of three dimensions.
The easiest way to enhance depth perception in the brain, then, is to provide the eyes of the viewer with two different images, representing two perspectives of the same object, with a minor deviation equal to the perspectives that both eyes naturally receive in binocular vision. References:
Stereographs - Holmes Style Stereoscope ViewerCards with pairs of stereo images - stereographs - were developed in the 1840s, popularized in the 1850s, and eventually supplanted by other media in the 1930. ("Stereo" is derived from the Greek for "solid," so a "stereograph" is a picture that depicts its subject so that it appears solid.)
Oliver Wendell Holmes invented a hand stereoscope viewer around 1860, which was inexpensive and easy to use with a folding handle. He chose not to patent his invention: "considering it as a quasi scientific improvement, I wished no pecuniary profit from it" [Philadelphia Photographer, 1869].
References:
Holmes style stereoscope viewers are still available in kits and assembled, as are new and collectable stereographs as cards, in books, and as scans.
View-MasterThe View-Master was introduced in 1939 as a device for viewing 3D images of scenic attractions. It is now primarily a children's toy with discs of licensed characters. The media for delivering View-Master content is paper discs with transparencies around the outer edge, containing seven pairs of stereo images, with each pair placed diametrically opposite. The lever is used to rotate the disk to step to the next image pair.
Tru-Vue was an earlier company that manufactured stereoscopic horizontal filmstrips beginning in 1931. It was purchased in 1951 by Sawyer's (the manufacturer of View-Master) because Tru-Vue had an exclusive contract to make children's filmstrips based on Disney characters. Competitors included the American company Novelview from the 1930s in America and Sightseer from the 1950s in England. References:
Free Viewing with the Naked EyeFree Viewing TechniquesSeeing stereo images in 3D does not necessarily require special equipment, although viewing devices are helpful to position and adjust the images for a comfortable viewing experience. You can also learn viewing techniques to perceive stereo by looking at a pair of left and right images side by side, and then controlling the focus of your eyes, forcing yourself to focus before or after the images to drift the images together to create the perception of stereo.
You can aid the process of focusing your eyes for these techniques by placing two dots above the image, and then adjusting your eyes until you see a third dot in the middle. The trick with these techniques is to first adjust your eyes so the left and right images overlap, and then let your eyes relax for a period of time to allow the 3D effect to snap in and strengthen. Once you have the 3D effect started, the eyes can typically hold on to it even as you look around the image. But the result still can be stressful to the eyes after extended viewing. Another clever viewing technique, Mirror Split, does require one simple piece of equipment -- a mirror held perpendicular to the screen. The stereo pair is then presented with the left side flipped horizontally, and then when you focus both eyes on the right image, you see the superimposed left image for a 3D effect. Mirror Split is supported by the YouTube 3D Channel (see below), which credits a video by REQ2010.
References:
Magic Eye - Random Dot AutostereogramsRandom dot autostereograms appear to be a collection of random colored dots -- but when viewed correctly an embedded 3D object emerges from the pattern. These were popularized, along with other similar designs, in the Magic Eye book series, which also used repeating patterns with recognizable shapes. Autostereograms are viewed with the free viewing techniques discussed above. The embedded image is more of a 3D shape or depth map than a textured and colored object, and the depth effect can be reversed (into vs. out of the page) depending on how you view it.
References:
Wiggle Stereoscopy
The parallax effect also occurs when you close one eye and move the head from side-to-side -- closer objects appear to move more than more distant objects. Wigging between two images can be done in a web browser using Flash animation or a Java applet, or even more easily with an animated gif image.(Wiggle Stereoscopy animated gif: http://en.wikipedia.org/wiki/File:Stereo_wiggle_3D.gif) The result can provide a sense of the depth in a scene that is easy to view and understand, since we are experienced at seeing parallax in motion video. However, this is not true binocular stereoscopic depth perception, and the constant and jittery motion can be annoying. Adding more images as a motion video sequence can smooth the motion and provide a wider view of the 3D scene. 3D Display Technology - Displays and Glasses3D Display MechanismsViewing devices like the Holmes style stereoscope and View-Master work great for presenting a 3D experience for an individual, but displaying 3D for a group requires a different approach -- a larger display, plus (for the moment at least) some form of eyewear for each viewer that can separate out the images for the left and right eyes. The challenge with displaying 3D to a group is to somehow simultaneously present two different images on the screen, in a way that glasses can separate the two signals. Generally, this can be done in two ways: spatial (dedicate half of the frame to each eye), or temporal (alternately display the left and right images).
Other approaches use holography, or spectral separation (as in Dolby 3D Digital Cinema), displaying different wavelengths of red, green, and blue for each eye. 3D Display TechnologyThese techniques then require corresponding glasses to separate out the left and right eye images as presented by the display. 3D glasses can be passive (just lenses mounted in the frame), or active (electrically active, and therefore battery powered), with costs ranging from pennies to over a hundred dollars. The hope for the future are autostereoscopic displays, which provide a 3D effect to the naked eye, without requiring special glasses.
References
Anaglyph 3D - Passive Red-Cyan GlassesAnaglyph is a spatial 3D display technique that creates a stereoscopic 3D effect by superimposing the two images using different colors which are offset to produce a depth effect. The two colors are usually chromatically opposite, typically red (for the left eye) and cyan (blue/green). When viewed using glasses with corresponding colored eyepieces, the eye merges the image into an integrated stereoscopic image. Anaglyph is used for mass market consumer applications, since it requires only inexpensive (cardboard) glasses, and works with existing imaging equipment and displays as well as printed material.
Older anaglyph imagery that you may remember from horror films may have used red/green or red/blue lenses, sometimes with significant loss of color accuracy. With better quality red/cyan glasses, anaglyph imagery can present quite good color range (including flesh colors) with minimal color ghosting.
The big advantage of anaglyph is that it does not require expensive equipment to get into stereo -- It just works with your existing image and video processing workflow. You can save anaglyph images in standard JPEG and other image file formats, and videos in standard video formats. You then can store and deliver them as usual -- including on computer disk, physical media such as DVD and Blu-ray, and online on sites including YouTube. And you can display anaglyph imagery on any existing display, including TVs, PCs, media players, and smartphones. References:
Polarized 3D - Passive Polarized GlassesPolarization is a spatial 3D display technique that creates a stereoscopic 3D effect by projecting the two images superimposed with different polarizing filters from slightly different perspectives. When viewed using glasses with corresponding polarizing filters, the eye merges the image into an integrated stereoscopic image. Polarization is typically used for 3D movies shown in theaters, since the polarized glasses are still relatively inexpensive (although with plastic frames for better durability than cardboard), and have fewer color artifacts than anaglyph. Since the polarization is independent of viewing position, the 3D effect works well for large audiences.
The filters for the left and right eyes are oriented differently to pass the light polarized for the corresponding eye and block the light for the other eye. The polarization is typically circular (clockwise/counterclockwise) or at 90 degree angles (45 and 135 degrees), so the effect continues to work even if the head is tilted. Like anaglyph and other spatial techniques, polarization splits the frame in half for each eye, reducing the horizontal resolution by half. The RealD Cinema Eyewear is lightweight and recyclable, comes individually packaged in adult and kid sizes, and fits over prescription glasses. References:
Alternate-Frame Sequencing - Active Shutter GlassesAlternate-frame sequencing is a temporal 3D display technique that creates a stereoscopic 3D effect by alternately displaying the two different perspectives for each eye. When viewed using active shutter glasses that alternate each eye between transparent and opaque in synch with the display, the brain merges the images into an integrated stereoscopic view.
The active shutter glasses are significantly more expensive, with a liquid crystal layer powered by a battery to apply voltage to switch between open and dark. This active system also requires a wireless emitter that sends a timing signal so the glasses can alternately darken each eye in synchronization with the refresh rate of the screen. To maintain the illusion of motion, active systems must boost the display rate. for example, alternately showing an image to each eye at 120 Hz, for an overall 240 Hz refresh rate. In addition, these systems have the result of darkening your view, since each eye is closed half the time. References:
Autostereoscopic Displays - Without GlassesWhile today's commercial 3D displays require wearing glasses, there are a variety of developing autostereoscopic display technologies that don't require that the viewer be encumbered by special eyewear. Instead, these Auto 3D displays present different images when viewed from different perspectives. For the moment, these technologies are being commercialized for use as small displays, as in the the LCD viewer on the Fujifilm FinePix REAL 3D W3 digital camera (see Using 3D) and the 3D display on the upcoming Nintendo 3DS handheld gaming system, due to be released in Japan in February 2011 for around $300 (which has a slider to turn the 3D display on and off).
Several common technologies use screens with vertical strips to separate the left and right portions of an image on an underlying display, and present the different views to your eyes based on their horizontal separation. Lenticular displays use a parallel array of microscopic lenses (also used for print, as familiar from 3D postcards), and parallax barrier displays use a layer of material with precision slits.
The Fujifilm FinePix REAL 3D W3 digital camera uses a lenticular sheet with an array of microscopic concave lenses to produce binocular disparity and therefore a 3D image to the naked eye.
References:
Storing 3D - File Formats3D images and videos require new conventions and file formats for storing the pair of left and right images. These can be stored as separate files, as separate images or frames in a single combined file, or be composed together into a single image or frame. Various 3D playback devices and 3D displays support various such file and frame formats, so the design of your 3D workflow will depend on your compatible equipment and software.
For the moment, 3D file formats are somewhat ad hoc, without widely-accepted standards, so expect to need to use dedicated 3D software tools to manipulate your 3D files, and even to view them. Separate FilesOne way to store pairs of stereo frames is in separate files for the left and right images or videos. These can be saved in standard formats, so they can be easily viewed and edited.
You then can import the separate photos into a 3D photo authoring tool to manipulate and export in other formats. For videos, you can import into a standard video editor to compose together into a single frame, or use a 3D video editor to edit and export in various formats. Single FrameAnother technique for storing 3D frames in existing file formats is to compose both the left and right images into a single image or video frame. This can be done in a wide variety of ways, including splitting the frame horizontally side by side, vertically over and under, or interlaced (alternating lines). Anaglyph is another form of stereo images composed into a single frame, distinguished by color.
Squeezing two images into a single frame requires resizing the original images to fit into the available space (reducing the resolution), and also distorting the aspect ratio as stored. When played back, a compatible display will then extract the separate frames, stretch as needed, and undo any distortion. (This is similar to the way DVDs use the same 720 x 480 video frame to store movies at either standard 4:3 aspect ratio, or squashed to expand horizontally for widescreen 16:9 aspect ratio.) This is the method described for the YouTube 3D Channel (see below) -- You prepare videos into the common YouTube format by editing the two streams side by side in a single frame, and then the frames are extracted and displayed in different way during playback. The RealD Format also is an enhanced version of a side-by-side format.
Again, since both images are composed into a single frame, the stereo data can be stored and played back in standard image and video formats such as JPEG for images and video for the PC, Web, and DVD. When using standard formats, however, there is no information in the file identifying it as 3D or describing how the frames are stored, so you need to explicitly define the frame format (including which part is the left and right image). Two examples of image formats in which stereo images are stored side by side using existing compression formats are JPEG Stereo (.JPS) and PNG Stereo (.PNS). These are standard file formats with the file type changed to indicate that the contents should be processed as (side-by-side) stereo data.
References:
Dual Stream - Single File with Both Frames.The other approach to storing 3D images is to include both the left and right frames together in a single file. This has the advantage of storing the full resolution original images. However, does require the use of non-standard file formats, or at least the extension of existing formats to include both images. As a result, viewing and editing these files does require dedicated 3D tools. For still images, the MPO (Multi-Picture) format is an extension of JPEG with both the left and right images. The files have the type .MPO, which means they are not immediately viewable with conventional image viewers. But you can rename the files as .JPG to view the file normally (and see one of the images). For videos, the 3D-AVI format similarly stores both the left and right sequences in a single file. And the file is playable with standard tools as a regular AVI movie (to view one of the streams). Both the MPO and 3D-AVI formats are used by the the Fujifilm FinePix REAL 3D W3 digital camera (see Using 3D) .
References:
3D ReferencesGeneral references for more on 3D stereoscopic imaging.
-> Part 2 - Using 3D: Products, Shooting, Editing
|
|
Avatar Special Edition - Courtesy RealD 
The most obvious issue with 3D TV, though, is the requirement to wear glasses. This is not the ideal cool fashion statement for the core young adult tech enthusiast market. Glasses are clunky, especially if you already use eyewear, and are another thing to lose around the house, hidden under the cushions with the remote. 
Find
the 
Random dot stereogram showing a 3D scene of a shark
A simple stereoscopic viewing technique is to quickly alternate between the left and right images of a stereogram. This works much as a flip book can animate a series of still drawings to suggest motion, and a pan shot in a movie can provide strong depth information from motion parallax, the perceived relative speed of different objects on the screen. The depth effect from panning is so strong that 3D glasses add little additional effect to a 3D movie during scenes where the camera is moving.
Nintendo 3DS with 3D top display
Lenticular display, courtesy Fujifilm
Combining 3D frames for YouTube upload
