Kartik Venkataraman, Pelican Imaging Corporation’s Founder and CTO, shares with us a technology that can revolutionize the way we capture images and manipulate them. In this interview from SIGGRAPH 2014, Kartik and Steve Waskul discuss how the Pelican array camera and its ability to capture depth information has great application for manufacturers and consumers alike.
As Kartik and Steve begin their conversation, Kartik shares what is meant by the term “computational array camera.” The first part, “computational,” is simply that the Pelican array camera must perform many more computations than your standard camera. The second, “array,” means that there are several cameras in the place of one allowing for the capture of both light and depth. This design is so “powerful,” Kartik enthuses, because it is like the human eye. Essentially, because all of the cameras on the array are designed the same way, they are able to capture an image from multiple viewpoints, much in the manner in which our brain triangulates what our eyes take in from the environment. Although, Kartik explains, “the whole approach is agnostic to the number of cameras,” greater depth and final resolution can be achieved with a greater number of cameras on the array. In short, the camera can be designed to meet your resolution needs.
As the interview progresses, Steve poses a query: What if somebody wanted an 8K by 8K image with a z-depth of 4,000 pixels? In reply, Kartik confirms that because this camera can calculate depth based on the parallax, or the disparity between cameras, an image such as the one Steve suggested could be created with the right setup. Elaborating on this idea, Kartik emphasizes three components needed to calibrate the system: 1) how far the point you want to capture is displaced from one camera with respect to the other, 2) the focal length of the cameras, and 3) the pixel sizes.
What enhances the usability of this device is its application to so many industries. Because the technology is compatible with smart phones and tablets, you can now take artistic photographs with a swipe of your finger. Kartik describes how the 10-20 minute task of highlighting one object in color while fading the background to grey in Photoshop is now the work of moment. With the kind of depth information accessible to the Pelican array camera, tone mapping can be restricted to a specified depth layer, allowing you to keep one focal object in color while objects at other depths are decolorized. This technology easily lends itself to enhancing social media applications, such as Instagram, in which a photograph of this caliber would expand and elevate the types of images users can create.
Moving forward, Steve asks if this technology can be applied to creating an animation sequence in between depth layers in Photoshop. It seems he is “catching on” as Kartik validates this line of thinking with an explanation of how the software allows you to basically splice something into a 3D scene with one exception: this is now possible with live images. Because the software gives you the depth and lighting information of every pixel in a scene, you not only know where exactly your object will go, but also will be able to have that object be re-lit as though it were captured in the original image.
After discussing the simplicity of the user interface, Kartik returns to other applications in which this technology may be seen in the future. Right now, he insists, drone navigation is a huge area of research, as well as is providing better feedback to drivers in technology-powered cars. The Pelican array camera has the potential to hugely impact both applications; both are dependent on utilizing technology with highly sensitive light and depth information. Because it is capable of working in any lighting level and does not rely on the traditional stereo approach to acquiring depth information, the Pelican array camera allows for greater navigational accuracy.
Excited by the possible applications of this technology, the conversation turns toward manufacturing and distribution of the product, in which Kartik emphasizes that while they are primarily a software company, they work hand in hand with manufacturing partners to produce a product that is optimally designed. In the spirit of demonstrating this partnership, Kartik brings out his tablet that has been retrofitted with the Pelican array camera. Utilizing the tablet’s CPU, DPU and DSP, the array camera- with 16 small cameras totaling 12 megapixels- is able to create an image with a final resolution of 6-8 megapixels.
Considering the usage of the tablet’s CPU and DPU software, Steve asks how long it will take between successive shots. The great thing about this camera, Kartik replies, is that there essentially is no autofocus or shutter focus lag, so essentially, you can continue snapping away as long as you have the system memory to store the data. Each camera has such a small aperture and focal length that no autofocus actuator is needed, reducing the autofocus lag to almost nothing.
As the interview comes to a close, Steve summarizes the camera’s ability to capture one object in focus, while selectively blurring others, at the time you are composing your shot. As Kartik points out, it is certainly much easier to capture something in focus and then selectively blur things out, rather than the other way around. With congratulations in developing this multi-faceted technology, we wish them the best in the future and look forward to utilizing the Pelican array camera on our smart phones and tablets.
