This is The Digital Story Podcast #602, Sept. 19, 2017. Today's theme is "iPhone X and the Rise of Computational Photography." I'm Derrick Story.

Opening Monologue

If I were to place my cameras for 2018 in a line on a table moving from left to right in terms of technical sophistication, it would go something like this: Pentax Super Program SLR, Olympus PEN-F mirrorless, and iPhone X. The Pentax can compute exposure, the PEN-F can stretch high dynamic range and add effects, and the iPhone X can build an image that is a creation of light and algorithms. The iPhone leverages software to process an image that would be otherwise difficult to capture with traditional cameras. And that's the focus of today's TDS podcast.

iPhone X and the Rise of Computational Photography

Cameras have been using computing power for quite some time. My Pentax Super Program 35mm SLR was European Camera of the Year in 1983. It uses a microprocessor to evaluate light and compute automatic exposure, setting both the shutter speed and the aperture.

The PEN-F takes exposure capability to new heights, allowing me to watch a long exposure develop on its LCD screen, then providing the option to end the exposure once the image looks the way that I want. Plus it has in-camera HDR processing, Art Filters, color adjustments, and more.

But smartphones are embracing computational photography in a very sophisticated way, using software for realtime image enhancement that goes well beyond exposure. And as I examine the iPhone X, there are some remarkable features that will literally change the face of consumer photography.

First, let's define computational photography: The theory was that software algorithms could do more than dutifully process photos, but actually make photos better in the process.

"The output of these techniques is an ordinary photograph, but one that could not have been taken by a traditional camera," is how the group described its efforts at the time. (Via CBC News.)

Now let's see how all of this plays out.

• Image Sensors and Optics - The iPhone X has two new image sensors that are 12 MP each, and both camera modules are equipped with an optical image stabilizer. The wide-angle lens offers a 28mm focal length and an aperture of f/1.8, while the telephoto lens is 56 millimeters and f/2.4. In addition, there is also a quad-LED flash. The sensor is reportedly bigger and thus, can capture 83 percent more light. Assuming a crop factor of 7 and a 1/2.9-inch sensor size on the iPhone 7, this would approximately mean a 1/2.0-inch format, which would be bigger than in other current smartphones.(via AndroidPit.)
• New Algorithms - As with previous iPhone generations, there is also a portrait mode in the new iPhones, which blurs the background, but the portrait lighting function is new. Portrait lighting works by detecting the face of the photographed person in detail and simulating several types of lighting. To process these elaborate effects, Apple has equipped its new A11 SoC with a standalone image signal processor. It will furthermore assist in focusing and noise reduction.
• Impressive Video - According to the presentation, Apple divides the captured image into a total of two million squares and analyzes their content. This compresses low-detail image areas more heavily while areas that are rich in detail are preserved as much as possible. Of course, this is nothing new--video encoders also work this way--but Apple wants to be exceptionally good at it. Apple uses HEVC for its video codec and, as usual, speaks confidently about having the best video quality of any smartphone.
  60 fps is possible at the maximum resolution, 4K. Furthermore, there are now slow-motion videos at a 1080p resolution and 240 images per second, which equals a slowdown by a factor of eight when played back at 30 fps. In comparison: The Samsung Galaxy Note 8 currently achieves only 1,280 x 720 pixels at 240 fps. (via AndroidPit.)
• Powerful Front Facing Camera - The iPhone X furthermore sports a 7 MP front camera with a 3D scanner for the new FaceID feature, allowing for portrait mode, including Bokeh effect and portrait lighting, to work with selfies as well. (via AndroidPit.)
  Hidden inside the small notch cutout at the top of the iPhone X is a significant number of new camera parts and sensors that do more than just transpose your face onto an emoji cat or scan it to unlock your phone. The front-facing camera module now contains an infrared camera, flood illuminator, proximity scanner, ambient light sensor, speaker, microphone, 7-megapixel camera, and dot projector. All of that together combines into what Apple calls its TrueDepth camera, used for Animoji, Face ID, and a number of cool camera tricks.(Via The Verge.)
• A Closer Look at Portrait Lighting Mode - Apple says it "brings dramatic studio lighting effects to iPhone." And it's all done in software, of course. Here's how an Apple press release describes it:
  "It uses the dual cameras and the Apple-designed image signal processor to recognize the scene, create a depth map and separate the subject from the background. Machine learning is then used to create facial landmarks and add lighting over contours of the face, all happening in real time."
  In other words, Apple is combining techniques used in augmented reality and facial recognition to create a photo that, to paraphrase the Stanford team, no traditional camera could take. On the iPhone X, the company is also using its facial recognition camera system, which can sense depth, to do similar tricks.
  While the underlying techniques behind many of these features aren't necessarily new, faster and more capable processors have made it feasible to do them on a phone. Apple says its new phones even have a dedicated chip for machine learning tasks. (Via CBC News.)

Take all of this and add an OLED screen: The edge-to-edge display is features Apple's proprietary advantages such as the coloration and brightness-adjusting True Tone display tech and a new Super Retina moniker that means the iPhone X sports a 2436 x 1125 resolution at 458 ppi across 5.8 inches of real estate. It's also Apple's first smartphone to come HDR-ready. All this adds up to an impressive display that is clearly the top differentiator between the iPhone X and the iPhone 8. (Via The Verge.)

Cascable Transfer with Photos for macOS "kas-ka-ball"

Here's a scenario that I faced. I had captured a handful of blue hour images of San Francisco the other evening from the top of the Metreon building. I wanted to transfer a few of them to my MacBook Air 11", but I had forgotten my card reader. (There isn't a SD card slot on the 11".) Fortunately, I had Cascable Transfer loaded, so I could send my selected images directly from the OM-D E-M10 Mark II to the laptop. And the best part is, they went directly into Photos for macOS, RAW files and Jpegs alike.

Cascable Transfer is a $15 app that supports WiFi enabled Canon, Nikon, Fuji, Olympus, Sony, and Panasonic cameras. It makes it easy to transfer RAW, Jpeg, or RAW+Jpeg to the Finder, Photos for macOS, or Lightroom (which I'll cover next week).

Each RAW+Jpeg pair took about a minute to transfer. Once the process was completed, my images were waiting for me in Photos. The procedure is easy. First, turn on WiFi in the camera, then connect to the network on your Mac, then launch Cascable Transfer and browse the images on the card.

This is a wonderful safety net for those times your card reader isn't available (or working) and you want to copy pictures from your camera to your laptop.

Updates and Such

Big thanks to all of our Patreon members!

Our next Nimble Classroom focuses on Luminar Pro Techniques on Saturday Sept. 23. If you want to master this amazing image editing application, join us online for this workshop. We chat among ourselves via a Slack classroom, and I'm teaching via live video and screen sharing. It's a blast!

B&H and Amazon tiles on www.thedigitalstory. If you click on them first, you're helping to support this podcast. And speaking of supporting this show, and big thanks to our Patreon Inner Circle members.

And finally, be sure to visit our friends at Red River Paper for all of your inkjet supply needs.

Texas-based Red River Paper recently announced a new fine art paper, Palo Duro Etching. The new paper is a 100 percent cotton rag paper and is free of optical brightener additives. The paper is designed to offer warm white tones, deep blacks and a subtle texture to accurately recreate traditional darkroom fine art prints.

See you next week!

More Ways to Participate

Want to share photos and talk with other members in our virtual camera club? Check out our Flickr Public Group. And from those images, I choose the TDS Member Photo of the Day.

Podcast Sponsors

Cascable - Cascable is the best tool available for working with your camera in the field.

Red River Paper - Keep up with the world of inkjet printing, and win free paper, by liking Red River Paper on Facebook.

The Nimbleosity Report

Do you want to keep up with the best content from The Digital Story and The Nimble Photographer? Sign up for The Nimbleosity Report, and receive highlights twice-a-month in a single page newsletter. Be a part of our community!

Want to Comment on this Post?

You can share your thoughts at the TDS Facebook page, where I'll post this story for discussion.