vision-feature-image

More resolution, High Dynamic Range (HDR), higher frame rates… How exactly do the limits of the human visual system impact our technology?

When is enough, enough? This is one of the most interesting questions brought to mind after a discussion that came up at the recent ARRI Broadcast Day held at Die Fernsehwerft studio facilities in the new creative district of East Berlin Harbour. In a studio filled with international guests from across the industry, Marc Shipman-Mueller – ARRI’s head of camera systems – opened up this topic in a way that has permanently realigned the way I view today’s core developments in digital cinema acquisition, post and delivery.

These are purely my opinions based on an evolving thought process rooted in an exploration of new technology, and not an absolute truth. I may well end up being wrong, and am happy to learn things that may change my views expressed here. In fact, I’d love to open this up to you, and hear some of your thoughts, so I encourage you to leave comments.

The Holy Grail

A decade ago, matching the performance and aesthetic of 35mm film was the “Holy Grail” of digital cinema. We have now met and surpassed the challenges laid down at the birth of digital cinema, matching 35mm emulsion technically, if not also in almost every other way. We are now far more concerned with the nuances of digital color science than arguing over film vs. digital. That battle has been won and, for the most part, the result is universally accepted.

So, what’s next for digital cinema? And how much of it is commercially driven by the need to sell new TVs and consumer gadgets rather than by a meaningful improvement of the visual experience? Perhaps more than we’d like to think. On the other hand, after a few years of feeling through the darkness, some core standards are finally being defined and adopted throughout the industry.

Digital Cinema 2.0

If the initial phase in the evolution of digital cinema saw us match and surpass 35mm film, the second phase has shifted gear and direction entirely. It is no longer a question of imitating what came before, but imagining and realising an unknown future. Film as the holy grail of the cinematic experience was a fixed, known goal, but the question of where to head next has seen us charge down a few blind alleys only to backtrack and change course.

The Cart before the Horse?

Perhaps it is this very lack of a fixed, known goal on the horizon that has allowed us to follow technology for technology’s sake to the detriment of both the art and the consumer of filmed entertainment. Television manufacturers have charged ahead, undeterred by the recent rise and fall of 3D TV, to UHD, SUHD, HDR and whatever other acronyms they can legitimately or illegitimately make up to dazzle consumers into parting with their cash. This was already taking place before formal standards were agreed, all the while content creators were finding themselves lost for technical direction, and content providers scrambling to launch their first UHD channels and services regardless of the fact that there is very little UHD content.

UHD, High Dynamic Range (HDR), wide color gamut, high frame rate… We’ve thrown everything we can at the screen, when perhaps fine tuning a more measured combination of technologies could give the best results for both creators and audience.

Just because we can do something doesn’t mean it’s the best (or only) solution to the problem. We may, in fact, not even fully understand the problem.

Resolution

Resolution_of_SD_Full_HD_4K_Ultra_HD__8K_Ultra_HD-640x361@2x

The resolution war has quietened over the past couple of years. We all know 4K is here to stay, and we’d be foolish not to acknowledge that 8K has already made it over the horizon and is charging toward us. Granted, I don’t believe we will see 8K adopted as a mainstream delivery resolution anytime soon, but RED (and of course NHK and partners on the broadcast side of things) has made 8K acquisition a reality, and it’s actually not as impractical as the cynics would like to think.

However, simply increasing pixel resolution has some far-reaching implications.

More pixels at a higher color bit-depth means a lot more data. This increases bandwidth requirements on location, through post and, of course, delivery. The whole point of increasing pixel resolution is to increase perceived sharpness and detail in the image, although there are other ways to achieve this without the huge overhead.

There is a difference between perceived resolution and actual pixel resolution – a difference which is important to distinguish. The only thing that matters to the viewer is “perceived” sharpness and detail and, as such, it is imperative to bring the response of human vision into the equation if we’re weighing the pros and cons of different technologies, advancements and their implications. Quite often, solving one issue by brute force – such as a 400% increase in actual pixels – brings up other problems.

All too often in these discussions it seems we don’t acknowledge how we see, process and perceive the illusion of moving images at all.

Resolution and Human Vision

We have only a small region of our retinas which packs cone cells at a very high density. This area is called the “fovea” and, while it comprises less than 1% of the retina, it takes up over 50% of the visual cortex in the brain. Only the central two degrees of the visual field is focussed on the fovea, approximately twice the width of your thumbnail at arm’s length.

Retinal_Image

By Ben Bogart – Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=31009153

When you observe an object in detail which occupies more than this central two degrees of your field of vision, your eye must scan the area and your brain fills in the gaps. Your peripheral field of view is effectively of far less resolution, but your brain is able to construct the overall image of the scene in front of you as your eye passes over it. You remember and recall detail in parts of the scene after your eye has moved focus, even when in motion, which is why you maintain real-time visual awareness over your entire field of view.

What you are seeing, however, is a constantly updated construct processed in your brain rather than the exact “real” projected image your eyes are seeing. You are perceiving a computationally constructed reality all the time, something far more complex and nuanced than a camera recording the entirety of an image focussed on an image sensor at a fixed, uniform pixel density and frame rate.

This is why your eye moves to scan your screen as you read this: for you to stop and again see full detail on another part of the screen you have to move your eyes to read it again. You are aware of the rest of the text and the overall layout and spatial position of everything else in your field of view, but to actually read the text you have to physically move your eyes.

A moving image on a screen is an illusion. It is a trick that our brain perceives as a fluid moving image. We should be careful not to break the illusion, which is more fragile than you might think.

In Conclusion

Sure, 4K and 8K HDR, HFR, wide color gamut images on a massive cutting-edge OLED display might make you feel like you’re looking through a window to a real-world scene on the other side. But is reality really what we want from our storytelling?

I don’t think so.

Let us know what you think of these technologies, trends and the impact on how we tell and experience cinema in the comments below.

The post Exploring the Limits – Resolution, HDR, HFR and Human Vision appeared first on cinema5D.