Interview with Mickael Ligot, CTO of Nodalview
Q: Mickael, tell us the secrets behind the visual quality of Nodalview panoramas. After all, you only use a simple smartphone.
A: There are several elements that need to be considered:
First and foremost, we use a special professional-quality wide-screen lens that we attach directly to the smartphone to increase the field of view, both up and down, and to reduce the sensitivity to “lens flares” when your lens is confronted with a direct or intense light source; like the sun or a projector.
This lens is mounted on the smartphone, not with a clamp, but thanks to a case that is specifically adapted to the smartphone model, to perfectly center and frame the lens. If your lens is poorly focused, you will never get good results.
Then, our panorama head stops 8 times to photograph 360 degrees when 5 or 6 stops would normally suffice: this higher coverage improves the quality of the assembly. Moreover, we cut the edges of the individual photos, as these often cause distortions. It is like having a 360° camera with 8 lenses! There are a few of them on the market, but we are in a completely different price range.
Moreover, when we increase the number of lenses on a 360° camera, other problems arise such as parallax errors: broken lines, blurred parts or even outright distortions in your result. These are caused by the gap between each lens, making it impossible to center on the nodal point.
Q: What do you mean by nodal point, parallax errors?
A: It is a physical limitation that is mentioned in every book about panoramic photography. Briefly explained: to avoid problems when merging photos taken indoors and where objects can be close to the lens, those photos must be taken from the Nodal point. This means: with the lens placed on the axis of rotation. And this is where the problem with classic 360 ° cameras comes up: with such cameras, the different lenses are physically separated. This makes it impossible for the lenses to be positioned on the nodal point as opposed to a rotating camera that is placed on that nodal point each time you take a picture. The greater the distance between the different lenses, the more parallax errors appear. This means that 360° cameras that have two lenses that are close together perform better, even if it is at the expense of resolution, than professional cameras with 6, 7 or 8, even 32 lenses! As you will have understood by now, we at Nodalview particularly paid attention to this: the panoramic head rotates perfectly on the nodal point.
Q: You are talking about a “High-Definition” image, about 10K, which is far better than the current mainstream 360° cameras. Images that compete with professional “High-End” cameras that cost a few thousand euros. Why offer such a high resolution, when 4K is not even that widespread?
A: If the image has a good resolution, you can view the panoramas not only on your smartphone, but also on the larger screen of your tablet or PC, without quality loss. You can still zoom in on details without these details getting blurred. You can also create photos and HD videos with your panoramas.
But you are right: everybody is used to watch TV in HD (2k) or even 4K, provided there is a good internet connection. That same quality will therefore be expected during a virtual visit. When taking panoramas, this means that you must have a minimum of 8K! To obtain an HD (2K) display in all 4 directions, it is necessary to have a resolution of at least 4 x 2 = 8K so that the panorama is equally sharp everywhere. So, to get a visual HD display, your panorama must have at least an 8K resolution.
Q: So, if I understand you correctly, high resolution = high image quality?
A: Not just that! The resolution alone is not enough, I would even say that the essential element lies elsewhere: namely a good dynamic range or color range. As soon as the natural lighting conditions are not optimal, cameras focus on a certain point to better adapt to the current conditions. This often results in gloomy or overexposed photos. You cannot achieve a perfect result with just one exposure.
Let us take the example of an image where a piece of furniture is placed under a window that lets the sunlight in. Without any technical adjustment, you will get a result where the furniture is displayed correctly, but the outside view is completely overexposed. Or you put the emphasis on the outside view and the details of your furniture become invisible due to underexposure. The solution to this problem is to take at least 2 photos, one that is correctly exposed for the outside view and the other for the furniture, and then to combine the best pixels of both photos. At Nodalview we take 5 “brackets” (= the same image, but differently exposed) per azimuth for one panorama: this means 40 photos per panorama, and up to 11 “brackets” per photo!
Q: You take up to 11 brackets for a photo, and no less than 8×5 = 40 brackets for panoramas. How do you explain this difference?
A: Indeed, for a panorama we take 40 brackets from the same scene instead of 88. In fact, we use a peculiarity that is specific to our recording system: a high coverage of almost 50%, with which we obtain the exposure information of the following and previous photo and allows us to double the available information.
Q: What is the difference between HDR and AEB?
A: High Dynamic Range (HDR) is a widely-used technique that ensures a better exposed photo, starting with the best pixels of usually 3 different exposures of the same scene. This requires a longer exposure time and it is important not to move during shooting. The dynamic range obtained depends on the power of the sensor and the processor to immediately create a “blended” image.
AEB (Automatic Exposure Bracketing) uses a similar procedure, namely different exposures for the same scene, which gives a much larger dynamic range. This is because a larger number of brackets (up to 11 at Nodalview) with a higher contrast difference between the images (from “almost completely black” to “almost completely white”) takes the entire light spectrum into account! This makes it possible to efficiently deal with bright backlight, as is often the case during a virtual tour. The camera takes individual photos (these are the “brackets”), which must then be “merged” by an external program. This technique provides much better results than the HDR technique, at the expense of a more complex post-processing that, for a high-resolution photo, cannot be done on a smartphone, but can be done on powerful servers.
Q: What technology do your servers use?
A: It is a technology with which we achieve results that look “better than reality” thanks to brackets. A whole range of filters and improvements are applied by our servers: level balance, optimized sharpness processing, etc. We vary the shutter speed of the sensor, which allows for a maximum amount of information about the image to be gathered. Next, algorithms are applied to the recording conditions thanks to data collected by the sensors of the smartphone, such as the amount of ambient light or the intensity of the backlight.
But the secret is the artificial intelligence that is used to enhance the image: the program has been trained with hundreds of thousands of examples of images shot outside and inside, and it continues to learn without any limitations in order to be able to apply the correct measurements to each photo. It is for this reason that you cannot obtain such results with your smartphone alone. Not without running them through our servers with AI first. And we are only at the start of new opportunities that IA offers.
A comparison between panoramas made with Nodalview and different 360° cameras is available here.