OnePlus recently reached out to us with an opportunity too good to be turned down. The company wanted us to visit their camera lab in Taipei and give us unlimited access to their test lab, test methodology, and engineers to explain everything to us. The information we received during this visit shed light on what goes behind the scenes to fine-tune multiple aspects of a camera’s performance. Here is what we learned.
Before getting into the specifics of testing, Taipei’s OnePlus Camera Lab is one of two entities and is responsible for tuning the camera’s algorithms using data generated by standardized test targets. The team consists of 32 engineers and is led by Simon Liu, head of the image development department at OnePlus. There is a second team in Shenzhen, China that does the same job, but using footage shot in the real world. Our story revolves around the installation of OnePlus in Taipei and a few ideas offered by the various engineers who work there, including Simon Liu.
How OnePlus tests its cameras
In 2017, OnePlus announced a partnership with DxO Mark, but when the OnePlus 5 was launched, they talked very little about it. The visit to the OnePlus Camera lab finally clarified what this partnership was all about. Within the walls of this test lab, we found numerous graphs that are used to measure various aspects of the capabilities of an imaging stack. Since then, the company has grown to utilize the resources of other test manufacturers as well, including Image Engineering and Imatest. The company has carried out tests with all established players and created its own setup to verify the quality of the algorithm that determines the nature of the camera.
It starts with the automated objective test lab, a room with over 20 test targets and a robotic arm. The whole system is automated, eliminating any user error or gap between photos. The system is calibrated so that OnePlus can test any focal length using the same graphics and simulate over a thousand light levels and multiple lighting conditions.
A robotic arm will lift the camera from its cradle to test it
The software will automatically align the camera with the test target
Robotic arm can be configured to shoot all three focal lengths with the push of a button
Automated setup eliminates human error or discrepancies between test shots
On the left of the sights are three dummy heads made of very high quality gel to simulate the texture of the skin. All three have distinct skin tones, with the aim of accommodating more colors than white. All three models also wear wigs. What we found missing was a male head with facial hair, which is surprising as it would also be a great way to collect data points when it comes to texture reproduction.
The gel-based heads that help OnePlus calibrate its cameras to different skin tones
Image data from the automated lens test lab is transmitted through one of a number of image analysis software, allowing OnePlus engineers to identify optical defects (such as different levels of sharpness across the frame) or color quirks (such as shifting white balance under a specific lighting condition).
With the OnePlus 7 Pro, Simon Liu says he is most proud of the HDR algorithm developed by the company. To test this algorithm, the company has a test box divided into two separate sections. In each section is printed a test scene, with varying color saturation, contrast, highlight information, and shadow information. Each of the two boxes is illuminated independently, allowing OnePlus engineers to create a lighting difference of up to 22 stops. This is a very big difference and allows OnePlus engineers to extract as much of the sensor as possible using software.
One of the configurations for evaluating color reproduction, dynamic range and sharpness
Data from the above two test labs is fed into a computer for analysis and after every aspect of image quality i.e. color accuracy, clarity (sharpness) and dynamic range, the acceptable set of images is sent to another computer dedicated to the learning machine. Here, the image parameters are incorporated into the imaging algorithms and therefore discarded as part of an OTA update.
Now seeing the setup and chatting with various imaging engineers, here are a few things we learned.
The secret of DxO partitions
Many of the OnePlus Imaging Lab’s test sets carried the DxO Mark mark, possibly from when OnePlus and DxO Mark entered into a partnership in 2017. DxO Mark as a company does not just evaluate imaging systems and publishes their results. , but a key part of their business is also selling the testing process to camera and lens manufacturers. This means that any company which partners with DxO to set up a camera test laboratory will naturally obtain good results from its tests. It’s a bit like having the quiz before taking an exam, but it’s not cheating. Let me explain.
DxO, over the years, has established a fairly rigorous testing protocol for imaging components. Their tests were quick to find flaws and inconsistencies in performance. Plus, their testing tends to offer some level of understanding of how this camera would perform in the real world. Therefore, if a smartphone company has used DxO’s setup to prototype and refine their camera, we can certainly be assured of a certain level of quality. This still means, however, that the scores should be taken with a slight pinch of salt.
HDR is the star of the show
We asked Simon Liu a simple question. What aspect of the OnePlus 7 (7 Pro) camera was he most proud of and his rather quick response was “the HDR algorithm”. He said the algorithm was the result of the company’s extensive work to create the lab setup not only to emulate HDR conditions, but also to incorporate moving targets into this plan. “The reason it’s special is because there are now a lot of scenes that fall in HDR. the lighting conditions are very, very complex. Simon also hints that their Shenzhen counterpart feeds a lot of real-world image data to simulate various stage and lighting conditions, all to ultimately train the HDR algorithm. He says it was a huge amount of work for the team to do.
OnePlus image engineer looks at code to polish HDR algorithm
On the unimpressive Pro mode
We asked Simon why the OnePlus 7 Pro’s Pro mode lacks critical features. If you’re wondering what this is, be aware that when shooting in Pro mode, you won’t have access to ultra-wide or telephoto lenses. Moreover, the captured RAW file is a 12 megapixel file instead of being natively 48MP. Simon tells Him that “based on the numbers we see, there are very, very few people who actually use Pro mode. Allowing the use of the other two lenses is not that difficult as it can be done, but it just takes a little time. He further adds that “on the basis of the time available, we have made the choice to put the resources on stage cover instead of the Pro mode”. When asked what it would take to make Pro mode a fully loaded feature, Simon simply replied, “a team 10 times the size”.
About 3X zoom gap
We finally asked Simon why there was a disagreement over the 3x optical zoom. He specifies that the zoom factor goes down to 2.86 (approximately), which is very close to the 3x number. Simon clarified the gap by saying that the company was aiming for a 3x field of view and instead of resorting to an optical setup which would have added thickness to the phone, they opted for capturing the image. from part of the sensor. The native focal length of the lens combined with an 8 MP capture area, the phone offers a true 3x field of view. He says this is by no means digital zoom and we’re inclined to agree. When asked why the EXIF data didn’t say the resulting image had a 3x field of view, he simply replied “EXIF data is what we program it”, alluding to the fact that ‘it could just be a bug. Simon agreed, however, that, according to conventional understanding, the way OnePlus achieves 3x zoom is not purely optical and, therefore, may not be the most accurate when referred to as “optical zoom.”
OnePlus 7 Pro camera hardware endurance test
While visiting the OnePlus Camera Lab, it’s nice to see the company investing so much in fine-tuning not only its hardware, but software as well. The equipment they use comes from three companies known to be at the forefront of image analysis tools, which should give many of us the peace of mind that the company is not skimping on the public. . One thing Simon said is that the camera team rely heavily on feedback from their community through the forums. So if you want Pro Mode to be “Pro” enough, the forum is where you ask. We have obviously shared our feedback with the engineering team and look forward to seeing it implemented in the months to come.