Camera based Head-Up-Display Brightness Control

Modern cars use head-up displays (HUD) to make information available to the driver without the necessity to take the eyes off the road. Current systems use a simple photo-diode illumination sensor to measure the overall intensity of the environmental light and adjust the overall brightness of the HUD, accordingly.
Alternatively, a camera can be used to measure the luminance of the scene in front of the car. Due to the higher resolution of the sensor, it’s possible to adjust the brightness of the HUD much more precisely and robust. For example, when driving at night a single photodiode is not able to differentiate between the headlights of an upcoming car or an actual change in the brightness of the environment. Also, reflective surfaces can deteriorate the contrast between the HUD and the background. The goal of this project is to develop a system to adjust the brightness of a HUD given a camera image of the scene in front of the car.

The challenges are:
• Develop a robust method to measure illuminance and reflectiveness of a scene from an image.
• Account for the different point of views of the camera and the driver, leveraging 3D information provided by a stereo camera system or monocular scene reconstruction.

The project is divided into two sub problems. The first one tackles the challenge of measuring the illumination of a scene with a camera. Given a camera image and exposure time, how can we measure the illuminance of every pixel? The second one focuses on the different viewpoints of the scene between camera and driver. Due to the fact that the position of the driver’s head depends on the driver and also changes continuously, another camera is used to track the head pose in real-time. Given the transformation between camera and head and the depth of the scene one can compensate for the different view-points. To estimate the depth of the scene different approaches are possible and shall be evaluated.

Part I: Measuring the illumination of the scene:
• Literature review of methods to measure illuminance and reflectiveness.
• Implement and validate approaches on example images and in the lab.
• Investigate the limits of accuracy and resolution of the approaches.

Part II: View-point compensation:
• Investigate the quality of the view point transformation between camera and head pose given different types of depth information
     o Stereo.
     o Structure-fromMotion.
     o Piecewise planar assumptions.
     o Object detections.