Sunday, June 2, 2024

One-Shot-Colour (OSC) and Debayering...

The sensor in the Seestar S50 is a 'one-shot-colour' (OSC) sensor. Unlike a monochrome sensor - where each pixel in the sensor is a light bucket over a wide range of colours in the spectrum - the OSC sensor has colour filters (red, green and blue) placed over a group of 4 pixels in a 2x2 Bayer matrix. The pattern of filters can vary from sensor to sensor - but in the Seestar S50 the order is GRBG starting from the top-left pixel and moving right - then the second row left-most moving right.

Seestar S50 Bayer Matrix

Note that there are two green pixels in the group of four, with the remaining two red and blue. A number of characteristics arise from this.

  • In an image of dimensions W x H, there are (W x H)/2 'green' pixels and (W x H)/4 'red' and 'blue' real physical pixels.
  • With three colours in RGB colour space, this number needs to be 'rounded' up to 4 in order to form a symmetrical repeating pattern.
  • Green is 'doubled-up' because of the history of OSC sensors is in normal photography and the eye is most sensitive to green.
  • In order to provide a value of all three colours for every pixel, a de-bayering (de-mosaicing) algorithm is used. This is a process of generating values from adjacent pixels. There are different de-bayering algorithms and different Bayer matrix ordering. Therefore - when exporting non-debayered image data the Bayer matrix pattern must accompany the data.
In summary - for every 'real' red or 'blue' pixel there are two 'green' pixels. That is, the 'real' resolution for green is twice that of red or blue. This can be seen in the image data.

To illustrate that I've taken a small tile of an image (outlined in red) from the Seestar S50 and examined it in detail. The data is from an image which has been 'stretched' using a CDF (cumulative density function) as described here.
The tile shown below is 25 x 25 pixels and is taken from an area of noisy data. The image data is displayed starting with the RGB composite, then the red, green and blue component magnitudes in row/column order.

What is immediately obvious is that the spatial resolution is much better in the Green channel. The Red and Blue channels are more 'blobby'. In other words - there is more high frequency data (detail) in the Green channel versus the Red and Blue channels.

It can be seen that there are areas in the red and blue data where the level is 'black'. In the corresponding areas in the green data there is fine features. When combined into the RGB composite this is the source of the 'green noise'.

It is interesting to see the effects of 'modulating' the red and blue data with the higher resolution green data. The results are shown below.

Here more detail has been added to the red and blue channels by multiplying their original values by the green channel data. The effect on the original image is shown below.

There is significantly more 'sharpness' in the detail - albeit with corruption of the colour balance. Nonetheless I find this an intriguing result.