﻿ Images and Channels

Images and Channels

Images are displayed on our computer monitor by varying the brightness of the red, green and blue dots on our monitor based on the numeric values in the channels that are assigned to red, green and blue respectively.  The idea of channels is a very simple idea that is easy to use.

RGB Explained

Much of how images are stored and manipulated in computers derives from the television technology that was first used to create computer graphics displays.

If we took look at our computer monitor’s display with a magnifying lens we would see that it consists of a very large number of triplets of red, green and blue dots. Each triplet of red, green and blue elements is a single pixel.  Depending on the monitor manufacturer the elements may be round dots or small squares or other shapes but each pixel will have some sort of triplet of red, green and blue elements.

In most monitors the dots are quite large and easy to see even with a weak lens.  If we have good eyesight we may be able to see the individual elements without a lens.  The dots are spaced closer to each other than in the illustration nearby, so that when viewed from a slight distance all the elements blend together without large amounts of black between them.

Images on the computer display are formed when the monitor precisely varies the brightness of the red, green and blue elements in each triplet. Because the dots are close together the human eye will fuse the three dots of varying red, green and blue brightness into what appears to be a single dot of color that is the spectral combination of the three levels of red, green and blue color. All colors perceived by most humans can be formed by the right brightness combination of red, green and blue color.

From here on we’ll refer to the Red, Green and Blue elements of images as "R", "G" and "B".

Images stored in computers are usually nothing more than a series of number triplets where each triplet of three numbers is intended to control a single triplet of R, G and B dots on the monitor.   To simplify display electronics, most dots are controlled in 256 brightness steps where a value of 0 means the dot has no brightness and a value of 255 means to increase brightness for that dot to the maximum.

A number triplet such as 67, 228, 180 means to turn the R element up to 67 brightness, the G element to 228 brightness and the B element to 180 brightness. The result will be seen by humans as a pretty shade of green-blue color. The RGB numbers may be called color values.

Image files are arranged so that there is a series of pixels in rows that correspond to the rows of tiny triplet dots on the monitor. When an image is displayed on the monitor so that each pixel triplet in the image file controls one color triplet on the monitor we say that the image is "natural size" or "100% zoom" or, as Manifold calls is, native zoom.

In most modern software packages we take for granted the ability to zoom in or zoom out of an image to see it smaller or larger than natural size. If we zoom in, our graphics software will take the RGB values in a pixel that were originally intended to control just one color triplet and will use that same value across and down as many triplets as is necessary to make the image bigger on the monitor. If we zoom out so that the image is much smaller than intended, then there will be more RGB values than there are color triplets to control. In that case our graphics software will average out the RGB pixel values to figure out what average value should be used to drive the color triplet for that spot in the image. Manifold  performs these functions automatically.

Channels

The word channels has several related meanings when used with images and data for images, with the context usually making it clear what precise meaning is intended.

Channels can refer to the number of physical elements that make up a display.   Images made up of a triplet of numbers such as R, G and B have three channels.  There is one set of numbers used to control the red elements, a second set of numbers used to control the green elements and a third set of numbers used to control the blue elements.   How these three sets of numbers are packaged and stored inside the computer, stored on disk or represented inside programming code does not matter when we are talking about the physical layout of the hardware that makes up a computer monitor.   Because it is confusing to use the word channels for two different, but closely related, things, Manifold uses outputs or display outputs to refer to the R, G and B output channels.

Channels can also refer to the sets of numbers inside an image.    If an image contains only triplets of numbers as in an image intended to control an RGB display as described above, the image will have three numbers for each pixel that directly correspond to the three R, G and B elements in the computer display. Such RGB images are said to have three channels.

Some images might have more than just three numbers for each pixel.   For example, an image might have four numbers for each pixel with three of those numbers indicating R, G and B values while the fourth number might be an alpha value that indicates what the transparency of that pixel should be if the image is combined in layers with other images.   Such an image would have four channels and is often referred to as an RGBa image.

Some images might have many channels per pixel.   For example, some sensors on satellites will record photos where simultaneously for each pixel several values are recorded.  Those values might be the intensity of light at different wavelengths, including values at wavelengths such as for infrared and ultraviolet that the human eye cannot see.   Such an image with multiple values per pixel is a multichannel image with each data channel containing the values for a given wavelength.

To view such images our viewing software will allow us to pick three of the channels out of many data channels per pixel to choose which channel should be used to control which output, that is, which channel should drive red elements, which channel should drive green and which channel should drive the blue output.  The Manifold Style dialog allows us to arrange which data channel is assigned to drive which output.

The resulting image is called a false color image because the resulting colors are not natural colors but are cobbled up from numbers that originally meant something other than real-life red, green and blue values.  But we can use the instant comprehension offered by color to see pixels where the numbers in the data are similar.

For example, the nearby illustration of Australia around the Gulf of Carpentaria was created by using gamma ray intensity numbers in three frequency ranges, one range characteristic of radioactive potassium that was used as a red output, a different frequency range characteristic of uranium used for a blue output and a frequency range characteristic of thorium used for the green output.

Even though the colors are not natural colors (Australia in that region as seen from space is mostly beige and khaki colors, not bright shades of blue, purple, reds and cyan), at a glance the false colors show how the three elements are distributed: all three elements abound in white regions, purple regions abound in uranium and potassium but are poor in thorium and turquoise regions abound in uranium and thorium but are poor in potassium.  Blue regions abound in uranium, the most interesting since it is but a bit of neutron cooking away from the Great Khan of all elements, plutonium.

Images collected by satellite sensors can contain hundreds of channels per pixel, with each channel being a number that represents some special quality collected by that sensor. Channels might code the height of that pixel, the temperature as seen from space or the reflectance as seen in various spectral bands invisible to human eyes. Since computer monitors only have three, RGB elements for each pixel, showing such multi-channel images requires software that maps the many channels into only three for display purposes. Such multi-channel images are often displayed using false color techniques that assign one or more channel values into a specific range of RGB values.

When we consider that an image that is 1000 pixels wide by 1000 pixels high contains a million pixels overall, if we have three data channels for each pixel (one number each to control the R, G and B dots) that can add up to very many bytes of data. If each number is just one byte, then a one million-pixel image will take three megabytes of space.  Larger images can easily require tens of gigabytes of space.   Not surprisingly, there have been many clever software schemes invented to reduce the amount of space required for an image.

Grayscale Images

A quick and dirty scheme is to use just one number that is the same for all three RGB dots. If the brightness of all three R, G and B dots is at the same level, we perceive no color but just different shades of gray.   That is a amazing but true, that the human eye can look at a sea of brightly colored dots but if they are small enough and all are close enough together as triplets, no color but only shades of gray will be seen.  Using one number per pixel in a million-pixel image reduces the size to only one megabyte, but at the price of seeing the image in shades of gray as a monochrome, that is, as a grayscale image.

Palette Images

Another type of one-channel image uses a single number per pixel to specify which color in a palette of 256 colors is to be used for that pixel. See the Palette Images topic for an explanation of what palettes are and how they are used. For the purposes of channels, it is important to note that images that use palettes are one channel images. This one channel is then expanded into three numbers per pixel via the palette to create three numbers to drive the R, G and B outputs.

Example: Channel Controls with an RGB Image

The Style dialog allows us to specify which channels in an image are assigned to Red, Green, Blue and Alpha outputs for display.   Using the procedure in the Example: Assign Channels using Source topic we can see how channels drive R, G and B outputs to create an image.  To save space the following illustrations show a sample image together with just the channel selection controls for R, G and B in the Style dialog.

If we open an image imported from a typical three channel format such as JPG we will see that by default Manifold automatically assigns Channels 0, 1 and 2 to their default meanings of controlling Blue, Green and Red outputs.    Using all three channels as intended shows the image in natural color.

We can turn off use of the B and G outputs by setting them to use a fixed value instead of using a channel from the data and then assigning a value of 0.   The result is an image that is displayed using only the red dots on the monitor, that is, an image shown using only a Red output.

We can set the Value of Red to 0 and the Value of Blue to 0 and instead use only the Green output to show an image entirely in shades of green.  The region of blue sky has darker red and green tones because it is predominately blue in appearance. The white clouds are bright in all three outputs because simultaneous brightness in R, G, and B results in white.

Using only the Blue output results in an image in shades of blue.

Using two outputs together results in combined colors as used in the printing industry. For example, using Blue and Red outputs with Green forced to a Value of 0 results in shades of magenta.

Blue and Green outputs together are cyan.

Surprisingly, using Green and Red outputs with the Blue output forced to a Value of 0 results in yellow tones.   Keep in mind that when mixing RGB outputs we are mixing light, not paint. Mixing green and red paint together results in muddy tones of brown, but mixing red and green light together results in yellow.

By default Manifold will assign channels as expected to R, G and B to form a natural color image when importing from formats used for such images.   However, just because the expected ordering is Channel 0 to Blue, Channel 1 to Green and Channel 2 to Red does not mean we cannot alter assign channels differently if we so choose.

In the image above we have swapped the assignments for Red and Blue outputs so that Channel 2 controls the Blue output and Channel 0 controls the Red output.  When applied to red color values, the greater intensity of values in Channel 0 in regions of the sky that is normally intended to create stronger shades of blue instead results in stronger reddish shades in those regions of the sky.  The green bronze stays about the same because it is mostly green tones and the assignment of Channel 1 to Green output has not changed.

Images

Palette Images

Style

Style: Presenting Images

Style: Autocontrast

Style: Palettes

Style: Formatting Drawings and Labels

Example: Change the Contrast of an Image - In this example we use the Edit - Style dialog to change the contrast of an image.

Example: Use the BGR Button to Assign Channels - The BGR button in the Edit - Style dialog for images allows us to assign channels in the data to B, G and R outputs that create the displayed image, using BGR, RGB or Grayscale ordering.

Example: Assign Channels using Source - The Source control in the Style dialog for images assigns channels to display outputs such as R, G, B or A.  .  This topic shows examples of using Source and the visual results.

Example: Set Image Transparency using Alpha - The A row in the Style dialog allows us to specify what transparency we want to apply to the image, either by applying the same value for A for all pixels or by using one of the other channels to also control the A value.

Example: Autocontrast and Hill Shading Images using Style - This example shows how the Edit - Style dialog can hill shade an image using the values of pixels as heights and generating shadows as if the Sun were located at the specified azimuth and altitude.   This capability is used most frequently with raster images to give an impression of three dimensionality in cases where the values of pixels represent terrain elevations.

Example: Style Applied to an Image Server Image - Because the Edit - Style dialog simply changes the way an image is displayed and not the data, it can operate on read-only data served by various web servers such as WMS REST servers.    In this example we look at every detail of creating a data source using an image server and then manipulating the appearance of the display with Style.  We will connect to a WMS server that provides LiDAR data in various forms, including as terrain elevation.