# 10D ISO 1600 is pushed one stop from 800

Discussion in 'Digital SLR' started by JPS, Dec 30, 2004.

1. ### Wolfgang WeisselbergGuest

In the lottery I'd assume (nearly) equal chances.
With sensors I don't.

-Wolfgang

Wolfgang Weisselberg, Jan 2, 2005

2. ### Jeremy NixonGuest

No, it doesn't. DNG doesn't require interpolation of the sensor data.
It *allows* it, but by default the DNG converter does not do it.
Up to 800, analog gain is used. After that it's a mathematical process.
This is not exactly news.

Jeremy Nixon, Jan 2, 2005

3. ### MarkHGuest

When you do not understand how Bayer interpolation works, it would be
better if you did not post about it. Thank you.

For those that are interested:
The idea that ¾ of the data has to be interpolated is a falsehood created
by a person on r.p.d under the name Geaorge Preddy. The fact is that each
pixel location on Bayer grid sensors contains a sensel that captures
information about one of the three colours (blue, red or green) and the
other two colour values are calculated based on information from several
neighbouring sensels as well as the measured data at that location. This
means that with a Bayer pattern sensor the final output will consist of 2/3
of the data being interpolated.

MarkH, Jan 2, 2005
4. ### Colin DGuest

With respect, John, lottery winning numbers aren't binary, and I cannot
see your analogy here.
I would think that if only 11 bits are being used as you suggest, then
it will be the most significant bit that will be dropped, not the least
significant, i.e. bit 0. But that seems to be what you are implying,
since whether or not a decimal conversion is odd or even depends on the
state of bit 0. At least that is what I think you are saying, but I may
have misinterpreted your earlier post.

A further point that has me confused is that, if the blackframe values
are represented by an 8- or 9-bit binary number, then even with 12 bits
available, the entire brightness range of the image has to be
represented with 3 or 4 bits, i.e. a range of 16:1 max. Yet the images
appear to have much greater range than that.

I wll have to read up some more on this, it appears.

Colin.

Colin D, Jan 2, 2005
5. ### JPSGuest

In message <>,
All of the data in an RGB output is interpolated (if the method of
interpolation is worth a dime), and the increase in data from the
interpolation is equivalent to interpolating 2/3 of it; not 3/4.
Which is totally irrelevant in a dark frame, as no light is passing
through the Bayer CFA. All the data is from sensor and circuit noise,
and a possible offset bias.
No, there was no bayer interpolation in the data I was looking at. It
was *RAW*.
You didn't read my post very clearly; did you? The stripes I'm getting
are vertical, not horizontal. Even if it were a difference between odd
and even lines for green, there are red and blue on those lines, as
well. You really wouldn't expect one line of green to be all odd
values, and the next to be all even, would you? There are solid runs of
odd pixels and solid runs of even pixels over multiple horizontal lines.
It doesn't compress the highlights. RAW data is RAW and it is linear!
Nope. You are just full of error, aren't you? ISOs 100 through 800 are
achived by amplifying the signal before the ADC stage.
There is no logical reason why this would not be so. The only
difference is the ISO setting; same Tv mode shutter speed, and same
wide-open aperture.
What for?
That would require some testing. The JPEG compression would probably
damage the shadows that you would want to boost.

In any event, I was talking about RAW, which is what people who want
maximum quality generally shoot.
If I can. If things are uncertain, I might play it safe with default
exposure.
This is RAW data.

--

JPS, Jan 3, 2005
6. ### JPSGuest

In message <>,
Of course there will be *ranges* of RAW data that are inlikely. Any
value less than 120 is unlikely with Canon DSLRs. However, every
*range* has both odd and even numbers, in equal distribution. When the
same pixels are always odd and others always even, neither has the
chance of being the other, and there is one bit of precision missing.
--

JPS, Jan 3, 2005
7. ### JPSGuest

In message <>,
Well, the popular conception is that up to 1600 is analog, and 3200 is
mathematical (based on 1600 analog). No one believed me in the past
when I said that "ISO 1600" is actually ISO 800 (analog), pushed
mathematically. This is sort of a proof, but it also contains a mystery
(why there are patterns of odd numbers). The only explanation I can
come up with is that Canon is trying to cheat the histograms. It's
clearly not visual dithering, because it is inconsistent in the image.
--

JPS, Jan 3, 2005
8. ### JPSGuest

In message <[email protected]>,
I believe that I said or implied that all the numbers should be even, if
only 11 bits are used. The RAW data is in 12-bit format, and the
numbers 0 through 2047 doubled are 0 through 4094, all even, and all
with 0 for the least significant bit, in binary.
Not at all. The camera has pixels which are not exposed to light
(masked). This "black" data is used to generate a "blackpoint" value,
probably from some position near the beginning of the expected
bell-curve histogram for the black pixels. This number is *subtracted*
from all RAW image values, before conversion to a full RGB bitmap. In
this case, the value might be 260. That means that the actual image
uses a range of 260 to 4095, or 0 to 3835 after the blackpoint
subtraction. That is only slightly less dynamic range than 0 to 4095
(which will never happen, as the blackpoint is lowest at ISO 100 and
with a 1/4000 shutter speed is about 126 at room temperature, on the
10D).
I don't know if much is written on this!
--

JPS, Jan 3, 2005
9. ### Jeremy NixonGuest

It seems to be that you get four stops of analog gain. Thus, the D70,
for example, has ISO 200-1600 for "real", but not 100, while the announced
D2x has 100-800 for "real" and 1600-3200 as digital boost. This seems
common enough to be expected at this point.

Jeremy Nixon, Jan 3, 2005
10. ### Wolfgang WeisselbergGuest

Naive me had assumed that the blue value on a blue pixel would
have been taken for it's value (excluding edges in the image).
And it could not be that red, green and blue sensors get any
different treatment inside the camera, like different gain,
say, because one of these colors happens to need that to get the
correct strength?
You used a converter. You have no knowledge what the
converter does _inside_.
The green pixels in different rows are also in different
columns in any common pattern.
On _alternating_ rows, and on _alternating_ columns.
I would expect them all to be zero, in a dark frame, at least
with dark frame substraction.

How about you photograph some object at ISO 800, 1600 and H,
while adjusting the shutter time to get equivalent exposure, run
the RAWs through dcraw -d (of which you can see what it does)
and examine the well-documented output format. See if you get
the same even-odd rows in 1600 and in H. If you are right, you
should see them there, too. If not, you have hit an artifact
of sensor noise in dark frame situations. Actually, if you are
right, you should see the 2 least significant bits in H being,
ah, unusual, and nothing similar in 800.
That could well be because of sensor noise in dark frame
situations.
If you have pure bitshifting, true.
Not having the source code of the firmware I cannot tell you what
the camera does. However, it _would_ be a pity to throw away
all that headroom by bitshifting, and it's not as if it would
hurt really, or would it?

By arguing how things should be we don't get answers how they are,
we get doctrines and should open a church.
I also have prejudices, half-knowledge, insults, attacks and a very
strong oppinion in me, so I scarcely can be *just* full of error.
Which is --- basically --- the analogue analog of bit shifting
(plus adding amplifier noise), isn't it?

The only difference is that the precious 12-bit converter
is brought into a better range (being exposed to the right,
basically) and that one hopes that the amplifier noise is below
the converter's threshold --- otherwise you get one (or more)
bits of noise and 11 (or less) bits of signal.

The only reason we bother with analog gain is that we don't
have better converters and we'd prefer more dynamics at the cost
of noise. Wait till we have 20-bit converters @ISO 100, then you
can have 12 bits at ISO 25,600 without any extra analog gain ---
and people will scream that the sensor 'cheats' and gets them
less than 20 bits at ISO 6,400.
As they say, the proof is in the pudding ...
Have you tested it?

After all, according to your logical reason, JPEG will always be
practically unusable, with most of the data being removed from
the picture, lossy compression and all that.
Same with Ogg Vorbis, mp3, DVDs (mpeg compressed), ...
If you don't use the headroom, what's it's use, except for
more work to polish the image?
JPEG compresses logarithmically.
And here _I_ thought that would be slides of at least medium
format, since 100+ megapixel cameras are still somewhat rare.
Does that mean no noise supression on the sensor chip itself?

-Wolfgang

Wolfgang Weisselberg, Jan 3, 2005
11. ### Wolfgang WeisselbergGuest

You assume that there is an equal distribution in the range,
concerning the physical sensor.
To show to a sufficienrt degree that these self-same pixels are
_always_ odd or even, you'd need far more than one picture.

Even if in the "dark frame" range that observation is true ---
that does not make it necessarily true in brighter ranges at the
same ISO setting.

-Wolfgang

Wolfgang Weisselberg, Jan 3, 2005
12. ### Wolfgang WeisselbergGuest

You are right, I thinko'ed.
On the average, this is true, and for the green channel only 1/2
of the pixels need interpolation.

-Wolfgang

Wolfgang Weisselberg, Jan 3, 2005
13. ### JPSGuest

In message <>,
That is a very safe assumption to make; about as safe as assuming that
the floor is there when I get out of my bed in the morning.
I should have said in "the same pattern". I have looked at other
images, and the pattern is the same, except when the data is clipped at
4095.
I looked at image RAW data also; the same thing.

I really don't understand why you are so stubborn about accepting this
pattern. It's as plain as the grid on a graph paper, and only exists at
ISO 1600 and 3200. At 800 and below, any pixel can have an odd or even
value.
--

JPS, Jan 4, 2005
14. ### Wolfgang WeisselbergGuest

Only assuming a sensor without non-random (coloured) noise and
a perfect digitizer.
Do I understand you correctly:
- you have looked at at least a dozen pictures with ISO
1600, of several opjects
- at least half a dozen of these pictures have been exposed
correctly for ISO1600
- all of these pictures are in RAW
- when converting the image file from camera RAW to readable
RAW again, the identical image comes out (why should any
'cheat at histograms' algorithm not be in the converter?)
- all these pictures show a pattern of stripes
- however, it's not always the same pixels that are even
and odd, this varies from picture to picture
- when you turn up the contrast way way high, you can
actually see the stripes
or am I missing something?

I do accept that you have seen some pattern in an HEX editor (and
up to now I only remembered you writing of one single dark frame).

However, the DNG format is not very trivial --- see for
yourself[1]. For example, what is the orientation --- it is
set as a single byte and is not optional. Could your stripes be
horizontal (page 13)?

What is the BlackLevel, and the BlackLevel pattern (page 19)
and BlackLevelDeltaH and BlackLevelDeltaV (page 20) --- and could
that pattern, as substracted (see page 37), unstripe your observed
'stripes'?
But whatever it is that you see --- is it what you think it is?

Without the raw data[2] (and not having a 10D, producing them is
kind of hard) I cannot see what you see, only what you show me,
interpreted through your eyes. Of course I am sceptic, I owe it
to myself.
But then you'd get only ..0 ..4 ..8 .12 .16 with 3200 (i.e. 2
unused bits), _unless_ 3200 has more analogue gain than 1600 ... do
you get that?

-Wolfgang

[2] .cr2, not the already processed .dng

Wolfgang Weisselberg, Jan 4, 2005
15. ### JPSGuest

In message <>,
I lost track of this thread, but now I have a better demo. No, the
stripes are different in every image; however, they are being applied
very evenly (odd and even number constitute 49.8 to 50.2% of the
images). This is an ISO 1600 image from the 10D, but showing only the
LSB (0=black; 1=white)

http://www.pbase.com/jps_photo/image/38841732

As far as it affecting the image is concerned, it is certainly too much
of an offset to be useful dithering, if in fact some are +1 and some are
-1. I believe they are offset all the same way, though, as a blurred
blackframe tends to have similarly-striped sections that look lighter
and darker than each other, after the blur is histogram-equalized (i.e.,
significant low-frequency content).
--

JPS, Jan 20, 2005