10D ISO 1600 is pushed one stop from 800

Discussion in 'Digital SLR' started by JPS, Dec 30, 2004.

  1. In the lottery I'd assume (nearly) equal chances.
    With sensors I don't.

    -Wolfgang
     
    Wolfgang Weisselberg, Jan 2, 2005
    #21
    1. Advertisements

  2. JPS

    Jeremy Nixon Guest

    No, it doesn't. DNG doesn't require interpolation of the sensor data.
    It *allows* it, but by default the DNG converter does not do it.
    Up to 800, analog gain is used. After that it's a mathematical process.
    This is not exactly news.
     
    Jeremy Nixon, Jan 2, 2005
    #22
    1. Advertisements

  3. JPS

    MarkH Guest

    When you do not understand how Bayer interpolation works, it would be
    better if you did not post about it. Thank you.

    For those that are interested:
    The idea that ¾ of the data has to be interpolated is a falsehood created
    by a person on r.p.d under the name Geaorge Preddy. The fact is that each
    pixel location on Bayer grid sensors contains a sensel that captures
    information about one of the three colours (blue, red or green) and the
    other two colour values are calculated based on information from several
    neighbouring sensels as well as the measured data at that location. This
    means that with a Bayer pattern sensor the final output will consist of 2/3
    of the data being interpolated.
     
    MarkH, Jan 2, 2005
    #23
  4. JPS

    Colin D Guest

    With respect, John, lottery winning numbers aren't binary, and I cannot
    see your analogy here.
    I would think that if only 11 bits are being used as you suggest, then
    it will be the most significant bit that will be dropped, not the least
    significant, i.e. bit 0. But that seems to be what you are implying,
    since whether or not a decimal conversion is odd or even depends on the
    state of bit 0. At least that is what I think you are saying, but I may
    have misinterpreted your earlier post.

    A further point that has me confused is that, if the blackframe values
    are represented by an 8- or 9-bit binary number, then even with 12 bits
    available, the entire brightness range of the image has to be
    represented with 3 or 4 bits, i.e. a range of 16:1 max. Yet the images
    appear to have much greater range than that.

    I wll have to read up some more on this, it appears.

    Colin.
     
    Colin D, Jan 2, 2005
    #24
  5. JPS

    JPS Guest

    In message <>,
    All of the data in an RGB output is interpolated (if the method of
    interpolation is worth a dime), and the increase in data from the
    interpolation is equivalent to interpolating 2/3 of it; not 3/4.
    Which is totally irrelevant in a dark frame, as no light is passing
    through the Bayer CFA. All the data is from sensor and circuit noise,
    and a possible offset bias.
    No, there was no bayer interpolation in the data I was looking at. It
    was *RAW*.
    You didn't read my post very clearly; did you? The stripes I'm getting
    are vertical, not horizontal. Even if it were a difference between odd
    and even lines for green, there are red and blue on those lines, as
    well. You really wouldn't expect one line of green to be all odd
    values, and the next to be all even, would you? There are solid runs of
    odd pixels and solid runs of even pixels over multiple horizontal lines.
    It doesn't compress the highlights. RAW data is RAW and it is linear!
    Nope. You are just full of error, aren't you? ISOs 100 through 800 are
    achived by amplifying the signal before the ADC stage.
    There is no logical reason why this would not be so. The only
    difference is the ISO setting; same Tv mode shutter speed, and same
    wide-open aperture.
    What for?
    That would require some testing. The JPEG compression would probably
    damage the shadows that you would want to boost.

    In any event, I was talking about RAW, which is what people who want
    maximum quality generally shoot.
    If I can. If things are uncertain, I might play it safe with default
    exposure.
    This is RAW data.

    --
     
    JPS, Jan 3, 2005
    #25
  6. JPS

    JPS Guest

    In message <>,
    Of course there will be *ranges* of RAW data that are inlikely. Any
    value less than 120 is unlikely with Canon DSLRs. However, every
    *range* has both odd and even numbers, in equal distribution. When the
    same pixels are always odd and others always even, neither has the
    chance of being the other, and there is one bit of precision missing.
    --
     
    JPS, Jan 3, 2005
    #26
  7. JPS

    JPS Guest

    In message <>,
    Well, the popular conception is that up to 1600 is analog, and 3200 is
    mathematical (based on 1600 analog). No one believed me in the past
    when I said that "ISO 1600" is actually ISO 800 (analog), pushed
    mathematically. This is sort of a proof, but it also contains a mystery
    (why there are patterns of odd numbers). The only explanation I can
    come up with is that Canon is trying to cheat the histograms. It's
    clearly not visual dithering, because it is inconsistent in the image.
    --
     
    JPS, Jan 3, 2005
    #27
  8. JPS

    JPS Guest

    In message <[email protected]>,
    I believe that I said or implied that all the numbers should be even, if
    only 11 bits are used. The RAW data is in 12-bit format, and the
    numbers 0 through 2047 doubled are 0 through 4094, all even, and all
    with 0 for the least significant bit, in binary.
    Not at all. The camera has pixels which are not exposed to light
    (masked). This "black" data is used to generate a "blackpoint" value,
    probably from some position near the beginning of the expected
    bell-curve histogram for the black pixels. This number is *subtracted*
    from all RAW image values, before conversion to a full RGB bitmap. In
    this case, the value might be 260. That means that the actual image
    uses a range of 260 to 4095, or 0 to 3835 after the blackpoint
    subtraction. That is only slightly less dynamic range than 0 to 4095
    (which will never happen, as the blackpoint is lowest at ISO 100 and
    with a 1/4000 shutter speed is about 126 at room temperature, on the
    10D).
    I don't know if much is written on this!
    --
     
    JPS, Jan 3, 2005
    #28
  9. JPS

    Jeremy Nixon Guest

    It seems to be that you get four stops of analog gain. Thus, the D70,
    for example, has ISO 200-1600 for "real", but not 100, while the announced
    D2x has 100-800 for "real" and 1600-3200 as digital boost. This seems
    common enough to be expected at this point.
     
    Jeremy Nixon, Jan 3, 2005
    #29
  10. Naive me had assumed that the blue value on a blue pixel would
    have been taken for it's value (excluding edges in the image).
    And it could not be that red, green and blue sensors get any
    different treatment inside the camera, like different gain,
    say, because one of these colors happens to need that to get the
    correct strength?
    You used a converter. You have no knowledge what the
    converter does _inside_.
    The green pixels in different rows are also in different
    columns in any common pattern.
    On _alternating_ rows, and on _alternating_ columns.
    I would expect them all to be zero, in a dark frame, at least
    with dark frame substraction.

    How about you photograph some object at ISO 800, 1600 and H,
    while adjusting the shutter time to get equivalent exposure, run
    the RAWs through dcraw -d (of which you can see what it does)
    and examine the well-documented output format. See if you get
    the same even-odd rows in 1600 and in H. If you are right, you
    should see them there, too. If not, you have hit an artifact
    of sensor noise in dark frame situations. Actually, if you are
    right, you should see the 2 least significant bits in H being,
    ah, unusual, and nothing similar in 800.
    That could well be because of sensor noise in dark frame
    situations.
    If you have pure bitshifting, true.
    Not having the source code of the firmware I cannot tell you what
    the camera does. However, it _would_ be a pity to throw away
    all that headroom by bitshifting, and it's not as if it would
    hurt really, or would it?

    By arguing how things should be we don't get answers how they are,
    we get doctrines and should open a church.
    I also have prejudices, half-knowledge, insults, attacks and a very
    strong oppinion in me, so I scarcely can be *just* full of error.
    Which is --- basically --- the analogue analog of bit shifting
    (plus adding amplifier noise), isn't it?

    The only difference is that the precious 12-bit converter
    is brought into a better range (being exposed to the right,
    basically) and that one hopes that the amplifier noise is below
    the converter's threshold --- otherwise you get one (or more)
    bits of noise and 11 (or less) bits of signal.

    The only reason we bother with analog gain is that we don't
    have better converters and we'd prefer more dynamics at the cost
    of noise. Wait till we have 20-bit converters @ISO 100, then you
    can have 12 bits at ISO 25,600 without any extra analog gain ---
    and people will scream that the sensor 'cheats' and gets them
    less than 20 bits at ISO 6,400.
    As they say, the proof is in the pudding ...
    Have you tested it?

    After all, according to your logical reason, JPEG will always be
    practically unusable, with most of the data being removed from
    the picture, lossy compression and all that.
    Same with Ogg Vorbis, mp3, DVDs (mpeg compressed), ...
    If you don't use the headroom, what's it's use, except for
    more work to polish the image?
    JPEG compresses logarithmically.
    And here _I_ thought that would be slides of at least medium
    format, since 100+ megapixel cameras are still somewhat rare.
    Does that mean no noise supression on the sensor chip itself?

    -Wolfgang
     
    Wolfgang Weisselberg, Jan 3, 2005
    #30
  11. You assume that there is an equal distribution in the range,
    concerning the physical sensor.
    To show to a sufficienrt degree that these self-same pixels are
    _always_ odd or even, you'd need far more than one picture.

    Even if in the "dark frame" range that observation is true ---
    that does not make it necessarily true in brighter ranges at the
    same ISO setting.

    -Wolfgang
     
    Wolfgang Weisselberg, Jan 3, 2005
    #31
  12. You are right, I thinko'ed.
    On the average, this is true, and for the green channel only 1/2
    of the pixels need interpolation.

    -Wolfgang
     
    Wolfgang Weisselberg, Jan 3, 2005
    #32
  13. JPS

    JPS Guest

    In message <>,
    That is a very safe assumption to make; about as safe as assuming that
    the floor is there when I get out of my bed in the morning.
    I should have said in "the same pattern". I have looked at other
    images, and the pattern is the same, except when the data is clipped at
    4095.
    I looked at image RAW data also; the same thing.

    I really don't understand why you are so stubborn about accepting this
    pattern. It's as plain as the grid on a graph paper, and only exists at
    ISO 1600 and 3200. At 800 and below, any pixel can have an odd or even
    value.
    --
     
    JPS, Jan 4, 2005
    #33
  14. Only assuming a sensor without non-random (coloured) noise and
    a perfect digitizer.
    Do I understand you correctly:
    - you have looked at at least a dozen pictures with ISO
    1600, of several opjects
    - at least half a dozen of these pictures have been exposed
    correctly for ISO1600
    - all of these pictures are in RAW
    - when converting the image file from camera RAW to readable
    RAW again, the identical image comes out (why should any
    'cheat at histograms' algorithm not be in the converter?)
    - all these pictures show a pattern of stripes
    - however, it's not always the same pixels that are even
    and odd, this varies from picture to picture
    - when you turn up the contrast way way high, you can
    actually see the stripes
    or am I missing something?

    I do accept that you have seen some pattern in an HEX editor (and
    up to now I only remembered you writing of one single dark frame).

    However, the DNG format is not very trivial --- see for
    yourself[1]. For example, what is the orientation --- it is
    set as a single byte and is not optional. Could your stripes be
    horizontal (page 13)?

    What is the BlackLevel, and the BlackLevel pattern (page 19)
    and BlackLevelDeltaH and BlackLevelDeltaV (page 20) --- and could
    that pattern, as substracted (see page 37), unstripe your observed
    'stripes'?
    But whatever it is that you see --- is it what you think it is?

    Without the raw data[2] (and not having a 10D, producing them is
    kind of hard) I cannot see what you see, only what you show me,
    interpreted through your eyes. Of course I am sceptic, I owe it
    to myself.
    But then you'd get only ..0 ..4 ..8 .12 .16 with 3200 (i.e. 2
    unused bits), _unless_ 3200 has more analogue gain than 1600 ... do
    you get that?

    -Wolfgang

    [1] http://www.adobe.com/products/dng/pdfs/dng_spec.pdf
    [2] .cr2, not the already processed .dng
     
    Wolfgang Weisselberg, Jan 4, 2005
    #34
  15. JPS

    JPS Guest

    In message <>,
    I lost track of this thread, but now I have a better demo. No, the
    stripes are different in every image; however, they are being applied
    very evenly (odd and even number constitute 49.8 to 50.2% of the
    images). This is an ISO 1600 image from the 10D, but showing only the
    LSB (0=black; 1=white)

    http://www.pbase.com/jps_photo/image/38841732

    As far as it affecting the image is concerned, it is certainly too much
    of an offset to be useful dithering, if in fact some are +1 and some are
    -1. I believe they are offset all the same way, though, as a blurred
    blackframe tends to have similarly-striped sections that look lighter
    and darker than each other, after the blur is histogram-equalized (i.e.,
    significant low-frequency content).
    --
     
    JPS, Jan 20, 2005
    #35
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.