How does noise reduction work?

Discussion in 'Digital SLR' started by Billy no mates, Jan 6, 2006.

  1. After looking in the manual i am none the wiser.
    Specifically I am trying to work out if i should use in camera noise
    reduction for long exposure astrophotography or not.

    I have read a review on noise reduction for high end Nikon D2 series
    cameras which described a method whereby the inherent noise on the
    sensor is cancelled by subtracting a same length exposure with the
    shutter closed from the image. This is intended to get rid of the
    effects of heat on the sensor etc.

    My lowly Nikon D50 pauses after a long exposure for the same(ish)
    length of time as the exposure while processing if I set in camera
    noise reduction to on. Is this doing the same (as in worthwhile) or is
    it looking for random hotspots (as in faint stars) and chucking them
    in the bin.

    TIA
    Jonathan
     
    Billy no mates, Jan 6, 2006
    #1
    1. Advertisements

  2. Billy no mates

    Mike King Guest

    It works like this. Assume a sensor with only nine pixels (!)

    Image mostly dark, night shot, x is a lit pixel o is a dark pixel.

    Your image:
    ooo
    oxo
    oox

    Shoot the black hole inside your camera, image is thus:
    ooo
    ooo
    oox

    Camera applies noise reduction and subtracts the lit pixels from the image
    of blackness from your stellar (pun intended!) image of the Horsehead Nebula
    (or whatever)

    final result:
    ooo
    oxo
    ooo

    Astrophotography one of the few sub-disciplines where it's OK to center your
    subject in the viewfinder.
     
    Mike King, Jan 6, 2006
    #2
    1. Advertisements

  3. Billy no mates

    Mike King Guest

    I should have added that the faint star images appear only on your photo not
    the subtraction mask so they should be safe.
     
    Mike King, Jan 6, 2006
    #3

  4. I was wondering if the dark subtraction in the D50 used a dark frame
    or software so that I could skip it and do the same in CS2?
    Jonathan
     
    Billy no mates, Jan 6, 2006
    #4
  5. Yes, you can do it in CS2 *if* you get a frame representative of the
    sensor dark output at the time of the exposure. Since the sensor dark
    noise will vary with temperature and exposure time, this is not usually
    a realistic option, but in some specialist cases it might be practical.
    (eg. if you keep the camera at a fixed temperature and capture dark
    reference frames for typical exposure times you use - it doesn't need to
    be every time, you can interpolate between dark frames to get the
    exposure you actually used.) Ideally, the dark frames will be averaged
    to reduce temporal noise, so that you don't degrade the image any
    further.)
     
    Kennedy McEwen, Jan 6, 2006
    #5

  6. I'll try to be more plain now.

    I have had two answers which, although perfectly accurate in their
    answer, have helped not one iota.
    My fault for being too vague in the question I guess.

    What I am seeking to find out, specifically with regard to the Nikon
    D50, is this:-

    When the camera reduces noise internally does it
    a) Do a full dark frame subtraction - saves me the bother of taking a
    dark frame with consequent work later on.
    b) reduce noise by software alone - I could do this, possibly with
    more sucess, or at least more control later on.
    c) None of the above.

    thanks guys for your replies
    Jonathan
     
    Billy no mates, Jan 6, 2006
    #6
  7. Billy no mates

    eawckyegcy Guest

    Your question is pointless.

    a) If you are concerned about this, then you will gather a master dark
    (average of 10 or more frames), and use that, and to hell with whatever
    the camera does. This will also expedite the collection of many
    long-exposure frames, since you can use the same dark over and over
    again (assuming exposure times are the same and roughly equal
    temperature; gather darks at the beginning and end of your session --
    this is all covered in various books on using CCD's for astronomy).

    b) Otherwise, who cares what the camera is doing? Enable the mode and
    march forward. Do you like the result or not?

    But if you feel you need an answer, here is how you obtain it:

    1) RTFM

    2) If (1) fails, given you have the camera in hand, a simple experiment
    would answer the question in less time you took to type in your
    posting. (To wit: set manual mode, shutter = 30s, turn the NR on, and
    note how long it takes to get a picture up to preview. If it's 60s,
    then (a) is your answer, and if 30s, then (b) is your answer, and if
    some other markedly difference value, the (c) is probably your answer.)
     
    eawckyegcy, Jan 6, 2006
    #7
  8. Billy no mates

    JPS Guest

    In message <>,
    It's probably taking a black picture to subtract it from the exposed
    one. This will get rid of any artifacts that are the same on successive
    exposure. Random sensor noise, however, is not removed, but actually
    increased by about 41%. So, it is usually only worth using with
    exposures long enough to have bright pixels that repeat every frame, or
    other artifacts, like amplifier glow, and some (but not all) types of
    banding.
    --
     
    JPS, Jan 6, 2006
    #8
  9. Pointless?

    How can you tell, from merely the time taken, what is happening inside
    the camera? It's 60 seconds give or take - this proves what exactly?

    I understand this attitude because you have assumed that I am an
    idiot. However, I have read the manual, you could too if you wanted
    but it's not your problem so I don't expect you to take the time to do
    things that I could so easily do (have done). If you don't want to
    assist, then don't. Simple as that. I am grateful that you have taken
    the time to reply and also sorry that your time was wasted.

    a) If I am concerned about this? Well yes I am - this is a standard
    method of removing unwanted noise used in many applications.

    b) I care that the camera might be getting rid of data that is signal
    rather than noise. it could also be doing a dark frame subtraction
    automatically. If it is, then I can miss out the manual subtraction
    later and probably get the camers to use a better dark frame to
    subtract. If it isn't, then I ought to do some dark frames to subtract
    at a later date in the nice warm office.

    As RTFM failed,all it (see extract below) I did take dark frame
    shots, there is less noise at iso400 and also at iso1600 with NR
    turned on than with it turned off, however there is still noise, more
    so at 1600 as expected. No real surprise there.

    If I do two consecutive dark frames at 30seconds iso1600 without NR
    and subtract them, then there is significantly more noise on the
    result than on the single dark frame, same conditions but NR switched
    on.

    The NR appears to get rid of noise, but at what expense? Is it simply
    dropping out all levels below a certain level? Is it doing a dark
    frame subtraction? Is it software just like Photoshop/Noise Ninja/
    Your favoourite software?

    So, to get back to my question - how does the in camera noise
    reduction work? Is it good or bad? Is it a trade secret?

    best regards
    Jonathan






    Excerpt from the manual
    If ON is selected, photos taken at shutter speeds slower than about 1s
    will be processed to reduce noise before being recorded to the memory
    card (the default setting is off). Note that the time required to
    process images more than doubles when noise reduction is on;.....
     
    Billy no mates, Jan 6, 2006
    #9
  10. Billy no mates

    JPS Guest

    In message <>,
    Software can not determine what a dark frame looks like. It needs to
    see one, because it needs to subtract what black looks like to the
    camera at that temperature, that time, with that exposure time, and that
    ISO. The subtraction is best done within the RAW data anyway; you can't
    currently do that with PSCS2's RAW converter on your own.

    There are programs like IRIS which will allow you to load RAW files,
    convert them literally to another format, add them or subract them, but
    it does not have a true RAW converter so you are stuck with unsaturated
    colors, and not a real demosaicing, so you will lose some sharpness and
    have some chromatic pixel effects. You would get the best results with
    such software, by taking multiple exposures, and multiple black frames
    with the same parameters (aperture is irrelevant to the dark frame, of
    course), and subtracting an equal number of black frame from exposure
    frames.
    --
     
    JPS, Jan 6, 2006
    #10
  11. Billy no mates

    eawckyegcy Guest

    Yes. I explained why. Did you read it?
    That the camera is collecting a dark and subtracting it. This is what
    all dSLR in-camera NR modes do, as it is the absolute simplest, and
    only removes bias (hot/cold pixels, etc). All other "noise reduction"
    techniques need _far_ more computation and do not require a dark frame;
    the bayesian is that the marketing people at Canon, Nikon, Minolta and
    other companies were told by their engineers that it's a better idea to
    depend on post-processing systems, for CPU/power reasons and many
    others.
    I offered an argument why you are chasing a red herring -- an argument
    you you didn't read or understand -- and now you draw a false
    conclusion. At the time I made no assumptions about your possible
    idiotness. Fortunately, now you've gone and proven it for me, thus
    relieving me the need of making any assumptions at all in this matter.
    Thanks!
    What is the standard method? Didn't you profess not to know? Or what?
    As I pointed out to you, if you are concerned, then don't use the mode
    and do everything yourself. Once you've made this decision, your
    question is then utterly moot. ie, "pointless".
    Ok, so you've tried the NR mode, and it appears you found you didn't
    like it. (Or did you?) Now, didn't I specifically advise you to do
    this? And now that you have, and found it wanting (or not), then why
    do you care what the camera is doing? Again, we come to the
    "pointless" issue: why ask questions whose answers are of no
    particular value?
    a) bias vs. variance
    b) no
    c) yes
    d) no
    1) what is called "NR" is dark frame subtraction; it removes the
    biases, but adds variance
    2) you tell me
    3) there is a vibrant reverse engineering community in Russia; they
    might know
     
    eawckyegcy, Jan 6, 2006
    #11
  12. SNIP
    I don't really know about the D50, but it probably does use an equal
    duration black frame subtraction (just like e.g. competitors like
    Canon do). It's a compromise, since it would be better to take the
    average of say at least 10 black frames, thus getting a lower noise
    black frame. It is obviously possible to collect those black-frame
    exposures yourself by covering the lens and viewfinder during (to
    approximate the average in-camera temperature) the shoot (which by
    itself might take several exposures).
    I've read reports about the D70 which indicate noise
    reduction/blurring before Raw storage, but I don't know if the D50
    acts similarly.

    SNIP
     
    Bart van der Wolf, Jan 7, 2006
    #12
  13. I can't answer for the Nikon D50 specifically, but generally what
    happens in the camera is (a) from your list above. The Nikon *may* also
    implement an element of (b) but I don't know how much, if I was a Nikon
    user, I would hope it was minimal. ;-)
     
    Kennedy McEwen, Jan 7, 2006
    #13
  14. I understand your frustration. Manuals are usually useless (often worse
    than) in terms of detailing the functions actually implemented. This is
    often because the company do not want to reveal the details of the
    particular process that they implement, so they simply use generic
    meaningless phrases like "noise reduction" or "noise processing".
    Sometimes language plays a part in it, but these days that isn't a
    problem with the big manufacturers. If they don't provide the details
    it is usually because they don't want to, not because that can't put it
    in words.
    My interpretation of that is as follows:
    1. Doubling of the time is due to capture of a single dark frame after
    the shutter has closed.
    2. More than double accounts for the process time required to subtract
    the single dark frame from the original exposed result.
    3. There is no way to determine if any additional processing is being
    implemented from the manual, although it may be possible by reverse
    engineering some test frames and comparing the result of enabled and
    disabled noise reduction.
    4. As a minimum, expect the temporal noise (ie. random noise) in the
    image to increase by 40% due to the subtraction of a single dark frame
    in the camera compared to what you can achieve in PS with an average of
    several (10 or more, but not hundreds - 1/f noise will then dominate the
    end result) dark frames.
     
    Kennedy McEwen, Jan 7, 2006
    #14
  15. Tes, of course
    Thank you. This is all I wanted to know, really. Is there a dark frame
    subtraction? Answer = yes!
    You offered an argument that i had not RTFM, nor done any test shots
    to experiment. I had done both with inconclusive results.
    I didn't profess anything of the sort. I professed only to not know
    what the camera does automatically.
    So I could do everything manually, down to drawing the pixels myself?
    The camera is a machine designed to do a specific job and I seek to
    get the best out of it. It kind of helps to know what the setting
    parameters actually do in order to acheive this. Astrophotography and
    other technical (high speed flash, rear curtain sync etc) methods
    which currently interest me require a knowledge of the mechanics of
    the machine. Pretty or not is irrelevant here.
    LIKE it? LIKE it? What has that got to do with it?
    I want to know which method best captures very low light without
    degradation of the image. What has like it got to do with it?

    I tried taking photos of the wifes Mother. I am supposed to like the
    result? No, not really, but I want it to look as much like her as is
    reasonable. (Well soft focus might help there - pass the foil I'm
    making a f/1.4 pinhole "lens" - there, that's better)

    Thank you, you did help me.
    Kindest Regards
    Jonathan
     
    Billy no mates, Jan 7, 2006
    #15
  16. Billy no mates

    bjw Guest

    Yes, but in general, if your total exposure time is limited, for
    example if you're doing astrophotography during the night,
    and you absolutely must take the darks during the night, and if
    your exposures are dark current limited, the best strategy is to
    put equal time into open-shutter and dark exposures.
    If you take one 30 sec open exposure and 9x30 sec darks,
    you're wasting open-shutter time - you could have split 5/5
    and gotten more signal on the faint source in your image.

    On the other hand, if you took 9x30 sec real exposures and
    only 1 dark, you have to subtract the same dark 9 times,
    so you're amplifying the noise in that particular frame by 9.
    (This is where it matters that the exposure is dark current
    limited.) Because the random noise in a sum of images
    goes up as the square root of the number of images, it
    turns out that the optimum combination in this simplified
    case is equal time on source and dark.

    Of course, the real optimum is to have a detector that is
    stable enough that you can take the dark frames before or
    after. This usually requires temperature control at a
    minimum and I have no experience in how feasible it is
    with general-purpose camera detectors, though I'm sure
    many others have explored it.
     
    bjw, Jan 7, 2006
    #16
  17. Billy no mates

    Mike King Guest

    I'm not an astrophotographer but read Astronomy magazine and Scientific
    American and I believe that this is exactly what was done in the earlier
    days of CCD Astrophotography.

    And in film based photography there really is an unsharp mask made on a
    piece of film that was sandwiched with the original image for improved
    sharpness as well as contrast masking, etc. before Photoshop. Photoshop
    borrowed (ahem) the terms so that wet darkroom photographers would have a
    leg up when first learning the program.
     
    Mike King, Jan 7, 2006
    #17
  18. Billy no mates

    Mike King Guest

    If you need that EXACT an answer for a specific new camera you should
    contact Nikon or ask a more specific question in the first place and the
    answer is:

    I don't own a Nikon D-50 and if I did own a D-50 my camera might not use the
    same firmware as your D-50.
     
    Mike King, Jan 7, 2006
    #18
  19. Billy no mates

    eawckyegcy Guest

    Christian Buil noticed that the D70 was using a simple median filter
    (or similar) to remove speckle noise (hot pixels), and did this on the
    so-called "raw" data. This would do a number on images of the sky
    (faint stars look like speckle noise), but probably a net improvement
    for pictures of Baby (if you could get it to sit still for the long
    exposure).

    http://www.astrosurf.org/buil/d70v10d/eval.htm

    I can't see any Nikon gear after his D70 tests though. Hm. Most of
    his subsequent work was with Canon equipment (when not his custom-made
    instruments):

    http://www.astrosurf.org/buil/index.htm
     
    eawckyegcy, Jan 8, 2006
    #19
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.