Image recovery: Recovering fragmented image files from flash memorycards

Discussion in 'UK Photography' started by Tonny Iversen, Sep 6, 2007.

  1. Hello. I'm trying to figure out if it exists any software that might be able to recover fragmented images from flash memory cards / digital cameras if the card was quick-formatted (i.e. if the card has a zeroed out FAT), or some details about the jpeg algorithm to figure out how difficult development of such would be.

    My main interest is for .jpg/.jpeg files.

    In the situation when the FAT contains only zeroes, recovery of non-fragmented image files seems to be easy for many (or maybe most?/all?) file / image /data recovery software products, but I haven't been able to find any that can recover anything more than the first fragment of the file if the file is fragmented.

    I don't know the specifics of jpg encoding, but maybe someone here knows if the different parts of the image part of the jpg files are numbered insome way, or contains some patterns/fields that can be used to connect later fragments together with the first fragment? If not, maybe something like FFT/Fouier transformations/frequency analysis or something similar/else could be used to do a qualified guess about which fragments belong toeach other and maybe present the user with some alternatives?

    I guess that if image files are fragmented on a memory card, the chance is pretty high that the files only has two fragments. At least this is my guess if the card was new or formatted before use, then used to take pictures and erase some in between, and then quick-formatted afterwards. And if the directory entries are intact and most files are defragmented, I guess it could be possible also to do an educated guess about which clusters are fragments belonging to other files. Also, comparing the starting clusters from all files could indicate if the file is actually fragmented, if the difference between the starting cluster of file n and n+1 is less than the number of clusters necessary to store the file. And if the card was new, zero-filled or filled with a specific pattern before use, it should be obvious if a specific cluster was in use or not before the quick-formatting. (Maybe a full formatting fills the cards with a specific pattern? I haven't tried...)

    Does anyone know of any software doing something like this or is anyone considering making something like this? I'm to pressed for time to consider developing something like this right now, but if someone else wants to look into this, I'd be more than happy to be involved in discussions around the development if you want (though not very much since I don't have much time available). And if someone could comment on these issues, it would be great if I or someone else should consider developing something like this.

    Best regards
    Tonny Iversen ()
    Tonny Iversen, Sep 6, 2007
    1. Advertisements

  2. First you'll have to find out the cluster size, then the directories,
    which give you the starting cluster, name and size of all the files.

    With fragmented files you'll have to combine every starting cluster with
    every other (not yet assigned) cluster on the medium, a very time
    consuming process. Also consider erased files, which can look like valid
    files, up to the point where a cluster was overwritten later. If you
    miss to detect the file as deleted, you'll spend much time in trying to
    find the no longer existing continuation clusters.

    This requires a "map" of all clusters, containing the characteristic
    information of each cluster. I made such a map already myself, for
    restoring segmented text files by manually pairing the ends and begins
    of the clusters. But with image data? At least you'll need an file
    checker, that can determine whether a chain of clusters *might* be a
    valid jpeg file.

    FAT32 allocates files strictly sequentially, until the medium is full.
    You can assume that no overlapped writes occured, to multiple files at
    the same time. In this case a recovery will work quite well. When files
    have been erased, and the clusters have been reused later, you'll have
    much more problems.

    Hans-Peter Diettrich, Sep 7, 2007
    1. Advertisements

  3. Tonny Iversen

    zac.badenoff Guest


    'ere ya go matey!

    It's been done so there's no need to try and develop something yourself.


    zac.badenoff, Sep 7, 2007
  4. Tonny Iversen

    Rob Guest

    Rob, Sep 7, 2007
  5. Tonny Iversen

    cmyk Guest

    Hi Hans,

    Under the FAT system, if you delete a file, as one might do with a poor shot, then it's space is simply marked as being available
    for re-use. If that file happens not to have been the last one written, then there is a good chance that either the next file
    written to the media, or the one after that, will be fragmented. For example (in simple terms, ignoring cluster sizes), if the
    deleted file occupied 1000K and the next two files written each occupied 800K, the second of those files would be fragmented into
    200K and 600K segments. Similarly, if the next file written was 1200K, it would be split into 1000K and 200K segments. Given the
    variability of jpeg file sizes and the tendency of many users to delete files when they review them, fragmentation over 3 or more
    clusters is quite possible.

    cmyk, Sep 7, 2007
  6. FAT32 appends files until the end of the medium is reached. This is an
    optimization to prevent file segmentation. Deleted files are overwritten
    only after the medium is full.

    Hans-Peter Diettrich, Sep 7, 2007
  7. Tonny Iversen

    Phil Carmody Guest

    I can think of few techniques _more_ likely to cause fragmentation!
    (Assuming that's what you meant by "segmentation".) Every temporary
    file (just think of your browser cache etc.) will create a hole, for
    example. Every hole is a place where the end of one file and the
    start of another, or even worse the middle of a file, will later be

    Phil Carmody, Sep 7, 2007
  8. Tonny Iversen

    cmyk Guest

    FAT32 does not do this - that is a task for the OS or, perhaps, the media controller. Whether a particular camera or memory card
    controller would do this when writing to the media is entirely up to the card or camera's maker. I know of none that have published
    anything to the effect that their's behaves this way. In fact, for digital media, doing so is pointless from an optimisation
    cmyk, Sep 7, 2007
  9. Right, I meant fragmentation.
    I suppose that image files are written sequentially, one after another.
    If not, the problem is very hard (NP?).

    Hans-Peter Diettrich, Sep 7, 2007
  10. Read the available documentation about the FAT32 related data structures.

    Hans-Peter Diettrich, Sep 7, 2007
  11. Tonny Iversen

    cmyk Guest

    I did.

    There is nothing in the FAT32 data structure that dictates the behaviour you claim.
    cmyk, Sep 8, 2007
  12. Look at FSInfo.FSI_Nxt_Free.

    Hans-Peter Diettrich, Sep 8, 2007
  13. Tonny Iversen

    cmyk Guest

    fsi.FSI_Nxt_Free specifies the lowest free cluster - there is nothing requiring this to occur after the last used cluster. It is
    simply there to speed up disk writes on magnetic/optical media.
    cmyk, Sep 8, 2007
  14. And consequently files are written contiguously, until the medium
    becomes full.

    Hans-Peter Diettrich, Sep 8, 2007
  15. Tonny Iversen

    cmyk Guest

    Not necessarily if the lowest free cluster occurs before the last occupied cluster.
    cmyk, Sep 8, 2007
  16. Tonny Iversen

    Phil Carmody Guest

    Are you sticking by your previous
    Deleted files are
    overwritten only after the medium is full.

    If so, consider this sequence:
    1) Create A
    2) Create B
    3) Delete A
    4) Create C
    After stage 3, what is the lowest free cluster? Is it after B,
    or where A was?

    If not, then can you point out where you retracted the claim?
    And maybe why you held that point of view in the first place.
    And what made you change your mind.

    Phil Carmody, Sep 8, 2007
  17. Yes, of course :-]
    Why bother with the lowest free cluster?

    Of course it's where A has been, but that doesn't affect the write
    pointer, which stays behind B.

    Hans-Peter Diettrich, Sep 8, 2007
  18. Tonny Iversen

    Phil Carmody Guest

    Each of the above operations is atomic, so what the **** is a write
    pointer? You're barely making sense, so it's hard to argue against

    The FAT32 system apparently (see posts by other posters upthread)
    writes each new file at the lowest cluster it can. That's from
    where A has been. This is in agreement with all the prior FAT
    systems that I have been familiar with.

    If you can't work out how and where you're wrong given this
    information, I simply don't have the time and patience to spoon
    feed you any more. Perhaps try on comp.i.dont.have.a.clue ?

    Phil Carmody, Sep 8, 2007
  19. You have read about FSInfo.FSI_Nxt_Free?

    Isn't that a write pointer?

    So you think that the system scans the FAT, each time a cluster has to
    be allocated to an file or directory?
    Then familiarize with the new features of FAT32, before posting nonsense ;-]

    Hans-Peter Diettrich, Sep 9, 2007
  20. Tonny Iversen

    cmyk Guest

    Then familiarize with the new features of FAT32, before posting nonsense ;-]

    And if you took the time to study how FSI_Nxt_Free is implemented, you wouldn't keep posting stuff like this. FSI_Nxt_Free points to
    the lowest available cluster. Nothing more & nothing less. Nowhere is is said that there can't be any used clusters after this. Get
    cmyk, Sep 9, 2007
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.