How do you STORE your RAW images and organize your JPG's????

Discussion in 'Digital SLR' started by infiniteMPG, Mar 20, 2008.

  1. infiniteMPG

    infiniteMPG Guest

    Just getting into the "real" digital photography world with my Sony
    Alpha A-100 and starting to use RAW and digitally edit, this takes
    some major storage space on the ol' PC. When I save my images from my
    camera I just use what I always have, I have a sub-folder under
    "IMAGES" by the name of the year (ie "2008"), then subfolders under
    that by the month (ie "2008-03") and then under that sub-folders by
    the date (ie "2008-0320") and just dump my pictures there. Under
    those I may create sub-folders for edited images and name them
    accordingly.

    I have a 720Gb USB drive I use to back stuff up but was wondering how
    others store, archive, and organize their images. Do you burn to DVD
    and store somewhere away from home? Do you use some database
    program?

    And what digital image editing software do you use and/or recommend?

    All info is helpful!
    Thanks.
     
    infiniteMPG, Mar 20, 2008
    #1
    1. Advertisements

  2. infiniteMPG

    Guest Guest

    i use two hard drives, and i'm working on an offsite solution that
    isn't overly inconvenient.
    adobe lightroom.
     
    Guest, Mar 20, 2008
    #2
    1. Advertisements

  3. infiniteMPG

    Sbc News Guest

    After years of trying different methods I settled on several different
    methods... depending on the use of the art.

    Raws - all raws I shoot end up, worst case, on two USB hard drives, that are
    only for original raw files.

    Show, gallery - Processed and Photoshoped tifs, ready to print. These also,
    worst case, on two USB hard drives that have only show and gallery files.

    I have a computer running Imageprint for my Epson 4800 and that is all that
    particular computer is used for. I have a couple of 500Gb drives as well as
    its boot drive. Every thing I print also resides on that computer for quite
    some time.

    Commissioned work - Virtually all is sold including all rights. I keep the
    originals on a USB drive and the computer used for processing/Photoshop till
    I need room for newer work. I've never had a client say a year later, "Hey,
    I lost that work you did for me."

    I've tried optical media and the retrieval time is unbearable, especially
    when you're browsing.

    I don't buy bargain drives. I usually purchase hi-end Seagates and the
    enclosures separately. I'll run new drives in my main water-cooled beast
    for a month or so as a hedge against infant mortality. After the exercise
    session I mount the drives in their USB enclosures. I purchase enclosures
    that hold two drives each.

    That's my system, your mileage may vary.
     
    Sbc News, Mar 20, 2008
    #3
  4. infiniteMPG

    Pat Guest

    I keep them on two disk drives. I also burn two DVD's. One stays home.
    The other gets taken to work.
     
    Pat, Mar 20, 2008
    #4
  5. I am currently using Bibble Pro from Bibble Labs. My OS is Linux so I have
    a few less choices for specialized software. I have a Canon Digital Rebel XT
    and the native Linux photo software did not handle the raw files very well.

    I am happy with Bibble Labs software other than for printing directly from
    the application. I currently make a jpg first and then print that.

    I have a directory structure some what like yours on my photo computer. I
    back the photo files up to a Seagate USB hard drive. I should have a second
    hard drive but I can't currently afford it.

    Have a good day.

    William
     
    William Hathaway via PhotoKB.com, Mar 21, 2008
    #5
  6. infiniteMPG

    Alan Browne Guest

    To get a 15 - 20% gain in space I convert all RAW images to DNG format
    using the free Adobe converter. After backup verification I wipe out
    the RAW files.

    I sort by collections within dates not unlike your description above.
    On the backup DVD's I write some info as to content. I should use a dB
    to track images, but I don't. Something to improve upon for sure.

    I have been backing up on DVD of late, but I may go entirely hard disk
    as capacities shoot up and prices tumble.

    I recently bought a 1 TB IOmega drive for this purpose. (But I'll keep
    on doing DVD backups as well for the time being).

    If you get another drive, try to get something faster than USB 2.0 such
    as Fireware 800 (1394B). Much faster xfer rates.

    I'll consider BluRay if I can record 50 GB on a long life ("gold")
    medium with benign condition 100+ year life. (As opposed to 5 yr life
    for benign condition ordinary DVD's/CD's).
    Photoshop CS3 or Lightroom. If RAW is part of your workflow then taking
    advantage of the 16 bit per color is important. These are not cheap.

    If you only work with the camera JPG's, then Photoshop Elements is more
    than sufficient for most amateur photographers.

    Cheers,
    Alan.
     
    Alan Browne, Mar 21, 2008
    #6
  7. infiniteMPG

    BobF Guest

    That's really all you have to do to place the actual files... I dump them in
    folders by the month. Except if I take a lot of pic's, then I limit each folder
    to 1000, so I may have June_A, June_B etc etc (I don't take that many in the
    winter!) All my edited files end up JPEG's so they're easy to find!

    Now get photo album software, such as Centico, and use that to keep track of
    things. You don't actually have to name the files, you can name them in the
    photo album. The tags in the album allow you to "store" each photo in dozens of
    places.. and search for them. You can edit the tags and the search "live" as you
    go, it's quite powerful..EG Search for 'flowers' + 'Canada' but NOT 'Montreal'
    etc etc... That's what I use... but at this moment I don't know if it can do
    Sony RAW files... I know it can do Nikon... so CHECK to make sure the album
    software you buy can see a Sony RAW file.

    I plan to contact the Centico company to check into available RAW converters...
    DON'T just rely on some hard drive!! I have a collection of dead hard drives,
    and not just old ones either... most are newer than my CD backups!! That's a
    case of CDR's outliving hard drives!

    I store on both CDR's and DVDR's, and I also put my photos on other computers.
    And I have a DVD collection in my locker at work... They're cheap, go crazy! (I
    plan on doing DL DVD's as well, and probably will also do blue ray when it
    becomes normal.

    I would imagine I have at least 10 copies of all my photos... I migrate them
    along to each new computer, each new storage format...
    The Sony RAW converter is an excellent start, it is more powerful than lightwave
    ( I have used both )... You could get a copy of adobe elements, it is useful for
    some tasks, but I seldom use it... the learning curve is very steep.

    A little practice with the Sony converter and you can do almost anything.

    The software I use myself is not available so that is no help to you!!
     
    BobF, Mar 21, 2008
    #7
  8. infiniteMPG

    AlanW Guest

    Lightroom question - I have PSE5 with about 3000 photos "tagged" using
    the PSE Organizer. If I get Lightroom, will I be able to convert the
    tagged photos or will I have to rename all of them using the Lightroom
    process?
     
    AlanW, Mar 21, 2008
    #8
  9. infiniteMPG

    Dave Busch Guest

    On Thu, 20 Mar 2008 10:20:24 -0700 (PDT), infiniteMPG

    I've gotten lazy in recent months and am way behind in making DVD
    backups. The big problem is that a DVD holds just 4.8GB of images,
    and I can shoot that much in a day, or twice that much if I am
    shooting sports. It's no fun having to get back and then immediately
    burn two DVDs, or double if I want to store one off-site.

    So I've been backing up everything from my computer's internal hard
    drives to a pair of 1TB NAS drives, giving me three magnetic copies.
    No RAID; just Acronis TrueImage copying to both drives regularly. I
    realize this provides absolutely no protection from fires or theft, so
    this solution is less than fully satisfactory. I expect prices will
    come down enough this year that I can add a third 1TB NAS drive and
    rotate one of them to a safer location once a week, minimizing loss.

    Computer has three 500GB EIDE drives mounted internally in slide-out
    trays. I have a recent duplicate of my C: in a tray, so if C:
    crashes, I just slide in the duplicate and reboot until I have time to
    restore from my most recent backup.

    There are also two 500GB SATA drives internally, plus two 500GB USB
    drives. RAW files, JPEGs, PSDs, and sidecars are stored by project in
    folders with date and subject (080304Segovia.) When backup up to DVD,
    I back up only the RAW and PSDs. The JPEGs can be restored from DVD
    from the RAW files in batches using a utility program if necessary. Or
    from magnetic storage.

    Oh, my Drive C: with operating system is backed up to one of the USB
    drives, so if something bad happens I can plug the USB drive in and
    restore without needing to have the network involved. No images are
    stored on Drive C: Images go on D:, E:, F:, and G:, the other
    internal drives.

    My current system replaced two USB drives with slide-out trays that
    let me hot swap eight or nine different EIDE drives as required. It
    was a little clumsy and the drives were always old ones with a mixture
    of capacities, so I ended up using 250GB, 300GB, 400GB, etc. drives
    that I'd pulled from the computer.

    Dave
     
    Dave Busch, Mar 21, 2008
    #9
  10. infiniteMPG

    Guest Guest

    i'm not familiar with pse5 and tagging but a quick google search
    revealed:
    <http://kb.adobe.com/selfservice/viewContent.do?externalId=kb400736>

    and if you used adobe camera raw, lightroom will pick up whatever
    settings you used.
     
    Guest, Mar 21, 2008
    #10
  11. infiniteMPG

    nickraosr Guest

    Great topic. I have been spending too much time managing my image
    library. I want to shoot, edit and print. Backup and Archival should
    be part of the workflow. Can anyone recommend a very good automated
    solution for backup and archival for technically challenged? I'm
    looking for something that is pretty seamless in getting to an image
    that may be stored or archived in the least amount of time. This
    question may require too lengthy a response, so a good URL to go to
    would suffice. Thanks.
     
    nickraosr, Mar 22, 2008
    #11
  12. infiniteMPG

    C J Campbell Guest

    I use Aperture, but right now I recommend Lightroom for its slightly
    more flexible editing controls. I particularly like the way Lightroom
    stores presets. However, I will continue to use Aperture simply because
    I don't want to try to convert my library to Lightroom.

    I import each set of images into a new project in Aperture. The images
    are renamed to image date and time while imported; no processing is
    done to them. I then export these images to a backup hard drive and
    burn them to DVD, storing the DVDs in a fireproof file cabinet. OS X's
    Time Machine makes an additional backup on another hard drive.
     
    C J Campbell, Mar 22, 2008
    #12
  13. infiniteMPG

    C J Campbell Guest

    Blu-Ray?
     
    C J Campbell, Mar 22, 2008
    #13
  14. infiniteMPG

    Dave Busch Guest

    Blu-Ray is promising, but not practical for me right now.
    A 2x Blu-Ray disk costs me about $20 for 25GB, and a 50GB is $35.
    While there may be sources for disks at a lower cost, prices would
    have to be at least 90 percent lower than that to compete with DVD.
    And the burners are hundreds of dollars.

    Meanwhile, I am paying 20 cents for DVD-Rs and can burn them at 16X in
    the $50 DVD burner I already have. Until Blu-Ray disks cost a dollar
    or two, I'd be paying a lot for convenience -- and then losing that
    convenience in slow write speeds.

    Except for the minor point that they only hold half as much as one of
    my 8GB memory cards, DVDs are, at least, economical.

    I paid $1000 for my first CD burner back in ancient times, and $8.00
    for a CD-R, so I already been there, done that with pricey media.

    Then there's the proven longevity thing, with neither media having a
    track record yet.

    Dave
     
    Dave Busch, Mar 22, 2008
    #14
  15. infiniteMPG

    Paul Furman Guest

    long answer about my workflow follows...
    I shoot raw+jpeg fine & only keep the raw files for images that make the
    cull, average about half the shots. When editing I still keep the
    original jpegs in a subfolder although that's a bit redundant with the
    raw files.

    _Pictures
    2008

    2008-03-17-moon
    orig
    raw
    seconds

    2008-03-18-birds
    orig
    raw
    seconds

    When the hard drive fills up I move the seconds, originals & raw files
    off to an external drive. The final edit jpegs stay on the C: drive
    (plus backup to external) and get sorted into shooting location folders
    (easier to find a particular subject) rather than chronological flat
    file like the raws which remain as above (easier to find individual
    files that way).

    _Pictures
    2008
    (new pics as above before sorting)
    Moon
    2006-01-22-new-moon
    2006-04-07-eclipse
    2008-03-17-moon
    Wildlife
    2005-12-25-woodpecker
    2007-08-29-hawk
    2008-03-18-birds

    This is also how I sort my web galleries (by location/subject). The web
    galleries include annotation in small text files. These are displayed by
    server side PHP pages and I create the files with the same custom
    setup. The .exif.txt files are generated with a batch file using exiftool.

    2008-03-18-birds
    _PBF2365.jpg
    _PBF2365.exif.txt
    _PBF2365.txt
    _PBF2366.jpg
    _PBF2366.exif.txt
    _PBF2366.txt

    The web files are how I search for an image initially because it's
    quick, the files are small and usually I can just google my own site to
    find, say, a good photo of a hawk. I also started to keep a 'Photo
    Update' blog section which is purely a chronological file of the best
    shots from each shoot every week or two. So I'll do a shoot, come home
    with 60 images, delete 10, delegate 40 as seconds, 10 as keepers,
    download those raw files & the Photo Update blog will probably only
    feature one or maybe 3 of those. It is comprised of 'shortcuts' to the
    originals in their respective folders. I can make more themed shortcut
    folders too but it's a lot of work.

    A pair of 500GB USB/firewire drives. Every couple years I have to get a
    bigger set of external drives :)
    Photoshop for editing, Irfanview for quick viewing & sorting. I'm still
    short of a good program for viewing that lets me flip through at full
    pixel zoomed into the same area. I had an old version of ACDSee which
    sort of came close to this. Irfanview is used to move files into the
    'seconds' folder by setting up the 'Move To' folder & tapping the f7
    key. This lets me work quickly as I can always go back & move a shot out
    of the seconds folder. Only badly messed up shots get deleted.

    I don't trust proprietary tagging and annotation. I had some bad
    experiences with the ACDSee database becoming corrupt. The little plain
    text files are easily searched on my hard drive or the web and could be
    imported to a database some day if need be. Embedded IPTC & EXIF data
    isn't a safe place to store tags either IMO.

    My setup is a bit of a mess. A 'real database' would be nice but there
    are advantages to a commonsense folder structure too. If you use a
    database & flat file approach, it's almost impossible to search by
    manual means.

    I don't bother saving layered photoshop edited files, just jpegs at
    quality 11/12. If I want to go back, I'll probably find a better way to
    redo the edits. Those files are huge. If I do a big final print, I'll
    up-size it to 300dpi at 13x19 & apply sharpening, do a test, maybe
    adjust colors or brightness & save as _PBF2365-print.jpg The camera is
    set to lowest contrast, saturation & sharpening and files don't get
    sharpened until a final use is determined.

    The web galleries include a full pixel crop and thumbnail of the
    uncropped, unedited original. This helps using the web gallery to find a
    suitable photo being able to see how well it will enlarge without
    digging up the original files on the external drive. It's a lot of work
    to create though that's where I study my work as I go, learning what
    settings give me the best results. Sorting the favorites helps find the
    best shots later. Most of my galleries include a 'full-set' and some
    select images as shortcuts in the folder above like this:

    2008-03-18-birds
    _PBF2365.link.txt (shortcut)
    _PBF2366.link.txt (shortcut)

    full-set
    _PBF2365.jpg
    _PBF2365.exif.txt
    _PBF2365.txt

    _PBF2366.jpg
    _PBF2366.exif.txt
    _PBF2366.txt

    _PBF2367.jpg
    _PBF2367.exif.txt
    _PBF2367.txt

    Then the Photo Update only shows the best shot from each shoot:

    photo-Uupdate
    2008-03-22
    _PBF2365.link.txt (shortcut)

    http://edgehill.net/gallery/photo-update
    Then I copy over some photos to my other web site where they are used in
    galleries for each plant species we sell:
    http://www.baynatives.com
    These get resized again and a custom thumbnail crop for some. It's a lot
    of work. The photo update goes to friends & magazine editors and I use
    it myself to find plant photos for the baynatives pages.

    And I copy a few over to flickr from the update pages. Flickr a nice
    tagging interface but I'd hate to have all my data in their proprietary
    database.
     
    Paul Furman, Mar 22, 2008
    #15
  16. infiniteMPG

    Guest Guest

    lightroom gives you the advantage of a database & keywording while
    honoring whatever file hierarchy you prefer. you can even browse by
    folders from within lightroom.
     
    Guest, Mar 22, 2008
    #16
  17. Most photo editing packages (e.g. Adobe Photoshop, MS PictureIt/Digital
    Image Suite, Nikon PictureProject, ...) come with a photo library component.
    When using those library functions the location of a photo on the HD becomes
    mostly irrelevant. Instead of having one sorting criteria (in your case the
    date the photo was taken) and then dig deep down many directory levels you
    assign as many custom attributes to a photo as you like. This information is
    stored in a database and allows you to easily search, filter, and manipulate
    any subset of photos. Some filters are predefined, others you can define
    yourself.
    E.g. it would be very simple to find all photos with aunt Betsy and nephew
    Bill, taken without flash, using camera X, at the wedding of your cousin
    John as long as you defined and correctly applied the attributes Betsy,
    Bill, and John's wedding.

    It doesn't matter where the photos are stored on the HD because all the
    relevant information about the photos is stored in that database. Even a
    flat folder is ok, although you may want to have some substructure for other
    reasons like e.g. easier archival.

    jue
     
    Jürgen Exner, Mar 23, 2008
    #17
  18. I switched to using dual-layer DVDs; they may cost 4x as much as a
    regular DVDs and don't quite hold twice as much, but that extra storage
    density means my growing stash of disks is relatively managable in its
    own right.

    I only use these for off-line backups; my on-line storage is RAID-5'd,
    so I can lose (and indeed, have lost on several occasions) a disk
    without losing all of my data.

    As for actually organizing my files, I have way too much (current count
    is 38,497 files in nearly 200GB) to rely on any manual technique. I
    switched to a database-driven solution several years ago and never
    looked back. See http://po.shaftnet.org

    - Solomon
     
    Stuffed Crust, Mar 23, 2008
    #18
  19. Hell, I'm using U320 SCSI drives for my storage needs. I know one thing; I
    find the USB interface to not be as reliable when going to external drives.
    Firewire is a bit better than USB, but SCSI still rules.




    Rita
     
    Rita Berkowitz, Mar 23, 2008
    #19
  20. infiniteMPG

    Mike Mills Guest

    I am sure this thread has been posted before, but here is my
    contribution.
    I tried shooting a few raw pics and I tried more than a few ways of
    editing and converting them.
    It was an interesting and technically superior way of doing things,
    but it was SLOW.
    I now store in jpg format.
    The vast majority of the pics I take are not worth ever printing.
    Many are thrown out.
    The majority are kept and displayed on screen only resolution.
    These pictures are ~10x smaller for storage purposes.
    The pics have comments added in the comments field by jhead. A dos
    commandline program.My Exif data is preserved.
    I can sometimes use a descript.ion file for additonal description of
    a file through XNview.
    Storage by date has worked for me with quick text search fairly easy
    to do.
    I keep my database updated with Indexyourfiles , a wonderful freeware
    file indexer , which works quickly and can be burned onto a backup cd
    for searching directly on the storage medium.
    I have about 4 or 5 years' photo collections on my main drive.
    Backing up to other machines, other drives, and to CD is a regular
    feature.
    This works for me but I am sure that your mileage will differ.
    No photoshop.
    Only Freeware tools.
    I have never printed a digital photo or had one printed.
    I have kept a few that may offer the possibility in future.
    I do not own a printer .
     
    Mike Mills, Mar 23, 2008
    #20
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.