# Why I don't erase/format a memory card......



## 480sparky (Apr 30, 2012)

....... right after uploading to my hard drive:

About a week after transferring roughly 300-odd photos, I discovered some had became corrupted.  They all (approx. 30 of 'em) looked like these:









That's part of the reason I have multiple cards, and don't erase images / reformat until I need the card again. Had I done so right after transfering, I may not have been able to restore them from the card.

.


----------



## table1349 (Apr 30, 2012)

If you had redundant backup you should be able to do the same thing.


----------



## 480sparky (Apr 30, 2012)

gryphonslair99 said:


> If you had redundant backup you should be able to do the same thing.



If they were corrupted from the git-go, the backup copy would simply have the same problem.


----------



## Bitter Jeweler (Apr 30, 2012)

Cool! Did you in those though Instagram?


----------



## Netskimmer (Apr 30, 2012)

After transferring the the images over to the HDD you could view the files in the folder as thumbnails. That would allow you to quickly scan a large number of pics to look for corruption.


----------



## 480sparky (Apr 30, 2012)

Netskimmer said:


> After transferring the the images over to the HDD you could view the files in the folder as thumbnails. That would allow you to quickly scan a large number of pics to look for corruption.



I ran through them, rating them all, the same time I transferred them. Never saw an issue then.   Just last night I noticed the corruption.


----------



## KmH (Apr 30, 2012)

Once I verify my upload (and backups) were successful, the card gets reformated to minimize the chances of corruption when new image files are written to the card.

The memory card FAT table on the card controller really should be reset/rewritten after each upload.


----------



## Netskimmer (Apr 30, 2012)

480sparky said:


> Netskimmer said:
> 
> 
> > After transferring the the images over to the HDD you could view the files in the folder as thumbnails. That would allow you to quickly scan a large number of pics to look for corruption.
> ...



So you reviewed them after copying them and they were fine and then became corrupted later?


----------



## 480sparky (Apr 30, 2012)

Netskimmer said:


> So you reviewed them after copying them and they were fine and then became corrupted later?



Yep.


----------



## nmoody (Apr 30, 2012)

At least the are pretty corruptions!

Not sure how true it is but I have read formatting in the card in a camera vs doing it in the computer reduces the chance of corruption. My guess is the camera formats it the way it likes it?


----------



## 480sparky (Apr 30, 2012)

Yeah, I reformat in-camera.  But I wait a week or two instead of doing so right after uploading.


----------



## bratkinson (May 2, 2012)

I typically wait until I am 'done' with all post processing on a set of photos (may be several days worth) before I erase the originals on the memory card - in camera, of course. That's =IF I REMEMBER= to erase them! (getting old is killing my memory). 

Like the OP, I have multiple memory cards. I just picked up my #3 card, another 16mb. That way, I can cycle-through the cards, one 'set'/'shoot' each and have a backup in case I inadvertently delete something during post that I want to get back.  I also shoot RAW+JPG these days, and download both, so I may just grab the JPG if I'm in a hurry.


----------



## Forkie (May 2, 2012)

480sparky said:


> ....... About a week after transferring roughly 300-odd photos, I discovered some had became corrupted.  They all (approx. 30 of 'em) looked like these:...



They turned out great!!  :lmao:

Obviously not what you were after, but I quite like them!  

Now, how to go about corrupting one of my cards........


----------



## Garbz (May 2, 2012)

480sparky said:


> Netskimmer said:
> 
> 
> > So you reviewed them after copying them and they were fine and then became corrupted later?
> ...



This is scary and I think it's time to review the reliability of your storage or your workflow.


----------



## 480sparky (May 2, 2012)

Garbz said:


> 480sparky said:
> 
> 
> > Netskimmer said:
> ...



That's also why I have 5 back-up hard drives going back 4-5 months.


----------



## Bitter Jeweler (May 2, 2012)

Bitter Jeweler said:


> Cool! Did you in those though Instagram?



This probably the stupidest sentence I ever posted.
Stupid auto correct.


----------



## MTVision (May 2, 2012)

Bitter Jeweler said:
			
		

> This probably the stupidest sentence I ever posted.
> Stupid auto correct.



LOL! Love auto correct


----------



## Buckster (May 2, 2012)

480sparky said:


> Garbz said:
> 
> 
> > 480sparky said:
> ...


Were the backups all corrupted too?


----------



## 480sparky (May 2, 2012)

Buckster said:


> 480sparky said:
> 
> 
> > Garbz said:
> ...



Only the most recent back-up.  The other 3 in the safe hadn't been used since ingestion of these frames.


----------



## table1349 (May 2, 2012)

Bitter Jeweler said:


> Bitter Jeweler said:
> 
> 
> > Cool! Did you in those though Instagram?
> ...



That's what happens when you use Auto......... Auto ISO........Auto Mode.......... :lmao:


----------



## marvinh (May 2, 2012)

I suggest not only formatting a card when there is some corruption of images, but use a WIPE utility.  A wipe utility is part of the san disk recovery software.  It completely ERASES the card and replaces the data with 1's and 0's. 

Formatting only resets the controller.  The wipe utility refreshes the card.  Of course, be careful. There is no chance of recovering your images after using the wipe utility.  

Marvinh


----------



## Garbz (May 3, 2012)

marvinh said:


> I suggest not only formatting a card when there is some corruption of images, but use a WIPE utility.  A wipe utility is part of the san disk recovery software.  It completely ERASES the card and replaces the data with 1's and 0's.
> 
> Formatting only resets the controller.  The wipe utility refreshes the card.  Of course, be careful. There is no chance of recovering your images after using the wipe utility.



There are so many things wrong with this comment. 

1) A format does no such thing. A format will simply re-write the file allocation table, data on the memory card itself. It makes the space as spare. It's is entirely controller agnostic. Hell it's technology agnostic and the process for formatting a memory card is no different from formatting a harddisk or any other block device.
2) A firmware wipe will reset a controller. A controller also happens to be responsible for marking unusable bad sectors on the memory card which it has uncovered previously. Resetting a memory card like this will make things worse not better.
3) NAND gates in the memory are likely to fail. Not the controller. They can not be repaired. 

4) All this is elementary since sparky's card works just fine. The pictures corrupted ON HIS COMPUTER. He still had the pictures on the card which were fine and used for recovery.


----------



## ClickandDestroy (May 4, 2012)

Sometimes I use a card so many times my camera tells me to change it cause it's unreadable. I do the classic eject blow and replace and works just fine. 

Going to start buying new cards. 4gb ones so I can cycle often


----------



## marvinh (May 4, 2012)

Garbz - you mention a firmware wipe above. What is that? I said a wipe/erase of all the data. 

Let me give a real world example: I took a photo class a few years ago. We passed around our memory cards to & from different cameras and even between Nikon and Canon without formatting first. When I went to use those cards again, I had many corrupted images that a format could not fix. The ONLY fix was to wipe all the data with the wipe utility included with the SanDisk recovery software.

That solved the issue. Marvinh


----------



## Skaperen (May 5, 2012)

Garbz said:


> There are so many things wrong with this comment.
> 
> 1) A format does no such thing. A format will simply re-write the file allocation table, data on the memory card itself. It makes the space as spare. It's is entirely controller agnostic. Hell it's technology agnostic and the process for formatting a memory card is no different from formatting a harddisk or any other block device.
> 2) A firmware wipe will reset a controller. A controller also happens to be responsible for marking unusable bad sectors on the memory card which it has uncovered previously. Resetting a memory card like this will make things worse not better.
> 3) NAND gates in the memory are likely to fail. Not the controller. They can not be repaired.


What I do for my cards is NOT reseting the controller, but "over wiping" the card.  This is TWO passes of sequentially writing every raw sector with binary zero.  I don't know how to do this in Windows.  If by chance you use Linux or BSD, I can tell you how.  This process leaves bad cells locked out, but cycles through the good ones and levels out the wear some.  Then I reformat and am good to go.  I've not had a bad card, yet.



Garbz said:


> 4) All this is elementary since sparky's card works just fine. The pictures corrupted ON HIS COMPUTER. He still had the pictures on the card which were fine and used for recovery.


Looks to me like he let some of the magnetic fields on his 4200 amp three phase circuits (while faulting) get around his hard drive :mrgreen:

This is what I keep backups for.  When I save photos to disk, I make 3 backup copies.  That's integrated in my archiving script.  I now have 53 GB of photos but only a pair of 16 GB CFs cards.  I reformat the cards to keep the camera fast.  But I have to reformat to make more space, anyway.


----------



## Garbz (May 5, 2012)

marvinh said:


> Garbz - you mention a firmware wipe above. What is that? I said a wipe/erase of all the data.
> 
> Let me give a real world example: I took a photo class a few years ago. We passed around our memory cards to & from different cameras and even between Nikon and Canon without formatting first. When I went to use those cards again, I had many corrupted images that a format could not fix. The ONLY fix was to wipe all the data with the wipe utility included with the SanDisk recovery software.



Honestly if this fixed the issue I'd be on the phone to SanDisk asking for a warranty replacement. NAND flash memory has known failure modes, but a controller does not. For a controller to have a problem reading and writing that is a scary prospect. Firmware wipe is probably what you did in this case. Memory cards and harddisks have complicated controllers in them. They do more than just read and write data. They cache frequently used data for faster access, and work around the problems of the device. For NAND flash the controller does wear levelling which is writing different bits to different places on the controller as you go. So when you open and save a file for instance the file will appear elsewhere in the physical memory so that repeated writes to the same spot don't wear out the memory. They also do things like align data in blocks of certain size so that if you for instance need to read out 3k worth of data it can do it in 1x 4k read rather than 2x 4k read where the data is spread around. In my opinion if a controller like this has issues there's a design flaw in the card. This isn't unheard of too. Resetting the firmware would restore it to factory defaults. The controller would loose track of how many times which memory address was written, a database of known bad addresses, etc. Typically this is not a good thing to need to do. 

Harddisk controllers are similarly complicated. Harddisks for instance actually have a larger capacity than the advertised by the controller to the computer. The controller on the disk uses this additional space for "remapping" bad sectors it finds completely transparently to the system. It also tracks all sorts of performance data of the drive, is responsible for powering it down, parking the head when it detects a sudden acceleration (dropping you laptop) etc.



Skaperen said:


> What I do for my cards is NOT reseting the controller, but "over wiping" the card.  This is TWO passes of sequentially writing every raw sector with binary zero.  I don't know how to do this in Windows.  If by chance you use Linux or BSD, I can tell you how.  This process leaves bad cells locked out, but cycles through the good ones and levels out the wear some.  Then I reformat and am good to go.  I've not had a bad card, yet.



You should trust your card more as I believe you're wasting your time. Both SD cards and CF cards implement a control system called TrueFFS which does wear levelling, error correction, and bad block remapping. Effectively you're just trying to confuse the poor controller and wear out your card at the same time (though given that 1000000+ rewrite capacity of flash memory and their incredibly large size these days I don't think they are as easy to wear out as they used to be). The only time I see what you're doing may be appropriate is if you're taking photos of something that the local government ... against the interest of national security. ... I say local government assuming you're a CIA agent working abroad wiping your memory card like that after each shoot


----------



## jake337 (May 5, 2012)

Nice triptych!


----------

