How to fix apparent ruined digital files
By: Thorsten Overgaard. August 21, 2024.
When there were pixels and stripes in my exported images, I quietly realized to myself, 'The end of the world has come.' I considered all possible scenarios, from a virus attack on my files, faulty cables, airport x-ray machines, a dirty SD card... and more. But then I found a simple solution
Re-editing a catalog from Paris 2021 with 1,507 images, I started seeing pixels on the screen during editing. At first, I decided it was likely my computer, an Intel MacBook Pro 16" that I had held onto for longer than usual to ensure that all software worked.
It also had a rough life with several repairs from hard travel, so it would make sense if it had taken a hit somewhere. It seemed like only a matter of time before it would end its life with a flame coming out the side from some loose connection. I ordered a new one.
The final file from Paris, which I thought had been corrupted, until I found out it was the Graphics Processing Unit (GPU) that caused the problems. Leica M10-P with Leica © Thorsten Overgaard.
On-screen editing in Capture One, I would see pixels appear and disappear when zooming into photos, as well as a smaller net of noise. It must be the graphics card that has a glitchy connection. The computer has taken too much of a beating. But then the pixels appeared in the exported final images as well!
The horror: There's piexls in my exported JPG files
But then it happened: I had large pixels in my exported images. Small JPGs for the web and large high-res JPG files had them too. And they moved around with each export. See them here on the right side and in the middle:
The corrups edition with pixels in the exported file. Leica M10-P with Leica © Thorsten Overgaard.
It wasn't just the one file. It was on several files, convincing me that something had happened to the folder at some point!
The pixel noise was even worse in Adobe Photoshop
It must be Capture One that created pixel noise, I thought. ,I opened the original DNG file in Photoshop, only to discover that it was even worse! The DNG file was completely corrupted and shattered with pixels, lines, and digital mayhem. And it seemed like there was nothing I could do to change it.
The DNG file opened in Photoshop looked like it was corrupt beyind repair.
Opening the corrupt file in another editing program: Adobe Lightroom Classic
Last resort: I opened the DNG file in Lightroom, and there was no noise, no pixels on the screen. I exported the files, and no pixels. Everything was clean.
Strange, but it did give hope. I thought, if this had happened to all the files on that hard drive, or that SD card, or whatever the reason was, I could to re-edit them in Lightroom. It could be a final resort to save the corrupted files.
The solution to remove the noise and pixels turned out to be simple
I decided to find out what happened. I would first contact Capture One, and then maybe Leica to hear if they had Leica M10-P corrupt files for some reason.
I recorded everything on the screen in a video, and sent that and the original files to Capture One support for help. In the process, they directed me to a support page that said to turn off the GPU.
In Capture One Pro, open Settings > General and set Hardware Acceleration to 'Never' and 'Never':
Remove the pixel noise from Photoshop
I decided to turn off the GPU in Photoshop as well. I opened the DNG file in Photoshop and saw the dialog screen. At the bottom of the screen, you click on the highlighted line (which will open Camera Raw Preferences):"
In the dialog box, you choose Performance > Use Graphic Processor set to "Custom" and then turn off both "Use GPU":
Once you click OK, you will see that your image is now clean:
Conclusion: It was the GPU creating the problem
While I thought I had bought the best of the best, optimized for speed and power, it was the GPU creating the problem. The Graphics Processing Unit (GPU) in your device helps handle graphics-related work, effects, and videos.
This means that if you experience weird video images, noise in videos, images, or any type of screen issue, you should look for ways to control the Graphics Processing Unit (GPU) in whichever computer you are using.
Other considerations:
As an epilogue to solving this problem – I was lucky the files weren't actually corrupted – this is where I start sounding like your mother:
I state again and again in my Capture One Survival Kit and my Lightroom Survival Kit that one must finish work. Just from common sense, completed work doesn't come back to haunt you. Also, you need to have final images ready to present. You're not actually a photographer if you don't have any images (by which I mean that if you have 50,000 unfinished images in software but none ready to show, you're just sotring image files but are not making any photos).
But more importantly, my argument is that you cannot be sure that the RAW and DNG formats used at the time you take a photo can also be read and edited in the future.
"Old love never rusts," as they say. This 50 year old Volvo 240 does a good job, but it will eventually come to and end. © Thorsten Overgaard.
"So far, so good," you may say
Lightroom Classic is still backward compatible with the original 2007 (that uses 'Process 1'), but it won't last forever.
History tells us that Word documents made in the 1990s can't be opened in current Microsoft Word, and QuarkXPress files made in version 4.1 and before cannot be opened in later versions of the software. Software made for 32bit Apple computers can't run on today's computers (FlexColor for example). And so on.
That is why you must finish work: Select photos, edit them to the final look you want, and then export the JPG sizes and any TIFF versions you consider the final look.
Then put it all in the archive and back up the archive.
An archive is where you put stuff so you can find it again. Pictures are not supposed to float around in software on your computer or in a cloud. They must be stored in an external hard drive archive, and then backed up on a separate hard drive that is in another location. Read more in my Survival Kit. © Thorsten Overgaard.
You keep the DNG (digital negative) files and the RAW files, naturally, but you know that you may not be able to open them in five or fifteen years. Or if you can, you may not expect to see the same. Because where is the software going to be that reads the raw data and the edits you made and present it that way? (The different Process 1, Peocess 2, etc in Lightroom all work differently, but is sofar supported. As long as it lasts. Capture One similarly have 'engines' for each generation). I have lots of video files of the kids and stuff I made with video cameras and phones 20 years ago I cannot read anymore.
Likely, in a few years, you cannot see your unfinished DNG and raw files. You have to finish them. Which means edit the exposure, colors, etc and export to JPG or TIFF.
By the time they have grown up, you may have lost the original file if you don't organize your files. My daughter Caroline's first school day in 2004. © Thorsten Overgaard.
Why secure DNG and raw files by exporting them as final JPG's?
Things can happen.
A virus could emerge that attacks raw files, or even picture files. I want to have some external hard drives stored far away with my final images. For me, there are currently 135,000 final images in my archive, which is about 1.2 TB. I can make a couple of SSD drives (2TB each) and give them to my kids.
The ultimate security would be to burn Archival Blu-ray disks of the 'final images.' These hold 100GB each. This would be 13 Blu-ray disks for my archive, and I would have to burn a new one once in a while with additional new photos. Not bad at all.
If I want to ensure that my most precious images are protected from any virus or digital warfare like Electromagnetic Pulse (EMP), I would have to make prints or have negatives made (this can be done from digital files, but is expensive).
EMP warfare or terrorism: Electromagnetic Pulse (EMP) destroys or corrupts all digital files on tape, SSDs, hard drives, etc. If the hard drives are put in a Faraday cage, they would be safe against EMP. A microwave oven acts as such, but it is not 100% secure (that's why they put their mobile phones in microwave ovens in spy movies; it cuts off signals). A metal container that is well-sealed and insulated is another 'economical' way of having a Faraday cage. Whether a bank box is safe depends on sealing and if they are grounded.
Faraday bags exist for phones, computers, etc., usually to keep signals from cell towers from reaching the phone. These can even be bought on Amazon for relatively small amounts. The key is that there is no leak or opening.You can also get Faraday drybags, which could be a way to transport some hard drives to a bank box or other storage. This would be for hard drives you store, not the ones you use (I would think so, as they generate heat and you wouldn't be sure if you covered all leaks on your office hard drives).
Professional boxes, as well as entire Faraday rooms to work in, exist, and are quite expensive. The solution should preferable be simple, not another big project.
EMP can happen from nuclear explosions 30 to 400 km above the earth's surface. It can happen from solar storms (geomagnetic storms), lightning strikes, EMP weapons (e-bombs without nuclear explosions), and even faulty electrical equipment (EMI - Electromagnetic Interference. this can be physical connections, such as power lines, but also electromagnetic waves from radio transmitters or a microwave oven or other - A Nespresson machine for example produce a lot of EMI when it makes coffee. In other words, keep your hard dives not next to the Nespresso machine!).
Church office in Italy. © Thorsten Overgaard.
Corrupt copying or storage: Hard drives, RAID drives, solid-state drives, tape drives—whatever could corrupt a file during copying, transport, or a faulty connection in a cable.This means you have to check files you copied, before you delete then from where they came; or at least be alert that you didn't get any error messages during copying.
Software use or updates that alter files: For a moment, I wondered if having a DNG previewed at some point in Adobe Lightroom Classic might have altered something in the file, making it corrupt for Capture One and Photoshop. This wasn't the case, but it wouldn't take much to imagine that some cataloging software, editing software, or other could alter a file or its structure, making it unreadable or damaged. Considering modern camera files are 50-100 million pixels, not much has to go wrong before it all looks bad. Who knows what happens when you drop a DNG file into Apple Photos. It copies it into it's own shell, but does it also change the file?
Most common error, the user error. You deleted a file you shouldn't have, or you 'fixed it,' you put it in 'a great new software' and didn't know it would convert it. Things like that. Backup should prevent that. Historical backup is a type of backup that stays untouched (contrary to continuous backup, for example, to Time Machine or a RAID mirroring system. If continuously backed up, so are errors and corrupted files). If you want to make sure current or future events don't corrupt your files, you need to keep historical backups, which are backups that are made and then left untouched and not connected to anything. These serve as a possibility to 'go back in time' if something should happen.
This and more is covered in detail in in my Capture One Survival Kit and my Lightroom Survival Kit that I wrote in 2010 and have continued to keep updated, now with video tutorials on editing and more.
Made by photographers for photographers, the Survival Kit is the one-of-a-kind tool, written so simple that anyone can understand and use it.
More to come
Bon voyage with it all. Sign up for the newsletter to stay in the know. As always, feel free to email me with suggestions, questions and ideas. And hope to see you in a workshop one day soon.
/Thorsten Overgaard
Sign up now and get instant access to the class on editing workflow at Overgaard-Academy.com
| |
|
|
|
|
| |
 |
|
| |
|
|
|
|
| |
"Photography is simple"
Thorsten Overgaard is one of the best to explain in simple terms how cameras works, and how to get excellent results.
Be competent. Buy and study this easy-to-use video class that explains how to get focus right, what sharpness is, how to get the exposure and the colors right, and more ...
"Easy to apply, photography finally told so everyone can become an expert user"
Normal price $498.00
Explained by Thorsten Overgaard by using Leica Q2, Leica SL2, Leica M11, Leica M10 Monochrom, 35mm film cameras, Leica Digilux 2 and more. |
|
Thorsten Overgaard's
"Camera Excellence Class"
For Computer,
iPad, Apple TV and smartphone.
Normal price $498.00
Only $198.00
Save $300.00
USE CODE: "COXY66" ON CHECKOUT
Brand new. Order now. Instant Delivery.
100% satisfaction or money back.
More info
Item #1847-0323
Released April 2023
|
|
| |
|
|
|
|
