Safely serving reader-submitted media files for news orgs

@akwizgran
Mon, 05 Sep 2016 11:33:05 +0000

The general approach I'd recommend is preserving the good rather than removing the bad. In other words, extract the content from the user-submitted file and use it to create a new file. This is easier and more reliable than trying to remove malware, metadata, etc, from the original file, because you know what the content's supposed to look like, and you can be sure that your newly created file contains only what you've put into it.

For example, convert an uploaded image to an uncompressed bitmap (BMP). If your images need to be served in a compressed format like JPEG or PNG, you can convert the clean bitmap into that format. Likewise, convert an uploaded MP3 file to an uncompressed WAV. Then you can recompress to MP3 if necessary, knowing that any extra data hidden in the original MP3 has been removed.

To convert a PDF to a folder full of bitmaps and then back to a PDF, you can use something like the following (on Linux with ImageMagick):

convert -density 300 dirty.pdf page-%02d.bmp
convert -density 300 page-*.bmp -compress lzw clean.pdf

LibreOffice can convert DOCX and other supported formats to PDF on the command line using something like the following:

loffice --headless --convert-to pdf dirty.docx

I second @jason_nstar's point that conversion should be done inside some kind of sandbox, otherwise malware in the original files may infect the system doing the conversion. Recent vulnerabilities in ImageMagick are very relevant here.