Photo by Kamil Pietrzak on Unsplash

Automating File Intake on a Linux Server

On Target

File uploads and intake are common administrative duties for teams of almost any size, and a shared upload folder can quickly become messy without a careful filing system to ensure that everything ends up where it belongs. It might start with one team member uploading a contract PDF, another dropping in some stock photos, and someone else saving an email attachment they think the team might find useful. Before long, filenames collide, documents are scattered across inconsistently organized folders, and no one is quite sure which files have been scanned, processed, or reviewed.

Typically, receiving the files isn't the hard part, especially not with the abundance of prebuilt solutions available in the modern web app landscape. Teams can choose from several robust, prebuilt solutions for shared storage and uploads, including traditional tools such as SFTP and Samba, modern platforms such as Nextcloud and Sync-in, and even email gateways or web forms. The real challenge begins after the files arrive. Validating uploads, checking them for security threats, preserving the originals, preventing duplicate names and overwrites, normalizing files into useful formats, and maintaining an audit trail often require manual intervention at some stage.

In this article, I show you how to build a small, automated, server-side drop zone that handles file intake through a repeatable, queue-based system. It uses a few standard components found on modern Linux systems, including a systemd path unit that monitors an incoming directory through the Linux inotify interface and starts the processor when work is waiting, along with a Bash processor that:

• claims each completed upload, creates a private service-owned snapshot, and scans that snapshot with ClamAV;
• optionally converts supported office documents to PDF with LibreOffice;
• Normalizes supported images with ImageMagick;
• assigns versioned names to duplicates and sorts the