C-Command Software Forum

Bug: Checksum verification fails on bundle-based file edits

This might not be something you can do anything about, but when a packaged file format is located in the EF library, and it is edited externally, EF does not correctly update the checksum when the file is closed in the editing application. I’m not sure how you do this with regular text and RTF files, but for whatever reason the checksum is not updated correctly with package files. I can manually update it, and then it will verify—so the underlying function must be working—it just isn’t getting triggered.

This doesn’t have to do with packages. EagleFiler never automatically updates checksums for files edited using another application. There’s no way for it to know whether the file is different because of a change you saved or because it became damaged.

Files edited within EagleFiler (including packages) do have their checksums automatically updated.

Okay, I was under the impression that individual text files were automatically updated; perhaps I was just confusing this with internal edits in my mind. It makes more sense that they are not. Would a chain of program events solve this?

  1. Edit in ___ initiated from within EagleFiler
  2. EagleFiler opens a thread which periodically checks the file for modification date changes.
  3. Updates checksum whenever this occurs
  4. Shuts down thread after an activity lapse (time from now to last mod greater than safety threshold).

You’d still miss some cases where the user has left a file open for days and then edits it again, but most external editing scenarios would result in a synchronised checksum value.

I’m thinking of applications like FTP clients which allow the user to open a file off of the server for edit. They download the file into a temp folder and then whenever that file is updated it is automatically re-uploaded to the server. On the other hand, I think I’ve seen some applications track when a file is closed by another application. I’m not sure how this is done, or if it can be anymore, but that would be a more reliable way of closing the check thread.

Yes, that’s a possibility for a future version that would probably work well for the common cases, at a slight risk of undetected corruption.

I think the FTP case is different, though. There’s an Apple-event protocol for the editor to notify the FTP client that it’s saved the file.

I don’t think checking for closed files is reliable on Mac OS X since most files are not actually edited but rather replaced.

Yes, quite slight I think. The only window of risk would be the time-out period after the user has closed the file or stopped editing it. While the file is actively being edited and saved, as you put, any potential in-place corruption will be overwritten by whatever is in the RAM when the user saves (with most applications; Scrivener bundle files do have a different state when opened vs closed, for example). I think the reduction of false-positive checksum error messages will actually be of more benefit than the slight risk described. Currently, at least for anyone that does all or most of their editing in external applications, the verification features are difficult to use due to the swarms of false-positives. Eliminate those which might arise from in-place editing, and this would help illuminate real issues with the archive.

Speaking of Bundles (and a bit off topic): Does EF bother itself with the internal components? Are all of those being tracked in the internal database, or does it just consider the parent bundle file to be a singular thing? I’ve been keeping working Scrivener projects in my EF library, which can generate a large quantity of sub-files which are updated and changed at a fairly rapid pace (two second auto-save). If that is not a good practice, perhaps I should just saved finalised and zipped versions.

Yes, that’s a good point.

The contents of packages are considered when calculating the checksum, but they’re otherwise ignored by EagleFiler.

Excellent, that sounds like an ideal solution.