DFArc15 is a archiver and extractor for the .dmod format. It "morphed"
in DFArc2, v1.08's default frontend for launching Dink and DMods, that
uses the extractor code for installing DMods, but does not provide the
.dmod generator.


The code
========

Andrew Reading's version (DFArc15) is portable, but buggy and not
tested under GNU/Linux. It can be compiled with a few fixes available
from the FreeDink repository. DFArc15 is using the portable FLTK GUI
toolkit.

DFArc2 switched to the portable wxWidgets GUI toolkit, and kept
DFArc15's untested non-Woe code as-is. It also uses new Win32-specific
constructs (eg in DFArcFrame.cpp), so it will need a bit a clean-up to
be portable. I tried compiling DFArc2 with 'g++ *.cpp `wx-config
--cxxflags`' - it triggers lots of "ambiguous overload for
'operator='" errors and uses the Win32 API unconditionaly; this needs
a deeper look. It may be ran under Wine.


The copyright
=============

You can find a DFArc15 release at
http://www.dinknetwork.com/file/dfarc/. It includes a source archive
with a copyright+license notices block based on the Berkeley DB
Database license (which, funnily, mentions "THIS SOFTWARE IS PROVIDED
BY SLEEPYCAT SOFTWARE"). Interestingly those notices were all removed
in DFArc2. Before working on the code, we need to make sure this isn't
some license violation (probably a legal mistake, Andrew Reading's is
properly credited in DFArc2 anyway).


.dmod format
============

DFArc introduces .dmod archives, which are similar to .tar.bz2
tarballs. I say "similar" because, unfortunately, the TAR code seems
written from scratch (it does not use a library) and produces archives
that are not completely compatible with POSIX Tar, so extracting those
archives with anything else than DFArc is likely to corrupt files (at
least DFArc15 is affected). BZip2 support, on the other hand, uses the
libbz2 2.1.0.3 (that's what is mentioned in the VC++ project file
include paths) and looks clean.

For example:
# Make sure bzip2 is not involved in the corruption:
$ bunzip2 -t marco_polo-v1_00.dmod
# <no error>
# Extract the tar archive:
$ LANG=C tar tjf marco_polo-v1_00.dmod
marco/DEBUG.TXT
tar: Skipping to next header
marco/DINK.DAT
[...]
marco/library.dat
tar: Skipping to next header
marco/MAP.DAT
[...]
marco/TILES/Ts41.bmp
tar: Unexpected EOF in archive

Those errors are not harmless:
# Using GNU Tar:
$ ls -l marco/TILES/Ts41.bmp
-rwxr-xr-x 1 me me 160768 2007-08-22 21:50 marco/TILES/Ts41.bmp
# Using DFArc2:
$ ls -l .wine/drive_c/Program\ Files/Dink\ Smallwood/marco/TILES/Ts41.bmp
-rw-r--r-- 1 me me 161080 2007-08-22 22:26 .wine/drive_c/Program Files/Dink Smallwood/marco/TILES/Ts41.bmp

So not using DFArc for extraction may truncate the files. In
particular, I never saw a .dmod that does not trigger a "Unexpected
EOF" error with GNU tar.


With this truncation, Ts41.bmp is still readable by GQView (for
example). However, SDL_LoadBMP is stricter and will refuse to load the
file. Since Ts41.bmp is necessary for the Dink engine to run, the game
engine won't start.


At first I feared that already released .dmod files would contain
corrupted files, so those files may be irremediably lost. Since
apparently DFArc's extrator can handle its encoder's mistakes, we can
contemplate fixing .dmods to make them back valid Tar+BZip2 files
without data loss.


The DFArc extractor also have troubles with extracting files it
generated. However it seems more foregiving than GNU Tar. When
properly fixed to work under GNU/Linux, DFArc15 will produce weird
'\377' one-letter files each time GNU Tar would warn about "Skipping
to next header".


Technical sum-up of the incompatibilities:

- DFArc defines the ustar version as "\0\0" instead of "00".

  (harmless, ignored by GNU Tar).

  Note that DFArc2 uses GNU-style "ustar  ".

- DFArc does not create entries for directories - only for files. This
  behavior can be reproduced with GNU Tar using:

  $ find marco -type f | xargs tar --format=ustar -b1 -cf dmod.tar

  (harmless, but directories can't have permissions).

- DFArc lacks the end-of-archive entry, its archives end straight
  after the last file, while
  http://www.gnu.org/software/tar/manual/tar.html#Standard
  mentions: "Physically, an archive consists of a series of file
  entries terminated by an end-of-archive entry, which consists of two
  512 blocks of zero bytes.".

  This makes GNU Tar exit, and hence truncate the last file.

  This can be fixed using:

  $ dd if=/dev/zero bs=512 count=2 >> dmod.tar

- DFArc adds a 512 block as content for empty files, while there
  should be no such block.

  GNU Tar skips the extra block with a "Skipping to next header"
  warning, and DFArc creates 'ÿ' files.

This should be fixable in existing archives, at first glance.



DFArc2 has a "Package" button that is not enabled by default. It
contains improved source code that fixes these issues.

The v1.08 RC5 indeed mentions:

* DFArc 2: When installing a D-Mod containing a file with a size of 0
  KB, DFArc 2 would get rather confused. It would create a file or
  folder called 'ÿ', and stop installing the D-Mod (such as with
  several D-Mods like Chaos, Bane of the Magi, and Mayhem). This issue
  has been fixed.

And v1.08 RC6:

* DFArc2: Properly fixed bugs related to D-Mods not extracting
  properly if they contained files without any content.


Incidentally RC5 also mentions:

* DFArc 2: When installing a D-Mod, DFArc would report a false error
  if the D-Mod had a folder with no files (just other folders) in it
  (such as in Terrania). It no longer reports an error.

While there's no note about it, DFArc2 also properly add the 'end of
archive' entry.
