source: WAeUP_Doc/ImportNew.txt @ 3188

==================================
Import of massdata into the portal
==================================
General procedure
----------------------------------

A record is read from the input-source-file and the data is checked. If the
record can be imported, it is **appended** to the file XXX_imported.csv,
otherwise it is **inserted** into the file XXX_pending.csv, if no record
with the exact same date already exists in the XXX_pending.csv. The check is
done by comparing the md5-digests of the new record with the existing records.

When doing the import in pending-only-mode the XXX_pending.csv is taken as
input-source-file and if a record can now be imported it is appended to the
XXX_imported.csv file and removed from XXX_pending.csv.

**Caveat**:

  if an record is inserted into XXX_pending, it will only be removed by
  importing in pending-only-mode. If a new input-file contains a record for
  the same object, with different data, that will not be detected at the
  moment.

The XXX_imported.csv and XXX_pending file has these columns added:

:imported_by:

 the member who did the import
 
:import_mode:

 edit, create, delete
 
:import_date:

 the date and time of the import
 
:imported_from:

 the filename of the input-source-file
 
:error:

 the error message (empty in XXX_imported)