What Is an LDIF File? The Text File That Lists a Directory and Its Password Hashes

7 How Do You Open an LDIF File? An editor, a server, an address book or a viewer

LDIF files are text, so an editor opens them. We tested ldapadd and the Python module on Linux. We read the pages of the tools below and installed none of them.

In a text editor

Any editor works. Watch three settings before you save. The encoding should be UTF-8, the byte order mark should be off and a blank line should be empty. In our copies a blank line with one space joined two entries (copy 21), and trailing spaces made ldapadd read no entry (copy 9).

One trap lives outside the editor. Microsoft says that Windows PowerShell 5.1 writes UTF-16LE when you redirect output with >. Both of our readers refused UTF-16 files (copies 5 and 6), so save the output as UTF-8.

In a directory server

Use ldapadd for entry files and ldapmodify for change files. The manual page for ldapadd says ldapadd is ldapmodify with the add flag on. Add -c to continue after an error, and read every message.

ldapadd -x -H ldap://localhost -D "cn=admin,dc=example,dc=com" -W -c -f directory.ldif
ldapmodify -x -H ldap://localhost -D "cn=admin,dc=example,dc=com" -W -f changes.ldif
ldapsearch -x -H ldap://localhost -b "dc=example,dc=com" -LLL "(objectClass=*)" > export.ldif
slapcat -l backup.ldif

ldapsearch -LLL writes entries as LDIF and leaves out the comments. slapcat reads the database files on the server itself and adds seven lines to every entry. Both wrote entry files without a version line in our test.

In address books and LDAP browsers

FileInfo on supported programs names Mozilla Thunderbird, Apple Contacts, Apple Directory Access, Active Directory Explorer, Apache Directory and two LDAP browsers as programs that open the format. Wikipedia on address books adds that Netscape Communicator and Mozilla used LDIF for address book import and export. We tested none of these.

Two Windows viewers

These cards link to two Windows programs that name LDIF files on their own pages. Both show contacts from address books. We read the pages and used neither program.

The SysTools page describes a free viewer for LDIF files from Mozilla and Netscape address books. It lists contacts and their attributes, sorts by nickname or email and has a find box. The page says it views any number of contacts and needs no other program.

The Freeviewer page describes a viewer for Thunderbird contacts. It shows every attribute, offers horizontal and vertical views and sorts by name, company or phone. The free version previews only, and the page lists the full version at $29. It needs .NET Framework 4.5 or later.

Neither page says how its tool treats damaged files or files from servers instead of address books. We did not use either tool on our 55 damaged LDIF files.

On the command line

Four commands show what an LDIF file is without importing it. They hash the LDIF file, print its first lines, name its encoding and start our 102 line script. The first three work on any Linux or macOS computer.

sha256sum directory.ldif
head -n 5 directory.ldif
file directory.ldif
python3 ldif_peek.py directory.ldif

The file command printed ASCII text for our intact LDIF file. That shows the encoding and nothing about the entries. The first lines show more, because version: 1 or a dn: line points to an entry file.

How the checks did on our 55 damaged LDIF files

8 How Do You Check an LDIF File? Six steps, and a 102 line script

The six steps

1. 1 Copy the LDIF file, write down its SHA256 hash and work on the copy.
2. 2 Read the first lines with head -n 5 file.ldif. An entry file starts with version: 1 or a dn: line. changetype lines mark change files.
3. 3 Name the encoding with file file.ldif. You want ASCII text or UTF-8 text. Byte order marks, UTF-16 and CR line endings need fixing first.
4. 4 Use the script below. It reads the structure and counts the entries.
5. 5 Read the last lines of its output. Every problem gets its own line, and the count of entries comes last.
6. 6 Load the copy into a test server with ldapadd -c, then compare the count and some entries with the LDIF file. Never test on the live directory.

A 102 line script that reads an LDIF file

The Python script below reads an LDIF file and does steps 3 to 5 in under a second. It checks the encoding, the line endings, the folds, every dn line, every name and value, every base64 value and the version line. It changes no file.

import re, sys, base64, binascii

NAME = re.compile(r'([A-Za-z][A-Za-z0-9;._-]*)(::|:<|:)([ \t]*)(.*)$')


def check(path):
    raw = open(path, 'rb').read()
    out = []
    if raw[:3] == b'\xef\xbb\xbf':
        out.append('byte order mark at the start')
        raw = raw[3:]
    if raw[:2] in (b'\xff\xfe', b'\xfe\xff') or b'\x00' in raw:
        return ['UTF-16 or NUL bytes: the file is not UTF-8 text']
    try:
        text = raw.decode('utf-8')
    except UnicodeDecodeError as e:
        return ['not valid UTF-8 near byte %d' % e.start]
    crlf, cr = text.count('\r\n'), text.count('\r') - text.count('\r\n')
    if cr:
        out.append('bare CR line endings (%d)' % cr)
    if crlf and crlf != text.count('\n'):
        out.append('mixed CRLF and LF line endings')
    if not text.endswith('\n'):
        out.append('no final line break (the file looks cut)')
    lines = text.replace('\r\n', '\n').replace('\r', '\n').split('\n')
    logical = []
    for i, l in enumerate(lines, 1):
        if l.strip() == '' and l != '':
            out.append('line %d looks blank but has spaces or tabs' % i)
        if l[:1] == '\t':
            out.append('line %d starts with a tab, not a space' % i)
        if l[:1] in (' ', '\t') and logical and logical[-1][1] != '':
            logical[-1] = (logical[-1][0], logical[-1][1] + l[1:])
        else:
            logical.append((i, l))
    recs, cur = [], []
    for i, l in logical:
        if l.startswith('#'):
            continue
        if l.strip() == '':
            recs += [cur] if cur else []
            cur = []
        else:
            cur.append((i, l))
    recs += [cur] if cur else []
    if recs and recs[0][0][1].lower().startswith('version:'):
        if recs[0][0][1].split(':', 1)[1].strip() != '1':
            out.append('version is not 1, and RFC 2849 allows only 1')
        recs[0] = recs[0][1:]
        recs = [r for r in recs if r]
    dns = []
    for rec in recs:
        n, first = rec[0]
        if not re.match(r'(?i)dn::?[ \t]', first):
            out.append('line %d: record does not start with dn' % n)
            continue
        dn = first.split(':', 1)[1].lstrip(': \t').lower()
        if dn in [d for _, d in dns]:
            out.append('line %d: dn appears twice' % n)
        dns.append((n, dn))
        names = set()
        for i, l in rec[1:]:
            m = NAME.match(l)
            if re.match(r'(?i)dn:', l):
                out.append('line %d: second dn inside one record (blank line missing?)' % i)
            if not m:
                out.append('line %d: not "name: value"' % i)
                continue
            name, kind, _, val = m.groups()
            names.add(name.lower())
            if name.lower() == 'version':
                out.append('line %d: version line is not at the start' % i)
            if kind == '::':
                try:
                    if not val.strip():
                        out.append('line %d: empty base64 value (%s)' % (i, name))
                    else:
                        base64.b64decode(val, validate=True)
                        assert len(val) % 4 == 0
                except (binascii.Error, AssertionError):
                    out.append('line %d: base64 value is not valid (%s)' % (i, name))
            elif kind == ':' and val[:1] in (':', '<'):
                out.append('line %d: value starts with : or < and needs base64 (%s)' % (i, name))
            elif kind == ':' and val != val.rstrip():
                out.append('line %d: value ends with a space (%s)' % (i, name))
        if 'objectclass' not in names:
            out.append('line %d: entry has no objectClass' % n)
    for k, (n, dn) in enumerate(dns):
        if ',' in dn and dn.split(',', 1)[1] in [d for _, d in dns[k + 1:]]:
            out.append('line %d: parent entry comes later in the file' % n)
    return out + ['%d entries found' % len(recs)]


if __name__ == '__main__':
    for path in sys.argv[1:]:
        print(path)
        groups = {}
        for msg in check(path):
            kind = re.sub(r' \(\w+\)$', '', re.sub(r'^line \d+:? ', '', msg))
            groups.setdefault(kind, []).append(msg)
        for g in groups.values():
            print('  ' + g[0] + ('  (and %d more like it)' % (len(g) - 1) if len(g) > 1 else ''))

On our intact LDIF file it finds 14 entries and no problem. On damaged LDIF files it names the line. It finds the missing blank line in copy 19 and the cut password in copy 38.

python3 ldif_peek.py directory.ldif
directory.ldif
  14 entries found

python3 ldif_peek.py copy03.ldif
copy03.ldif
  bare CR line endings (200)
  no final line break (the file looks cut)
  14 entries found

python3 ldif_peek.py copy14.ldif
copy14.ldif
  line 28: base64 value is not valid (userPassword)  (and 11 more like it)
  14 entries found

python3 ldif_peek.py copy19.ldif
copy19.ldif
  line 77: second dn inside one record (blank line missing?)
  13 entries found

python3 ldif_peek.py copy38.ldif
copy38.ldif
  no final line break (the file looks cut)
  line 152: base64 value is not valid (userPassword)
  11 entries found

What the script cannot see

The script reads text and not meaning. It printed nothing for copy 28, where four characters are cut off a password hash. It printed nothing for copy 31 either, where one letter changed. It also cannot know which attributes your directory allows, so it missed the misspelled name in copy 42.

python3 ldif_peek.py copy28.ldif
copy28.ldif
  14 entries found

The script never executes an LDIF file. It checks that base64 values decode and does not read what they say. We wrote it from RFC 2849 and from our 55 damaged LDIF files, so it knows the faults we tested and no others.

Want the causes of an error? Go to the causes.

9 Is an LDIF File Safe to Share? It can store password hashes and file URLs

Opening an LDIF file in an editor is not the risk. The risk is what the LDIF file contains and who gets a copy. We scanned no file for malware, so we cannot say what a downloaded LDIF file contains. We can say what LDIF files can store.

What an LDIF file can contain

The file URL trap

RFC 2849 on file URLs lets values point to files with :< and a URL. The reader then includes the LDIF file. Its security section warns about LDIF files from untrusted sources for this reason.

We tested it. We saved a one line file in /tmp and loaded an entry whose description pointed at it. ldapadd read the LDIF file and stored its text in the description. A command like grep -n ":<" file.ldif finds such lines before you import.

Base64 is not encryption

Every base64 value in our LDIF file decoded in one line. Treat password hashes like passwords. Remove the userPassword lines before you send an LDIF file for support, and keep the original.

Before you share or import an LDIF file

1. 1 Ask who needs it. A support request rarely needs password lines.
2. 2 Remove the userPassword lines and any other secret, and keep a copy of the original.
3. 3 Hash the LDIF file and send the hash on another channel, so the other side can check it.
4. 4 Search for :< lines before you import files from strangers.
5. 5 Import into a test server first, and compare the entry count with our script.

11 How Do You Export, Import or Convert an LDIF File? What we tested in OpenLDAP and what the vendors say

This part reports what we tested in OpenLDAP 2.6.10 and what the vendors say about the rest. We read the pages named in every paragraph and used no Windows tool.

Export from OpenLDAP

ldapsearch -LLL writes entries as LDIF without comments. slapcat reads the database files on the server and wrote 298 lines for our 14 entries, because it adds seven operational lines to every entry. Neither wrote a version line.

Import into OpenLDAP

ldapadd loads entry files and takes the version: 1 line. slapadd writes into the database files, and 2.6.10 refused our LDIF file at line 1 with the message that the attribute type version is undefined. It loaded all 14 entries after we deleted that line.

Active Directory and ldifde

Microsoft’s ldifde page describes a command line tool that creates, modifies and deletes directory objects in Active Directory. The flag -i selects import and -f names the LDIF file. The flag -m leaves out attributes that only Active Directory objects have, like objectGUID.

Wikipedia on modify records adds that ldifde needs a trailing hyphen line in modify records. We tested none of this, because we had no Active Directory.

Convert an LDIF file to another format

A converter converts the entries into rows or cards. The card below links to the one Windows converter that names LDIF files on its own page. We read the page and did not use the program.

The SysTools page lists CSV, VCF, PDF and PST as outputs. It says the tool also reads LDF and LDAP files made by an LDAP server. It needs Windows 7 or later and .NET Framework 4.5. The page lists no file size limit, and it offers a 30 day money back refund.

The page gives two demo limits. One passage says the free version converts up to 15 items per file, and the comparison table says something else. Test the demo on your own file before you buy.