In many cases, if you decide to use LDAP to interact with users, groups, and application data, that decision may be significantly influenced by an LDAP environment that already exists for other purposes. But if you're involved in the process of choosing the data repository, there are a number of compelling reasons to choose LDAP over an old standby like SQL or one of the newer NoSQL repositories.
Lots of people make fun of the use of “lightweight” to describe LDAP, but they usually go something like “LDAP is lightweight like SNMP is simple”. And while this isn't the appropriate place to get into a discussion about whether SNMP is simple (or good for network management, for that matter), it's hard to understand the basis for this joke because LDAP is extremely lightweight. Perhaps the intent is to equate lightweight with simple, and they're saying that LDAP isn't all that simple, but that's debatable that as well.
Even though the “lightweight” moniker is primarily intended as a comparison to its predecessor, the X.500 Directory Access Protocol, LDAP really is lightweight compared with just about any other TCP-based protocol. The binary ASN.1 syntax that is used to encode LDAP messages is both very compact and very efficient to parse, especially when compared with text-based protocols like HTTP where the data being transferred is often further bloated by being formatted in XML or JSON (and in fact the term “fast web services” refers to replacing XML with ASN.1 as a means of both compacting the data and improving encoding and decoding efficiency). LDAP clients can use persistent connections in order to eliminate the need to create a new connection for each request, and the server can maintain state information for the connection so that it's not necessary to repeat things (e.g., authentication credentials, session cookies, etc.) in each request.
Further, if you're security conscious and want to encrypt all of your network traffic, it's worth pointing out that the most expensive part of SSL/TLS communication occurs in the asymmetric encryption that's done during the initial handshake. The symmetric encryption that is used to protect the data being transferred after the handshake is completed is much cheaper, so using long-lived persistent connections dramatically reduces the performance overhead of encrypting everything.
LDAP is a well-defined public standard, with the core of the protocol defined in RFCs, and numerous other RFCs and IETF drafts describing other capabilities. Anyone can read the specs and create their own client and/or server (and in fact, if you want to really learn the ins and outs of the protocol, that's a great way to do it), and any standards-compliant client should be able to interact with any standards-compliant server. This means that there are several good LDAP directory servers out there, and there are LDAP client libraries out there for just about whatever programming language you want to use.
While SQL does have a standardized textual representation, this really isn't the same as having a standard wire protocol because different relational databases have different mechanisms for encoding their communication with clients, and as such if you want to be able to develop your application to support different types of relational databases, you will either include multiple database driver libraries with your application or require whoever is installing the application to provide their own driver (and hopefully the database vendor provides a library that's compatible with the language used to write the application). And despite the standardized textual representation, not all database systems interpret SQL the same way (e.g., the sizes of different data types), so it's often necessary to customize the SQL for each type of database.
The situation is even more complicated with the newer NoSQL databases because each basically has its own API and its own way of doing things. There are some very interesting NoSQL technologies, but they come at the cost of locking yourself into that particular database software. If you want to change what database you're using after the fact, it can be a significant investment to alter your application. There are some persistence frameworks that may be able to help with this by hiding the low-level interaction with the database behind an abstraction layer, but these frameworks often suffer from the “lowest common denominator” problem that prevent you from leveraging some of the most interesting and unique features of the database you've selected.
LDAP has been around since the early 1990s and is a simplified subset of X.500 which has been around since the 1980s. LDAP is a staple of enterprise IT infrastructures for authenticating and storing data about users, and it's also successfully made the transition to Internet-scale platforms. LDAP has shown itself to be quite versatile, and its deployment runs the gamut of business types, including heavy use in areas like telecommunications, finance, manufacturing, retail, education, and government. Even though LDAP is a relatively simple protocol in terms of the number of operations it provides, you don't need to worry about finding yourself out in the cold because it's missing some vital capability.
Unfortunately, the fact that LDAP has been around for a while can sometimes hurt its image because some people may have outdated impressions. For example, many years ago, LDAP servers primarily offered single-master replication and weren't always adept at handling heavy write loads. Despite having nothing to do with the protocol itself, this led to the impression that LDAP was primarily for read-heavy workloads and that applications that needed to perform a lot of writes should look elsewhere. As such, if you're considering ruling out LDAP because of experiences that you or others have had in the past, it would be wise to check out the current situation to determine whether those problems still exist.
Information security is a very big deal, and it needs to be done right. All too often, whenever we hear that a company has been hacked and user data stolen, we find that they didn't take adequate steps to protect sensitive data. That's why it's so surprising that most kinds of databases don't really offer much in the way of security features, whereas most directory servers offer quite a bit in the way of security functionality. Virtually all kinds of network databases support some kind of secure communication mechanism (often via SSL/TLS), but there are a number of security features that are pretty much standard across LDAP directory servers that are practically nonexistent in other kinds of data repositories:
Support for automatically encoding passwords with one-way digests or encryption so that they are stored in a way that makes it easy to authenticate users while making it very difficult to determine what their clear-text credentials really are.
Support for extensible authentication via the SASL framework. This provides for a range of authentication options that include non-password-based mechanisms like certificates and Kerberos tickets, and some servers provide a number of options for multi-factor authentication.
Support for numerous password policy features like password expiration, password quality validation, and account lockout from too many failed attempts.
Fine-grained access control that can impose restrictions on the data that is available to various classes of users, and the expectation that data access is often going to be performed as an end user rather than via an administrative account.
LDAP servers may have a lot of interesting features that make them well-suited to a wide range of applications, but that doesn't mean they lack a lot of the important features that are available in other kinds of repositories. Some of those features include:
General-purpose data storage. LDAP servers are often used to hold information about users and groups (primarily because they're so good at performing authentication), but they're equally well-suited to holding information about other things as well. LDAP servers have extensible schemas that allow you to define whatever kinds of objects you want.
Scalability. A data repository isn't any good if it can't cope with the volume of data or amount of load that your applications demand. Many modern LDAP servers can scale vertically to take advantage of systems with tons of memory and a lot of CPUs, and horizontally to spread the load across multiple systems.
High availability. Redundancy is a very good thing. All LDAP servers of any consequence support replication, and most support some form of multimaster replication that allow writes to any node in the environment. A number of vendors also offer LDAP proxy servers, which can help further improve performance by balancing traffic across multiple servers (while also preferring local servers to those in other data centers), and can improve availability by automatically routing around servers that become unavailable or start to misbehave. Further, some LDAP client libraries (like the UnboundID LDAP SDK for Java) also provide load balancing and failover capabilities.
Disaster Recovery. No software is perfect, and neither is the hardware that it runs on. Redundancy is good for hiding failures from end users, but backup and restore capabilities are essential for recovering from them. Not only do all LDAP servers offer some kind of backup and restore mechanism, but they also support exporting data in the standard LDIF format that can be read by not only other LDAP servers (including those from other vendors), but also by a wide range of client software.