In part 1 of this blog series, we talked about centralized authentication in Linux and covered some basic use cases. In today’s post, we want to talk about System Security Services Daemon (SSSD) and why it’s becoming so important to centralized authentication in the Linux space. But first, we will touch upon Pluggable Authentication Module (PAM) Lightweight Directory Access Protocol (LDAP).
A Quick Primer on PAM LDAP
In the younger days of Linux, attempting centralized authentication of hosts could be an incredibly cumbersome affair — particularly when attempting to authenticate to a system that wasn’t a strict LDAP v3 implementation (e.g. Active Directory). At the most basic level, you could directly use PAM LDAP to perform authentication and authorization as we discussed in part 1. However, PAM LDAP itself has some fairly big issues that can be problematic:
Supports only LDAP:
If your organization absolutely wants to use Kerberos (KRB5)-based authentication, simply using LDAP won’t help. Of course, KRB5 has its own pros and cons (which are outside of the scope of this blog, but when properly implemented KRB5 can be a useful tool in an SSO strategy).
Failover is a pain:
While it is true that multiple servers can be defined in PAM LDAP configurations in case one goes down, failover options are crude. Each system entered for failover is restricted to a “bind time limit” which is configured and tested at each login, which is by default 30 seconds. What this means to us is that in an improperly configured environment, or an environment that has seen catastrophic failure, logins may become impossible due to this time limit being improperly scoped despite having active servers. Proper load balancing mechanisms can help mitigate this, but it is still a significant issue to contend with.
Doesn’t handle system outages gracefully:
In the event that all centralized authentication hosts are down, PAM LDAP will fail to allow a user in — it has no concept of creating tokens based upon successful authentication.
Sudo implementation can get hairy:
When using PAM LDAP there are two ways to get a user to be a sudoer. The first way (and the one you’re most likely to find on a web search) involves the use of a sudo-specific schema for LDAP, more information for which can be found here.
So the key takeaway is this — PAM LDAP doesn’t scale well, it doesn’t directly integrate with Active Directory (Which has become the de facto LDAP directory in many enterprise environments), and there is no real way to integrate SSO on its own.
To be fair to PAM LDAP, there are tools that integrate within the stack (Samba, Winbind, PAM KRB5, and so on) that can extend functionality. But this is where the Unix/Linux credo of “do one thing and do it well” becomes problematic — the workload of configuring, testing, tuning, and deploying all of these additional tools (which do their respective jobs well) can result in an untenable pace for system administrators and developers alike. Not to mention that with each application added, there is an increased chance that one link in the chain may be exploitable and permit an attacker entry. Then, additional time and energy must be placed into patch management and mitigation of potential weaknesses in the enterprise environment.
The sum of solutions leads to a great deal of complexity, and if that complexity can be reduced, it will make systems easier to understand, administer, and manage at scale. Essential complexity is key — the simpler we can make a solution that meets a given use case, the better.
What Does it Look Like?
We’ll begin by going back to our initial use case around centralized authentication:
And narrow our focus to the guts of step 2 as it might be implemented in PAM LDAP. The following diagram is a simple overview of how the components work, you can find a more in-depth analysis of the various code and configuration elements here.
As you can see, the flow here shows PAM being controlled by a series of configuration files on the system. pam_ldap.so, one of many PAM libraries has additional configuration files. When properly configured, PAM (using a set of shared libraries focused on LDAP, via pam_ldap.so) will reach out to the provided LDAP server. The LDAP server takes the data presented and returns a result (good or bad) to the client. The client, using that information, relays it back through PAM to provide an appropriate system response.
This flow, while we have modeled it for usage with an LDAP server, is the same general flow for other PAM modules as well — client performs an action based on prior configuration, action is assessed either locally or remotely, and produces an accept or deny response based on the assessment.
So What About SSSD?
SSSD seeks to streamline the tangled mess of applications and configurations outlined above. Understandably, it’s a bit more complex than a single application, so let’s take the time to step through what it looks like.
SSSD is a bit more expansive in how it deals with the use case of providing authentication and authorization. The following diagram is an overview of how it all comes together (for a more in-depth discussion on the architecture of SSSD, Jakub Hrozek wrote an article on the topic available here). Additionally, SUSECon 2015 had a session on SSSD, slides for that talk (which have architectural diagrams) which can be found here.
While the architecture behind SSSD is expansive (by necessity), it’s not overly complicated. Clients on the host (such as PAM and NSS) interface with SSSD directly through what are called “Responders.” These responders, depending on what is asked for and the state of the data providers, check either the local caches or reach out to the data providers through plugins. For the sake of efficiency, data is typically cached so as to reduce network chatter, and SSSD will check the local cache first (taking into account the cache age) before reaching out to external data providers. These data providers can take on three roles (or a combination of them). They can provide identity, authentication, and authorization. For instance, you might use Kerberos as an authentication provider within SSSD, and rely on LDAP to act as an authorization provider.
The key takeaway here is that SSSD interfaces with PAM and NSS, and depending on what actions those take, SSSD interfaces with the backend to perform identity, authentication, and authorization needs and do so securely. In fact, if we look back at the issues we had with PAM LDAP, we see that SSSD:
~ increases the scope of what we can use for authentication (removing the need for, say, Samba/Winbind to be added on top for KRB5)
~ offers better failover configuration if you read the man file to compare, (check the section on “Failover” near the bottom for more information)
~ offers a graceful means of resolving outage crises
We’ll talk about the sudoer issue in another the next installment of the series.
Configuring and Managing SSSD
As with any software implementation, there is no prescriptive “magic bullet” for all configurations and use cases. However, there are many good sources for moving forward with an SSSD implementation, and if you are unfamiliar with the service, you will want to spend a fair amount of time reading and testing in a development environment before moving forward. To that end, we have put together a condensed “Reading List” of resources at the bottom of this article that you may find helpful moving forward with SSSD. Something to keep in mind — SSSD is very, very flexible if used properly, and can work in environments with custom objectClasses and custom schema if sufficient attention and time are put toward its configuration.
A common problem with any centralized authentication solution is getting the host to talk to the authentication server. In an environment with a small footprint of Linux servers, this can be done manually. The problem with manual, of course, is that with each system that is spun up there comes an increased chance of errors and omissions. Scriptwork is helpful in this regard, but it can be difficult to account for version drift as well as to implement idempotence in scripts. It’s highly recommended that with any centralized authentication setup for Linux (not just SSSD) a configuration management/deployment/continuous delivery tool (Chef, Ansible, Saltstack, etc.) be used in conjunction to ensure configurations are tested, sensible, idempotent, and so on. There are a number of freely available implementations to install and configure SSSD for these DevOps tools. If they do not meet the needs of your organization outright, they can be food for thought in creating your own configurations.
In the final installment blog in this series, we will be discussing a major issue that arises from managing users centrally — defining who can run elevated functions on a given host, and who cannot. We’ll detail a few ways of performing this: one using SSSD, and one using some simple enumeration tricks.
A final note, most of the icons used in the above diagrams can be found at the Draw.io Github repository.
|Resource Name||Description||Bitly/Shortened Link|
|Pavel Reichl’s SSSD Blog||An Article on SSSD Cached Credentials, and why cached_auth_timeout is good.||https://bit.ly/2LXg44O|
|RedHat Access Documentation||entry_cache_timeout tuning.||https://red.ht/2OAB4Ah|
|RedHat Access Documentation||Troubleshooting SSSD, common issues, and resolutions.||https://red.ht/2ATxYPG|
|RedHat Access Documentation||Using AuthConfig to generate home directories for new users automatically in RHEL and RHEL derivatives.||https://red.ht/2nr61vn|
|Linuxthrill’s Blog||Configuring SSSD in Redhat, using Authconfig. This can save a lot of time potentially.||https://bit.ly/2VoEojl|
|Jakub Hrozek’s Blog||Group restriction in SSSD, and why we need to limit the scope to group memberships.||https://bit.ly/31WM3HR|
|Sven’s Blog||Potential integration path for SSSD with groupOfUniqueNames||https://bit.ly/35iw09i|
|SSSD Team||How to troubleshoot SSSD issues, straight from the developers.||https://bit.ly/33go86H|
|Rootusers Blog||Clearing the SSSD cache, the easy and hard way.||https://bit.ly/2OumyKj|
|Oracle A-Team Blog||Part 1 of a 4 part series on SSSD Configuration, while geared to Oracle Linux lessons can be applied to every distro.||https://bit.ly/35gJgvd|
|Oracle A-Team Blog||Part 2 of a 4 part series on SSSD Configuration, while geared to Oracle Linux lessons can be applied to every distro.||https://bit.ly/35frM2q|
|Oracle A-Team Blog||Part 3 of a 4 part series on SSSD Configuration, while geared to Oracle Linux lessons can be applied to every distro.||https://bit.ly/2Ix8Mm5|
|Oracle A-Team Blog||Part 4 of a 4 part series on SSSD Configuration, while geared to Oracle Linux lessons can be applied to every distro.||https://bit.ly/33c4vwj|