Auth Series 1: Sessions and Passwords

13 Nov 2024 by cthos
5016 words

Hello and welcome to the first part of a multipart post series where I'll dig through the depths of authentication knowledge and share all of that with you, including some deep lore of how we got from the early days of the internet to today (though, I'm a millennial, so I wasn't exactly a web professional back then. Neopets all the way).

Part 1 will cover the following:

1. How do web servers identify requests?
2. Session storage and state management
3. Passwords and how to store them as securely as possible
4. Handling forgotten passwords
5. Signing out and Account removal considerations

I've also created a repository on Gitlab with a couple of examples for session state: https://gitlab.com/cthonic-studios/authentication-examples

Now, there are going to be a lot of caveats to this topic - authentication is a deep topic and has a lot of gotchas. I'll cover some of them, but I'd need a full book to cover everything. That said, let's get right in!

This post assumes some familiarity with web servers and web technologies. HTTP, cookies, methods of data storage, the concept of multiple web servers working in coordination. If you need some grounding in web servers, Digital Ocean has a good guide.

So the very first thing you need to consider when dealing with web authentication is that web servers are stateless (more specifically, the HTTP protocol is stateless). Whenever your browser makes a request to a website, the server has no idea that one request is connected to another. There will be some telemetry, like your IP address or the browser you're using, but as far as the web server is concerned, each request is totally independent of any other request.

This is a problem if you want the server to understand a given series of requests are actually related to a given account. Without some unique identifier to pass along to the server, you can't associate a given page load with a given account, which makes persistent web applications kinda difficult.

To deal with this, we have the concept of a browser session. The very basic premise is that the browser will store some bit of information and send it along with every HTTP request that the server can then use to uniquely identify the "session" in question. There are several ways that one could do this so we're going to talk a bit about some of those techniques first and then talk about best practices.

I'm going to use PHP's session management as an example because it has a lot of built-in functions for managing session state, and they're quite illustrative of how it works under the hood. They even have a best practices page for proper session storage.

So, to start, PHP has a function called session_start() that will initiate a session for a given request, which allows you to store parameters in the $_SESSION global which will persist across requests. Just like magic, you've got a storage area where you can store information about the session in question. But how does this magic function work?

The very first thing PHP does when starting a session is generate a random session_id() for the session, and sends that back down to the browser in the response. How it sends it depends on the configuration. By default, it sets a cookie via the Set-Cookie response header, with the name of PHPSESSID. There are a number of configuration values for this, allowing you to disable session cookies, control their behavior, change the name of the session variable, and so on.

The generated session ID was a derived ID based on a number of factors until PHP 7.1, but now it's generated using random_bytes. Either way it results in an alphanumeric ID which isn't guessable in a reasonable amount of time given current computing power.

We'll talk about the cookie thing in a minute, but for the moment let's talk about what happens if you don't send a cookie back. How else might we get that session information? Well, if you disable the cookie setting, and enable session.use_trans_sid, the session information will be transparently appended to every anchor tag on the page so that ?PHPSESSID= will be appended to the target URL, thus preserving your session across requests through the power of $_GET variables. If you'd like to see this in action, I've included some simple examples in the aforementioned code repo.

Now, this isn't great, because that makes your session ID directly visible in the URL. GET params are encrypted over https:// but anyone who wants to look over your shoulder could take a picture of your session ID and then hijack your session simply by using the same ID. This is why basically all modern systems use Cookies to store your session identifier. We'll talk about GET parameters more when we discuss other auth systems (OIDC and SAML both use GET parameters for various things), but for now, just internalize "use Cookies to store session info".

Okay! Now that we know how to start a session and how the server identifies a stateless request, what can we do with that? Well, now you can store information about that user in the session storage. In PHP, you can directly assign things to $_SESSION['example_key'] which will persist across requests. PHP's basic session storage is file-based (meaning it stores your session data on the hard drive of the server you're interacting with), which is generally unsuitable for distributed applications because you'll have to either ensure the user is pinned to the server where their session information is stored, or* you'll need to synchronize the file system across all web servers. Neither of those is ideal, but it's definitely a thing we used to do many years ago.

So, you can also store your session information in some other system, like Memcached, or a database which is what modern systems tend to do. Indeed, if you're working in a language without a built-in session mechanism, this is what you're going to want to do. Let's talk about that a bit more.

Storing session state is a complicated topic and depends a lot on your server topology. If you just have a single web server, storing state on the file system is going to look "fine". If you have multiple web servers handling requests, you'll need your state to be available to them. There are several ways to handle this, and if you want a quick overview of the options Laravel's session storage page has a good series of the typical options. Including:

• Storing data on the file system on a single server
• Storing data on a networked file system (block storage on a cloud provider, typically)
• A database, either the application database or a separate database
• In-memory storage like Redis or Memcached
• Directly in browser cookies, typically signed or encrypted.

The most common options I've seen implemented for session storage are either "just store it in the database" attached to the session identifier, or "store it in memcached/redis" also attached to a session identifier. Over the course of my career I've seen the progression from file → database → memory cache storage several times now, and that's directly correlated to scaling applications. File storage (even on a networked drive) is slower than the db which is slower than memory, so when you're wanting to squeeze more performance out, this is where you tend to go.

The other option which requires very little configuration is an encrypted session cookie. Most frameworks handle this for you because if you put session information in a cookie the client can tamper with it, which is usually something you don't want to have to worry about.

That said, you also generally want to minimize the amount of data you're storing in the session in the first place.

At minimum, for an authenticated user, you'll need to store their user_id or another primary key so you can identify that session to the user in question. We'll talk about that more in a second.

For anonymous sessions, you might do something like store a shopping cart (for e-commerce) or preferences, or any number of other things that you need persistent across page loads.

This can also be things like error or status messages that need to be shown on subsequent page loads, for example if there's an error on a POST request and you need to redirect back to the prior page with a message. You could include that in the URL, but it's commonly stored to session storage instead (because someone else could send you a link with a misleading message via that same URL).

This really depends on your application, but I'd recommend storing as little data in the session as you can get away with.

As the OWASP Session sheet calls out, you have a few different options:

• Keep the session alive indefinitely, until the user takes an affirmative action (logging out)
• Expire the session after a period of time.
• Expire the session after the user has been inactive for a period of time.
• Expire the session when the browser is closed.

Which one you'll choose depends on your application and how sensitive it is. I've encountered a lot of blanket "you must log the user out after 15 minute" corporate requirements, and I need to tell you that is less secure than you might think it is, especially if it isn't an idle timeout.

In general, I recommend:

• Expire the session when the browser tab is closed (expiration time of 0 on the session cookie), but give the user the option to stay logged in across visits. In that case, set the expiration of the cookie to some sensible time in the future (weeks / months).

For more sensitive contexts:

• Expire the session after a sensible idle timeout (yes, 15 minutes works)
• Do some basic anomaly detection (IP addresses changing suddenly, etc), require re-authentication for sensitive operations, etc.

A popular choice for storing ephemeral data on Single Page Apps (SPAs), which can be a good option for data that's tolerant to being modified by the client (meaning, the server cannot inherently trust that data, it has to be verified).

While I don't recommend doing this (use a library if you can), the basics of how sessions work boil down to this:

1. Create a session identifier which is unique and impractical to guess (ideally long and random) or cryptographically signed.
2. Sending that session identifier securely to the browser (over TLS, encrypted, etc.) so that it cannot be intercepted in transit.
3. Inducing the browser to send back that session ID to the server (also securely so it's not intercepted over the wire) ideally via cookies.
4. Storing information associated with that session in some sort of storage that's accessible by all the web servers that could potentially serve the request.
5. Deciding how long you want that session to last and removing the session when it expires.

There's a lot of nuance that I'm eliding over in those posts, but those are the core elements of a session storage mechanism. If there's enough interest we can do a roll-your-own example of session storage, but I'd never encourage you to do this yourself other than to understand the mechanics. Much like cryptography, there are a lot of footguns, and there are many good libraries that handle sessions.

That said, for more detailed information, check out the OWASP Session Cheat Sheet.

For an even deeper dive into proper session storage, NIST's SP-800-63B document is an extremely long technical document that covers a lot of information about authentication.

You should generally have the web server manage the session with a cookie, and unless you have a very good reason to allow JavaScript to read that cookie, you should send it with the HttpOnly flag (which prevents JS from accessing it).

Likewise, you should set SecureOnly so it will only be sent over HTTPS and the SameSite attribute so it will not be sent along to other domains on the same root domain (unless, of course, you want to do that and you know the trade-offs).

Session hijacking is a technique whereby an attacker gains access to the session for a given target user and then uses that to act on their behalf. OWASP covers that in this article, but the basics are "Gain token, use token". There's also a reverse attack where you use an XSS vulnerability to inject your own session token into another user's session and get them to do some target action.

Either way, you want to prevent that session token from leaking. Here are some steps you should take to prevent session hijacking:

• Never send the session identifier over http://, only over https:// connections. If using cookies, ensure SecureOnly is set.
• If you can help it, do not let JavaScript access the session identifier, like with HttpOnly cookies.
• Ensure that your session ID is impractical to guess (length, randomness).
• Change the default session identifier variable for your language (for PHP, don't use PHPSESSID). This'll make it harder for a malicious user to guess or target in bulk.
• Implement Content Security Policies that prevent cross-site requests to prevent JS from sending session information.

_{Song_about_summer @ Shutterstock #1794130912}

Right, so we've now talked at length about how sessions work, but currently we still don't know anything about the person on the other end of the internet other than they initiated a session in a browser. We can reliably identify the browser that started the session is still the browser interacting with that session, But what happens if the human on the other side of the screen needs to change computers? Or they shut down their browser? Or they clear their cookies? How do we go past identifying the browser and identify the user?

Well, that's why usernames exist. Very basically, as a website operator, we're asking you directly to tell us who you are. So, when you provide registration, you'd ask a user who they are, and that most often is a memorable username chosen by the human who's signing up for the service.

But, if all we ask for is a username, then anyone with that username is able to log in as the person in question. Because usernames are typically also displayed publically, this would be very bad.

This is where passwords come in. Passwords are an ancient invention, you might have to utter a password to a guard at a door in ancient Rome to gain access to a place. Modern passwords are directly derived from that idea, you need to give a password that ideally only you know so that we, the service operator, know that you are who you claim to be. Or at least, that you have the requisite knowledge (which is where impersonation and multifactor authentication come in).

Now you want to collect an email address and use it for account recovery or for the username. How do you ensure that this email address is an email address? You can use a regular expression to validate that it's an email, right?

No! Email addresses are...remarkably complicated in how they can be formatted. There is a regular expression you can use and catch most of the cases you'd find in RFC 5322, but you're not going to catch everything. My advice is to use a simple regular expression to ensure the format is in the ballpark (ensuring there's a local and remote part with an @, for example) and then send an email to the provided email with a one-time code in it to verify delivery of the email address. There are also services like Neverbounce that do validation by sending test emails and maintaining giant lists, but you can get by with simply making the user validate their email address.

You should also do this before allowing the user to change their email, and you should put that operation behind a second factor. If you haven't implemented MFA or the user hasn't turned it on, send a code to the existing email first (following the guidelines for password resets, presented later) and then again to the new email address to confirm the change.

So, the combination of username (or email address, commonly) and password are what basic applications use to identify users with the service. Great! We can just collect that and store it directly in the database, right?

Wrong! You may be tempted to store passwords in plain text in your database, how else will you send the password to the user when they forget it? This is a really bad idea. For one, if your database is compromised, you've now given malicious users every single user's password. Humans tend to reuse passwords across services (even though we've been trying to get folks to not do that for the entire history of application passwords), which means you might have compromised a bunch of user's security across any number of sites.

We don't want that. Okay, so maybe we'll encrypt those passwords in the database! Then, if you steal the database, you can't decrypt the passwords! This is also a bad idea, because in order to encrypt the password in the first place, your application server needs to be able to access the encryption key. Which means if your application is compromised, then you can still steal plain text passwords. It might be slightly harder, but it's still possible.

So, what do we do? You hash the passwords, with a one-way algorithm so that the server cannot get the plain text password back out of it. You'll be able to check that what the user has given you matches what you have on file when you run it through the hashing algorithm, but anyone with the database cannot just get that password back out.

In the early days, we'd often just use an algorithm like MD5 without any other mechanisms in place and call it good. As computing power has increased drastically, MD5 hashes can be checked in a short amount of time rendering them insufficient for modern security. Likewise, there are entire databases of hashes called "Rainbow tables" which correlate MD5 hashes to plaintext equivalents.

So, instead, we now have better algorithms and other techniques to slow down the checking of a hash. The most common mechanism I've seen in the wild is still bcrypt, while NIST recommends using PBKDF2 which is specifically for passwords. OWASP recommends using Argon2id. These algorithms increase security in several ways, the first is they each run through a number of iterations (which is configurable) to both increase the time taken to generate the hash and limiting the speed at which you can brute-force it (bcrypt is much better at this than PBKDF2) and to make it harder to do if you do not know how many iterations to use. Also, because of the rainbow tables we mentioned before, each of these algorithms includes a "salt" value, which is unique to the generated hash. This ensures you can't just build a giant database of hashes.

For a brief introduction to password cracking, check out this article by Matt Miller on Beyond Trust. You can also hop on over to TryHackMe's Crack the Hash room to try out password hash cracking for yourself to learn how to defend against it.

_{mapo_japan @ Shutterstock #2490729947}

This is a thing we've been fighting for a long time. To forget things is human. In an ideal world (where passwords have to exist) everyone would be using a password manager, and they would never ever forget their password. And they'd always have access to that password manager. And they'd always be able to tell their loved ones how to access that password manager in the event of an accident.

Okay, since none of those things are guaranteed out there in the world, we need to have a way for the user to get in if they don't have access to their password. How do we securely(ish) verify they are who they say they are when they request a password reset?

The most common way to handle this is by sending a one-time link or code to the email address or the phone number they gave us when they signed up that they can then click to prove they have control of that communications channel. We also generally assume that if the user's email address has been compromised, they have way bigger problems than a password reset on our service, so we'll let you in if you possess that one-time code. But! Critically, this requires us to have collected those communication channels up front. Which requires us to store more information about the user. This is why the username also tends to be the email address in services (but this is, IMO, a bad user experience because changing it often becomes problematic).

Related point, we could also send that code to SMS — most folks have a phone after all. But this is a fairly weak form of authentication because of sim swapping attacks. A sim swap, briefly, is where someone calls your phone company, pretends to be you, and then gets the phone company to activate their phone with your number. You lose access to your phone number, they gain it, and can intercept these kinds of codes (this is also true of multifactor). CISA recommends only using SMS or voice as a last-resort option for MFA, and that holds true for forgotten password requests (for the same reasons).

So, assuming we're going to send a reset request to some other communication channel for the user, what do we send them? Well, in order to be secure, it should have the following properties:

1. Impossible to enumerate in a reasonable amount of time.
   1. This means that you should rate-limit the number of requests to the confirmation page.
   2. It should also be longer than a few characters, if you're using a PIN, at least 6 digits.
2. Should only be valid for a short period of time.
   1. Generally between 5 and 30 minutes is acceptable. Use the shortest period of time that is reasonable for your user base.
3. Should only be usable a single time. Once a reset code / pin / URL has been used successfully, it should be invalid for subsequent requests.

For more reading on the topic, I recommend OWASP's Forgot Password Cheat Sheet.

Multifactor Authentication (MFA) is a large topic which I'll cover in more detail in a different post, but for its worth mentioning here: MFA both increases account security and the complexity of ensuring legitimate users don't lose access to their accounts.

So, stay tuned for the deeper dive on how to implement MFA in another post.

One thing that Banks in particular do that drives me to irrational anger is to ask "security" questions, like "What was the make and model of your first car?"

These questions are trying to give you something easy to remember as either a recovery method or a MFA factor, but they invariably ask things that are easy to find out in public or social engineer. Like, anyone can figure out what high school I went to. So instead of answering these in a memorable way, I use a password manager to generate random words to fill out the answers. That way you can't guess them, and I have a shot at remembering them (but otherwise just use my password manager). It's pointless, just give me a proper MFA factor and call it a day.

Stop using these, they're a bad idea.

Just ask NIST:

Verifiers and CSPs SHALL NOT prompt subscribers to use knowledge-based authentication (KBA) (e.g., “What was the name of your first pet?”) or security questions when choosing passwords. - NIST SP-800-63B

There's an old security requirement still floating around (especially in corporate settings) that you must force a password reset every {n} days. This is no longer a good requirement, and you should not force the user to change their password. This has been true for a long time now.

Verifiers SHOULD NOT require memorized secrets to be changed arbitrarily (e.g., periodically). However, verifiers SHALL force a change if there is evidence of compromise of the authenticator. - NIST SP-800-63B

To summarize, your baseline level of secure password storage:

1. Do not store passwords in plaintext. Use Argon2id, scrypt, bcrypt, or PBKDF2. If you're using a framework it's probably doing this for you, but understand what the framework is doing and which hashing mechanism it has chosen.
   1. If you're going for a NIST certification, that will determine which algorithm you must use.
2. Consider how you're going to reset the user's passwords when they lose access. This requires collecting an email, phone number, or some other mechanism up front.
3. Resetting the password should require possession of something else (email, SMS, backup tokens).
4. MFA adds additional security but also additional account lockout considerations.
5. Please for the love of the gods do not ask for "security questions".

Alright! So we're getting close to the end of the topic for this post, but what happens if you want to log the user out. You've got a bunch of questions you should answer.

When the user clicks a "Log Out" button, you should do the following things:

1. Delete their session storage (the cookie and any data you've got on the session server-side)
2. Redirect them back to a public page.

You do not need to do things like remove their history, because your server should redirect to an authentication page if the session is missing. Most frameworks encapsulate best practices into a logout() method somewhere on your user or session model which will do these things for you.

As a matter of practice, you should offer your users the ability to delete their account. How you implement this depends on local regulations: you may need to anonymize the user's data and remove personal identifiers (but continue to store logs based on retention requirements - this is common in health care settings). However, if you do not have retention requirements, you should remove all related user data from your database that you do not require.

In practice, most applications will need to store a "stub" of the user record. For example, in an eCommerce application you'll want to retain archival records for your orders (which may be synced to other systems), but "what data do we need to retain" is a huge topic and industry-dependent.

Regardless, read up on GDPR or CCPA under the right to be forgotten for further guidelines on how you can delete user data.

Okay, this is getting long, and I'm sure there are some things I missed. If there are bits that you'd like to see me touch on I can update this post later, just leave me a comment below or ping me on Mastodon or Bluesky.

Part 2 will be on Multifactor Authentication and the various methods for how that works, along with the why, how, and what to do when you drop your phone in a river.

authentication blog