The minimal number of concepts required for you to understand how HTTPS works
For some jobs, having a credential — like a medical degree — is necessary to ensure that the person has the proper training to perform the job. This situation begs the question: how do you verify whether a person has a credential?
When a company wants to confirm whether a job applicant has the credentials they say they do, it may hire a “trusted” background check agency to contact institutions holding the job applicant’s credentials to verify that they have what they do. In other words, a trusted third party confirms the relationship.
Once the job applicant becomes an employee of the company, they typically gain access to company-provided accounts — like an email address — that allows them to send private messages to other members of the company.
Verifying and sending private data over the public internet to web servers (like amazon.com) proceeds similarly. Clients, eg, web browsers like Firefox or Google Chrome, verify that they’ve connected to a legitimate web server through a trusted third party, known as a certificate authority (CA) which can prove that the web server the browser is talking to is actually who it says it is.
Once verified, the client and browser can share private information — like login information — between each other since they’ve established a relationship.
In summary, Hypertext Transfer Protocol Secure (HTTPS), is a protocol by which a client, eg a browser, can:
- Verify that it’s talking to the correct web server via a CA
- Send secure data between web server and browser via encryption
We’ll go over the details of this protocol in the remainder of this article.
To understand how HTTPS works, we first must understand a few encryption techniques since these are the essential tools of HTTPS. I won’t dive deep into each topic but will provide a basic summary so you can grasp the general concepts.
A message, file, etc., is encrypted with a private key. This same private key decrypts the message.
- Alice and Bob share a secret key.
- Alice can use the secret key to encrypt a file.
- Bob can decrypt that encrypted file with the same secret key to return the original message.
### Symmetric Encryption with GPG ###### Adapted from https://www.baeldung.com/linux/encrypt-decrypt-files# create a random file
echo hi > symmetric-example.txt# encrypt it with the phrase "mysecret"
--symmetric symmetric-example.txt# decrypt it with the phrase
--decrypt symmetric-example.txt.gpg# check the differences
diff symmetric-example.txt symmetric-example.out.txt
A key pair includes a public and private key. The public key can encrypt messages decryptable only with its private key.
- Alice generates a public and private key pair.
- Alice shares her public key with Bob, which he uses to encrypt a message. Note: Bob cannot decrypt the message using the Alice’s public key.
- The encrypted message is decrypted back into the original message using Alice’s private key.
### Asymmetric Encryption with GPG ###### Adapted from https://www.baeldung.com/linux/encrypt-decrypt-files# generate the key pair as a key-ring
gpg --batch --generate-key <<EOF
EOF# Create a shareable public key, alicepubkey.gpg
gpg --keyring ./alicepubring.kbx
--export# Import alice's pubkey into bob's keyring file
gpg --keyring ./bobpubring.kbx
--import alicepubkey.gpg# Encrypt a file with alice's pubkey
echo "Hi Alice!" > alice.txtgpg --keyring ./bobpubring.kbx
--recipient "email@example.com" alice.txt# Decrypt the file with alice's private key
gpg --keyring ./alicepubring.kbx
--decrypt alice.txt.gpg # Compare the messages
diff -s alice.txt alice-decrypted.txt
A digital certificate is an identity file that a “trusted” third party issues (for HTTPS, this is an x.509 certificate). Digital certificates are like passports; They contain identity and certification information about an entity and have a way to verify that this information is correct through the certifying body.
In the case of a passport, the certifying body is a governmental institution (like the United States Department of State), and the entity is a person. The identity information within the passport includes the person’s first name, last name, date of birth, etc., which a customs and border officer can then verify with the institution.
For websites, the certifying body are organizations like Entrust, and the entity is a web server. The certificate includes information about the web server, like its common name (ie domain name), organization, country, etc., which a browser can verify.
Programmatically, a digital certificate follows an interface like this:
.public_key # attribute, webserver pubkey
.identity_information # attribute, webserver info
.issuing_CA # attribute, CA info
.verify(CA_public_key) # method, verify it's signed by CA
HTTPS uses the symmetric, asymmetric, and digital certificates to enable secure communication over a network. The following diagram details the data flow from when a web server wants to register with a CA (steps 1–4) to when the web server and browser (ie client) can securely transmit data to each other (steps 5–9):
- The web server generates a certificate request (CSR) which includes identity information about the web server as and its private key. It sends the CSR to a CA.
- The CA, once it approves the CSR, signs the CSR, thereby converting it into a digital certificate.
- The CA sends the certificate back to the web server.
- Upon a GET request to the web server from a client, eg, a web browser, the web server sends the digital certificate from step 3 to the browser.
- Upon receiving the certificate, the browser verifies that the certificate was signed by a known and trusted CA using the CA’s public key. Note: browsers have a database of CA public keys built into them.
- Once verified, the browser generates a symmetric session key for encrypted communication between the web server and the browser. This symmetric session key is encrypted using the web server’s public key from the certificate and sent back to the web server.
- The web server receives the message which includes the symmetric session key from the client, and decrypts the symmetric session key using its private key
- The web server can now symmetrically encrypt messages like HTML files, etc., and send the encrypted message over the internet.
- Once the client receives the encrypted message, it can decrypt it and view the message.
You may be wondering, why does HTTPS use both asymmetric and symmetric encryption when this whole process could be satisfied with just asymmetric encryption? That’s because symmetric encryption is fast relative to asymmetric encryption. Once verification between client and server completes, there’s no need for slow, back-and-forth asymmetric communication between client and server.