Web technologies

Web server

A web server serves documents to web browsers via HTTP, which is a network protocol.

A protocol is an agreed upon way of talking between computers.

HTTP is the HyperText Transfer Protocol
Web servers usually serves web pages, written in HTML (HyperText Markup Language)
And sometimes program code, written in Javascript

HTTP

HTTP uses URLs to find web pages. These are what you see in the address bar near the top of a web browser.

Protocol: https (HTTP Secured with TLS)
Host: hackintheclass.nl (connect to the web server at address "hackintheclass.nl")
Request URI: /index.php?show=Workshops&page=1
File/Script name: /index.php
Query String: show=Workshops&page=1
- Some fields from forms end up here:
  - show: Workshops
  - page: 1

Path to the document

For a path of /admin/news/index.php, it looks in the file system tree:
First it finds admin in / (the root), then it finds news in admin/, then it finds index.php in news.

What happens when..

Visiting web page https://bitlair.nl/Workshops:

Web browser looks up "bitlair.nl" in the Domain Name System (DNS)
- Which is kind of a phone book for internet names
DNS says bitlair.nl is at IP address 2001:470:d1e7:1337:5054:ff:fe49:a94
Connect to web server with address 2001:470:d1e7:1337:5054:ff:fe49:a94
Set up secure connection (S for secure in HTTPS)
Do a GET request for the web page /Workshops

HTTP GET request

GET forms put data in the URL, visible in the address bar and stored in the browser history and can be bookmarked.

GET /logon.php?username=hax0r&password=supersecret123 HTTP/1.1
Host: bitlair.nl
Cookie: PHPSESSID=f1a1d9715b3491bbc2d5203c88ac67fb
Referer: https://bitlair.nl/index.php?action=showloginform
User-Agent: Mozilla/5.0 >--snip--< Chrome/69.0.4453.96

>-- end of request --<

It helps to use the developer tools (press F12) - network tab to look at the network traffic.

HTTP POST request

POST forms put data in the request content, not visible anywhere else.

POST /logon.php HTTP/1.1
Host: bitlair.nl
Cookie: PHPSESSID=f1a1d9715b3491bbc2d5203c88ac67fb
Referer: https://bitlair.nl/index.php?action=showloginform
User-Agent: Mozilla/5.0 >--snip--< Chrome/69.0.4453.96
Content-Type: application/x-www-form-urlencoded
Content-Length: 38

username=hax0r&password=supersecret123
>-- end of request --<

It helps to use the developer tools (press F12) - network tab to look at the network traffic.

Mistake: Not using encryption

Websites should always use HTTPS (HTTP Secured). When they don't use encryption:

Local attackers can eavesdrop on the network and see passwords and other secrets
The website can be hijacked (through BGP hijacks)
Proxy servers see all the secrets

Mistake: Information disclosure

Web server and PHP version numbers may give away that it is running old versions
Error messages may show paths that can help attackers find things or tune their exploits
Indexes in subdirectories may show the existence of files that were accidentally published
There may be backup files (.bak or ~ extensions), .svn or .git directories may exist, that can give away the existence of files or even source code
A robots.txt file may exist in the root directory. It is designed to make sure search engine spiders don't visit certain files.
- Why would people keep files from google with a robots.txt file?
  - Could it be that they tried to make sure google doesn't follow links with delete_user=something ?
  - Could it be that they thought it would hide something?

Play HTTP challenges

They get more difficult as you reach higher levels, look for version numbers, indexes, backup files, subversion work trees and git work trees.

If you get stuck, no problem, just continue the tutorial and come back here later.

HTML

All of the layout and content of every web page is controlled through HTML

Go to a random web page, right click on a background area, then click on "View page source".
View source right mouse click
Also available by pressing and holding Ctrl then pressing U (Ctrl-U)

Basic HTML

HyperText Markup Language: markup with tags: <open tag>content</close tag>

    <!doctype html>
    <html>
    <head>
      <title>Example HTML website</title>
    </head>
    <body>
      <h1>Top level header</h1>
      <ul>
        <li><a href="http://www.google.nl">Google</a></li>
        <li><a href="https://bitlair.nl>Bitlair</a></li>
      </ul>
    </body>
    </html>

HTML tutorial at w3schools

HTML Forms

    <form method="POST">
      <table>
        <tr>
          <td>Username:</td>
          <td><input type="text" name="username"></td>
        </tr>
        <tr>
          <td>Password:</td>
          <td><input type="password" name="password"></td>
        </tr>
      </table>
      <input type="submit" value="Log me in">
    </form>

Try changing the method from POST to GET on the Play around with HTML forms slide, then submit and see what happens HTML forms tutorial at w3schools

Play around with HTML forms

HTML code - Change me

HTTP information

HTML: Common mistakes

Information leaking
- Look at the source, comments may be available
- Also look at any scripts that may come with the HTML in <script> tags
Encoding user content issues (Cross Site Scripting/XSS)
- Try putting something like this in a comment field: <script>alert("vulnerable");</script>
- This should be translated to <script>alert("vulnerable&);</script> so it does not become valid HTML
- If this encoding does not happen, you'll get a popup message saying "vulnerable"

Mistake: Information leaking

Some websites leave HTML comments in the code <-- like this comment. -->.
- Could contain information like default passwords or vulnerable version numbers.

Mistake: not encoding user content

Many websites show user input in some way. Like comment sections or number of items you're about to order
If this is not HTML encoded, it can include HTML tags or double quotes
- This attack is called Cross Site Scripting (XSS)
Example HTML code: You are about to order 10<img src="https://i.imgur.com/R1g3YTc.jpg"> items.
- This shows an image, not just the number 10
Example HTML code:
<input type="text" name="item_count" value="10"><img src="https://i.imgur.com/R1g3YTc.jpg">">.
- This broke out of the item_count input field and shows an image
This can be countered by using templates, or in PHP with htmlentities() on all output of user input.
- <img> tags will become <img> so the browser knows it's not HTML
- Also true for other html tags

Javascript: example

This script finds the username HTML input and makes it available in the variable usernameInput.
It then checks if what the user typed is the same in the username field as in the password field

    <input type="text" id="username">
    <script type="text/javascript">
        // Finds the username HTML input field above and puts it in variable usernameInput
        var usernameInput = document.getElementById("username");

        // Compare what the user entered in username to the password input.
        if (usernameInput.value == passwordInput.value) {
            // Username and password are the same.
            ...
        }
    </script>

Javascript: Common mistakes

Because Javascript runs on the client, clients can bypass or manipulate it. As a result, these mistakes are common:

Not checking the values of form fields again on the server.
Checking passwords or permissions in the client, not on the server.
Allowing users to inject Javascript into the page (HTML escaping vulnerability: Cross Site Scripting/XSS).
Allowing Javascript access to the session cookie (httpOnly not set).
Including Javascript from a server managed by somebody else (like googleapis.com, jquery.org, github.io).
Using frames with external content (like advertisements) without preventing access to information on the page (sandboxing).

Mistake: Not checking input

Javascript runs on the client. Validating user input like form fields in javascript is not enough to protect the server. All input always has to be checked on the server.

Example: Age check on credit card application form
- Disable javascript and any age is accepted?
Example: Encoding HTML entities
- Disable javascript and no encoding happens?

Mistake: Checking the password in JS

Javascript runs on the client. Any check can be bypassed or inspected. This includes password checks, but also permission checks (authentication and authorisation).

Look at the source code, find <script> tags or .js files.
Is password checking done in javascript?
What criteria does the password need to comply with?
What action is taken when it matches?
Any password validation that is only done on the client is insecure.

Mistake: Not encoding input, allowing JS

If HTML code can be injected, so can Javascript.
In XSS level 1, you injected an image into an HTML page.
What if you could run scripts? <script>top.location.href='https://evil.host/';</script>
- This redirects users to an evil web host.

This is actually an HTML mistake: Not encoding user content

Mistake: Including JS from other sites

Including javascript from a server managed by somebody else (like googleapis.com, jquery.org, github.io) is not good practice.

If that site gets compromised, then they can also take over that website.
If that site becomes evil, then they can take over that website.
If the IP or DNS gets compromised (BGP hijack), then attackers can get a certificate through Let's Encrypt and attack your website.
Security-minded people disallow javascript from other servers, because the javascript can be malicious.
- The website will not load for them.

Mistake: Not restricting JS in frames

Using frames with external content (like advertisements) without preventing access to information on the page (sandboxing) opens the website up to hijacking.

Without iframe restrictions, javascript can access information on the top/parent frame
Solution, the sandbox attribute: <iframe name="advertisement" src="https://ad.someadparty.net/" sandbox>
The attack: <script>alert(top.document.body.innerHTML);</script>

Web application

Javascript can run program code in the web browser. But the web server can also run program code.

When visiting https://bitlair.nl/index.php?page=Workshops
Server runs PHP script "/index.php"
PHP script sees the query string (page=Workshops)
PHP script builds up the Workshops page

Other programming languages can be run on the server server as well, like Python, Java, ASP.NET and even Javascript.

Sessions

HTTP has no memory (state). Every request is independent of other requests. There are no sessions.
To implement sessions on the server, random numbers are given to visitors
Visitors store this random number in a cookie.
The cookie is sent along with every web page visit.
The server can look up who the visitor is, based on the cookie.
This is called a session ID cookie.

Mistake: Predictable session ID

If the session cookie for user ID 1 is always something like sha1(user1), or md5(1) or sessions are handed out like sessionid=10, sessionid=11, it is considered predictable.

Is your session ID just a relatively small number? That's vulnerable.
Is your session ID the hash of a number below 1 million? That's likely vulnerable
Try setting the cookie to a (hash of a) different number.. (enumeration).
- One of those sessions may have different privileges.

Play session challenges

Get the admin cookie with Cross Site Scripting, log in with the admin cookie:

Try our Cookie monster to steal the admin's cookie.
Cross Site Scripting (XSS) level 3

PHP

PHP is a programming language for building up HTML web pages.

It runs on the web server
It often gets information from files or a database
It may show different web page options to different people

Other programming languages can be run on the server server as well, like Python, Java, ASP.NET and even Javascript.

Web application: Common mistakes

Having default passwords
Allowing weak / leaked passwords
Not limiting login attempts over time
Trusting the web browser to keep secrets
Predictable interactions (Enabling Cross Site Request Forgery/CSRF)
Not updating components
Poor permission checks
All mistakes mentioned under every other mistake section in this tutorial

Mistake: Having default passwords

A lot of applications and devices come with default passwords. This is not smart. It is better to require the user to set a password on first use.

Most common default usernames: admin, root, user, manager, system, operator, guest, setup, private
Most common default passwords: Empty password, admin, ADMIN, password, PASSWORD, pass, system, SYSTEM, sys, user, manager, operator, guest, setup, private, letmein, calvin, forgetme!, changeme, welcome1, Welcome01!, 123, 1234, 123456
Sometimes default usernames / passwords are not easy, but eventually leak anyway. These are sometimes called backdoor accounts and then require a security update to fix.
Do a web search to find the default password for a device when you need it. You'll often get in that way.

Mistake: Allowing weak / leaked passwords

Weak passwords can be easily cracked if hashes somehow get compromised
Weak passwords are in rainbow tables
Attackers have lists of used passwords from leaks or just common weak passwords
They could just try to login with all common usernames / passwords (brute force attack)

Mistake: Trusting the browser with secrets

Some web applications don't use sessions, but store the username and password in the cookies

This credential expires only when the password does.
This secret is stored in the browser in plain text, it can be read by local attackers

Mistake: Predictable interactions

Any form that does not have something random in it is predictable
- Example: https://example.com/admin/delete_user.php?userid=1
- If you give that link to somebody logged in with admin access, it'll delete user 1
This attack is called Cross Site Request Forgery (CSRF)
- Especially problematic when combined with Cross Site Scripting (XSS)

Mistake: Not updating components

When using components from others, always keep an eye on vulnerabilities that come to light.
For example: old versions of PHP are vulnerable to NUL-byte (HTTP encoded as %00) termination of strings.
- So file_get_contents($_GET['page'] . '.html') becomes vulnerable.
- Using ../../../etc/passwd%00 as requested page removes .html that's appended, because NUL-byte ends the string.
- It ends up serving not an HTML page, but the /etc/passwd file.
Keeping software up-to-date (patch management) and tracking vulnerabilities (vulnerability management) both help to mitigate the issue.
For attackers it helps to look for vulnerabilities in the webapp components as well.

Mistake: poor permission checks

Enforcing permissions on the webapp, but not on uploaded documents/static content.
Enforcing permissions with a Location header redirect, but still serving the content (very briefly).

Database

A data store that servers data to other applications. There are many types, but the most common are:

SQL - Structured Query Language
Key-value stores, like memcached, redis, berkeley database, lmdb, dbm
Hierarchical stores, like File systems
Hierarchical with attributes, like LDAP and X.500
Document stores (key value stores that also index and parse content in the value), like mongoDB and couchDB

Data retrieval

Most web applications use SQL databases for data storage

Structured Query Language
Look at MySQL SELECT syntax
SELECT fieldname1, fieldname2 FROM tablename WHERE condition

SQL comments - MySQL

MySQL uses # as comment character. Most other SQL implementations use " -- " (spaces important)
Everything after the comment character on that line is a comment.
Comments will be ignored by the SQL server.

            SELECT password     # Password field
            FROM users          # Users table
            WHERE id=1 OR id=2  # User ID 1 or 2
            ORDER BY id ASC     # Sort by id, ascending
            LIMIT 2             # Show only two records
            ;                   # End of query

SQL data is stored in tables

Table "users":

userid	username	password	email
1	synnack	Supersecret123!	synnack@example.com
2	admin	Adm1ns3cr3t!	admin@example.com

Use SELECT userid, username, password, email FROM users WHERE userid='1'

* Note: It is bad to store plaintext passwords, use a password-based key derivation function like PBKDF2 or bcrypt.

Enumeration

If data is accessed like /page.php?id=1, can we also get page 2, 3, 4 and 5?

id	page
1	AMERSFOORT Today we witnessed a...
2	AMSTERDAM It was an historic even...
3	Unpublished - Top secret files leaked...

SQL injection

SELECT userid, password FROM users WHERE username='$USERNAME'
What if $USERNAME is user input?
What if $USERNAME is " ' OR 1=1 # "?
SELECT userid, password FROM users WHERE username='' OR 1=1 # '
It will match on any record, because 1 is always equal to 1

Try SQL injection

Common patterns (do try at Play around with SQL):

' OR 1=1 #
- This makes the SQL statement unconditional, it cancels out everything after WHERE, because 1 is always 1
' OR 1=1 LIMIT 0,1 #
- Unconditional, but also show only one record, the first (show 1 result from offset 0)
' AND 1=0 UNION SELECT 1,2,3,4 #
- The original query returns nothing and a union of nothing and what you want, is what you want.
- Note that with a UNION query, the number of columns on both SELECT statements must be identical
- Try with UNION SELECT 1 #, UNION SELECT 1,2 #, UNION SELECT 1,2,3 #, etc

Study the Pentestmonkey cheatsheet for inspiration on what to type behind UNION SELECT, like finding table names and field names in a certain table.

Play SQL injection challenges 1 and 2

Open the Pentestmonkey cheatsheet and The play around slide in new tabs, then open the levels in a new tab.

Play SQL injection challenges 3

For this level you will need to know about HTTP requests as well, see the web server section

Find the database query, there is only one and it is vulnerable
Find out which tables there are
Find out which fields the relevant tables have
Find the flag
Good luck! Start here: SQL injection level 3

Path Traversal

download.php?file=pentest-report.pdf

All operating systems have special directories . and .. for current and parent directory
download.php?file=../index.php (POSIX-based operating systems)
- Downloads index.php from the parent directory!
download.php?file=..\index.php or download.asp?file=..\index.asp (DOS-like operating systems)
Sometimes poorly mitigated.. removes all occurrences of '../'
- What about ....//? remove all ../ in ....// and the result is ../!

Play path traversal challenges 3

Some programs that are written in C end their strings when they encounter a NUL-byte (encoded as %00 in URLs).
- Software such as PHP versions before 5.3
This allows us to strip the .html part of $_GET['page_name'] . ".html" and access all files.
You can now play Path traversal level 3

Play combining skills challenges

If you've done all the above challenges, you can try combining skills
To play Combining skills 2, you should have completed login mistakes and all SQL injection and path traversal challenges
You can now play Combining skills level 2

Web techniques

Web server

HTTP

Path to the document

What happens when..

HTTP GET request

HTTP POST request

More information about HTTP

HTTP: common mistakes

Mistake: Not using encryption

Mistake: Information disclosure

Play HTTP challenges

HTML

Basic HTML

HTML Forms

More information about HTML

Play around with HTML forms

HTML: Common mistakes

Mistake: Information leaking

Mistake: not encoding user content

Play HTML challenges

Javascript

Javascript: example

More information about Javascript.

Javascript: Common mistakes

Mistake: Not checking input

Mistake: Checking the password in JS

Mistake: Not encoding input, allowing JS

Mistake: Allowing JS to access cookies

Mistake: Including JS from other sites

Mistake: Not restricting JS in frames

Play Javascript challenges

Play Basic Web challenges

Web application

Sessions

Cookie login session

Session mistakes

Mistake: Predictable session ID

Mistake: Bad cookie settings