DΛΣMӨП

// HackTricks · Network Services

80,443 - Pentesting Web Methodology

80,443 - Pentesting Web Methodology

Basic Info

Web services are among the most common and extensive attack surfaces, and many different vulnerability classes can affect them.

Default ports: 80 (HTTP) and 443 (HTTPS)

PORT    STATE SERVICE
80/tcp  open  http
443/tcp open  ssl/https
nc -v domain.com 80 # GET / HTTP/1.0
openssl s_client -connect domain.com:443 # GET / HTTP/1.0

Web API Guidance

Web Api Pentesting

Methodology summary

This methodology assumes that you are assessing one domain or subdomain at a time. Apply it to every discovered in-scope domain, subdomain, or IP address that exposes an unidentified web server.

  • Start by identifying the technologies used by the web server. Look for tricks to keep in mind during the rest of the test if you can successfully identify the tech.
    • Any known vulnerability of the version of the technology?
    • Is it using any well-known technology? Are there useful techniques for extracting more information?
    • Any specialised scanner to run (like wpscan)?
  • Launch general-purpose scanners. They may identify a vulnerability or provide useful information for manual testing.
  • Start with the initial checks: robots, sitemap, 404 error and SSL/TLS scan (if HTTPS).
  • Start spidering the web page: It’s time to find all the possible files, folders and parameters being used. Also, check for special findings.
    • Note that anytime a new directory is discovered during brute-forcing or spidering, it should be spidered.
  • Directory Brute-Forcing: Try to brute force all the discovered folders searching for new files and directories.
    • Note that anytime a new directory is discovered during brute-forcing or spidering, it should be Brute-Forced.
  • Backups checking: Test if you can find backups of discovered files appending common backup extensions.
  • Brute-Force parameters: Try to find hidden parameters.
  • Once you have identified all possible endpoints that accept user input, check them for every relevant vulnerability class.

Server Version (Vulnerable?)

Identify

Check if there are known vulnerabilities for the server version that is running.
The HTTP headers and cookies of the response could be very useful to identify the technologies and/or version being used. Nmap scan can identify the server version, but it could also be useful the tools whatweb, webtech or https://builtwith.com/:

whatweb -a 1 <URL> #Stealthy
whatweb -a 3 <URL> #Aggresive
webtech -u <URL>
webanalyze -host https://google.com -crawl 2

Search for vulnerabilities of the web application version

Check if any WAF

Web tech tricks

Some tricks for finding vulnerabilities in different well known technologies being used:

Take into account that the same domain can be using different technologies in different ports, folders and subdomains.
If the web application is using any well known tech/platform listed before or any other, don’t forget to search on the Internet new tricks (and let me know!).

Source Code Review

If the application’s source code is available on GitHub, perform a white-box review and use the following information to improve black-box testing:

  • Is there a Change-log or Readme or Version file or anything with version info accessible via web?
  • How and where are credentials stored? Is any accessible file exposing usernames or passwords?
  • Are passwords in plain text, encrypted or which hashing algorithm is used?
  • Is it using any master key for encrypting something? Which algorithm is used?
  • Can you access any of these files exploiting some vulnerability?
  • Do GitHub issues (open or closed) or the commit history contain useful information, such as a password committed in an older revision?

Code Review Tools

Automatic scanners

General purpose automatic scanners

nikto -h <URL>
whatweb -a 4 <URL>
wapiti -u <URL>
W3af
zaproxy #You can use an API
nuclei -ut && nuclei -target <URL>

# https://github.com/ignis-sec/puff (client side vulns fuzzer)
node puff.js -w ./wordlist-examples/xss.txt -u "http://www.xssgame.com/f/m4KKGHi2rVUN/?query=FUZZ"

CMS scanners

If a CMS is used don’t forget to run a scanner, maybe something juicy is found:

Clusterd: JBoss, ColdFusion, WebLogic, Tomcat, Railo, Axis2, Glassfish
CMSScan: WordPress, Drupal, Joomla, vBulletin websites for Security issues. (GUI)
VulnX: Joomla, Wordpress, Drupal, PrestaShop, Opencart
CMSMap: (W)ordpress, (J)oomla, (D)rupal or (M)oodle
droopscan: Drupal, Joomla, Moodle, Silverstripe, Wordpress

cmsmap [-f W] -F -d <URL>
wpscan --force update -e --url <URL>
joomscan --ec -u <URL>
joomlavs.rb #https://github.com/rastating/joomlavs

At this point you should already have some information of the web server being used by the client (if any data is given) and some tricks to keep in mind during the test. If you are lucky you have even found a CMS and run some scanner.

Step-by-step Web Application Discovery

From this point we are going to start interacting with the web application.

Initial checks

Default pages with interesting info:

  • /robots.txt
  • /sitemap.xml
  • /crossdomain.xml
  • /clientaccesspolicy.xml
  • /.well-known/
  • Check also comments in the main and secondary pages.

Forcing errors

Web servers may behave unexpectedly when malformed data is sent to them. This may expose vulnerabilities or disclose sensitive information.

  • Access fake pages like /whatever_fake.php (.aspx,.html,.etc)
  • Add ”[]”, ”]]”, and ”[[” in cookie values and parameter values to create errors
  • Generate error by giving input as /~randomthing/%s at the end of URL
  • Try different HTTP Verbs like PATCH, DEBUG or wrong like FAKE

Check if you can upload files (PUT verb, WebDav)

If you find that WebDav is enabled but you don’t have enough permissions for uploading files in the root folder try to:

  • Brute Force credentials
  • Upload files via WebDav to the rest of found folders inside the web page. You may have permissions to upload files in other folders.

SSL/TLS vulnerabilities

  • If the application does not enforce HTTPS for sensitive functionality, it may be vulnerable to man-in-the-middle attacks.
  • If the application is sending sensitive data (passwords) using HTTP. Then it’s a high vulnerability.

Use testssl.sh to check for vulnerabilities and a2sv to verify findings. Many bug-bounty programs do not accept low-impact TLS configuration reports:

./testssl.sh [--htmlfile] 10.10.10.10:443
#Use the --htmlfile to save the output inside an htmlfile also

# You can also use other tools, by testssl.sh at this momment is the best one (I think)
sslscan <host:port>
sslyze --regular <ip:port>

Information about SSL/TLS vulnerabilities:

Spidering

Launch a spider against the application. Combine crawling with external sources to discover as many valid paths as possible.

  • gospider (go): HTML spider, LinkFinder in JS files and external sources (Archive.org, CommonCrawl.org, VirusTotal.com).
  • hakrawler (Go): HTML spider with LinkFinder-style JavaScript discovery and Archive.org as an external source.
  • dirhunt (python): HTML spider, also indicates “juicy files”.
  • evine (go): Interactive CLI HTML spider. It also searches in Archive.org
  • meg (go): This tool isn’t a spider but it can be useful. You can just indicate a file with hosts and a file with paths and meg will fetch each path on each host and save the response.
  • urlgrab (go): HTML spider with JS rendering capabilities. However, it looks like it’s unmaintained, the precompiled version is old and the current code doesn’t compile
  • gau (go): HTML spider that uses external providers (wayback, otx, commoncrawl)
  • ParamSpider: This script will find URLs with parameter and will list them.
  • galer (go): HTML spider with JS rendering capabilities.
  • LinkFinder (python): HTML spider, with JS beautify capabilities capable of search new paths in JS files. It could be worth it also take a look to JSScanner, which is a wrapper of LinkFinder.
  • goLinkFinder (go): To extract endpoints in both HTML source and embedded javascript files. Useful for bug hunters, red teamers, infosec ninjas.
  • JSParser (python2.7): A python 2.7 script using Tornado and JSBeautifier to parse relative URLs from JavaScript files. Useful for easily discovering AJAX requests. Looks like unmaintained.
  • relative-url-extractor (ruby): Given a file (HTML) it will extract URLs from it using nifty regular expression to find and extract the relative URLs from ugly (minify) files.
  • JSFScan (bash, several tools): Gather interesting information from JS files using several tools.
  • subjs (go): Find JS files.
  • page-fetch (go): Load a page in a headless browser and print out all the urls loaded to load the page.
  • Feroxbuster (rust): Content discovery tool mixing several options of the previous tools
  • Javascript Parsing: A Burp extension to find path and params in JS files.
  • BurpJSLinkFinder Enhanced: Burp extension (Jython) that passively analyzes JavaScript responses (by MIME type and /js paths) to extract endpoints/links and optionally flag embedded secrets with severity.[1]
  • Sourcemapper: Given a .js.map URL, retrieves the beautified JavaScript source.
  • xnLinkFinder: This is a tool used to discover endpoints for a given target.
  • waymore: Discover links from the wayback machine (also downloading the responses in the wayback and looking for more links)
  • HTTPLoot (go): Crawl (even by filling forms) and also find sensitive info using specific regexes.
  • SpiderSuite: Spider Suite is an advance multi-feature GUI web security Crawler/Spider designed for cyber security professionals.
  • jsluice (go): It’s a Go package and command-line tool for extracting URLs, paths, secrets, and other interesting data from JavaScript source code.
  • ParaForge: A Burp Suite extension that extracts parameters and endpoints from requests to create custom wordlists for fuzzing and enumeration.
  • katana (go): Awesome tool for this.
  • Crawley (go): Print every link it’s able to find.

Brute Force directories and files

Start brute-forcing from the root folder and be sure to brute-force all the directories found using this method and all the directories discovered by the Spidering (you can do this brute-forcing recursively and appending at the beginning of the used wordlist the names of the found directories).
Tools:

  • Dirb / Dirbuster - Included in Kali; old and slow, but functional. They support self-signed certificates and recursive search.
  • Dirsearch (python): It doesn’t allow auto-signed certificates but allows recursive search.
  • Gobuster (go): It allows auto-signed certificates, it doesn’t have recursive search.
  • Feroxbuster - Fast, supports recursive search.
  • wfuzz wfuzz -w /usr/share/seclists/Discovery/Web-Content/raft-medium-directories.txt https://domain.com/api/FUZZ
  • ffuf - Fast: ffuf -c -w /usr/share/wordlists/dirb/big.txt -u http://10.10.10.10/FUZZ
  • uro (Python): Not a spider; it normalizes a list of discovered URLs and removes functionally duplicated entries.
  • Scavenger: Burp Extension to create a list of directories from the burp history of different pages
  • TrashCompactor: Remove URLs with duplicated functionalities (based on js imports)
  • Chamaleon: It uses wapalyzer to detect used technologies and select the wordlists to use.

Recommended dictionaries:

Note that anytime a new directory is discovered during brute-forcing or spidering, it should be Brute-Forced.

What to check on each file found

Special findings

While performing the spidering and brute-forcing you could find interesting things that you have to notice.

Interesting files

403 Forbidden/Basic Authentication/401 Unauthorized (bypass)

403 And 401 Bypasses

502 Proxy Error

A 502 response may indicate a misconfigured reverse proxy. Test whether it improperly accepts an absolute-form request target such as GET https://google.com HTTP/1.1; if it fetches the supplied host, investigate the behavior as potential SSRF.

NTLM Authentication - Info disclosure

If the running server asking for authentication is Windows or you find a login asking for your credentials (and asking for domain name), you can provoke an information disclosure.
Send the header: “Authorization: NTLM TlRMTVNTUAABAAAAB4IIAAAAAAAAAAAAAAAAAAAAAAA=” and due to how the NTLM authentication works, the server will respond with internal info (IIS version, Windows version…) inside the header “WWW-Authenticate”.
You can automate this using the nmap plugin “http-ntlm-info.nse”.

HTTP Redirect (CTF)

It is possible to put content inside a Redirection. This content won’t be shown to the user (as the browser will execute the redirection) but something could be hidden in there.

Web Vulnerabilities Checking

Now that a comprehensive enumeration of the web application has been performed it’s time to check for a lot of possible vulnerabilities. You can find the checklist here:

Web Vulnerabilities Methodology

Find more info about web vulns in:

Monitor Pages for changes

You can use tools such as https://github.com/dgtlmoon/changedetection.io to monitor pages for modifications that might insert vulnerabilities.

HackTricks Automatic Commands

HackTricks Automatic Commands 
Protocol_Name: Web    #Protocol Abbreviation if there is one.
Port_Number:  80,443     #Comma separated if there is more than one.
Protocol_Description: Web         #Protocol Abbreviation Spelled out

Entry_1:
  Name: Notes
  Description: Notes for Web
  Note: |
    https://book.hacktricks.wiki/en/network-services-pentesting/pentesting-web/index.html

Entry_2:
  Name: Quick Web Scan
  Description: Nikto and GoBuster
  Command: nikto -host {Web_Proto}://{IP}:{Web_Port} &&&& gobuster dir -w {Small_Dirlist} -u {Web_Proto}://{IP}:{Web_Port} && gobuster dir -w {Big_Dirlist} -u {Web_Proto}://{IP}:{Web_Port}

Entry_3:
  Name: Nikto
  Description: Basic Site Info via Nikto
  Command: nikto -host {Web_Proto}://{IP}:{Web_Port}

Entry_4:
  Name: WhatWeb
  Description: General purpose auto scanner
  Command: whatweb -a 4 {IP}

Entry_5:
  Name: Directory Brute Force Non-Recursive
  Description:  Non-Recursive Directory Brute Force
  Command: gobuster dir -w {Big_Dirlist} -u {Web_Proto}://{IP}:{Web_Port}

Entry_6:
  Name: Directory Brute Force Recursive
  Description: Recursive Directory Brute Force
  Command: python3 {Tool_Dir}dirsearch/dirsearch.py -w {Small_Dirlist} -e php,exe,sh,py,html,pl -f -t 20 -u {Web_Proto}://{IP}:{Web_Port} -r 10

Entry_7:
  Name: Directory Brute Force CGI
  Description: Common Gateway Interface Brute Force
  Command: gobuster dir -u {Web_Proto}://{IP}:{Web_Port}/ -w /usr/share/seclists/Discovery/Web-Content/CGIs.txt -s 200

Entry_8:
  Name: Nmap Web Vuln Scan
  Description: Tailored Nmap Scan for web Vulnerabilities
  Command: nmap -vv --reason -Pn -sV -p {Web_Port} --script=`banner,(http* or ssl*) and not (brute or broadcast or dos or external or http-slowloris* or fuzzer)` {IP}

Entry_9:
  Name: Drupal
  Description: Drupal Enumeration Notes
  Note: |
    git clone https://github.com/immunIT/drupwn.git for low hanging fruit and git clone https://github.com/droope/droopescan.git for deeper enumeration

Entry_10:
  Name: WordPress
  Description: WordPress Enumeration with WPScan
  Command: |
    ?What is the location of the wp-login.php? Example: /Yeet/cannon/wp-login.php
    wpscan --url {Web_Proto}://{IP}{1} --enumerate ap,at,cb,dbe && wpscan --url {Web_Proto}://{IP}{1} --enumerate u,tt,t,vp --passwords {Big_Passwordlist} -e

Entry_11:
  Name: WordPress Hydra Brute Force
  Description: Need User (admin is default)
  Command: hydra -l admin -P {Big_Passwordlist} {IP} -V http-form-post '/wp-login.php:log=^USER^&pwd=^PASS^&wp-submit=Log In&testcookie=1:S=Location'

Entry_12:
  Name: Ffuf Vhost
  Description: Simple Scan with Ffuf for discovering additional vhosts
  Command: ffuf -w {Subdomain_List}:FUZZ -u {Web_Proto}://{Domain_Name} -H "Host:FUZZ.{Domain_Name}" -c -mc all {Ffuf_Filters}

References