2024-3-11 21:57:52 Author: lab.wallarm.com(查看原文) 阅读量:15 收藏
Since 1991, Web Application Firewall, commonly referred to as WAF, has become one of the most common application security technologies available on the market. Since the last century, WAFs have evolved by incorporating the cloud and using Machine Learning instead of RegExp.
Currently, few technologies, such as NG-WAF, RASP, WAAP, and a few others, have internal WAF capabilities, which prevent web applications and API threats.
Majority of the fintech, health tech, and e-commerce companies have had WAFs installed for years to protect their APIs, but also due to PCI DSS, SOC2, and HIPAA compliance requirements, bot mitigation, and OWASP Top-10 attack prevention needs.
How good is my WAF?
WAF-like technologies have already been in place for a while, but how good are they? Given the notorious issue of false positives that WAFs have always been known for, people often focus on evaluating false positive rates while ignoring the testing of false negative rates. In addition, it is not easy to test and check the actual level of WAF or RASP protection.
So how can we really test how good your current WAF is? What attacks can it really stop and where can application and API attacks still hit even with a WAF in place? Where are the blind spots? There isn’t a simple tool that any developer, QA, or security engineer can run to get a PDF report on your WAF coverage.
Meet GoTestWAF
To address the issue, we are open-sourcing a project called GoTestWAF.
GoTestWAF generates requests with predefined, basic payloads as well as attacks specific to different APIs (REST, SOAP, XMLRPC, gRPC). Afterwards, it sends them to the application and analyzes the responses to generate a detailed report in the console output or as a PDF.
The usage is pretty simple. It only requires that you run this tool against your WAF protected application.
docker run --rm -it -v "$(pwd)/reports:/app/reports" wallarm/gotestwaf --url=http://the-waf-you-wanna-test/
It gives clear results and indicates which of the attacks are detected with your existing appsec solution and what ways attackers can still hit your apps:
INFO[0000] GoTestWAF started version=v0.4.14 INFO[0000] Test cases loading started INFO[0000] Test cases loading finished INFO[0000] Test cases fingerprint fp=23c3ae919db5e6edcb62815de1a09fdf INFO[0000] Try to identify WAF solution INFO[0000] WAF was not identified INFO[0000] WAF pre-check url="http://host.docker.internal:8080" INFO[0000] WAF pre-check blocked=true code=403 status=done INFO[0000] WebSocket pre-check status=started url="ws://host.docker.internal:8080" INFO[0000] WebSocket pre-check connection="not available" error="websocket: bad handshake" status=done INFO[0000] gRPC pre-check status=started INFO[0000] gRPC pre-check connection="not available" status=done INFO[0000] Scanning started url="http://host.docker.internal:8080" INFO[0012] Scanning finished duration=12.856448447s True-Positive Tests: +-----------------------+-------------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+ | TEST SET | TEST CASE | PERCENTAGE, % | BLOCKED | BYPASSED | UNRESOLVED | SENT | FAILED | +-----------------------+-------------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+ | community | community-128kb-rce | 0.00 | 0 | 0 | 1 | 1 | 0 | | community | community-128kb-sqli | 0.00 | 0 | 0 | 1 | 1 | 0 | | community | community-128kb-xss | 0.00 | 0 | 0 | 1 | 1 | 0 | | community | community-16kb-rce | 100.00 | 1 | 0 | 0 | 1 | 0 | | community | community-16kb-sqli | 100.00 | 1 | 0 | 0 | 1 | 0 | | community | community-16kb-xss | 100.00 | 1 | 0 | 0 | 1 | 0 | | community | community-32kb-rce | 100.00 | 1 | 0 | 0 | 1 | 0 | | community | community-32kb-sqli | 100.00 | 1 | 0 | 0 | 1 | 0 | | community | community-32kb-xss | 100.00 | 1 | 0 | 0 | 1 | 0 | | community | community-64kb-rce | 100.00 | 1 | 0 | 0 | 1 | 0 | | community | community-64kb-sqli | 100.00 | 1 | 0 | 0 | 1 | 0 | | community | community-64kb-xss | 100.00 | 1 | 0 | 0 | 1 | 0 | | community | community-8kb-rce | 100.00 | 1 | 0 | 0 | 1 | 0 | | community | community-8kb-sqli | 100.00 | 1 | 0 | 0 | 1 | 0 | | community | community-8kb-xss | 100.00 | 1 | 0 | 0 | 1 | 0 | | community | community-lfi | 100.00 | 8 | 0 | 0 | 8 | 0 | | community | community-lfi-multipart | 0.00 | 0 | 0 | 9 | 9 | 0 | | community | community-rce | 83.33 | 10 | 2 | 0 | 12 | 0 | | community | community-sqli | 100.00 | 32 | 0 | 0 | 32 | 0 | | community | community-user-agent | 70.00 | 7 | 3 | 0 | 10 | 0 | | community | community-xss | 95.80 | 502 | 22 | 0 | 524 | 0 | | community | community-xxe | 0.00 | 0 | 2 | 0 | 2 | 0 | | owasp | crlf | 77.78 | 7 | 2 | 0 | 9 | 0 | | owasp | ldap-injection | 3.13 | 2 | 62 | 0 | 64 | 0 | | owasp | mail-injection | 12.50 | 3 | 21 | 0 | 24 | 0 | | owasp | nosql-injection | 0.00 | 0 | 70 | 0 | 70 | 0 | | owasp | path-traversal | 24.77 | 27 | 82 | 1 | 110 | 0 | | owasp | rce | 33.33 | 22 | 44 | 0 | 66 | 0 | | owasp | rce-urlparam | 33.33 | 3 | 6 | 0 | 9 | 0 | | owasp | shell-injection | 27.08 | 13 | 35 | 0 | 48 | 0 | | owasp | sql-injection | 24.36 | 38 | 118 | 0 | 156 | 0 | | owasp | ss-include | 37.50 | 15 | 25 | 0 | 40 | 0 | | owasp | sst-injection | 18.75 | 12 | 52 | 0 | 64 | 0 | | owasp | xml-injection | 0.00 | 0 | 12 | 1 | 13 | 0 | | owasp | xss-scripting | 33.20 | 167 | 336 | 1 | 504 | 0 | | owasp-api | graphql | 0.00 | 0 | 6 | 0 | 6 | 0 | | owasp-api | graphql-post | 50.00 | 2 | 2 | 0 | 4 | 0 | | owasp-api | grpc | 0.00 | 0 | 0 | 0 | 0 | 0 | | owasp-api | non-crud | 100.00 | 2 | 0 | 0 | 2 | 0 | | owasp-api | rest | 23.08 | 3 | 10 | 0 | 13 | 0 | | owasp-api | soap | 23.08 | 3 | 10 | 0 | 13 | 0 | +-----------------------+-------------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+ | DATE: | PROJECT NAME: | TRUE-POSITIVE SCORE: | BLOCKED (RESOLVED): | BYPASSED (RESOLVED): | UNRESOLVED (SENT): | TOTAL SENT: | FAILED (TOTAL): | | 2024-03-04 | GENERIC | 49.12% | 890/1812 (49.12%) | 922/1812 (50.88%) | 15/1827 (0.82%) | 1827 | 0/1827 (0.00%) | +-----------------------+-------------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+ True-Negative Tests: +-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+ | TEST SET | TEST CASE | PERCENTAGE, % | BLOCKED | BYPASSED | UNRESOLVED | SENT | FAILED | +-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+ | false-pos | texts | 85.65 | 31 | 185 | 0 | 216 | 0 | +-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+ | DATE: | PROJECT NAME: | TRUE-NEGATIVE SCORE: | BLOCKED (RESOLVED): | BYPASSED (RESOLVED): | UNRESOLVED (SENT): | TOTAL SENT: | FAILED (TOTAL): | | 2024-03-04 | GENERIC | 85.65% | 31/216 (14.35%) | 185/216 (85.65%) | 0/216 (0.00%) | 216 | 0/216 (0.00%) | +-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+-----------------------+ Summary: +-----------------------------+-----------------------------+-----------------------------+-----------------------------+ | TYPE | TRUE-POSITIVE TESTS BLOCKED | TRUE-NEGATIVE TESTS PASSED | AVERAGE | +-----------------------------+-----------------------------+-----------------------------+-----------------------------+ | API Security | 26.32% | n/a | 26.32% | | Application Security | 49.61% | 85.65% | 67.63% | +-----------------------------+-----------------------------+-----------------------------+-----------------------------+ | SCORE | 46.97% | +-----------------------------+-----------------------------+-----------------------------+-----------------------------+ Do you want to include payload details to the report? ([y/N]): y INFO[0022] Export full report filename=reports/waf-evaluation-report-2024-March-04-15-56-05.pdf Email to send the report (ENTER to skip): INFO[0024] Skip report sending to email code gtw.txt Displaying code gtw.txt.
In conclusion, the GoTestWAF is a tool to test WAFs, RASPs, and WAAP for application and API attacks, not just CGI payloads from 90th.
How it works?
The main idea is to encode and place attack payloads in different parts of an HTTP request: its body, headers, URL parameters, etc. To simplify things, we decided to implement the following logic:
Payload -> Encoder -> Placeholder
It means that every payload sample (malicious attack sample such as an XSS string like “<script>alert(1)</script>”) will be first encoded in some way then placed into an HTTP request. There is also an option to use a plain encoder that keeps the string as-is.
To make tests readable, we introduced a YAML DSL with an equal structure (payload->encoder->placeholder), where all the fields are arrays and make a test by permutations between them.
As a result of the permutation of each 4 payloads, 2 encoders, and 4 placeholders, this test will send 4*2*4=32 requests.
Note: be careful with the YAML. If you need to send a few binary payloads, then it is best to use !!binary attribute on the payload fields.
How to run
There are 3 options to run GoTestWAF:
1) using the official docker image - https://hub.docker.com/r/wallarm/gotestwaf
2) independently according to the instructions:
There is an Alpine-based Docker image available. It is ready to start out-of-the-box with a single command-line right after “git clone” and “docker build”:
If you need the report, you are required to map it from your host machine via -v docker attribute. Otherwise, all the stats will be available in your console only.
If you need the opportunity to answer a couple of questions from GTW (do we want to insert payloads into the report, do we want to send the report by email), use the -it attribute
$ git clone https://github.com/wallarm/gotestwaf $ cd gotestwaf $ make gotestwaf $ docker run --rm -it -v "$(pwd)/reports:/app/reports" gotestwaf --url=http://the-waf-you-wanna-test/
If you need to configure some more parameters, here are the command-line arguments and config.yaml:
Options:
--addDebugHeader Add header with a hash of the test information in each request
--addHeader string An HTTP header to add to requests
--blockConnReset If true, connection resets will be considered as block
--blockRegex string Regex to detect a blocking page with the same HTTP response status code as a not blocked request
--blockStatusCodes ints HTTP status code that WAF uses while blocking requests (default [403])
--configPath string Path to the config file (default "config.yaml")
--email string E-mail to which the report will be sent
--followCookies If true, use cookies sent by the server. May work only with --maxIdleConns=1
--grpcPort uint16 gRPC port to check
--idleConnTimeout int The maximum amount of time a keep-alive connection will live (default 2)
--ignoreUnresolved If true, unresolved test cases will be considered as bypassed (affect score and results)
--includePayloads If true, payloads will be included in HTML/PDF report
--logFormat string Set logging format: text, json (default "text")
--logLevel string Logging level: panic, fatal, error, warn, info, debug, trace (default "info")
--maxIdleConns int The maximum number of keep-alive connections (default 2)
--maxRedirects int The maximum number of handling redirects (default 50)
--noEmailReport Save report locally
--nonBlockedAsPassed If true, count requests that weren't blocked as passed. If false, requests that don't satisfy to PassStatusCodes/PassRegExp as blocked
--openapiFile string Path to openAPI file
--passRegex string Regex to a detect normal (not blocked) web page with the same HTTP status code as a blocked request
--passStatusCodes ints HTTP response status code that WAF uses while passing requests (default [200,404])
--proxy string Proxy URL to use
--quiet If true, disable verbose logging
--randomDelay int Random delay in ms in addition to the delay between requests (default 400)
--renewSession Renew cookies before each test. Should be used with --followCookies flag
--reportFormat string Export report to one of the following formats: none, pdf, html, json (default "pdf")
--reportName string Report file name. Supports `time' package template format (default "waf-evaluation-report-2006-January-02-15-04-05")
--reportPath string A directory to store reports (default "reports")
--sendDelay int Delay in ms between requests (default 400)
--skipWAFBlockCheck If true, WAF detection tests will be skipped
--skipWAFIdentification Skip WAF identification
--testCase string If set then only this test case will be run
--testCasesPath string Path to a folder with test cases (default "testcases")
--testSet string If set then only this test set's cases will be run
--tlsVerify If true, the received TLS certificate will be verified
--url string URL to check
--version Show GoTestWAF version and exit
--wafName string Name of the WAF product (default "generic")
--workers int The number of workers to scan (default 5)
--wsURL string WebSocket URL to check
Command-line flags have priority over configuration files. As you can see, the most important thing here is how to detect a blocking page. Currently, you can do it with a response status or RegExp that will be applied to the request body. By default, the 403 response status code will be used for this.
You can also choose test cases between two embedded: OWASP Top-10, OWASP-API, or your own.
3) Using Free Online WAF Tester Tool - https://www.wallarm.com/gotestwaf/overview
The report sample
Again, the main goal of this tool is to easily generate readable reports to check the current state of protection according to OWASP guidelines.
To see what it looks like, we can install mod_security WAF and run it on the 8080 port:
Next, we can launch the GoTestWAF utility:
docker run -v /tmp:/tmp/report gotestwaf --url=http://the-waf-you-wanna-test/
The results will be printed in your console as such:
owasp path-traversal 6/18 (0.33) owasp sql-injection 4/16 (0.25) owasp ss-include 2/8 (0.25) owasp xml-injection 6/8 (0.75) owasp xss-scripting 4/12 (0.33) owasp ldap-injection 0/8 (0.00) owasp mail-injection 3/12 (0.25) owasp nosql-injection 0/18 (0.00) owasp shell-injection 3/8 (0.38) owasp sst-injection 5/20 (0.25)
The PDF version will also contain bupasses with details, screenshot below:
We updated the 2023-2024 GoTestWAF tool with the following features:
- Email Report Sending: GoTestWAF interactively prompts for an email address to send the report. If the desired behavior is not selected through options, it will ask for an email. The email can be set using the options --email or --noEmailReport.
- OpenAPI Specification Testing: Testing can be performed using the OpenAPI specification (specified through the --openapiFile option, detailed in the readme).
- Debug Header Addition: Ability to add a debug header (activated through the --addDebugHeader option). GoTestWAF will transmit payload information hash in the X-GoTestWAF-Test header, enabling the unique identification of the payload-encoder-placeholder combination during traffic analysis.
- gRPC Support: Added support for gRPC (via the --grpcPort flag). GoTestWAF sends vectors to this port where gRPC is specified as a placeholder in the test case.
- New Placeholders: Numerous new placeholders have been added, listed in the readme's "How It Works" section. The RawRequest placeholder is described, allowing precise request creation.
- Multiple Status Codes: Multiple status codes can be specified (options --blockStatusCodes, --passStatusCodes), enabling the inclusion of an array of status codes instead of just one. This is useful when applications react differently to different status codes
- Regular Expressions for WAF Response Analysis: Regular expressions (--blockRegex and --passRegex) can be used to analyze WAF responses.
- Automatic Vendor Identification: With session handling improvements, GoTestWAF can automatically identify multiple (2 or 3) vendors and include them in the report. This feature works by sending a test vector and observing the response. It operates by default, but can be disabled with --skipWAFIdentification
- Report Changes: The report format has changed, and now there's an option to receive a full report with all bypassed vectors (--includePayloads option). Alternatively, a short version with graphs and statistical data is available by default.
- JSON Report and Output: JSON reports and output are introduced to integrate GoTestWAF into pipelines. Log output format during execution is controlled by the --logFormat option (text, json), and the report format is set through --reportFormat (default is pdf, with other options available like none, html, json).
- Specific Test Set and Test Case: The ability to specify a particular test set (--testSet) or test case (--testCase) to narrow down the vectors during testing.
- Cookie Handling: GoTestWAF can consider cookies during scanning (--followCookies) and update the session before each request (--renewSession). This allows scanning hosts that require specific WAF-specific cookies, as otherwise, requests are blocked.
- OWASP Core Rule Set Testing: A script is added to generate test sets from the OWASP Core Rule Set regression testing suite. These vectors are not available by default and require additional steps as outlined in the readme.
The new report looks like this:
Graphs from the new report:
For the technical details about payload, encoders, and responses, you can check the CSV file that will be generated in the same folder as PDF report.
The new report has a comparison with other solutions:
Future improvements
We invite everyone to join our mission and commit to the project, so that we can collectively improve the task at hand. The current roadmap includes support of WebSockets and .NET specific attack improvements, as well as some bots-emulations and other new ideas recommended by current users. Any pull requests and improvement suggestions are greatly appreciated.
Frequently Asked Questions
How do you test WAF rules?
To test WAF rules, you can use penetration testing tools such as Burp Suite, OWASP ZAP, or Nmap to simulate attacks on your website. These tools can help identify vulnerabilities and ensure that WAF rules are functioning correctly.
How do I check my WAF application?
One way to check your WAF application is to perform regular security audits and vulnerability assessments. This can help identify any potential WAF issues that need to be addressed.
How do you audit a WAF?
Auditing a WAF involves evaluating the configuration, log files, and network traffic to ensure that the system is functioning correctly. Additionally, it is important to review attack detection and mitigation policies to determine their effectiveness.
What is WAF testing?
WAF testing is a process of evaluating the effectiveness of a web application firewall (WAF) against potential web-based attacks. This involves testing the WAF configuration, monitoring system logs, and assessing the response to known vulnerabilities.
Explore more:
Wallarm Cloud-Native WAAP - Product
如有侵权请联系:admin#unsafe.sh