Teler - Real-time HTTP Intrusion Detection
2020-11-19 05:30:00 Author: www.blogger.com(查看原文) 阅读量:172 收藏

tag:blogger.com,1999:blog-8317222231133660547.post-3665650164511052742020-11-18T17:30:00.014-03:002020-11-18T17:30:01.626-03:00Teler - Real-time HTTP Intrusion Detection<div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-3Um6ClDZsp4/X7Nfn7hvM0I/AAAAAAAAUaM/cKbbrPWZTB4pe437iQXkfCqL7GBghokuACNcBGAsYHQ/s800/teler_1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="800" data-original-width="800" height="640" src="https://1.bp.blogspot.com/-3Um6ClDZsp4/X7Nfn7hvM0I/AAAAAAAAUaM/cKbbrPWZTB4pe437iQXkfCqL7GBghokuACNcBGAsYHQ/w640-h640/teler_1.png" width="640" /></a></div><p><br /></p> <p><code>teler</code> is an <strong>real-time intrusion detection</strong> and threat alert based on web log that runs in a <strong>terminal</strong> with resources that we collect and provide by the community.</p><span><a name='more'></a></span><p><br /></p><span style="font-size: x-large;"><b>Features</b></span><br /> <ul> <li> <p><strong>Real-time</strong>: Analyze logs and identify suspicious activity in real-time.</p> </li> <li> <p><strong>Alerting</strong>: teler provides <a href="https://www.kitploit.com/search/label/Alerting" target="_blank" title="alerting">alerting</a> when a threat is detected, <a href="https://www.kitploit.com/search/label/Push%20Notifications" target="_blank" title="push notifications">push notifications</a> include Slack, Telegram and Discord.</p> </li> <li> <p><strong>Monitoring</strong>: We've our own metrics if you want to monitor threats easily, and we use <a href="https://www.kitploit.com/search/label/Prometheus" target="_blank" title="Prometheus">Prometheus</a> for that.</p> </li> <li> <p><strong>Latest resources</strong>: Collections is continuously up-to-date.</p> </li> <li> <p><strong>Minimal configuration</strong>: You can just run it against your log file, write the log format and let teler analyze the log and show you alerts!</p> </li> <li> <p><strong>Flexible log formats</strong>: teler allows any custom log format string! It all depends on how you write the log format in configuration file.</p> </li> <li> <p><strong>Incremental log processing</strong>: Need data persistence rather than <a href="https://linux.die.net/man/1/stdbuf" rel="nofollow" target="_blank" title="buffer stream">buffer stream</a>? teler has the ability to process logs incrementally through the on-disk persistence options.</p> </li> </ul> <br /><span style="font-size: x-large;"><b>Why teler?</b></span><br /> <p>teler was designed to be a fast, terminal-based threat analyzer. Its core idea is to quickly analyze and hunt threats in real time!</p> <br /><span style="font-size: x-large;"><b>Installation</b></span><br /> <br /><span style="font-size: large;"><b>from Binary</b></span><br /> <p>The installation is easy. You can download a prebuilt binary from <a href="https://github.com/kitabisa/teler/releases" rel="nofollow" target="_blank" title="releases page">releases page</a>, unpack and run! or run with:</p> <div><pre><code>▶ curl -sSfL 'https://ktbs.dev/get-teler.sh' | sh -s -- -b /usr/local/bin</code></pre></div> <br /><span style="font-size: large;"><b>using Docker</b></span><br /> <p>Pull the Docker image by running:</p> <div><pre><code>▶ docker pull kitabisa/teler</code></pre></div> <br /><span style="font-size: large;"><b>from Source</b></span><br /> <p>If you have go1.14+ compiler installed and configured:</p> <div><pre><code>▶ GO111MODULE=on go get -v -u ktbs.dev/teler/cmd/teler</code></pre></div> <p>In order to update the tool, you can use <code>-u</code> flag with <code>go get</code> command.</p> <br /><span style="font-size: large;"><b>from GitHub</b></span><br /> <div><pre><code>▶ git clone https://github.com/kitabisa/teler<br />▶ cd teler<br />▶ make build<br />▶ mv ./bin/teler /usr/local/bin</code></pre></div> <br /><span style="font-size: x-large;"><b>Usage</b></span><br /> <p>Simply, teler can be run with:</p> <div><pre><code>▶ [buffers] | teler -c /path/to/config/teler.yaml<br /># or<br />▶ teler -i /path/to/access.log -c /path/to/config/teler.yaml</code></pre></div> <p>If you've built teler with a Docker image:</p> <div><pre><code>▶ [buffers] | docker run -i --rm -e TELER_CONFIG=/path/to/config/teler.yaml teler<br /># or<br />▶ docker run -i --rm -e TELER_CONFIG=/path/to/config/teler.yaml teler --input /path/to/access.log</code></pre></div> <br /><span style="font-size: large;"><b>Flags</b></span><br /> <div><pre><code>▶ teler -h</code></pre></div> <p>This will display help for the tool.</p> <p align="center"><br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-WwDr8KG0Mzs/X7NfxQYIPqI/AAAAAAAAUaQ/44bDUAiWg5suh3Pe91mVue1sALTjxvUMQCNcBGAsYHQ/s800/teler_11.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="631" data-original-width="800" height="504" src="https://1.bp.blogspot.com/-WwDr8KG0Mzs/X7NfxQYIPqI/AAAAAAAAUaQ/44bDUAiWg5suh3Pe91mVue1sALTjxvUMQCNcBGAsYHQ/w640-h504/teler_11.png" width="640" /></a></div><p align="center"><span style="text-align: left;">&nbsp;</span></p> <p>Here are all the switches it supports.</p> <table> <tr> <th>Flag</th> <th>Description</th> <th>Examples</th> </tr> <tr> <td>-c,<br /> --config</td> <td>teler configuration file</td> <td>kubectl logs nginx | teler -c /path/to/config/teler.yaml</td> </tr> <tr> <td>-i,<br /> --input</td> <td>Analyze logs from data persistence rather than buffer stream</td> <td>teler -i /var/log/nginx/access.log</td> </tr> <tr> <td>-x,<br /> --concurrent</td> <td>Set the concurrency level to analyze logs<br />(default: 20)</td> <td>tail -f /var/log/nginx/access.log | teler -x 50</td> </tr> <tr> <td>-o,<br /> --output</td> <td>Save detected threats to file</td> <td>teler -i /var/log/nginx/access.log -o /tmp/threats.log</td> </tr> <tr> <td>--json</td> <td>Display threats in the terminal as JSON format</td> <td>teler -i /var/log/nginx/access.log --json</td> </tr> <tr> <td>--rm-cache</td> <td>Remove all cached resources</td> <td>teler --rm-cache</td> </tr> <tr> <td>-v,<br /> --version</td> <td>Show current teler version</td> <td>teler -v</td> </tr> </table> <br /><b>Config</b><br /> <p>The <code>-c</code> flag is to specify teler configuration file.</p> <div><pre><code>▶ tail -f /var/log/nginx/access.log | teler -c /path/to/config/teler.yaml</code></pre></div> <p>This is <strong>required</strong>, but if you have defined <code>TELER_CONFIG</code> environment you don't need to use this flag, e.g.:</p> <div><pre><code>▶ export TELER_CONFIG="/path/to/config/teler.yaml"<br />▶ tail -f /var/log/nginx/access.log | teler<br /># or<br />▶ tail -f /var/log/nginx/access.log | TELER_CONFIG="/path/to/config/teler.yaml" teler</code></pre></div> <br /><b>Input</b><br /> <p>Need log analysis incrementally? This <code>-i</code> flag is useful for that.</p> <div><pre><code>▶ teler -i /var/log/nginx/access.log</code></pre></div> <br /><b>Concurrency</b><br /> <p>Concurrency is the number of logs analyzed at the same time. Default value teler provide is 20, you can change it by using <code>-x</code> flag.</p> <div><pre><code>▶ teler -i /var/log/nginx/access.log -x 50</code></pre></div> <br /><b>Output</b><br /> <p>You can also save the detected threats into a file with <code>-o</code> flag.</p> <div><pre><code>▶ teler -i /var/log/nginx/access.log -o threats.log</code></pre></div> <br /><b>JSON Format</b><br /> <p>If you want to display the detected threats as JSON format, switch it with <code>--json</code> flag.</p> <div><pre><code>▶ teler -i /var/log/nginx/access.log --json</code></pre></div> <p>Please note this will also apply if you save it to a file with <code>-o</code> flag.</p> <br /><b>Remove Caches</b><br /> <p>It will removes all stored resources in the user-level cache directory, see <a href="https://github.com/kitabisa/teler#cache" rel="nofollow" target="_blank" title="cache">cache</a>.</p> <div><pre><code>▶ teler --rm-cache</code></pre></div> <br /><span style="font-size: x-large;"><b>Configuration</b></span><br /> <p><code>teler</code> requires a minimum of configuration to process and/or log analysis, and execute threats and/or alerts. See <a href="https://github.com/kitabisa/teler/blob/development/teler.example.yaml" rel="nofollow" target="_blank" title="teler.example.yaml">teler.example.yaml</a> for an example.</p> <br /><span style="font-size: large;"><b>Log Formats</b></span><br /> <p>Because we use <code>gonx</code> package to parse the log, you can write any log format. As an example:</p> <br /><b>Apache</b><br /> <div><pre><code>log_format: |<br /> $remote_addr - $remote_user [$time_local] "$request_method $request_uri $request_protocol" $status $body_bytes_sent</code></pre></div> <br /><b>Nginx</b><br /> <div><pre><code>log_format: |<br /> $remote_addr $remote_user - [$time_local] "$request_method $request_uri $request_protocol" <br /> $status $body_bytes_sent "$http_referer" "$http_user_agent"</code></pre></div> <br /><b>Nginx Ingress</b><br /> <div><pre><code>log_format: |<br /> $remote_addr - [$remote_addr] $remote_user - [$time_local] <br /> "$request_method $request_uri $request_protocol" $status $body_bytes_sent <br /> "$http_referer" "$http_user_agent" $request_length $request_time <br /> [$proxy_upstream_name] $upstream_addr $upstream_response_length $upstream_response_time $upstream_status $req_id</code></pre></div> <br /><b>Amazon S3</b><br /> <div><pre><code>log_format: |<br /> $bucket_owner $bucket [$time_local] $remote_addr $requester $req_id $operationration $key <br /> "$request_method $request_uri $request_protocol" $status $error_code $body_bytes_sent - <br /> $total_time - "$http_referer" "$http_user_agent" $version_id $host_id <br /> $signature_version $cipher_suite $http_auth_type $http_host_header $tls_version</code></pre></div> <br /><b>Elastic LB</b><br /> <div><pre><code>log_format: |<br /> $time_local $elb_name $remote_addr $upstream_addr $request_processing_time <br /> $upstream_processing_time $response_processing_time $status $upstream_status $body_received_bytes $body_bytes_sent <br /> "$request_method $request_uri $request_protocol" "$http_user_agent" $cipher_suite $tls_version</code></pre></div> <br /><b>CloudFront</b><br /> <div><pre><code>log_format: |<br /> $date $time $edge_location $body_bytes_sent $remote_addr <br /> $request_method $http_host_header $requst_uri $status <br /> $http_referer $http_user_agent $request_query $http_cookie $edge_type $req_id <br /> $http_host_header $ssl_protocol $body_bytes_sent $response_processing_time $http_host_forwarded <br /> $tls_version $cipher_suite $edge_result_type $request_protocol $fle_status $fle_encrypted_fields <br /> $http_port $time_first_byte $edge_detail_result_type <br /> $http_content_type $request_length $request_length_start $request_length_end</code></pre></div> <br /><span style="font-size: large;"><b>Threat rules</b></span><br /> <br /><b>Cache</b><br /> <p>By default, <code>teler</code> will fetch external resources every time you run it, but you can switch external resources to be cached or not.</p> <div><pre><code>rules:<br /> cache: true</code></pre></div> <p>If you choose to cache resources, it's stored under user-level cache directory of cross-platform and will be updated every day, see <a href="https://github.com/kitabisa/teler#resources" rel="nofollow" target="_blank" title="resources">resources</a>.</p> <br /><b>Excludes</b><br /> <p>We include resources for predetermined threats, including:</p> <ul> <li>Common Web Attack</li> <li>Bad IP Address</li> <li>Bad Referrer</li> <li>Bad Crawler</li> <li>Directory Bruteforce</li> </ul> <p>You can disable any type of threat in the <code>excludes</code> configuration <em>(case-sensitive)</em>.</p> <div><pre><code>rules:<br /> threat:<br /> excludes:<br /> - "Bad IP Address"</code></pre></div> <p>The above format detects threats that are not included as bad IP address, and will not analyze logs/ send alerts for that type.</p> <br /><b>Whitelists</b><br /> <p>You can also add <a href="https://www.kitploit.com/search/label/Whitelists" target="_blank" title="whitelists">whitelists</a> to teler configuration.</p> <div><pre><code>rules:<br /> threat:<br /> whitelists:<br /> - "(curl|Go-http-client|okhttp)/*"<br /> - "^/wp-login\\.php"</code></pre></div> <p>It covers the entire HTTP request and processed as <em>regExp</em>, please write it with caution!</p> <br /><span style="font-size: large;"><b>Notification</b></span><br /> <p>We provide alert notification options:</p> <ul> <li>Slack,</li> <li>Telegram</li> <li>Discord</li> </ul> <p>Configure the notification alerts needed on:</p> <div><pre><code>notifications:<br /> slack:<br /> token: "xoxb-..."<br /> color: "#ffd21a"<br /> channel: "G30SPKI"<br /><br /> telegram:<br /> token: "123456:ABC-DEF1234...-..."<br /> chat_id: "-111000"<br /><br /> discord:<br /> token: "NkWkawkawkawkawka.X0xo.n-kmZwA8aWAA"<br /> color: "16312092"<br /> channel: "700000000000000..."</code></pre></div> <p>You can also choose to disable alerts or want to be sent where the alerts are.</p> <div><pre><code>alert:<br /> active: true<br /> provider: "slack"</code></pre></div> <br /><span style="font-size: large;"><b>Metrics</b></span><br /> <p><code>teler</code> also supports metrics using Prometheus.</p> <br /><b>Prometheus</b><br /> <p>You can configure the host, port and endpoint to use Prometheus metrics in the configuration file.</p> <div><pre><code>prometheus:<br /> active: true<br /> host: "localhost"<br /> port: 9099<br /> endpoint: "/metrics"</code></pre></div> <p>Here are all the metrics we collected &amp; categorized.</p> <table> <tr> <th>Metric</th> <th>Description</th> </tr> <tr> <td><code>teler_threats_count_total</code></td> <td>Total number of detected threats</td> </tr> <tr> <td><code>teler_cwa</code></td> <td>Get lists of Common Web Attacks</td> </tr> <tr> <td><code>teler_badcrawler</code></td> <td>Get lists of Bad Crawler requests</td> </tr> <tr> <td><code>teler_dir_bruteforce</code></td> <td>Get lists of Directories Bruteforced</td> </tr> <tr> <td><code>teler_bad_referrer</code></td> <td>Get lists of Bad Referrer requests</td> </tr> <tr> <td><code>teler_badip_count</code></td> <td>Total number of Bad IP Addresses</td> </tr> </table> <br /><br /><span style="font-size: large;"><b>Resources</b></span><br /> <p>All external resources used in this teler are <strong>NOT</strong> provided by us. See all peoples who involved in this resources at <a href="https://github.com/kitabisa/teler-resources" rel="nofollow" target="_blank" title="teler Resource Collections">teler Resource Collections</a>.</p> <br /><br /><br /><div style="text-align: center;"><b><span style="font-size: x-large;"><a class="kiploit-download" href="https://github.com/kitabisa/teler" rel="nofollow" target="_blank" title="Download Teler">Download Teler</a></span></b></div>Zion3R[email protected]

文章来源: http://www.blogger.com/feeds/8317222231133660547/posts/default/366565016451105274
如有侵权请联系:admin#unsafe.sh