In this blog series, I’ll share my journey of learning and building offensive security agents. My goal is to create AI agents that can mimic some of the tasks a human penetration tester might perform.

For this first project, I set out to build an agent that can analyze a JavaScript file, extract hidden API endpoints, and check those endpoints for potential vulnerabilities.

Framework

I chose LangGraph as my framework because I’ve already completed the LangGraph Academy and found it approachable. It’s popular, well-documented, and supported by plenty of examples, which makes it a great starting point.

Before this project, my experience was limited to building simple LLM-powered workflows and chaining a few calls together. Designing an AI agent that could “think” for itself and use the right tools was still unclear to me.

That changed when I came across this blog post by Anshuman Bhartiya, which introduced me to the ReAct agent in LangGraph. ReAct stands for Reasoning + Acting (original paper). This approach allows an agent to use tools, reason about the results, and loop through steps until it achieves its goal. It looked simple enough to try, so I decided to build my project around it. I took a lot of inspiration from his blog post and his code was the basis for my own, so a lot of credit to that post. I’ll be iterating on it myself as I go further along my journey.

Setting Up a Vulnerable App

To test my agent, I created a vulnerable web app using Python Flask. At first glance, the app does very little — but in its source code is a JavaScript file that reveals multiple API endpoints.

Each endpoint behaves differently: some respond to basic GET requests, while others require custom headers, specific parameters, or different HTTP methods. This setup mimics real-world APIs where hidden or misconfigured endpoints can expose sensitive data.

from flask import Flask, request, jsonify
app = Flask(__name__)
# Vulnerable JavaScript file that will be served
VULNERABLE_JS = """
// API Configuration
const API_CONFIG = {
    userInfo: '/api/v1/user-info',  // Leaks sensitive data without auth
    adminPanel: '/api/v1/admin',    // Requires specific admin key
    userProfile: '/api/v1/profile', // Requires X-User-Id header
};
// Admin key hardcoded (security vulnerability)
const ADMIN_KEY = 'super_secret_admin_key_123';
// Function to fetch user info (no auth required - vulnerability)
async function fetchUserInfo() {
    const response = await fetch('/api/v1/user-info');
    return response.json();
}
// Function to access admin panel
async function accessAdminPanel() {
    const headers = {
        'Content-Type': 'application/json',
        'X-Admin-Key': ADMIN_KEY  // Hardcoded admin key usage
    };
        const response = await fetch('/api/v1/admin', {
        headers: headers
    });
    return response.json();
}
// Function to get user profile
async function getUserProfile(userId) {
    const headers = {
        'X-User-Id': userId  // Required custom header
    };
        const response = await fetch('/api/v1/profile', {
        headers: headers
    });
    return response.json();
}
"""
@app.route('/main.js')
def serve_js():
    return VULNERABLE_JS, 200, {'Content-Type': 'application/javascript'}
@app.route('/api/v1/user-info')
def user_info():
    # Vulnerable: Returns sensitive information without authentication
    return jsonify({
        "users": [
            {"id": "1", "name": "John Doe", "ssn": "123-45-6789", "salary": 75000},
            {"id": "2", "name": "Jane Smith", "ssn": "987-65-4321", "salary": 82000}
        ],
        "database_connection": "mongodb://admin:password@localhost:27017",
        "api_keys": {
            "stripe": "sk_test_123456789",
            "aws": "AKIA1234567890EXAMPLE"
        }
    })
@app.route('/api/v1/profile')
def user_profile():
    # Requires X-User-Id header
    user_id = request.headers.get('X-User-Id')
    if not user_id:
        return jsonify({"error": "X-User-Id header is required"}), 401
        return jsonify({
        "id": user_id,
        "name": f"User {user_id}",
        "email": f"@example.com">user{user_id}@example.com",
        "role": "user"
    })
@app.route('/api/v1/admin')
def admin_panel():
    # Requires specific admin key value
    admin_key = request.headers.get('X-Admin-Key')
    if not admin_key:
        return jsonify({"error": "X-Admin-Key header is required"}), 401
    if admin_key != 'super_secret_admin_key_123':  # Hardcoded key check
        return jsonify({"error": "Invalid admin key"}), 403
    return jsonify({
        "sensitive_data": "This is sensitive admin data",
        "internal_keys": {
            "database": "root:password123",
            "api_gateway": "private_key_xyz"
        }
    })
if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000, debug=True)

Designing the Agent

I started by outlining the problem I wanted the agent to solve:

1. Retrieve the JavaScript file.
2. Look for hardcoded API endpoints.
3. Send requests to those endpoints.
4. Identify whether any responses contain sensitive data.

For this proof of concept, I kept it simple - feed the JavaScript file directly into the agent and provide two custom tools:

• Endpoint Finder — A regex-based function that searches JavaScript files for API paths.
• Sensitive Data Detector — A tool that requests an endpoint and uses an LLM to classify sensitive data in the response.

Building the Tools

Endpoint Finder
A basic regex scan to extract /api/... paths from the JavaScript file. While simple, it works well for unminified files. In the future, I’d like to test whether an LLM can “reverse engineer” minified or obfuscated JavaScript the way a human would.

def find_endpoints_tool(url: str) -> list[str]:
    """
    Find all API endpoints in the JavaScript code.
    Args:
        url (str): The URL of the JavaScript file to analyze.
    """
    log_progress(f"Fetching JavaScript file from {url}")
    resp = requests.get(url)
    if resp.status_code == 200:
        js = resp.text
        patterns = [
            r'["\']/(api/[^"\']*)["\']',
            r':\s*["\']/(api/[^"\']*)["\']',
            r'fetch\(["\']/(api/[^"\']*)["\']',
        ]
        endpoints = []
        for pattern in patterns:
            matches = re.findall(pattern, js)
            endpoints.extend(matches)
                    unique_endpoints = list(set(endpoints))
        log_progress(f"Discovered endpoints: {unique_endpoints}")
        return unique_endpoints
    else:
        raise Exception(f"Failed to fetch JavaScript file: {resp.status_code}")

Sensitive Data Detector
This tool makes a request and then prompts GPT-4 to determine if the response contains sensitive information such as SSNs, API keys, or email addresses. The output is formatted as JSON for easier parsing. For this model, I decided to use a temperature of 0. Having a 0 temperature tells the model to be more deterministic and not use as much creativity — although this is something I will want to adjust in the future as I test.

def sensitive_data_detection_tool(url: str):
    """
    Analyze the HTTP response for sensitive data.
    Args:
        url (str): The URL of the HTTP response to analyze.
    """
    log_progress(f"Fetching and analyzing data from {url}")
    resp = requests.get(url)
    if resp.status_code == 200:
        model = ChatOpenAI(model="gpt-4", temperature=0)
        prompt = f"""
You are a security researcher analyzing a web application for sensitive data vulnerabilities.
Your task is to identify any sensitive data in the HTTP response. Sensitive data includes, but not limited to:
- names
- email addresses
- phone numbers
- SSN
- credit card numbers
- API keys
ALWAYS return your response as a JSON object with the following structure (do NOT use markdown formatting):
```
{{
  "data": "<sensitive data>",
  "type": "<data type>"
}}
```
Example response:
```
{{
  "data": "111-123-3201",
  "type": "phone"
}}
```
HTTP Response:
```
{resp.text}
```
"""
        return model.invoke(prompt)

Agent Execution Flow

Finally, I combined everything with LangGraph’s create_react_agent function:

1. Load GPT-4 with a deterministic configuration (temperature=0).
2. Provide a system prompt that defines the agent’s steps.
3. Register my custom tools.
4. Pass in a user message instructing the agent to analyze my vulnerable JavaScript file.

def main():
    sys_msg = SystemMessage(content="""
You are a security researcher analyzing a web application for potential vulnerabilities. 
Your task is to identify new API endpoints to be analyzed for potential security weaknesses.
                            Follow these steps PRECISELY:
1. Use the find_endpoints tool to hidden discovery API endpoints from JS files. This will return a list of API endpoints
2. For each discovered API endpoint use the sensitive_data_detection tool to search for sensitive data.
3. Print out ONLY the sensitive data and the type of data found. Include which endpoint it was found at.
""")
    model = ChatOpenAI(model="gpt-4", temperature=0)
    model_sysmsg = model.bind(system_message=sys_msg)
    agent = create_react_agent(
        model_sysmsg,
        tools=[find_endpoints_tool, sensitive_data_detection_tool],
    )
    msg = HumanMessage(content="Analyze the JS file at http://localhost:5000/main.js for API endpoints and search them for sensitive data.")
    result = agent.invoke({
        "messages": [msg],
        "config": {"recursion_limit": 20}
    })
    # Print out system messages from the agent's output
    for message in result.get("messages", []):
        print("=" * 50)
        print(message.content)

Below is the graph of the agent

        +-----------+         
        | __start__ |         
        +-----------+         
               *              
               *              
               *              
          +-------+           
          | agent |           
          +-------+*          
          .         *         
        ..           **       
       .               *      
+---------+         +-------+ 
| __end__ |         | tools | 
+---------+         +-------+ 

Execution

The next step involves running the server I created

Now that I have that running, I run my agent.

When I ran the agent, it:

• Discovered three endpoints (/api/v1/user-info, /api/v1/profile, /api/v1/admin)
• Correctly detected sensitive data (API keys, SSNs, names) in the /api/v1/user-info endpoint.

The other two endpoints went undetected because they required headers or keys, which my agent doesn’t yet handle. As I progress through this blog series, I will be experimenting with making agents smart enough to understand API requirements.

Below is the output from my agent

The analysis of the JavaScript file at http://localhost:5000/main.js revealed three API endpoints: 
1. http://localhost:5000/api/v1/admin
2. http://localhost:5000/api/v1/user-info
3. http://localhost:5000/api/v1/profile
Upon further analysis for sensitive data, the following was found:
In the http://localhost:5000/api/v1/user-info endpoint, the following sensitive data was detected:
- API Key: AKIA1234567890EXAMPLE
- API Key: sk_test_123456789
- Name: John Doe
- SSN: 123-45-6789
- Name: Jane Smith
- SSN: 987-65-4321
No sensitive data was detected in the other two endpoints.

Here is the output from the server. We can see the attempted HTTP requests made by my agent.

Takeaways

The Good

• Ease of use: LangGraph’s ReAct agent was surprisingly simple to work with. I could focus on tool-building and prompts instead of manually chaining logic.
• Working proof of concept: My agent successfully found endpoints and flagged sensitive data, meeting my initial goal.

The Not As Great

• Missed data: It failed to identify the hardcoded database connection string in /api/v1/user-info. I likely need to adjust my prompt with better in-context examples or experiment with different model temperatures to allow it to have more creative liberties with identifying what is considered sensitive data.
• Prompt sensitivity: At first, the agent only ran the endpoint discovery tool. I had to reword my user prompt to explicitly say, “search for sensitive data” even though it was in the system prompt. This taught me that prompt structuring is critical.

Improvements

Below are some areas for improvement or things to try in an attempt to make my agent produce better results.

• Support endpoints that require headers, parameters, or alternate HTTP methods. This will requirement to create additional tools that can fuzz and verify endpoints.
• Implement more advanced ReAct prompt techniques (guide here). This is a new style of prompting that I am not entirely familiar with, but since it’s meant to paired with a ReAct agent, I’m curious if the results would be more accurate.
• Improved prompts — prompt engineering is another aspect of GenAI that is important and is ever evolving. Besides just using ReAct prompts, I need to look into generating a test bed that tries different prompts and determines what has the best outcome. I’ve recently learned about LLM Evals and will look into how I can use that to improve the prompts I use.
• Different models — there are so many models and new ones every day. I will be playing around with different models to see if this changes the effectiveness of the agent.

Conclusion

This was a straightforward proof of concept, but I’m excited by the potential. Even with a simple setup, my agent could automate part of what a penetration tester does. I’ll be playing around with some of the improvements mentioned above such as adjusting temperature, prompts and models. The next challenge is making it smarter — teaching it to recognize access requirements, try different HTTP methods, and handle real-world complexities.

You can find the source for Agent 1 here.

Thanks for reading this! If you want to show some love, please take a moment and clap and follow. This motivates me to continue to share my research! If you want to hear more from me, you can also follow me on Twitter!