API discovery is a thing. 

Articles on API security often highlight that finding and documenting APIs is crucial for understanding an organization’s attack surface.

Developers have spent years arguing about the best methods to implement this. Mainly because the more interesting proposals come from people who have a vested interest in producing a standard for it. 

Usually, from API tooling vendors. No surprise there.

And no one standard has been widely adopted… yet. 

But I say we shouldn’t wait for API builders and evangelists to figure out the “right way” to do it. Instead, let’s build a framework for using API discovery metadata and create a recon methodology around it.

In this article, I will demonstrate how I used two popular specifications for API discovery metadata to detect and catalog APIs and then weaponized the information for my own recon process during security testing.

I will also introduce you to a new extension I have written called “API Discovery” that taps directly into the Burp Suite Web Vulnerability Scanner to passively detect this API metadata and alert you when it finds anything. 

You can use this as part of your API recon process. I welcome and encourage you to submit pull requests on GitHub to update the extension with support for newer specifications and formats as they are introduced.

With that out of the way, let me show you the approach I used.

API discovery metadata specifications

I have used two specifications for API discovery metadata. 

The first is APIs.json, which is led by Kin Lane from Bloomberg and is being worked on in this GitHub repo.

The second is api-catalog, which is led by Kevin Smith from Vodafone and is being worked on as an IETF draft for consideration in this GitHub repo.

Both specifications provide metadata that we can use to find useful API artifacts, such as the OpenAPI specification documentation, and extract information to help attack the APIs.  

APIs.json

Kin has been working on finding ways to publish API discovery metadata for over a decade. In fact, the GitHub repo goes back that far. There was a gap of several years with no work done, leading people to believe the project was dead.

But it has recently seen a resurgence. 

APIs.json serves as a standardized format for documenting APIs, making them easier to discover and manage. By providing a consistent structure for API metadata, including endpoints, descriptions, and versions, APIs.json facilitates the automation of API discovery and integration processes. 

This standardization helps developers quickly find and understand available APIs, reducing onboarding time and enhancing efficiency. 

It helps us on offense in the very same way. 

The value of APIs.json extends to improve API management and ecosystem integration. It allows API providers to maintain up-to-date documentation in a centralized, machine-readable format, ensuring that consumers always have access to the latest information.

This metadata is usually provided at the root of the API publisher’s server at /apis.json.

You can read the latest version of the format of this metadata here.

The Properties collection inside the APIs collection stores the most useful information. It is where API producers may publish information on their Swagger, OpenAPI, or Postman Collection URLs.     

api-catalog

The IETF’s api-catalog specification, developed by Kevin Smith, aims to provide a standardized method for publishing and discovering API metadata. This specification leverages a well-known URL path, /.well-known/api-catalog, to centralize API descriptions in a machine-readable format. 

Unlike APIs.json, it doesn’t mandate the format for the API catalog document itself; in fact, one of the formats suggested is APIs.json. However, they recommend considering a linkset [RFC9264] of API endpoints and information to facilitate API usage.

It’s this linkset that holds the most promise for recon purposes. The specification recommends three key elements for linkset relations:

1. “service-desc”: Used to link to a description of the API that is primarily intended for machine consumption.
2. “service-doc”: Links to API documentation; this is primarily intended for human consumption.
3. “service-meta”: Points to additional metadata about the API and is primarily intended for machine consumption.

Within the service-meta collection, we can expect to find useful recon data alongside the service-doc, which usually points to API spec docs.

The API Discovery Burp Extension

The API Discovery extension leverages the Burp Web Vulnerability Scanner to identify and extract API discovery metadata from target websites. By scanning for standardized API metadata files, such as /apis.json and /.well-known/api-catalog, the extension automates the discovery process. It analyzes the responses for these well-known paths to map out the API landscape, revealing available endpoints and their corresponding documentation.

This functionality is invaluable for developers, testers, and security professionals. For developers and testers, it provides a clear overview of the APIs a web host exposes, facilitating integration and testing processes. 

For security professionals, it highlights potential attack surfaces by identifying all API endpoints, thus enabling comprehensive security assessments and pentesting. This automated discovery process significantly enhances the efficiency and accuracy of identifying APIs, making it an essential tool for thorough API security testing.

You have to do nothing. It’s tied into the passive scanner in Burp Suite Pro and will automatically issue an advisory in the Dashboard and the Issues pane of the Site Map.  

It will look something like this:

Limitations of API Discovery

Currently, the extension has a few limitations. These include:

• It currently only parses the JSON-formatted version of APIs.json. YAML is a perfectly acceptable format structure supported by the specification, but I haven’t had the time to write a YAML parser for it yet.
• It currently only supports Linksets based on the common structure defined in the IETF draft for api-catalog. I wasn’t looking to write a robust parser to account for all the flexibility the specification allows for. Frankly, I find the fact that the specification doesn’t clearly articulate an API catalog format and structure a weakness of the specification. But that’s a debate for another day.
• The extension has had little exposure to a lot of production API metadata, which makes the parsers still untested and possibly brittle.

That last point is a whopper. We should talk about that.  

API discovery metadata adoption is still in its early stages.

Let me be blunt. The debate over API discovery metadata has been going on for some time now. Kin’s specification for APIS.json has been out for something like a decade now, and api-catalog has been worked on for a few years now.

But you know what? There is very little production API discovery metadata in the field.

Check for yourself. Go Google dork inurl:/apis.json and inurl:/.well-known/api-catalog. Very little is getting indexed.

APIs.json publishes a catalog at APIs.io, but it’s far from discoverable or even useful. And it’s not actually published or maintained by the authors of the actual APIs.

Until the tooling in the CICD pipeline for these API producers includes a generation step for API discovery metadata, I don’t think you need to fret much about it. Install the extension, and let Burp tell you when it starts to see the metadata.  

It’ll come. And you’re ready for it.

Conclusion

API discovery metadata represents a powerful way for identifying and documenting the myriad of APIs that underpin modern web applications. 

By leveraging proposed standards like APIs.json and IETF’s api-catalog, you can systematically map out the API landscape of any given target, uncovering endpoints and associated documentation that may be crucial for both integration and security assessment.

The API Discovery extension for Burp Suite demonstrates how you can weaponize this metadata for API security testing. Automating the detection of this metadata streamlines the process of identifying an API’s potential attack surface during recon. 

As your targets adopt these specifications, tools like the API Discovery extension will help you stay ahead of your work. While the adoption of API discovery metadata is still in its infancy, preparing and being proactive will position you at the forefront of this evolving field.

Install the extension, stay alert, and be ready to leverage the metadata as it becomes more prevalent.

One last thing…

Have you joined The API Hacker Inner Circle yet? It’s my FREE weekly newsletter where I share articles like this, along with pro tips, industry insights, and community news that I don’t tend to share publicly. If you haven’t, subscribe today at https://apihacker.blog

The post Weaponizing API discovery metadata appeared first on Dana Epp's Blog.

*** This is a Security Bloggers Network syndicated blog from Dana Epp's Blog authored by Dana Epp. Read the original post at: https://danaepp.com/weaponizing-api-discovery-metadata