How to scan an API server

Any HTTP POST/PUT requests that send JSON data to an API server should also have headers with the string “Content-type: application/json”. When this header field is present, VAddy will scan the parameters in the JSON data. AngularJS and other modern JavaScript frameworks should generate headers in this format, so VAddy can scan their requests without any issues.

If you are developing a mobile or single-page application, you may decide to start building your API server first. How do you scan it with VAddy? We’ll cover this topic next.

VAddy can only scan URLs and parameters that have been registered in advance as crawl data. Ordinarily, you would register crawl data by configuring your web browser to use VAddy’s proxy server and then navigating to different pages on your web application. If you haven’t created a view for a given URL, however, you would instead use a command like curl.

First confirm that you can access your web application with the curl command.

A GET request

curl http://www.example.com/blog/Api/view.json?id=1

A POST request (with JSON data)

curl -v -H "Accept: application/json" -H "Content-type: application/json" 
-X POST -d '{"Articles":{"title":"post_aa", "body":"foobar"}}' 
http://www.example.com/blog/Api/add.json

If you can run commands like these without encountering any errors, the next step is to create a shell script such as the following.

#!/bin/sh
PROXY='--proxy http://54.92.84.100:10080'

## (1)
curl $PROXY http://www.example.com/vaddy-abc12345.html?action=begin

## (2) 
curl $PROXY http://www.example.com/blog/Api/view.json?id=1

## (3)
curl $PROXY -v -H "Accept: application/json" 
-H "Content-type: application/json" 
-X POST -d '{"Articles":{"title":"post_aa", "body":"foobar"}}' 
http://www.example.com/blog/Api/add.json

## (4)
curl $PROXY http://www.example.com/vaddy-abc12345.html?action=commit

Notice that you first configure the PROXY variable with the IP address and port number of VAddy’s proxy server so that all HTTP requests will pass through VAddy’s proxy before accessing your web application. By simply setting this variable equal to an empty string (e.g. PROXY=’’), you can also access your web application directly.

In steps 1 and 4, you access specific URLs to start and stop crawling, respectively, with VAddy. The filename in these URLs (vaddy-abc12345.html above) is unique to your user account. In steps 2 and 3, you add VAddy’s proxy settings to the same curl commands that you first used earlier to confirm that you could access your application.

As your web application grows, you can scan new URLs and parameters by simply adding them to this shell script and then running it, thus updating VAddy’s crawl data. Although you can initiate a scan from VAddy’s admin console, you can also do so via Jenkins and CircleCI using the Jenkins plugin and Web API that we have already provided for you. If you configure your Jenkins server or CircleCI instance to initiate a scan after it has run a shell script containing the curl commands above, you can push code, register crawl data, and scan all at once.

We continue to welcome your feedback and strive to make VAddy a service that you will love to use.