Amass, go deep in the sea with free APIs

Amass, go deep in the sea with free APIs

There are several types of Subdomains scanning tools. Amass, Subfinder, findomain, etc… In my opinion, the tool at its peak is Amass, and many Bugbounty hunters have automated systems through Amass. Today I’m going to talk about ways to expand Amass’ datasources and get more results.

Subdomains 스캐닝 도구에는 여러가지가 있습니다. 대표적으로 Amass, Subfinder, findomain, etc… 개인적인 생각으론 정점에 있는 도구는 Amass 이며, 많은 버그바운티 헌터들이 Amass를 통한 자동화 시스템을 가지고 있습니다.

오늘은 Amass의 datasources를 확장을 통해 좀 더 많은 결과를 얻어오기 위한 방법에 대해 이야기하려고 합니다.

Amass datasource

All of the tools mentioned above call multiple APIs to get a list of subdomains quickly. Of course, it has its own testing function, but as a result, it’s a necessary process to get a lot of domains in a short time. Amass also has this logic and supports a number of services.

위에서 이야기한 도구들 모두 여러 API를 호출하여 서브도메인 리스트를 빠르게 얻어옵니다. 물론 자체적인 테스팅 기능도 갖추고 있지만, 결과적으로 단시간에 많은 도메인을 확보하기 위해선 꼭 필요한 과정이죠. Amass 또한 이러한 로직을 가지고 있고 여러가지 서비스들을 지원하고 있습니다.

Know yourself. default available datasources

​Know yourself

from Socrates

First, we need to know our Amass status. The currently installed Amass version is v3.10.3, and the currently active datasources can be obtained through the amass enum -list command.

먼저 우리의 Amass 상태를 알아야합니다. 현재 설치된 Amass 버전은 v3.10.3이며 amass enum -list 명령을 통해 현재 활성화된 datasources 를 얻어올 수 있습니다.

$ amass -version
v3.10.3

​ default available apis

$ amass enum -list
Data Source               | Type                    | Available
--------------------------------------------------------------------------------
AlienVault                  api                         *
Alterations                 alt                         *
ArchiveIt                   archive                     *
Ask                         scrape                      *
Baidu                       scrape                      *
BinaryEdge                  api
Bing                        scrape                      *
Brute Forcing               brute                       *
BufferOver                  api                         *
BuiltWith                   scrape                      *
C99                         api
CIRCL                       api
Censys                      cert                        *
CertSpotter                 cert                        *
Chaos                       api
Cloudflare                  api                         *
CommonCrawl                 api                         *
Crtsh                       cert                        *
DNSDB                       api
DNSDumpster                 scrape                      *
FacebookCT                  cert
GitHub                      api
GoogleCT                    cert                        *
HackerOne                   scrape                      *
HackerTarget                api                         *
IPToASN                     api                         *
LoCArchive                  archive                     *
Mnemonic                    api                         *
NetworksDB                  scrape                      *
PassiveTotal                api
Pastebin                    api                         *
RADb                        api                         *
RapidDNS                    scrape                      *
ReconDev                    api
Riddler                     scrape                      *
Robtex                      api                         *
SecurityTrails              api
ShadowServer                api                         *
Shodan                      api
SiteDossier                 scrape                      *
Spyse                       api
Sublist3rAPI                api                         *
TeamCymru                   api                         *
ThreatCrowd                 api                         *
ThreatMiner                 api                         *
Twitter                     api
UKGovArchive                archive                     *
URLScan                     api                         *
Umbrella                    api
ViewDNS                     scrape                      *
VirusTotal                  api                         *
Wayback                     archive                     *
WhoisXML                    none
Yahoo                       scrape                      *
ZETAlytics                  api
ZoomEye                     api

find unabled apis​..

이제 활성화되지 않은 api 들만 뽑아봅시다.

$ amass enum -list | grep -v "\*"
Data Source                        | Type                             | Available
--------------------------------------------------------------------------------
BinaryEdge                           api
C99                                  api
CIRCL                                api
Chaos                                api
DNSDB                                api
FacebookCT                           cert
GitHub                               api
PassiveTotal                         api
ReconDev                             api
SecurityTrails                       api
Shodan                               api
Spyse                                api
Twitter                              api
Umbrella                             api
WhoisXML                             none
ZETAlytics                           api
ZoomEye                              api

​ 17 datasource is disabled, and let’s go enable it. 17개의 datasource가 비활성화 상태이고, 이를 활성화 시켜봅시다.

Amass -config flag and automatically tries to discover config file

It’s a good idea to know an option before looking at how to apply it. This is the -config flag, and you can define and use Amass settings in advance. You can also specify API Token here, so you can eliminate the hassle of giving an option every time you scan.

적용 방법을 알아보기 전에 옵션을 하나 알고가면 좋습니다. -config flag 이고 미리 Amass의 설정을 정의하고 사용할 수 있는데, 여기에 API Token 또한 지정할 수 있어서, 매번 스캐닝 시 옵션을 주어야하는 번거로움을 없앨 수 있습니다. ​

$ amass enum -d target.com -config ~/.conf/your_amass_conf.ini

https://github.com/OWASP/Amass/blob/master/examples/config.ini

The path below is the default config that references without the -config option. I think recommend using config separately for services with api limit.

그리고 아래 경로에 config 생성 시 -config 옵션이 없어도 해당 config를 참조하게 됩니다. 저는 개인적으로 api limit이 있는 서비스를 위해서 config를 분리해서 사용하는걸 추천합니다.

Operating System Path
Linux / Unix $XDG_CONFIG_HOME/amass/config.ini or $HOME/.config/amass/config.ini
Windows %AppData%\amass\config.ini
OSX $HOME/Library/Application Support/amass/config.ini

Price and limit?

Summarize the price and limit of the remaining inactive APIs, it looks like this. 우선 나머지 비활성화 되어있던 APIs들을 가격과 Limit을 정리해보면 이렇습니다. ​

Name Price Memo
BinaryEdge - Free (Free)
- $10/Month (Starter)
- $500/Month (Business)
- X (Enterprise)
https://www.binaryedge.io/pricing.html
C99 - $5/Month
- $25/Year
http://api.c99.nl/dashboard/shop
CIRCL    
Chaos - Free https://chaos.projectdiscovery.io/#/
but contributed nuclei-template or public-bugbounty-programs
DNSDB - Free (500q/Month) https://www.farsightsecurity.com/get-started/
FacebookCT - Free https://developers.facebook.com/docs/certificate-transparency-api/
Github - Free  
PassiveTotal - Free https://community.riskiq.com/
ReconDev - Free (30q/Month)
- 1$/10q
- 20$/Month (200q/Month)
https://recon.dev/pricing.html
SecurityTrails - Free (50q/Month)
- $50/Month (1500q/m)
- $500/Month (20000q/m)
- $1500/Month (65000q/m)
https://securitytrails.com/corp/pricing#api
Shodan - 7$ (Event!)
- $59/Month (Freelancer)
- $299/Month (business)
- $899/Month (corp)
https://developer.shodan.io/pricing
Spyse - Free
- $50/Month (Standard)
- $250/Month (Pro)
https://spyse.com/user/subscription
Twitter - Free  
Umbrella   https://umbrella.cisco.com/
WhoisXML - Free (500q/Month)
- $15/Month (2000q/Month)
~
- $1800/Month (2000000/m)
https://whois.whoisxmlapi.com/pricing
ZETAlytics - $199/Month (4000q/m) https://zetalytics.com/#pricetable
ZoomEye - Free (10000/Month) https://www.zoomeye.org/business

All Free? and how to get key

Chaos , FacebookCT, Github, PassiveTotal, Twitter, ZoomEye ​

Chaos

​contribute nuclei-template or public-bugbounty-programs and get key. I am an early Chaos beta user, so I already have a api key

Chaos는 현재 nuclei 템플릿과 pbp에 컨트리뷰트 하는 사용자에게만 키를 발급해주고 있습니다. 이는 어려운게 아니기 때문에 컨트리뷰트하고 키를 받아서 사용하시면 됩니다. 물론 Amass API가 아니여도 Chaos는 정말 좋은 도구에요 :D (아 저는 베타 초기 사용자라 따로 키를 받았습니다.)

[data_sources.Chaos]
ttl = 4320
[data_sources.Chaos.Credentials]
apikey = 89

FacebookCT

Open https://developers.facebook.com and ​Sign in facebook account(developer) ​ Go apps

First, create app

Create app > Your app page

Get apikey

setting > advance setting > security > client token

Get Secret

setting > basic setting > app secret code1414

[data_sources.FacebookCT]
ttl = 4320
[data_sources.FacebookCT.app1]
apikey = your_api_key
secret = yuur_secret_key

Github

Open your github setting tokens

Generate Token > if you added token, get token string

[data_sources.GitHub]
ttl = 4320
[data_sources.GitHub.accountname]
apikey = your_api_key

Amass simply use search queries, so we don’t need to add additional permissions.

단순하게 검색 쿼리만 사용하는거라 추가 권한은 안넣어줘도 되는 것 같습니다. ​

PassiveTotal

https://community.riskiq.com/settings

Profile > User > Secret

[data_sources.PassiveTotal]
ttl = 10080
[data_sources.PassiveTotal.Credentials]
username = your_name
apikey = your_api_key

Twitter

​https://developer.twitter.com/en/apps

Create an app > Choose any type > If confirmed app, get key!

[data_sources.Twitter]
[data_sources.Twitter.account1]
apikey = your_api_key
secret = your_api_secret

1415

ZoomEye

​I don’t think I can tell you in detail because most of this datasource is in Chinese. I can’t speak Chinese.and I don’t use ZoomEye datasource either.

Shodan

Shodan sometimes put a big discount. Aim for that moment!

[data_sources.Shodan]
ttl = 10080
[data_sources.Shodan.Credentials]
apikey = your_api_key

Finally, let’s apply it!

Add your config data / 설정 넣어주면 잘 나옵니다. 이후 스캔부턴 해당 datasource를 사용합니다.

$ vim ~/.config/amass/config.ini
$ amass enum -list -config ~/.config/amass/config.ini

1416

Other source? and conclusion

In fact, there are many other free datasource. However, API limit is short, so it is not included, but if you make and use various configs depending on the conditions, you can get good scan results depending on the situation.

사실 다른 무료 소스들도 많이 있습니다. 다만 API limit이 짧아서 포함하진 않았는데요, 조건에 따라서 여러가지 config를 만들고 정해서 사용하면 상황에 따라 좋은 스캔 결과를 얻을 수 있습니다.

e.g

for automation scanning

$ amass enum -d [TARGET] -config ~/.config/amass/auto.ini

for manual scanning

$ amass enum -d [TARGET] -config ~/.config/amass/all.ini