How to find End-point URL in Javascript with LinkFinder

@ZracheSs-AnasZ 의 트윗을 보다가 아주 괜찮은 툴을 알게되어 공유합니다. I’ve seen the tweet from @ZracheSs-AnasZ yesterday, and I got to know awesome tool and I share it now

LinkFinder?

이름 그대로 링크를 찾아주는 도구인데, 일반적인 크롤러와 다른점은 Javascript 내부에 정의된 URL 패턴도 찾아준다는 겁니다. It’s tool for finding link(url), but what’s different from a typical crawler is it also looks for URL patterns defined inside Javascript.

보통 버그바운티 대상 사이트들은 굉장히 많은 테스팅을 거쳐서 단단하기 때문에 취약점이 쉽게 나오진 않습니다. 특히 뻔한 주소, 뻔한 취약점일 수록 중복 확률도 매우 높죠. Usually, bugbounty sites are hard after a lot of testing, so they’re not easily vulnerable. The common urls, the common the vulnerability is the more likely duplicated

저 또한 느끼고 있고 다른 해커들도 동일하게 생각하는 점은 Javascript 를 꼼꼼하게 봐야한다는 점입니다. Js에서 deprecated 된 주소를 찾을수도 있고 다른 사람이 아직 식별하지 못한 URL들을 찾아낼 수 있습니다. What I feel and what other hackers think the same is that you have to look at Javascript carefully. Js can find addresses that have been de-preceded or URLs that others have not yet identified.

그래서, 이 툴이 좋은 이유입니다. So this tool is awesome!

(제가 예전에 SMT Solver 보고 Js단에서 흐름을 추적하는걸 만들어보고 싶단 생각을 했었는데, 어찌보면 이런쪽으로도 활용해볼 여지가 있었네요.. 그러나 어차피 못만듬..)

How to Install?

git clone https://github.com/GerbenJavado/LinkFinder
cd LinkFinder
python3 setup.py install

finish!

python3 linkfinder.py

Options

linkfinder -h
usage: linkfinder.py [-h] [-d] -i INPUT [-o OUTPUT] [-r REGEX] [-b]
                     [-c COOKIES]

optional arguments:
  -h, --help            show this help message and exit
  -d, --domain          Input a domain to recursively parse all javascript
                        located in a page
  -i INPUT, --input INPUT
                        Input a: URL, file or folder. For folders a wildcard
                        can be used (e.g. '/*.js').
  -o OUTPUT, --output OUTPUT
                        Where to save the file, including file name. Default:
                        output.html
  -r REGEX, --regex REGEX
                        RegEx for filtering purposes against found endpoint
                        (e.g. ^/api/)
  -b, --burp
  -c COOKIES, --cookies COOKIES
                        Add cookies for authenticated JS files

Scan endpoint url in Javascript

linkfinder -i https://pagead2.googlesyndication.com/pagead/js/r20191205/r20190131/show_ads_impl_fy2019.js -o cli
text/javascript
https://pagead2.googlesyndication.com/getconfig/sodar
//tpc.googlesyndication.com/sodar/%{basename}.js
http://googleads.g.doubleclick.net
http://pagead2.googlesyndication.com
https://googleads.g.doubleclick.net
https://pagead2.googlesyndication.com
amp4ads-host-v0.js
https://cdn.ampproject.org/
//fonts.googleapis.com/css
text/css
https://pagead2.googlesyndication.com/pagead/gen_204
/pagead/gen_204?id=
/images/icons/material/system/2x/close_white_24dp.png
//www.google.com/settings/ads/anonymous
/pagead/js/logging_library.js
/r20100101
//www.googletagservices.com/activeview/js/current/osd.js?cb=
//pagead2.googlesyndication.com/pagead/gen_204?id=osd&amp;r=om
/r20190131
//www.google.com/adsense/search/ads.js
https://www.gstatic.com/adsense/autoads/icons/gpp_good_24px_blue_600.svg
https://www.gstatic.com/adsense/autoads/icons/close_24px_grey_700.svg
https://fundingchoicesmessages.google.com/uf/%{externalId}
/pagead/js/
/r20190131/reactive_library.js
/r20190131/debug_card_library.js
/pagead/js/adsbygoogle.js
https://pagead2.googlesyndication.com/pagead/gen_204?id=imerr
/pagead/html/
/r20190131/zrt_lookup.html#
/pagead/ads?
/pagead/lopri?
/pagead/blank.gif#?
/pagead/js/r20191205/r20190131/rum.js
/pagead/js/r20191205/r20190131/creativetoolset/xpc_expansion_embed.js

Scan endpoint url in Javascript on Target domain

linkfinder -i https://www.hahwul.com -d -o cli | more
Running against: https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js

process/browser.js
./auth.js
../vendor/es6-promise.js
../vendor/easyXDM.js
./util.js
./common.js
./api.authType
/v1/api/story/upload/multi
/v2/api/talk/message/image/upload
/v1/user/signup
....
https://www.hahwul.com/2018/08/install-kakaotalk-on-ubuntu-18.04.html
https://www.hahwul.com/2019/10/find-subdomain-takeover-with-amass-and-subjack.html
https://www.hahwul.com/2018/08/git-credential-helper.html
https://www.hahwul.com/2017/08/hacking-frida-hooking-to-multi-platform.html
https://www.hahwul.com/2016/08/debian-apt-get-could-not-get-lock.html
https://www.hahwul.com/2016/02/coding-git-push-push-error-failed-to.html
https://www.hahwul.com/2016/04/coding-git-pull-pull.html
https://www.hahwul.com/2019/11/how-to-diable-detectportal-firefox.html
https://www.hahwul.com/search/label/Hacking
https://www.hahwul.com/search/label/Web%20Hacking
https://www.hahwul.com/search/label/%23Hacking
https://www.hahwul.com/search/label/Coding

Output options

기본적으론 html output을 지원하고 -o 옵션 중 cli 를 통해 console output도 뿌려줄 수 있습니다. 개인적으론 pipeline 하기 위해선 cli 옵션쪽이 좀 더 현명한 선택 같네요 :)

Conclusion

아무튼 Javascript를 유심히 살펴보는건 웹이나 모바일(웹뷰, 인앱브라우저 등?)에서도 굉장히 중요한 작업니다. 슬슬 Js를 심도있게 분석해줄 수 있는 도구의 필요성이 점점 느껴지네요.