Mlget
Mlget is designed to fetch malware from a variety of sources so that you don’t have to manually hunt for a hash everywhere.
| Source | Releases |
| License Type | MIT License |
| Version This ReadMe Matches | v3.4.1 |
Sources
These are the services that Mlget can query. The version number corresponds to the version release number on github.
| Service | Added in Version | MD5 | SHA1 | SHA256 | Flag | Notes | |
|---|---|---|---|---|---|---|---|
| 1 | Assembly Line | v3.2.1 | x | x | x | al | |
| 2 | Cape Sandbox | v1.1 | x | x | x | cs | Sometimes hangs when querying the public instance due to Cloudflare |
| 3 | FileScan IO | v2.5 | x | x | x | fs | |
| 4 | Hybrid Analysis | v1.1 | x | x | x | ha | |
| 5 | Inquest Labs | v1.1 | x | x | x | iq | |
| 6 | Joe Sandbox | v1.2 | x | x | x | js | Works for paid instances only |
| 7 | Malpedia | v2.4 | x | x | mp | ||
| 8 | Malshare | v1.1 | x | x | x | ms | |
| 9 | Malware Bazaar | v1.1 | x | x | x | mb | |
| 10 | MWDB | v1.1 | x | x | x | mw | |
| 11 | Objective See | v2.3 | x | os | List is redownloaded upon each run. Recommend not adding unless needed. | ||
| 12 | Polyswarm | v1.1 | x | x | x | ps | |
| 13 | Triage | v1.1 | x | x | x | tr | Sometimes the hash is found in the analysis archive. Mlget will extract, identify, and save the file. The archive along with the rest of the files extracted are deleted. |
| 14 | UnpacMe | v2.4 | x | x | x | um | |
| 15 | URLScan IO | v.3.0.0 | x | us | |||
| 16 | Virus Exchange | v3.4.0 | x | ve | VXUnderground’s malware repository | ||
| 17 | Virus Total | v1.1 | x | x | x | vt | |
| 18 | VxShare | v2.5 | x | x | x | vx | This endpoint is heavily rate limited, so if downloading in bulk expect to hit the limit quickly |
For the best mileage, use sha256 hashes.
Mlget also has the ability to upload to MWDB and AssemblyLine. Those are configured separately from the download sources.
Additional Features Baked In
- Auto archive extraction
- Will delete the archive upon extraction
- Will automatically hash the file downloaded to make sure they match.
- If not a match, will delete the downloaded file and try the next service
Basic Usage
Setup
Adding services to query to the config
mlget --addtoconfig
The file created can be found at ~/.mlget.yml
Dumping the config to the console
mlget --config
Downloading Sample(s)
Download a single hash
mlget <hash>
Downloading multiple hashes
mlget <hash> <hash> <hash>
Pass in a file of hashes (one has per line) to download
mlget --read <filename>
Read in a file over the Internet containing a list of hashes
This works great for companies like ESET who put their hashes in files on github.
mlget --read https://raw.githubusercontent.com/eset/malware-ioc/master/king_tut/samples.sha256
Specify the Download Source
mlget --from [al,cs,fs,ha,iq,js,mp,ms,mb,mw,os,ps,tr,um,us,ve,vt,vx] <hash>
or
mlget --from [al,cs,fs,ha,iq,js,mp,ms,mb,mw,os,ps,tr,um,us,ve,vt,vx] --read <filename>
Advance Usage
Download hashes from a file and record the ones not found into another file
mlget --read <filename> --output
Download hashes from a file and have the file updated to leave just the ones not found
mlget --readupdate <filename>
Check current directory for the hash(es) prior to querying the different web services
If the hash is found locally, it will skip querying the web services
mlget --precheckdir <hash>
Don’t extract the downloaded file
This will cause the hash validation routine not to run.
mlget --noextraction <hash>
To force mlget to query the MWDB upload instance as well for the hash(es)
Mlget will not query UploadMWDB or UploadAssemblyLine unless this flag is set.
mlget --downloadonly <hash>
Sample Uploading
After adding UploadMWDB and/or UploadAssemblyLine to the config, mlget can upload the downloaded files to MWDB and/or AssemblyLine (respectively).
Uploading to Assembly Line
mlget --uploadal <hash>
Force the file to be resubmitted to Assembly Line if it already exists
mlget --uploadal --f <hash>
Upload to MWDB
mlget --uploadmwdb <hash>
Upload to MWDB with tags
mlget --uploadmwdb --tag <tag0> --tag <tag1> <hash>
Upload to MWDB with comments
mlget --uploadmwdb --comment <comment0> --comment <comment1> <hash>
Future Work
I’m alway looking for additional services to add to this. Happy to take suggestions. Please submit bug/feature requests to the github project.
Appendix
- any.run does not appear to support sample downloading at this time; if this changes, please reach out.
- If someone has VMRay access and is willing to test, let me know and I’ll add it.
