November 12, 2019

VirusTotal APK Malware Detection Data - Week 45: 20191104-201901110

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20191104_20191110.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.69% 0.02% 47580 34 168050 150
Trustlook 99.58% 0.24% 47531 404 167680 199
K7GW 99.47% 0.10% 47478 175 167909 252
Avira 98.93% 0.00% 47220 0 168084 510
McAfee 98.15% 0.01% 46845 18 168066 885
Avast-Mobile 98.07% 0.12% 46807 205 167879 923
ZoneAlarm 96.46% 0.03% 46039 44 168040 1691
Ikarus 96.24% 0.16% 45933 268 167816 1797
F-Secure 95.99% 0.00% 45815 8 168076 1915
AhnLab-V3 94.88% 0.03% 45285 52 168032 2445
Kaspersky 94.82% 0.01% 45258 11 168073 2472
Fortinet 93.96% 0.02% 44849 31 168053 2881
DrWeb 93.78% 0.22% 44763 367 167717 2967
Symantec 93.29% 0.05% 44525 87 167997 3205
Qihoo-360 81.05% 0.02% 38687 41 168043 9043
Sophos 80.91% 0.05% 38620 76 168008 9110
NANO-Antivirus 73.35% 0.05% 35009 87 167997 12721
McAfee-GW-Edition 72.59% 0.00% 34648 3 168081 13082
CAT-QuickHeal 63.62% 0.09% 30368 154 167930 17362
AVG 51.54% 0.05% 24601 77 168007 23129
Avast 50.69% 0.04% 24194 73 168011 23536
MAX 48.11% 0.00% 22964 6 168078 24766
Cyren 46.21% 0.01% 22058 11 168073 25672
Tencent 32.08% 0.00% 15310 0 168084 32420
Rising 7.82% 0.01% 3731 21 168063 43999
TrendMicro-HouseCall 5.77% 0.21% 2754 350 167734 44976
Antiy-AVL 3.69% 0.00% 1763 8 168076 45967
BitDefender 3.49% 0.00% 1665 0 168084 46065
Ad-Aware 0.22% 0.00% 104 0 168084 47626
Baidu 0.18% 0.03% 88 43 168041 47642
Babable 0.00% 0.00% 0 0 168084 47730
TotalGoodware 168084
TotalMalware 47730
TotalSample 215814

Please send an email to lxu@trustlook.com if you have any comments. Thanks.