December 16, 2019

VirusTotal APK Malware Detection Data - Week 50: 20191209-201901215

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20191209_20191215.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.64% 0.03% 22292 14 55361 80
K7GW 99.31% 0.12% 22218 69 55306 154
Fortinet 98.78% 0.01% 22100 3 55372 272
ZoneAlarm 98.57% 0.04% 22053 23 55352 319
Kaspersky 98.07% 0.04% 21940 21 55354 432
Ikarus 97.97% 0.20% 21917 109 55266 455
Trustlook 97.51% 0.25% 21816 137 55238 556
DrWeb 97.17% 0.13% 21738 74 55301 634
F-Secure 96.99% 0.02% 21698 10 55365 674
Avira 96.56% 0.00% 21602 2 55373 770
AhnLab-V3 96.46% 0.06% 21581 32 55343 791
McAfee 91.70% 0.01% 20515 8 55367 1857
Symantec 83.28% 0.04% 18631 22 55353 3741
Qihoo-360 82.61% 0.06% 18481 32 55343 3891
Avast-Mobile 81.54% 0.22% 18242 122 55253 4130
Sophos 73.82% 0.02% 16515 9 55366 5857
CAT-QuickHeal 73.22% 0.04% 16381 24 55351 5991
NANO-Antivirus 61.80% 0.04% 13827 22 55353 8545
McAfee-GW-Edition 55.28% 0.00% 12367 2 55373 10005
Cyren 42.19% 0.00% 9438 1 55374 12934
AVG 36.98% 0.07% 8273 38 55337 14099
Avast 36.30% 0.06% 8122 34 55341 14250
MAX 28.53% 0.00% 6382 0 55375 15990
Tencent 25.58% 0.00% 5723 0 55375 16649
Rising 4.85% 0.03% 1086 15 55360 21286
TrendMicro-HouseCall 4.00% 0.03% 894 18 55357 21478
BitDefender 3.42% 0.00% 766 0 55375 21606
Antiy-AVL 2.99% 0.00% 670 0 55375 21702
Baidu 0.09% 0.01% 20 5 55370 22352
Ad-Aware 0.08% 0.00% 18 0 55375 22354
Babable 0.00% 0.00% 0 0 55375 22372
TotalGoodware 55375
TotalMalware 22372
TotalSample 77747

Please send an email to lxu@trustlook.com if you have any comments. Thanks.