October 14, 2019

VirusTotal APK Malware Detection Data - Week 41: 20191007-201901013

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20191007_20191013.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.38% 0.08% 47318 339 399109 296
K7GW 99.35% 0.26% 47306 1021 398427 308
Avast-Mobile 98.20% 0.08% 46756 324 399124 858
ZoneAlarm 96.96% 0.04% 46168 165 399283 1446
Avira 96.51% 0.00% 45951 13 399435 1663
Trustlook 96.24% 0.21% 45824 848 398600 1790
DrWeb 95.23% 0.58% 45343 2333 397115 2271
McAfee 94.59% 0.01% 45036 20 399428 2578
AhnLab-V3 93.72% 0.04% 44626 155 399293 2988
Symantec 91.44% 0.06% 43538 244 399204 4076
Kaspersky 91.13% 0.02% 43393 66 399382 4221
NANO-Antivirus 90.93% 0.11% 43294 428 399020 4320
Ikarus 88.86% 0.32% 42310 1273 398175 5304
Fortinet 87.26% 0.02% 41549 96 399352 6065
F-Secure 85.83% 0.01% 40865 23 399425 6749
Sophos 85.09% 0.09% 40517 356 399092 7097
Qihoo-360 84.55% 0.02% 40257 84 399364 7357
CAT-QuickHeal 77.29% 0.27% 36802 1095 398353 10812
Tencent 76.60% 0.08% 36470 320 399128 11144
McAfee-GW-Edition 76.21% 0.00% 36288 5 399443 11326
MAX 56.83% 0.00% 27060 18 399430 20554
Cyren 53.61% 0.02% 25527 73 399375 22087
AVG 46.56% 0.02% 22168 70 399378 25446
Avast 44.09% 0.02% 20995 62 399386 26619
Rising 7.97% 0.02% 3796 68 399380 43818
Antiy-AVL 5.82% 0.01% 2773 39 399409 44841
TrendMicro-HouseCall 5.25% 0.06% 2501 240 399208 45113
BitDefender 3.57% 0.00% 1700 2 399446 45914
Baidu 0.24% 0.01% 115 41 399407 47499
Ad-Aware 0.22% 0.00% 107 0 399448 47507
Babable 0.00% 0.00% 2 2 399446 47612
TotalGoodware 399448
TotalMalware 47614
TotalSample 447062

Please send an email to lxu@trustlook.com if you have any comments. Thanks.