October 12, 2020

VirusTotal APK Malware Detection Data - Week 41: 20201005-20201011

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20201005_20201011.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.71% 0.01% 9150 14 96727 27
K7GW 99.47% 0.11% 9128 111 96630 49
Trustlook 98.68% 0.21% 9056 199 96542 121
Avira 98.54% 0.00% 9043 0 96741 134
DrWeb 98.40% 0.23% 9030 219 96522 147
Fortinet 98.34% 0.01% 9025 11 96730 152
Avast-Mobile 97.26% 0.18% 8926 177 96564 251
AhnLab-V3 96.38% 0.02% 8845 23 96718 332
CAT-QuickHeal 95.98% 0.01% 8808 13 96728 369
McAfee 95.33% 0.00% 8748 2 96739 429
ZoneAlarm 95.16% 0.01% 8733 10 96731 444
Ikarus 94.42% 0.11% 8665 109 96632 512
Kaspersky 94.04% 0.00% 8630 1 96740 547
McAfee-GW-Edition 93.41% 0.01% 8572 6 96735 605
F-Secure 84.77% 0.00% 7779 0 96741 1398
NANO-Antivirus 83.57% 0.04% 7669 40 96701 1508
Qihoo-360 71.89% 0.01% 6597 12 96729 2580
Symantec 71.61% 0.01% 6572 14 96727 2605
Sophos 66.18% 0.02% 6073 17 96724 3104
AVG 35.94% 0.01% 3298 12 96729 5879
Ad-Aware 2.75% 0.00% 252 0 96741 8925
TotalGoodware 96741
TotalMalware 9177
TotalSample 105918

Please send an email to lxu@trustlook.com if you have any comments. Thanks.