VirusTotal APK Malware Detection Data - Week 48: 20201123-20201129

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20201123_20201129.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.34% 0.07% 15718 85 130502 105
Trustlook 99.18% 0.08% 15694 110 130477 129
ESET-NOD32 99.12% 0.01% 15683 15 130572 140
Avast-Mobile 98.89% 0.26% 15647 346 130241 176
Fortinet 97.94% 0.01% 15497 13 130574 326
Avira 97.90% 0.00% 15491 2 130585 332
AhnLab-V3 97.47% 0.01% 15422 12 130575 401
DrWeb 96.74% 0.10% 15307 128 130459 516
ZoneAlarm 96.42% 0.01% 15256 8 130579 567
Ikarus 96.20% 0.07% 15222 94 130493 601
Kaspersky 95.94% 0.00% 15180 1 130586 643
F-Secure 93.93% 0.00% 14862 3 130584 961
CAT-QuickHeal 92.75% 0.01% 14676 12 130575 1147
Sophos 85.61% 0.02% 13546 22 130565 2277
McAfee 78.90% 0.00% 12485 1 130586 3338
NANO-Antivirus 77.76% 0.02% 12304 27 130560 3519
Qihoo-360 75.82% 0.00% 11997 6 130581 3826
McAfee-GW-Edition 70.89% 0.01% 11217 15 130572 4606
AVG 69.25% 0.01% 10958 16 130571 4865
Symantec 58.06% 0.01% 9187 12 130575 6636
Ad-Aware 0.76% 0.00% 121 0 130587 15702
TotalGoodware 130587
TotalMalware 15823
TotalSample 146410

Please send an email to lxu@trustlook.com if you have any comments. Thanks.