VirusTotal APK Malware Detection Data - Week 39: 20200921-20200927

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200921_20200927.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.74% 0.07% 13650 29 43558 36
ESET-NOD32 99.68% 0.01% 13642 3 43584 44
Trustlook 99.02% 0.10% 13552 42 43545 134
Avira 98.83% 0.00% 13526 1 43586 160
Fortinet 98.66% 0.02% 13503 8 43579 183
CAT-QuickHeal 98.60% 0.03% 13494 11 43576 192
DrWeb 98.22% 0.14% 13443 59 43528 243
Avast-Mobile 98.11% 0.15% 13428 67 43520 258
AhnLab-V3 97.84% 0.05% 13391 21 43566 295
ZoneAlarm 96.16% 0.00% 13160 2 43585 526
Kaspersky 95.72% 0.01% 13100 3 43584 586
Ikarus 94.61% 0.16% 12949 68 43519 737
F-Secure 93.91% 0.00% 12852 0 43587 834
McAfee 92.52% 0.00% 12662 2 43585 1024
NANO-Antivirus 78.46% 0.03% 10738 13 43574 2948
Sophos 71.61% 0.05% 9800 21 43566 3886
Qihoo-360 71.59% 0.01% 9798 5 43582 3888
Symantec 65.91% 0.01% 9020 5 43582 4666
McAfee-GW-Edition 59.16% 0.00% 8097 1 43586 5589
AVG 46.57% 0.01% 6373 6 43581 7313
Ad-Aware 0.74% 0.00% 101 0 43587 13585
TotalGoodware 43587
TotalMalware 13686
TotalSample 57273

Please send an email to lxu@trustlook.com if you have any comments. Thanks.