VirusTotal APK Malware Detection Data - Week 27: 20200629-20200705

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200629_20200705.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.33% 0.11% 19319 73 64201 130
Fortinet 98.88% 0.05% 19231 29 64245 218
CAT-QuickHeal 98.65% 0.13% 19186 86 64188 263
ESET-NOD32 98.60% 0.04% 19177 26 64248 272
Trustlook 97.60% 0.15% 18982 98 64176 467
Avira 97.47% 0.00% 18957 0 64274 492
Avast-Mobile 97.23% 0.17% 18910 107 64167 539
ZoneAlarm 96.58% 0.01% 18783 5 64269 666
DrWeb 96.10% 0.31% 18691 197 64077 758
AhnLab-V3 95.56% 0.03% 18585 22 64252 864
McAfee 95.43% 0.01% 18561 6 64268 888
F-Secure 95.35% 0.01% 18545 4 64270 904
Kaspersky 95.23% 0.00% 18522 2 64272 927
Ikarus 94.16% 0.17% 18314 109 64165 1135
NANO-Antivirus 84.74% 0.04% 16482 23 64251 2967
Symantec 84.24% 0.04% 16384 25 64249 3065
Sophos 83.74% 0.23% 16286 151 64123 3163
Qihoo-360 81.89% 0.02% 15927 12 64262 3522
AVG 63.67% 0.07% 12383 48 64226 7066
Ad-Aware 0.54% 0.00% 105 0 64274 19344
McAfee-GW-Edition 0.00% 0.00% 0 0 64274 19449
TotalGoodware 64274
TotalMalware 19449
TotalSample 83723

Please send an email to lxu@trustlook.com if you have any comments. Thanks.