VirusTotal APK Malware Detection Data - Week 42: 20201012-20201018

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20201012_20201018.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.74% 0.02% 8663 11 56219 23
K7GW 99.36% 0.15% 8630 87 56143 56
Avira 98.38% 0.00% 8545 1 56229 141
DrWeb 98.19% 0.30% 8529 167 56063 157
ZoneAlarm 98.01% 0.01% 8513 6 56224 173
Fortinet 97.79% 0.02% 8494 11 56219 192
Kaspersky 96.89% 0.00% 8416 2 56228 270
CAT-QuickHeal 96.88% 0.02% 8415 9 56221 271
Trustlook 96.32% 0.13% 8366 75 56155 320
AhnLab-V3 96.25% 0.03% 8360 18 56212 326
Avast-Mobile 96.22% 0.31% 8358 173 56057 328
McAfee 95.52% 0.01% 8297 4 56226 389
Ikarus 95.13% 0.21% 8263 120 56110 423
McAfee-GW-Edition 94.46% 0.02% 8205 14 56216 481
F-Secure 86.75% 0.00% 7535 2 56228 1151
NANO-Antivirus 82.34% 0.05% 7152 30 56200 1534
Symantec 67.90% 0.02% 5898 9 56221 2788
Qihoo-360 66.24% 0.02% 5754 11 56219 2932
Sophos 65.90% 0.03% 5724 18 56212 2962
AVG 32.44% 0.03% 2818 19 56211 5868
Ad-Aware 1.52% 0.00% 132 0 56230 8554
TotalGoodware 56230
TotalMalware 8686
TotalSample 64916

Please send an email to lxu@trustlook.com if you have any comments. Thanks.