December 15, 2020

VirusTotal APK Malware Detection Data - Week 50: 20201207-20201213

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20201207_20201213.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.71% 0.11% 12477 63 58785 36
Trustlook 99.44% 0.12% 12443 70 58778 70
ESET-NOD32 99.29% 0.01% 12424 6 58842 89
Avast-Mobile 99.08% 0.25% 12398 146 58702 115
Fortinet 97.99% 0.02% 12261 13 58835 252
Avira 97.93% 0.00% 12254 0 58848 259
ZoneAlarm 97.39% 0.00% 12186 0 58848 327
AhnLab-V3 97.37% 0.01% 12184 8 58840 329
Kaspersky 96.66% 0.00% 12095 0 58848 418
DrWeb 96.37% 0.20% 12059 115 58733 454
Ikarus 95.73% 0.13% 11979 75 58773 534
CAT-QuickHeal 95.68% 0.01% 11973 6 58842 540
F-Secure 93.35% 0.00% 11681 1 58847 832
Sophos 85.93% 0.03% 10752 18 58830 1761
NANO-Antivirus 79.06% 0.05% 9893 27 58821 2620
Qihoo-360 77.42% 0.04% 9688 24 58824 2825
AVG 70.34% 0.02% 8802 14 58834 3711
McAfee 68.07% 0.00% 8518 1 58847 3995
McAfee-GW-Edition 58.37% 0.01% 7304 8 58840 5209
Symantec 46.07% 0.01% 5765 8 58840 6748
Ad-Aware 0.82% 0.00% 102 0 58848 12411
TotalGoodware 58848
TotalMalware 12513
TotalSample 71361

Please send an email to lxu@trustlook.com if you have any comments. Thanks.