VirusTotal APK Malware Detection Data - Week 26: 20200622-20200628

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200622_20200628.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.84% 0.03% 11945 27 81740 19
K7GW 99.65% 0.13% 11922 104 81663 42
Fortinet 99.61% 0.05% 11917 42 81725 47
Trustlook 99.36% 0.17% 11888 140 81627 76
Avira 98.65% 0.00% 11803 0 81767 161
Avast-Mobile 98.38% 0.13% 11770 110 81657 194
ZoneAlarm 98.33% 0.01% 11764 7 81760 200
DrWeb 97.98% 0.17% 11722 143 81624 242
Kaspersky 97.91% 0.01% 11714 5 81762 250
AhnLab-V3 97.73% 0.03% 11693 26 81741 271
CAT-QuickHeal 97.55% 0.02% 11671 15 81752 293
F-Secure 97.27% 0.01% 11637 6 81761 327
Ikarus 96.30% 0.13% 11521 103 81664 443
McAfee 93.69% 0.00% 11209 2 81765 755
Sophos 93.29% 0.03% 11161 21 81746 803
NANO-Antivirus 84.39% 0.03% 10097 27 81740 1867
Qihoo-360 83.41% 0.02% 9979 19 81748 1985
Symantec 81.40% 0.02% 9739 14 81753 2225
AVG 71.21% 0.04% 8520 34 81733 3444
McAfee-GW-Edition 47.01% 0.00% 5624 1 81766 6340
Ad-Aware 0.46% 0.00% 55 0 81767 11909
TotalGoodware 81767
TotalMalware 11964
TotalSample 93731

Please send an email to lxu@trustlook.com if you have any comments. Thanks.