March 2, 2020

VirusTotal APK Malware Detection Data - Week 9: 20200224-20200301

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200224_20200301.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
ESET-NOD32 99.83% 0.04% 17155 33 88809 30
K7GW 99.53% 0.07% 17105 62 88780 80
Trustlook 98.97% 0.31% 17008 274 88568 177
ZoneAlarm 98.49% 0.01% 16925 11 88831 260
McAfee 98.20% 0.01% 16876 8 88834 309
AhnLab-V3 97.65% 0.01% 16782 11 88831 403
Kaspersky 97.65% 0.01% 16781 9 88833 404
Ikarus 97.28% 0.13% 16717 119 88723 468
DrWeb 97.20% 0.08% 16703 71 88771 482
Avira 96.95% 0.00% 16660 1 88841 525
F-Secure 95.04% 0.00% 16332 1 88841 853
Qihoo-360 89.74% 0.03% 15422 23 88819 1763
Sophos 88.85% 0.02% 15269 20 88822 1916
Avast-Mobile 88.12% 0.21% 15144 190 88652 2041
Tencent 80.65% 0.10% 13859 89 88753 3326
CAT-QuickHeal 72.38% 0.02% 12439 18 88824 4746
NANO-Antivirus 65.96% 0.02% 11335 16 88826 5850
Symantec 65.39% 0.03% 11238 31 88811 5947
McAfee-GW-Edition 62.64% 0.00% 10765 3 88839 6420
AVG 59.79% 0.07% 10275 62 88780 6910
Avast 59.06% 0.07% 10149 61 88781 7036
MAX 56.19% 0.00% 9656 3 88839 7529
Cyren 42.97% 0.00% 7384 0 88842 9801
Fortinet 37.42% 0.00% 6431 0 88842 10754
Rising 5.62% 0.02% 966 14 88828 16219
Antiy-AVL 3.85% 0.00% 661 0 88842 16524
BitDefender 3.65% 0.00% 627 0 88842 16558
TrendMicro-HouseCall 2.68% 0.02% 461 14 88828 16724
Ad-Aware 0.57% 0.00% 98 0 88842 17087
Baidu 0.04% 0.00% 7 1 88841 17178
Babable 0.00% 0.00% 0 0 88842 17185
TotalGoodware 88842
TotalMalware 17185
TotalSample 106027

Please send an email to lxu@trustlook.com if you have any comments. Thanks.