VirusTotal APK Malware Detection Data - Week 52: 20201221-20201227

by Lifan Xu

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its detection results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20201221_20201227.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
Avast-Mobile 99.84% 0.30% 110798 138 45575 180
Trustlook 99.64% 0.09% 110581 42 45671 397
ZoneAlarm 99.50% 0.01% 110428 3 45710 550
CAT-QuickHeal 99.39% 0.01% 110301 3 45710 677
ESET-NOD32 99.28% 0.01% 110176 4 45709 802
Kaspersky 99.28% 0.00% 110176 1 45712 802
Fortinet 99.20% 0.01% 110095 4 45709 883
Sophos 98.71% 0.02% 109547 11 45702 1431
DrWeb 98.06% 0.23% 108827 105 45608 2151
Avira 97.40% 0.00% 108091 0 45713 2887
Ikarus 95.97% 0.11% 106503 48 45665 4475
AVG 95.92% 0.03% 106448 13 45700 4530
F-Secure 93.69% 0.00% 103975 0 45713 7003
K7GW 93.44% 0.05% 103695 22 45691 7283
NANO-Antivirus 90.75% 0.03% 100718 14 45699 10260
AhnLab-V3 89.66% 0.04% 99498 16 45697 11480
McAfee 81.94% 0.01% 90931 3 45710 20047
Qihoo-360 81.03% 0.01% 89928 6 45707 21050
McAfee-GW-Edition 75.79% 0.02% 84113 7 45706 26865
Symantec 53.47% 0.00% 59337 1 45712 51641
Ad-Aware 0.27% 0.00% 301 0 45713 110677
TotalGoodware 45713
TotalMalware 110978
TotalSample 156691

Please send an email to lxu@trustlook.com if you have any comments. Thanks.