March 10, 2020

VirusTotal APK Malware Detection Data - Week 10: 20200302-20200308

At Trustlook, we monitor live feed from VirusTotal (VT). On a daily basis, we collect APK samples from VT along with detection results from Anti-Virus (AV) vendors hosted on VT. Using a conservative labeling policy, we are able to select thousands of benign and malicious APK samples from millions of live feed samples. Then we look at detection results from AV vendors and rate them by how many malware they have detected and how many benign samples they have misclassified.

We generate a CSV file recording the detection results everyday. In the CSV file, from left to right, the columns are MD5 hash of the APK, label where 1 means positive (malicious) and 0 means negative (benign), and one column for each vendor showing its deteciton results where 1 means positive and 0 means negative.

On a weekly basis, we publish the detection results and zip the CSV files to AWS S3. For this week, you can download the detection data from:

https://virustotal-results.s3-us-west-1.amazonaws.com/VirusTotal_Results_20200302_20200308.zip

The weekly results are summarized in the table below and here is a simple explanation of the columns in the table:

  • Vendor: AV engine vendor
  • TPR: True Positive Rate, percentage of positive (malware) samples being correctly classified as positive
  • FPR: False Positive Rate, percentage of negative (goodware) samples being misclassified as positive
  • TP: True Positive, number of positive (malware) samples being correctly classified as positive
  • FP: False Positive, number of negative (goodware) samples being misclassified as positive
  • TN: True Negative, number of negative (goodware) samples being correctly classified as negative
  • FN: False Negative, number of positive (malware) samples being misclassified as negative
Vendor TPR FPR TP FP TN FN
K7GW 99.32% 0.16% 15021 118 74515 103
ESET-NOD32 99.05% 0.08% 14980 61 74572 144
Avira 97.68% 0.00% 14773 1 74632 351
ZoneAlarm 97.64% 0.01% 14767 5 74628 357
Ikarus 97.36% 0.18% 14725 136 74497 399
Kaspersky 96.95% 0.00% 14663 1 74632 461
DrWeb 96.51% 0.15% 14596 109 74524 528
F-Secure 95.97% 0.01% 14515 4 74629 609
McAfee 95.95% 0.01% 14511 9 74624 613
Trustlook 95.69% 0.33% 14472 246 74387 652
AhnLab-V3 92.87% 0.04% 14046 33 74600 1078
CAT-QuickHeal 87.31% 0.10% 13205 72 74561 1919
Avast-Mobile 86.38% 0.27% 13064 202 74431 2060
Qihoo-360 81.69% 0.04% 12355 30 74603 2769
Sophos 80.01% 0.04% 12100 31 74602 3024
NANO-Antivirus 75.20% 0.04% 11374 30 74603 3750
Tencent 69.26% 0.11% 10475 80 74553 4649
McAfee-GW-Edition 65.19% 0.00% 9860 3 74630 5264
Symantec 58.75% 0.05% 8885 34 74599 6239
AVG 51.09% 0.08% 7727 61 74572 7397
Avast 50.30% 0.08% 7607 57 74576 7517
Cyren 49.82% 0.01% 7535 4 74629 7589
MAX 38.00% 0.01% 5747 7 74626 9377
Fortinet 36.35% 0.00% 5497 0 74633 9627
Rising 7.84% 0.03% 1186 20 74613 13938
Antiy-AVL 6.28% 0.00% 950 2 74631 14174
BitDefender 5.25% 0.00% 794 0 74633 14330
TrendMicro-HouseCall 3.15% 0.02% 476 12 74621 14648
Baidu 0.11% 0.00% 16 1 74632 15108
Ad-Aware 0.10% 0.00% 15 0 74633 15109
Babable 0.00% 0.00% 0 0 74633 15124
TotalGoodware 74633
TotalMalware 15124
TotalSample 89757

Please send an email to lxu@trustlook.com if you have any comments. Thanks.