2019 |
Luckner, Marcin; Gad, Michal; Sobkowiak, Pawel Antyscam-Practical web spam classifier Journal Article International Journal of Electronics and Telecommunications, 65 (4), pp. 713–722, 2019, ISSN: 23001933. Abstract | Links | BibTeX | Tagi: Automatic classification, Imbalanced sets classification, Machine learning, Spam detection, Web spam detection @article{Luckner2019b, title = {Antyscam-Practical web spam classifier}, author = {Marcin Luckner and Michal Gad and Pawel Sobkowiak}, doi = {10.24425/ijet.2019.130255}, issn = {23001933}, year = {2019}, date = {2019-01-01}, journal = {International Journal of Electronics and Telecommunications}, volume = {65}, number = {4}, pages = {713--722}, abstract = {To avoid of manipulating search engines results by web spam, anti spam system use machine learning techniques to detect spam. However, if the learning set for the system is out of date the quality of classification falls rapidly. We present the web spam recognition system that periodically refreshes the learning set to create an adequate classifier. A new classifier is trained exclusively on data collected during the last period. We have proved that such strategy is better than an incrementation of the learning set. The system solves the starting-up issues of lacks in learning set by minimisation of learning examples and utilization of external data sets. The system was tested on real data from the spam traps and common known web services: Quora, Reddit, and Stack Overflow. The test performed among ten months shows stability of the system and improvement of the results up to 60 percent at the end of the examined period.}, keywords = {Automatic classification, Imbalanced sets classification, Machine learning, Spam detection, Web spam detection}, pubstate = {published}, tppubtype = {article} } To avoid of manipulating search engines results by web spam, anti spam system use machine learning techniques to detect spam. However, if the learning set for the system is out of date the quality of classification falls rapidly. We present the web spam recognition system that periodically refreshes the learning set to create an adequate classifier. A new classifier is trained exclusively on data collected during the last period. We have proved that such strategy is better than an incrementation of the learning set. The system solves the starting-up issues of lacks in learning set by minimisation of learning examples and utilization of external data sets. The system was tested on real data from the spam traps and common known web services: Quora, Reddit, and Stack Overflow. The test performed among ten months shows stability of the system and improvement of the results up to 60 percent at the end of the examined period. |
Luckner, Marcin; Gad, Michal; Sobkowiak, Pawel Antyscam-Practical web spam classifier Journal Article International Journal of Electronics and Telecommunications, 65 (4), pp. 713–722, 2019, ISSN: 23001933. Abstract | Links | BibTeX | Tagi: Automatic classification, Imbalanced sets classification, Machine learning, Spam detection, Web spam detection @article{Luckner2019c, title = {Antyscam-Practical web spam classifier}, author = {Marcin Luckner and Michal Gad and Pawel Sobkowiak}, doi = {10.24425/ijet.2019.130255}, issn = {23001933}, year = {2019}, date = {2019-01-01}, journal = {International Journal of Electronics and Telecommunications}, volume = {65}, number = {4}, pages = {713--722}, abstract = {To avoid of manipulating search engines results by web spam, anti spam system use machine learning techniques to detect spam. However, if the learning set for the system is out of date the quality of classification falls rapidly. We present the web spam recognition system that periodically refreshes the learning set to create an adequate classifier. A new classifier is trained exclusively on data collected during the last period. We have proved that such strategy is better than an incrementation of the learning set. The system solves the starting-up issues of lacks in learning set by minimisation of learning examples and utilization of external data sets. The system was tested on real data from the spam traps and common known web services: Quora, Reddit, and Stack Overflow. The test performed among ten months shows stability of the system and improvement of the results up to 60 percent at the end of the examined period.}, keywords = {Automatic classification, Imbalanced sets classification, Machine learning, Spam detection, Web spam detection}, pubstate = {published}, tppubtype = {article} } To avoid of manipulating search engines results by web spam, anti spam system use machine learning techniques to detect spam. However, if the learning set for the system is out of date the quality of classification falls rapidly. We present the web spam recognition system that periodically refreshes the learning set to create an adequate classifier. A new classifier is trained exclusively on data collected during the last period. We have proved that such strategy is better than an incrementation of the learning set. The system solves the starting-up issues of lacks in learning set by minimisation of learning examples and utilization of external data sets. The system was tested on real data from the spam traps and common known web services: Quora, Reddit, and Stack Overflow. The test performed among ten months shows stability of the system and improvement of the results up to 60 percent at the end of the examined period. |
2014 |
Filasiak, Robert; Grzenda, Maciej; Luckner, Marcin; Zawistowski, Pawel On the testing of network cyber threat detection methods on spam example Journal Article Annales des Telecommunications/Annals of Telecommunications, 69 (7-8), pp. 363–377, 2014, ISSN: 19589395. Abstract | Links | BibTeX | Tagi: Flow analysis, Network data sets, Network Intrusion Detection Systems (NIDS), Spam detection @article{Filasiak2014, title = {On the testing of network cyber threat detection methods on spam example}, author = {Robert Filasiak and Maciej Grzenda and Marcin Luckner and Pawel Zawistowski}, url = {http://dx.doi.org/10.1007/s12243-013-0412-5}, doi = {10.1007/s12243-013-0412-5}, issn = {19589395}, year = {2014}, date = {2014-01-01}, journal = {Annales des Telecommunications/Annals of Telecommunications}, volume = {69}, number = {7-8}, pages = {363--377}, publisher = {Springer Paris}, abstract = {As a response to the increasing number of cyber threats, novel detection and prevention methods are constantly being developed. One of the main obstacles hindering the development and evaluation of such methods is the shortage of reference data sets. What is proposed in this work is a way of testing methods detecting network threats. It includes a procedure for creating realistic reference data sets describing network threats and the processing and use of these data sets in testing environments. The proposed approach is illustrated and validated on the basis of the problem of spam detection. Reference data sets for spam detection are developed, analysed and used to both generate the requested volume of simulated traffic and analyse it using machine learning algorithms. The tests take into account both the accuracy and performance of threat detection methods under real load and constrained computing resources. textcopyright 2014 The Author(s).}, keywords = {Flow analysis, Network data sets, Network Intrusion Detection Systems (NIDS), Spam detection}, pubstate = {published}, tppubtype = {article} } As a response to the increasing number of cyber threats, novel detection and prevention methods are constantly being developed. One of the main obstacles hindering the development and evaluation of such methods is the shortage of reference data sets. What is proposed in this work is a way of testing methods detecting network threats. It includes a procedure for creating realistic reference data sets describing network threats and the processing and use of these data sets in testing environments. The proposed approach is illustrated and validated on the basis of the problem of spam detection. Reference data sets for spam detection are developed, analysed and used to both generate the requested volume of simulated traffic and analyse it using machine learning algorithms. The tests take into account both the accuracy and performance of threat detection methods under real load and constrained computing resources. textcopyright 2014 The Author(s). |
Filasiak, Robert; Grzenda, Maciej; Luckner, Marcin; Zawistowski, Pawel On the testing of network cyber threat detection methods on spam example Journal Article Annales des Telecommunications/Annals of Telecommunications, 69 (7-8), pp. 363–377, 2014, ISSN: 19589395. Abstract | Links | BibTeX | Tagi: Flow analysis, Network data sets, Network Intrusion Detection Systems (NIDS), Spam detection @article{Filasiak2014b, title = {On the testing of network cyber threat detection methods on spam example}, author = {Robert Filasiak and Maciej Grzenda and Marcin Luckner and Pawel Zawistowski}, url = {http://dx.doi.org/10.1007/s12243-013-0412-5}, doi = {10.1007/s12243-013-0412-5}, issn = {19589395}, year = {2014}, date = {2014-01-01}, journal = {Annales des Telecommunications/Annals of Telecommunications}, volume = {69}, number = {7-8}, pages = {363--377}, publisher = {Springer Paris}, abstract = {As a response to the increasing number of cyber threats, novel detection and prevention methods are constantly being developed. One of the main obstacles hindering the development and evaluation of such methods is the shortage of reference data sets. What is proposed in this work is a way of testing methods detecting network threats. It includes a procedure for creating realistic reference data sets describing network threats and the processing and use of these data sets in testing environments. The proposed approach is illustrated and validated on the basis of the problem of spam detection. Reference data sets for spam detection are developed, analysed and used to both generate the requested volume of simulated traffic and analyse it using machine learning algorithms. The tests take into account both the accuracy and performance of threat detection methods under real load and constrained computing resources. textcopyright 2014 The Author(s).}, keywords = {Flow analysis, Network data sets, Network Intrusion Detection Systems (NIDS), Spam detection}, pubstate = {published}, tppubtype = {article} } As a response to the increasing number of cyber threats, novel detection and prevention methods are constantly being developed. One of the main obstacles hindering the development and evaluation of such methods is the shortage of reference data sets. What is proposed in this work is a way of testing methods detecting network threats. It includes a procedure for creating realistic reference data sets describing network threats and the processing and use of these data sets in testing environments. The proposed approach is illustrated and validated on the basis of the problem of spam detection. Reference data sets for spam detection are developed, analysed and used to both generate the requested volume of simulated traffic and analyse it using machine learning algorithms. The tests take into account both the accuracy and performance of threat detection methods under real load and constrained computing resources. textcopyright 2014 The Author(s). |
2013 |
Luckner, Marcin; Filasiak, Robert Reference data sets for spam detection: Creation, analysis, propagation Inproceedings Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 212–221, 2013, ISSN: 03029743. Abstract | Links | BibTeX | Tagi: Anomaly detection, Flow analysis, Hybrid classifiers, Reference sets, Spam detection @inproceedings{Luckner2013, title = {Reference data sets for spam detection: Creation, analysis, propagation}, author = {Marcin Luckner and Robert Filasiak}, doi = {10.1007/978-3-642-40846-5_22}, issn = {03029743}, year = {2013}, date = {2013-01-01}, booktitle = {Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)}, volume = {8073 LNAI}, pages = {212--221}, abstract = {A reference set is a set of data of network traffic whose form and content allows detecting an event or a group of events. Realistic and representative datasets based on real traffic can improve research in the fields of intruders and anomaly detection. Creating reference sets tackles a number of issues such as the collection and storage of large volumes of data, the privacy of information and the relevance of collected events. Moreover, rare events are hard to analyse among background traffic and need specialist detection tools. One of the common problems that can be detected in network traffic is spam. This paper presents the methodology for creating a network traffic reference set for spam detection. The methodology concerns the selection of significant features, the collection and storage of data, the analysis of the collected data, the enrichment of the data with additional events and the propagation of the set. Moreover, a hybrid classifier that detects spam on relatively high level is presented. textcopyright 2013 Springer-Verlag.}, keywords = {Anomaly detection, Flow analysis, Hybrid classifiers, Reference sets, Spam detection}, pubstate = {published}, tppubtype = {inproceedings} } A reference set is a set of data of network traffic whose form and content allows detecting an event or a group of events. Realistic and representative datasets based on real traffic can improve research in the fields of intruders and anomaly detection. Creating reference sets tackles a number of issues such as the collection and storage of large volumes of data, the privacy of information and the relevance of collected events. Moreover, rare events are hard to analyse among background traffic and need specialist detection tools. One of the common problems that can be detected in network traffic is spam. This paper presents the methodology for creating a network traffic reference set for spam detection. The methodology concerns the selection of significant features, the collection and storage of data, the analysis of the collected data, the enrichment of the data with additional events and the propagation of the set. Moreover, a hybrid classifier that detects spam on relatively high level is presented. textcopyright 2013 Springer-Verlag. |
Luckner, Marcin; Filasiak, Robert Flow-level Spam Modelling using separate data sources Inproceedings Computer Science and Information Systems (FedCSIS), 2013 Federated Conference on, pp. 91 – 98, IEEE, 2013. Abstract | Links | BibTeX | Tagi: Detection, Flow analysis, Intrusion, Network, Network data sets, Spam detection, Systems (NIDS) @inproceedings{Luckner2013b, title = {Flow-level Spam Modelling using separate data sources}, author = {Marcin Luckner and Robert Filasiak}, url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6643981}, year = {2013}, date = {2013-01-01}, booktitle = {Computer Science and Information Systems (FedCSIS), 2013 Federated Conference on}, pages = {91 -- 98}, publisher = {IEEE}, abstract = {Spam detection based on flow-level statistics is a new approach in anti-spam techniques. The approach reduces number of collected data but still can obtain relative good results in a spam detection task. The main problems in the approach are selection of flow-level features that describe spam and detection of discrimination rules. In this work, flow-level model of spam is presented. The model describes spam subclasses and brings information about major features of a spam detection task. The model is the base for decision trees that detect spam. The analysis of detectors, which was learned from data collected from different mail servers, results in the universal spam description consists of the most significant features. Flows described by selected features and collected on Broadband Remote Access Server were analysed by an ensemble of created classifiers. The ensemble detected major sources of spam among senders IP addresses.}, keywords = {Detection, Flow analysis, Intrusion, Network, Network data sets, Spam detection, Systems (NIDS)}, pubstate = {published}, tppubtype = {inproceedings} } Spam detection based on flow-level statistics is a new approach in anti-spam techniques. The approach reduces number of collected data but still can obtain relative good results in a spam detection task. The main problems in the approach are selection of flow-level features that describe spam and detection of discrimination rules. In this work, flow-level model of spam is presented. The model describes spam subclasses and brings information about major features of a spam detection task. The model is the base for decision trees that detect spam. The analysis of detectors, which was learned from data collected from different mail servers, results in the universal spam description consists of the most significant features. Flows described by selected features and collected on Broadband Remote Access Server were analysed by an ensemble of created classifiers. The ensemble detected major sources of spam among senders IP addresses. |
Luckner, Marcin; Filasiak, Robert Flow-level Spam Modelling using separate data sources Inproceedings Computer Science and Information Systems (FedCSIS), 2013 Federated Conference on, pp. 91 – 98, IEEE, 2013. Abstract | Links | BibTeX | Tagi: Detection, Flow analysis, Intrusion, Network, Network data sets, Spam detection, Systems (NIDS) @inproceedings{Luckner2013bb, title = {Flow-level Spam Modelling using separate data sources}, author = {Marcin Luckner and Robert Filasiak}, url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6643981}, year = {2013}, date = {2013-01-01}, booktitle = {Computer Science and Information Systems (FedCSIS), 2013 Federated Conference on}, pages = {91 -- 98}, publisher = {IEEE}, abstract = {Spam detection based on flow-level statistics is a new approach in anti-spam techniques. The approach reduces number of collected data but still can obtain relative good results in a spam detection task. The main problems in the approach are selection of flow-level features that describe spam and detection of discrimination rules. In this work, flow-level model of spam is presented. The model describes spam subclasses and brings information about major features of a spam detection task. The model is the base for decision trees that detect spam. The analysis of detectors, which was learned from data collected from different mail servers, results in the universal spam description consists of the most significant features. Flows described by selected features and collected on Broadband Remote Access Server were analysed by an ensemble of created classifiers. The ensemble detected major sources of spam among senders IP addresses.}, keywords = {Detection, Flow analysis, Intrusion, Network, Network data sets, Spam detection, Systems (NIDS)}, pubstate = {published}, tppubtype = {inproceedings} } Spam detection based on flow-level statistics is a new approach in anti-spam techniques. The approach reduces number of collected data but still can obtain relative good results in a spam detection task. The main problems in the approach are selection of flow-level features that describe spam and detection of discrimination rules. In this work, flow-level model of spam is presented. The model describes spam subclasses and brings information about major features of a spam detection task. The model is the base for decision trees that detect spam. The analysis of detectors, which was learned from data collected from different mail servers, results in the universal spam description consists of the most significant features. Flows described by selected features and collected on Broadband Remote Access Server were analysed by an ensemble of created classifiers. The ensemble detected major sources of spam among senders IP addresses. |
Luckner, Marcin; Filasiak, Robert Reference data sets for spam detection: Creation, analysis, propagation Inproceedings Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 212–221, 2013, ISSN: 03029743. Abstract | Links | BibTeX | Tagi: Anomaly detection, Flow analysis, Hybrid classifiers, Reference sets, Spam detection @inproceedings{Luckner2013f, title = {Reference data sets for spam detection: Creation, analysis, propagation}, author = {Marcin Luckner and Robert Filasiak}, doi = {10.1007/978-3-642-40846-5_22}, issn = {03029743}, year = {2013}, date = {2013-01-01}, booktitle = {Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)}, volume = {8073 LNAI}, pages = {212--221}, abstract = {A reference set is a set of data of network traffic whose form and content allows detecting an event or a group of events. Realistic and representative datasets based on real traffic can improve research in the fields of intruders and anomaly detection. Creating reference sets tackles a number of issues such as the collection and storage of large volumes of data, the privacy of information and the relevance of collected events. Moreover, rare events are hard to analyse among background traffic and need specialist detection tools. One of the common problems that can be detected in network traffic is spam. This paper presents the methodology for creating a network traffic reference set for spam detection. The methodology concerns the selection of significant features, the collection and storage of data, the analysis of the collected data, the enrichment of the data with additional events and the propagation of the set. Moreover, a hybrid classifier that detects spam on relatively high level is presented. textcopyright 2013 Springer-Verlag.}, keywords = {Anomaly detection, Flow analysis, Hybrid classifiers, Reference sets, Spam detection}, pubstate = {published}, tppubtype = {inproceedings} } A reference set is a set of data of network traffic whose form and content allows detecting an event or a group of events. Realistic and representative datasets based on real traffic can improve research in the fields of intruders and anomaly detection. Creating reference sets tackles a number of issues such as the collection and storage of large volumes of data, the privacy of information and the relevance of collected events. Moreover, rare events are hard to analyse among background traffic and need specialist detection tools. One of the common problems that can be detected in network traffic is spam. This paper presents the methodology for creating a network traffic reference set for spam detection. The methodology concerns the selection of significant features, the collection and storage of data, the analysis of the collected data, the enrichment of the data with additional events and the propagation of the set. Moreover, a hybrid classifier that detects spam on relatively high level is presented. textcopyright 2013 Springer-Verlag. |
Publikacje
2019 |
Antyscam-Practical web spam classifier Journal Article International Journal of Electronics and Telecommunications, 65 (4), pp. 713–722, 2019, ISSN: 23001933. |
Antyscam-Practical web spam classifier Journal Article International Journal of Electronics and Telecommunications, 65 (4), pp. 713–722, 2019, ISSN: 23001933. |
2014 |
On the testing of network cyber threat detection methods on spam example Journal Article Annales des Telecommunications/Annals of Telecommunications, 69 (7-8), pp. 363–377, 2014, ISSN: 19589395. |
On the testing of network cyber threat detection methods on spam example Journal Article Annales des Telecommunications/Annals of Telecommunications, 69 (7-8), pp. 363–377, 2014, ISSN: 19589395. |
2013 |
Reference data sets for spam detection: Creation, analysis, propagation Inproceedings Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 212–221, 2013, ISSN: 03029743. |
Flow-level Spam Modelling using separate data sources Inproceedings Computer Science and Information Systems (FedCSIS), 2013 Federated Conference on, pp. 91 – 98, IEEE, 2013. |
Flow-level Spam Modelling using separate data sources Inproceedings Computer Science and Information Systems (FedCSIS), 2013 Federated Conference on, pp. 91 – 98, IEEE, 2013. |
Reference data sets for spam detection: Creation, analysis, propagation Inproceedings Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 212–221, 2013, ISSN: 03029743. |